* [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
@ 2020-09-09 15:11 Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 01/13] monitor: Add Monitor parameter to monitor_set_cpu() Kevin Wolf
` (14 more replies)
0 siblings, 15 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Some QMP command handlers can block the main loop for a relatively long
time, for example because they perform some I/O. This is quite nasty.
Allowing such handlers to run in a coroutine where they can yield (and
therefore release the BQL) while waiting for an event such as I/O
completion solves the problem.
This series adds the infrastructure to allow this and switches
block_resize to run in a coroutine as a first example.
This is an alternative solution to Marc-André's "monitor: add
asynchronous command type" series.
v7:
- Added patch 2 to add a Monitor parameter to monitor_get_cpu_index(),
too [Markus]
- Let monitor_set_cur() return the old monitor [Markus]
- Fixed comment about linking stub objects in test-util-sockets [Markus]
- More detailed commit message for per-coroutine current monitor and
document that monitor_set_cur(NULL) must be called eventually [Markus]
- Resolve some merge conflicts on rebase
v6:
- Fixed cur_mon behaviour: It should always point to the Monitor while
we're in the handler coroutine, but be NULL while the handler
coroutines has yielded. [Markus]
- Give HMP handlers the coroutine option, too, because they will call
QMP handlers, and life is easier when we know whether we are in
coroutine context or not.
- Fixed block_resize for block devices in iothreads and for HMP
- Resolved some merge conflict with QAPI generator and monitor
refactorings that were merged in the meantime
v5:
- Improved comments and documentation [Markus]
v4:
- Forbid 'oob': true, 'coroutine': true [Markus]
- Removed Python type hints [Markus]
- Introduced separate bool qmp_dispatcher_co_shutdown to make it clearer
how a shutdown request is signalled to the dispatcher [Markus]
- Allow using aio_poll() with iohandler_ctx and use that instead of
aio_bh_poll() [Markus]
- Removed coroutine_fn from qmp_block_resize() again because at least
one caller (HMP) calls it outside of coroutine context [Markus]
- Tried to make the synchronisation between dispatcher and the monitor
thread clearer, and fixed a race condition
- Improved documentation and comments
v3:
- Fix race between monitor thread and dispatcher that could schedule the
dispatcher coroutine twice if a second requests comes in before the
dispatcher can wake up [Patchew]
v2:
- Fix typo in a commit message [Eric]
- Use hyphen instead of underscore for the test command [Eric]
- Mark qmp_block_resize() as coroutine_fn [Stefan]
Kevin Wolf (13):
monitor: Add Monitor parameter to monitor_set_cpu()
monitor: Add Monitor parameter to monitor_get_cpu_index()
monitor: Use getter/setter functions for cur_mon
hmp: Update current monitor only in handle_hmp_command()
qmp: Assert that no other monitor is active
qmp: Call monitor_set_cur() only in qmp_dispatch()
monitor: Make current monitor a per-coroutine property
qapi: Add a 'coroutine' flag for commands
qmp: Move dispatcher to a coroutine
hmp: Add support for coroutine command handlers
util/async: Add aio_co_reschedule_self()
block: Add bdrv_co_move_to_aio_context()
block: Convert 'block_resize' to coroutine
qapi/block-core.json | 3 +-
docs/devel/qapi-code-gen.txt | 12 +++
include/block/aio.h | 10 ++
include/block/block.h | 6 ++
include/monitor/monitor.h | 7 +-
include/qapi/qmp/dispatch.h | 5 +-
monitor/monitor-internal.h | 7 +-
audio/wavcapture.c | 8 +-
block.c | 10 ++
blockdev.c | 13 ++-
dump/dump.c | 2 +-
hw/core/machine-hmp-cmds.c | 2 +-
hw/scsi/vhost-scsi.c | 2 +-
hw/virtio/vhost-vsock.c | 2 +-
migration/fd.c | 4 +-
monitor/hmp-cmds.c | 4 +-
monitor/hmp.c | 44 ++++++--
monitor/misc.c | 38 ++++---
monitor/monitor.c | 101 ++++++++++++++++--
monitor/qmp-cmds-control.c | 2 +
monitor/qmp-cmds.c | 2 +-
monitor/qmp.c | 130 +++++++++++++++++-------
net/socket.c | 2 +-
net/tap.c | 6 +-
qapi/qmp-dispatch.c | 61 ++++++++++-
qapi/qmp-registry.c | 3 +
qga/main.c | 2 +-
softmmu/cpus.c | 2 +-
stubs/monitor-core.c | 10 +-
tests/test-qmp-cmds.c | 10 +-
tests/test-util-sockets.c | 12 +--
trace/control.c | 2 +-
util/aio-posix.c | 8 +-
util/async.c | 30 ++++++
util/qemu-error.c | 6 +-
util/qemu-print.c | 3 +-
util/qemu-sockets.c | 1 +
scripts/qapi/commands.py | 10 +-
scripts/qapi/doc.py | 2 +-
scripts/qapi/expr.py | 10 +-
scripts/qapi/introspect.py | 2 +-
scripts/qapi/schema.py | 12 ++-
tests/qapi-schema/test-qapi.py | 7 +-
hmp-commands.hx | 1 +
tests/qapi-schema/meson.build | 1 +
tests/qapi-schema/oob-coroutine.err | 2 +
tests/qapi-schema/oob-coroutine.json | 2 +
tests/qapi-schema/oob-coroutine.out | 0
tests/qapi-schema/qapi-schema-test.json | 1 +
tests/qapi-schema/qapi-schema-test.out | 2 +
50 files changed, 481 insertions(+), 143 deletions(-)
create mode 100644 tests/qapi-schema/oob-coroutine.err
create mode 100644 tests/qapi-schema/oob-coroutine.json
create mode 100644 tests/qapi-schema/oob-coroutine.out
--
2.25.4
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v7 01/13] monitor: Add Monitor parameter to monitor_set_cpu()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 02/13] monitor: Add Monitor parameter to monitor_get_cpu_index() Kevin Wolf
` (13 subsequent siblings)
14 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_set_cpu().
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
include/monitor/monitor.h | 2 +-
monitor/hmp-cmds.c | 2 +-
monitor/misc.c | 10 +++++-----
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 1018d754a6..0dcaefd4f9 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -33,7 +33,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
GCC_FMT_ATTR(2, 0);
int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
void monitor_flush(Monitor *mon);
-int monitor_set_cpu(int cpu_index);
+int monitor_set_cpu(Monitor *mon, int cpu_index);
int monitor_get_cpu_index(void);
void monitor_read_command(MonitorHMP *mon, int show_prompt);
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index 7711726fd2..f608b5b27b 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -996,7 +996,7 @@ void hmp_cpu(Monitor *mon, const QDict *qdict)
/* XXX: drop the monitor_set_cpu() usage when all HMP commands that
use it are converted to the QAPI */
cpu_index = qdict_get_int(qdict, "index");
- if (monitor_set_cpu(cpu_index) < 0) {
+ if (monitor_set_cpu(mon, cpu_index) < 0) {
monitor_printf(mon, "invalid CPU index\n");
}
}
diff --git a/monitor/misc.c b/monitor/misc.c
index e847b58a8c..b4f779b8d3 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -129,7 +129,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
cur_mon = &hmp.common;
if (has_cpu_index) {
- int ret = monitor_set_cpu(cpu_index);
+ int ret = monitor_set_cpu(&hmp.common, cpu_index);
if (ret < 0) {
cur_mon = old_mon;
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
@@ -255,7 +255,7 @@ static void monitor_init_qmp_commands(void)
}
/* Set the current CPU defined by the user. Callers must hold BQL. */
-int monitor_set_cpu(int cpu_index)
+int monitor_set_cpu(Monitor *mon, int cpu_index)
{
CPUState *cpu;
@@ -263,8 +263,8 @@ int monitor_set_cpu(int cpu_index)
if (cpu == NULL) {
return -1;
}
- g_free(cur_mon->mon_cpu_path);
- cur_mon->mon_cpu_path = object_get_canonical_path(OBJECT(cpu));
+ g_free(mon->mon_cpu_path);
+ mon->mon_cpu_path = object_get_canonical_path(OBJECT(cpu));
return 0;
}
@@ -285,7 +285,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)
if (!first_cpu) {
return NULL;
}
- monitor_set_cpu(first_cpu->cpu_index);
+ monitor_set_cpu(cur_mon, first_cpu->cpu_index);
cpu = first_cpu;
}
assert(cpu != NULL);
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 02/13] monitor: Add Monitor parameter to monitor_get_cpu_index()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 01/13] monitor: Add Monitor parameter to monitor_set_cpu() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon Kevin Wolf
` (12 subsequent siblings)
14 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_get_cpu_index().
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/monitor/monitor.h | 2 +-
hw/core/machine-hmp-cmds.c | 2 +-
monitor/hmp-cmds.c | 2 +-
monitor/misc.c | 20 ++++++++++----------
softmmu/cpus.c | 2 +-
5 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 0dcaefd4f9..a64502fbb2 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -34,7 +34,7 @@ int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap)
int monitor_printf(Monitor *mon, const char *fmt, ...) GCC_FMT_ATTR(2, 3);
void monitor_flush(Monitor *mon);
int monitor_set_cpu(Monitor *mon, int cpu_index);
-int monitor_get_cpu_index(void);
+int monitor_get_cpu_index(Monitor *mon);
void monitor_read_command(MonitorHMP *mon, int show_prompt);
int monitor_read_password(MonitorHMP *mon, ReadLineFunc *readline_func,
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index 39999c47c5..da7d2e27b6 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -34,7 +34,7 @@ void hmp_info_cpus(Monitor *mon, const QDict *qdict)
for (cpu = cpu_list; cpu; cpu = cpu->next) {
int active = ' ';
- if (cpu->value->cpu_index == monitor_get_cpu_index()) {
+ if (cpu->value->cpu_index == monitor_get_cpu_index(mon)) {
active = '*';
}
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index f608b5b27b..1e5e611ed9 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -1007,7 +1007,7 @@ void hmp_memsave(Monitor *mon, const QDict *qdict)
const char *filename = qdict_get_str(qdict, "filename");
uint64_t addr = qdict_get_int(qdict, "val");
Error *err = NULL;
- int cpu_index = monitor_get_cpu_index();
+ int cpu_index = monitor_get_cpu_index(mon);
if (cpu_index < 0) {
monitor_printf(mon, "No CPU available\n");
diff --git a/monitor/misc.c b/monitor/misc.c
index b4f779b8d3..972726061c 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -269,23 +269,23 @@ int monitor_set_cpu(Monitor *mon, int cpu_index)
}
/* Callers must hold BQL. */
-static CPUState *mon_get_cpu_sync(bool synchronize)
+static CPUState *mon_get_cpu_sync(Monitor *mon, bool synchronize)
{
CPUState *cpu = NULL;
- if (cur_mon->mon_cpu_path) {
- cpu = (CPUState *) object_resolve_path_type(cur_mon->mon_cpu_path,
+ if (mon->mon_cpu_path) {
+ cpu = (CPUState *) object_resolve_path_type(mon->mon_cpu_path,
TYPE_CPU, NULL);
if (!cpu) {
- g_free(cur_mon->mon_cpu_path);
- cur_mon->mon_cpu_path = NULL;
+ g_free(mon->mon_cpu_path);
+ mon->mon_cpu_path = NULL;
}
}
- if (!cur_mon->mon_cpu_path) {
+ if (!mon->mon_cpu_path) {
if (!first_cpu) {
return NULL;
}
- monitor_set_cpu(cur_mon, first_cpu->cpu_index);
+ monitor_set_cpu(mon, first_cpu->cpu_index);
cpu = first_cpu;
}
assert(cpu != NULL);
@@ -297,7 +297,7 @@ static CPUState *mon_get_cpu_sync(bool synchronize)
CPUState *mon_get_cpu(void)
{
- return mon_get_cpu_sync(true);
+ return mon_get_cpu_sync(cur_mon, true);
}
CPUArchState *mon_get_cpu_env(void)
@@ -307,9 +307,9 @@ CPUArchState *mon_get_cpu_env(void)
return cs ? cs->env_ptr : NULL;
}
-int monitor_get_cpu_index(void)
+int monitor_get_cpu_index(Monitor *mon)
{
- CPUState *cs = mon_get_cpu_sync(false);
+ CPUState *cs = mon_get_cpu_sync(mon, false);
return cs ? cs->cpu_index : UNASSIGNED_CPU_INDEX;
}
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index e3b98065c9..ae643447e9 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -2224,7 +2224,7 @@ exit:
void qmp_inject_nmi(Error **errp)
{
- nmi_monitor_handle(monitor_get_cpu_index(), errp);
+ nmi_monitor_handle(monitor_get_cpu_index(cur_mon), errp);
}
void dump_drift_info(void)
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 01/13] monitor: Add Monitor parameter to monitor_set_cpu() Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 02/13] monitor: Add Monitor parameter to monitor_get_cpu_index() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-10-02 7:51 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 04/13] hmp: Update current monitor only in handle_hmp_command() Kevin Wolf
` (11 subsequent siblings)
14 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
cur_mon really needs to be coroutine-local as soon as we move monitor
command handlers to coroutines and let them yield. As a first step, just
remove all direct accesses to cur_mon so that we can implement this in
the getter function later.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/monitor/monitor.h | 3 ++-
audio/wavcapture.c | 8 ++++----
dump/dump.c | 2 +-
hw/scsi/vhost-scsi.c | 2 +-
hw/virtio/vhost-vsock.c | 2 +-
migration/fd.c | 4 ++--
monitor/hmp.c | 11 +++++------
monitor/misc.c | 13 +++++++------
monitor/monitor.c | 24 +++++++++++++++++++++++-
monitor/qmp-cmds-control.c | 2 ++
monitor/qmp-cmds.c | 2 +-
monitor/qmp.c | 7 ++-----
net/socket.c | 2 +-
net/tap.c | 6 +++---
softmmu/cpus.c | 2 +-
stubs/monitor-core.c | 5 ++++-
tests/test-util-sockets.c | 12 +++++-------
trace/control.c | 2 +-
util/qemu-error.c | 6 +++---
util/qemu-print.c | 3 ++-
util/qemu-sockets.c | 1 +
21 files changed, 72 insertions(+), 47 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index a64502fbb2..7b0ad1de12 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -5,7 +5,6 @@
#include "qapi/qapi-types-misc.h"
#include "qemu/readline.h"
-extern __thread Monitor *cur_mon;
typedef struct MonitorHMP MonitorHMP;
typedef struct MonitorOptions MonitorOptions;
@@ -13,6 +12,8 @@ typedef struct MonitorOptions MonitorOptions;
extern QemuOptsList qemu_mon_opts;
+Monitor *monitor_cur(void);
+Monitor *monitor_set_cur(Monitor *mon);
bool monitor_cur_is_qmp(void);
void monitor_init_globals(void);
diff --git a/audio/wavcapture.c b/audio/wavcapture.c
index 17e87ed6f4..c60286e162 100644
--- a/audio/wavcapture.c
+++ b/audio/wavcapture.c
@@ -1,5 +1,5 @@
#include "qemu/osdep.h"
-#include "monitor/monitor.h"
+#include "qemu/qemu-print.h"
#include "qapi/error.h"
#include "qemu/error-report.h"
#include "audio.h"
@@ -94,9 +94,9 @@ static void wav_capture_info (void *opaque)
WAVState *wav = opaque;
char *path = wav->path;
- monitor_printf (cur_mon, "Capturing audio(%d,%d,%d) to %s: %d bytes\n",
- wav->freq, wav->bits, wav->nchannels,
- path ? path : "<not available>", wav->bytes);
+ qemu_printf("Capturing audio(%d,%d,%d) to %s: %d bytes\n",
+ wav->freq, wav->bits, wav->nchannels,
+ path ? path : "<not available>", wav->bytes);
}
static struct capture_ops wav_capture_ops = {
diff --git a/dump/dump.c b/dump/dump.c
index 383bc7876b..232027e92c 100644
--- a/dump/dump.c
+++ b/dump/dump.c
@@ -1986,7 +1986,7 @@ void qmp_dump_guest_memory(bool paging, const char *file,
#if !defined(WIN32)
if (strstart(file, "fd:", &p)) {
- fd = monitor_get_fd(cur_mon, p, errp);
+ fd = monitor_get_fd(monitor_cur(), p, errp);
if (fd == -1) {
return;
}
diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c
index a83ffeefc8..4d70fa036b 100644
--- a/hw/scsi/vhost-scsi.c
+++ b/hw/scsi/vhost-scsi.c
@@ -177,7 +177,7 @@ static void vhost_scsi_realize(DeviceState *dev, Error **errp)
}
if (vs->conf.vhostfd) {
- vhostfd = monitor_fd_param(cur_mon, vs->conf.vhostfd, errp);
+ vhostfd = monitor_fd_param(monitor_cur(), vs->conf.vhostfd, errp);
if (vhostfd == -1) {
error_prepend(errp, "vhost-scsi: unable to parse vhostfd: ");
return;
diff --git a/hw/virtio/vhost-vsock.c b/hw/virtio/vhost-vsock.c
index c8f0699b4f..f9db4beb47 100644
--- a/hw/virtio/vhost-vsock.c
+++ b/hw/virtio/vhost-vsock.c
@@ -143,7 +143,7 @@ static void vhost_vsock_device_realize(DeviceState *dev, Error **errp)
}
if (vsock->conf.vhostfd) {
- vhostfd = monitor_fd_param(cur_mon, vsock->conf.vhostfd, errp);
+ vhostfd = monitor_fd_param(monitor_cur(), vsock->conf.vhostfd, errp);
if (vhostfd == -1) {
error_prepend(errp, "vhost-vsock: unable to parse vhostfd: ");
return;
diff --git a/migration/fd.c b/migration/fd.c
index 0a29ecdebf..6f2f50475f 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -26,7 +26,7 @@
void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error **errp)
{
QIOChannel *ioc;
- int fd = monitor_get_fd(cur_mon, fdname, errp);
+ int fd = monitor_get_fd(monitor_cur(), fdname, errp);
if (fd == -1) {
return;
}
@@ -55,7 +55,7 @@ static gboolean fd_accept_incoming_migration(QIOChannel *ioc,
void fd_start_incoming_migration(const char *fdname, Error **errp)
{
QIOChannel *ioc;
- int fd = monitor_fd_param(cur_mon, fdname, errp);
+ int fd = monitor_fd_param(monitor_cur(), fdname, errp);
if (fd == -1) {
return;
}
diff --git a/monitor/hmp.c b/monitor/hmp.c
index d598dd02bb..561e32d02f 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -1300,12 +1300,11 @@ cleanup:
static void monitor_read(void *opaque, const uint8_t *buf, int size)
{
- MonitorHMP *mon;
- Monitor *old_mon = cur_mon;
+ MonitorHMP *mon = container_of(opaque, MonitorHMP, common);
+ Monitor *old_mon;
int i;
- cur_mon = opaque;
- mon = container_of(cur_mon, MonitorHMP, common);
+ old_mon = monitor_set_cur(&mon->common);
if (mon->rs) {
for (i = 0; i < size; i++) {
@@ -1313,13 +1312,13 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size)
}
} else {
if (size == 0 || buf[size - 1] != 0) {
- monitor_printf(cur_mon, "corrupted command\n");
+ monitor_printf(&mon->common, "corrupted command\n");
} else {
handle_hmp_command(mon, (char *)buf);
}
}
- cur_mon = old_mon;
+ monitor_set_cur(old_mon);
}
static void monitor_event(void *opaque, QEMUChrEvent event)
diff --git a/monitor/misc.c b/monitor/misc.c
index 972726061c..a5eb3025f8 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -125,13 +125,12 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
monitor_data_init(&hmp.common, false, true, false);
- old_mon = cur_mon;
- cur_mon = &hmp.common;
+ old_mon = monitor_set_cur(&hmp.common);
if (has_cpu_index) {
int ret = monitor_set_cpu(&hmp.common, cpu_index);
if (ret < 0) {
- cur_mon = old_mon;
+ monitor_set_cur(old_mon);
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
"a CPU number");
goto out;
@@ -139,7 +138,7 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
}
handle_hmp_command(&hmp, command_line);
- cur_mon = old_mon;
+ monitor_set_cur(old_mon);
qemu_mutex_lock(&hmp.common.mon_lock);
if (qstring_get_length(hmp.common.outbuf) > 0) {
@@ -297,7 +296,7 @@ static CPUState *mon_get_cpu_sync(Monitor *mon, bool synchronize)
CPUState *mon_get_cpu(void)
{
- return mon_get_cpu_sync(cur_mon, true);
+ return mon_get_cpu_sync(monitor_cur(), true);
}
CPUArchState *mon_get_cpu_env(void)
@@ -1232,6 +1231,7 @@ static void hmp_acl_remove(Monitor *mon, const QDict *qdict)
void qmp_getfd(const char *fdname, Error **errp)
{
+ Monitor *cur_mon = monitor_cur();
mon_fd_t *monfd;
int fd, tmp_fd;
@@ -1272,6 +1272,7 @@ void qmp_getfd(const char *fdname, Error **errp)
void qmp_closefd(const char *fdname, Error **errp)
{
+ Monitor *cur_mon = monitor_cur();
mon_fd_t *monfd;
int tmp_fd;
@@ -1361,7 +1362,7 @@ AddfdInfo *qmp_add_fd(bool has_fdset_id, int64_t fdset_id, bool has_opaque,
const char *opaque, Error **errp)
{
int fd;
- Monitor *mon = cur_mon;
+ Monitor *mon = monitor_cur();
AddfdInfo *fdinfo;
fd = qemu_chr_fe_get_msgfd(&mon->chr);
diff --git a/monitor/monitor.c b/monitor/monitor.c
index b385a3d569..be3839a7aa 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -66,13 +66,31 @@ MonitorList mon_list;
int mon_refcount;
static bool monitor_destroyed;
-__thread Monitor *cur_mon;
+static __thread Monitor *cur_monitor;
+
+Monitor *monitor_cur(void)
+{
+ return cur_monitor;
+}
+
+/**
+ * Sets a new current monitor and returns the old one.
+ */
+Monitor *monitor_set_cur(Monitor *mon)
+{
+ Monitor *old_monitor = cur_monitor;
+
+ cur_monitor = mon;
+ return old_monitor;
+}
/**
* Is the current monitor, if any, a QMP monitor?
*/
bool monitor_cur_is_qmp(void)
{
+ Monitor *cur_mon = monitor_cur();
+
return cur_mon && monitor_is_qmp(cur_mon);
}
@@ -209,6 +227,8 @@ int monitor_printf(Monitor *mon, const char *fmt, ...)
*/
int error_vprintf(const char *fmt, va_list ap)
{
+ Monitor *cur_mon = monitor_cur();
+
if (cur_mon && !monitor_cur_is_qmp()) {
return monitor_vprintf(cur_mon, fmt, ap);
}
@@ -217,6 +237,8 @@ int error_vprintf(const char *fmt, va_list ap)
int error_vprintf_unless_qmp(const char *fmt, va_list ap)
{
+ Monitor *cur_mon = monitor_cur();
+
if (!cur_mon) {
return vfprintf(stderr, fmt, ap);
}
diff --git a/monitor/qmp-cmds-control.c b/monitor/qmp-cmds-control.c
index 8f04cfa6e6..a456762f6a 100644
--- a/monitor/qmp-cmds-control.c
+++ b/monitor/qmp-cmds-control.c
@@ -69,6 +69,7 @@ static bool qmp_caps_accept(MonitorQMP *mon, QMPCapabilityList *list,
void qmp_qmp_capabilities(bool has_enable, QMPCapabilityList *enable,
Error **errp)
{
+ Monitor *cur_mon = monitor_cur();
MonitorQMP *mon;
assert(monitor_is_qmp(cur_mon));
@@ -119,6 +120,7 @@ static void query_commands_cb(const QmpCommand *cmd, void *opaque)
CommandInfoList *qmp_query_commands(Error **errp)
{
CommandInfoList *list = NULL;
+ Monitor *cur_mon = monitor_cur();
MonitorQMP *mon;
assert(monitor_is_qmp(cur_mon));
diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
index 864cbfa32e..c7bf6bb4dc 100644
--- a/monitor/qmp-cmds.c
+++ b/monitor/qmp-cmds.c
@@ -327,7 +327,7 @@ void qmp_add_client(const char *protocol, const char *fdname,
Chardev *s;
int fd;
- fd = monitor_get_fd(cur_mon, fdname, errp);
+ fd = monitor_get_fd(monitor_cur(), fdname, errp);
if (fd < 0) {
return;
}
diff --git a/monitor/qmp.c b/monitor/qmp.c
index d433ceae5b..bb2d9d0cc7 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -139,12 +139,9 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
QDict *rsp;
QDict *error;
- old_mon = cur_mon;
- cur_mon = &mon->common;
-
+ old_mon = monitor_set_cur(&mon->common);
rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
- cur_mon = old_mon;
+ monitor_set_cur(old_mon);
if (mon->commands == &qmp_cap_negotiation_commands) {
error = qdict_get_qdict(rsp, "error");
diff --git a/net/socket.c b/net/socket.c
index 2d21fddd9c..15b410e8d8 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -727,7 +727,7 @@ int net_init_socket(const Netdev *netdev, const char *name,
if (sock->has_fd) {
int fd, ret;
- fd = monitor_fd_param(cur_mon, sock->fd, errp);
+ fd = monitor_fd_param(monitor_cur(), sock->fd, errp);
if (fd == -1) {
return -1;
}
diff --git a/net/tap.c b/net/tap.c
index 14dc904fca..688dcfafdf 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -696,7 +696,7 @@ static void net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
if (vhostfdname) {
int ret;
- vhostfd = monitor_fd_param(cur_mon, vhostfdname, &err);
+ vhostfd = monitor_fd_param(monitor_cur(), vhostfdname, &err);
if (vhostfd == -1) {
if (tap->has_vhostforce && tap->vhostforce) {
error_propagate(errp, err);
@@ -802,7 +802,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
return -1;
}
- fd = monitor_fd_param(cur_mon, tap->fd, errp);
+ fd = monitor_fd_param(monitor_cur(), tap->fd, errp);
if (fd == -1) {
return -1;
}
@@ -856,7 +856,7 @@ int net_init_tap(const Netdev *netdev, const char *name,
}
for (i = 0; i < nfds; i++) {
- fd = monitor_fd_param(cur_mon, fds[i], errp);
+ fd = monitor_fd_param(monitor_cur(), fds[i], errp);
if (fd == -1) {
ret = -1;
goto free_fail;
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index ae643447e9..aa1b6956f3 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -2224,7 +2224,7 @@ exit:
void qmp_inject_nmi(Error **errp)
{
- nmi_monitor_handle(monitor_get_cpu_index(cur_mon), errp);
+ nmi_monitor_handle(monitor_get_cpu_index(monitor_cur()), errp);
}
void dump_drift_info(void)
diff --git a/stubs/monitor-core.c b/stubs/monitor-core.c
index 6cff1c4e1d..0cd2d864b2 100644
--- a/stubs/monitor-core.c
+++ b/stubs/monitor-core.c
@@ -3,7 +3,10 @@
#include "qemu-common.h"
#include "qapi/qapi-emit-events.h"
-__thread Monitor *cur_mon;
+Monitor *monitor_cur(void)
+{
+ return NULL;
+}
void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
{
diff --git a/tests/test-util-sockets.c b/tests/test-util-sockets.c
index af9f5c0c70..6c50dbf051 100644
--- a/tests/test-util-sockets.c
+++ b/tests/test-util-sockets.c
@@ -52,6 +52,7 @@ static void test_fd_is_socket_good(void)
static int mon_fd = -1;
static const char *mon_fdname;
+__thread Monitor *cur_mon;
int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
{
@@ -66,15 +67,12 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
/*
* Syms of stubs in libqemuutil.a are discarded at .o file granularity.
- * To replace monitor_get_fd() we must ensure everything in
- * stubs/monitor.c is defined, to make sure monitor.o is discarded
- * otherwise we get duplicate syms at link time.
+ * To replace monitor_get_fd() and monitor_cur(), we must ensure that we also
+ * replace any other symbol that is used in the binary and would be taken from
+ * the same stub object file, otherwise we get duplicate syms at link time.
*/
-__thread Monitor *cur_mon;
+Monitor *monitor_cur(void) { return cur_mon; }
int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) { abort(); }
-void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp) {}
-void monitor_init_hmp(Chardev *chr, bool use_readline, Error **errp) {}
-
static void test_socket_fd_pass_name_good(void)
{
diff --git a/trace/control.c b/trace/control.c
index c63a4de732..b35e512dce 100644
--- a/trace/control.c
+++ b/trace/control.c
@@ -176,7 +176,7 @@ void trace_enable_events(const char *line_buf)
{
if (is_help_option(line_buf)) {
trace_list_events();
- if (cur_mon == NULL) {
+ if (monitor_cur() == NULL) {
exit(0);
}
} else {
diff --git a/util/qemu-error.c b/util/qemu-error.c
index 3ee41438e9..aa30f03564 100644
--- a/util/qemu-error.c
+++ b/util/qemu-error.c
@@ -171,7 +171,7 @@ static void print_loc(void)
int i;
const char *const *argp;
- if (!cur_mon && progname) {
+ if (!monitor_cur() && progname) {
fprintf(stderr, "%s:", progname);
sep = " ";
}
@@ -208,7 +208,7 @@ static void vreport(report_type type, const char *fmt, va_list ap)
GTimeVal tv;
gchar *timestr;
- if (error_with_timestamp && !cur_mon) {
+ if (error_with_timestamp && !monitor_cur()) {
g_get_current_time(&tv);
timestr = g_time_val_to_iso8601(&tv);
error_printf("%s ", timestr);
@@ -216,7 +216,7 @@ static void vreport(report_type type, const char *fmt, va_list ap)
}
/* Only prepend guest name if -msg guest-name and -name guest=... are set */
- if (error_with_guestname && error_guest_name && !cur_mon) {
+ if (error_with_guestname && error_guest_name && !monitor_cur()) {
error_printf("%s ", error_guest_name);
}
diff --git a/util/qemu-print.c b/util/qemu-print.c
index e79d6b8396..69ba612f56 100644
--- a/util/qemu-print.c
+++ b/util/qemu-print.c
@@ -20,6 +20,7 @@
*/
int qemu_vprintf(const char *fmt, va_list ap)
{
+ Monitor *cur_mon = monitor_cur();
if (cur_mon) {
return monitor_vprintf(cur_mon, fmt, ap);
}
@@ -48,7 +49,7 @@ int qemu_printf(const char *fmt, ...)
int qemu_vfprintf(FILE *stream, const char *fmt, va_list ap)
{
if (!stream) {
- return monitor_vprintf(cur_mon, fmt, ap);
+ return monitor_vprintf(monitor_cur(), fmt, ap);
}
return vfprintf(stream, fmt, ap);
}
diff --git a/util/qemu-sockets.c b/util/qemu-sockets.c
index b37d288866..40c18ba142 100644
--- a/util/qemu-sockets.c
+++ b/util/qemu-sockets.c
@@ -1092,6 +1092,7 @@ fail:
static int socket_get_fd(const char *fdstr, int num, Error **errp)
{
+ Monitor *cur_mon = monitor_cur();
int fd;
if (num != 1) {
error_setg_errno(errp, EINVAL, "socket_get_fd: too many connections");
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 04/13] hmp: Update current monitor only in handle_hmp_command()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (2 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 05/13] qmp: Assert that no other monitor is active Kevin Wolf
` (10 subsequent siblings)
14 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
The current monitor is updated relatively early in the command handling
code even though only the command handler actually needs it.
The current monitor will become coroutine-local later, so we can only
update it when we know in which coroutine the command will be exectued.
Move it to handle_hmp_command() where this information will be
available.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
monitor/hmp.c | 10 +++++-----
monitor/misc.c | 5 -----
2 files changed, 5 insertions(+), 10 deletions(-)
diff --git a/monitor/hmp.c b/monitor/hmp.c
index 561e32d02f..896c670183 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -1061,6 +1061,7 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
QDict *qdict;
const HMPCommand *cmd;
const char *cmd_start = cmdline;
+ Monitor *old_mon;
trace_handle_hmp_command(mon, cmdline);
@@ -1079,7 +1080,11 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
return;
}
+ /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+ old_mon = monitor_set_cur(&mon->common);
cmd->cmd(&mon->common, qdict);
+ monitor_set_cur(old_mon);
+
qobject_unref(qdict);
}
@@ -1301,11 +1306,8 @@ cleanup:
static void monitor_read(void *opaque, const uint8_t *buf, int size)
{
MonitorHMP *mon = container_of(opaque, MonitorHMP, common);
- Monitor *old_mon;
int i;
- old_mon = monitor_set_cur(&mon->common);
-
if (mon->rs) {
for (i = 0; i < size; i++) {
readline_handle_byte(mon->rs, buf[i]);
@@ -1317,8 +1319,6 @@ static void monitor_read(void *opaque, const uint8_t *buf, int size)
handle_hmp_command(mon, (char *)buf);
}
}
-
- monitor_set_cur(old_mon);
}
static void monitor_event(void *opaque, QEMUChrEvent event)
diff --git a/monitor/misc.c b/monitor/misc.c
index a5eb3025f8..ba1063024c 100644
--- a/monitor/misc.c
+++ b/monitor/misc.c
@@ -120,17 +120,13 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
int64_t cpu_index, Error **errp)
{
char *output = NULL;
- Monitor *old_mon;
MonitorHMP hmp = {};
monitor_data_init(&hmp.common, false, true, false);
- old_mon = monitor_set_cur(&hmp.common);
-
if (has_cpu_index) {
int ret = monitor_set_cpu(&hmp.common, cpu_index);
if (ret < 0) {
- monitor_set_cur(old_mon);
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "cpu-index",
"a CPU number");
goto out;
@@ -138,7 +134,6 @@ char *qmp_human_monitor_command(const char *command_line, bool has_cpu_index,
}
handle_hmp_command(&hmp, command_line);
- monitor_set_cur(old_mon);
qemu_mutex_lock(&hmp.common.mon_lock);
if (qstring_get_length(hmp.common.outbuf) > 0) {
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 05/13] qmp: Assert that no other monitor is active
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (3 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 04/13] hmp: Update current monitor only in handle_hmp_command() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch() Kevin Wolf
` (9 subsequent siblings)
14 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
monitor_qmp_dispatch() is never supposed to be called in the context of
another monitor, so assert that monitor_cur() is NULL instead of saving
and restoring it.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
monitor/qmp.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index bb2d9d0cc7..8469970c69 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -140,8 +140,11 @@ static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
QDict *error;
old_mon = monitor_set_cur(&mon->common);
+ assert(old_mon == NULL);
+
rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
- monitor_set_cur(old_mon);
+
+ monitor_set_cur(NULL);
if (mon->commands == &qmp_cap_negotiation_commands) {
error = qdict_get_qdict(rsp, "error");
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (4 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 05/13] qmp: Assert that no other monitor is active Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-14 15:10 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property Kevin Wolf
` (8 subsequent siblings)
14 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
The correct way to set the current monitor for a coroutine handler will
be different than for a blocking handler, so monitor_set_cur() needs to
be called in qmp_dispatch().
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/qapi/qmp/dispatch.h | 3 ++-
monitor/qmp.c | 8 +-------
qapi/qmp-dispatch.c | 8 +++++++-
qga/main.c | 2 +-
stubs/monitor-core.c | 5 +++++
tests/test-qmp-cmds.c | 6 +++---
6 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 5a9cf82472..0c2f467028 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -14,6 +14,7 @@
#ifndef QAPI_QMP_DISPATCH_H
#define QAPI_QMP_DISPATCH_H
+#include "monitor/monitor.h"
#include "qemu/queue.h"
typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
@@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
bool qmp_has_success_response(const QmpCommand *cmd);
QDict *qmp_error_response(Error *err);
QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
- bool allow_oob);
+ bool allow_oob, Monitor *cur_mon);
bool qmp_is_oob(const QDict *dict);
typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
diff --git a/monitor/qmp.c b/monitor/qmp.c
index 8469970c69..922fdb5541 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
{
- Monitor *old_mon;
QDict *rsp;
QDict *error;
- old_mon = monitor_set_cur(&mon->common);
- assert(old_mon == NULL);
-
- rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
-
- monitor_set_cur(NULL);
+ rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
if (mon->commands == &qmp_cap_negotiation_commands) {
error = qdict_get_qdict(rsp, "error");
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 79347e0864..2fdbc0fba4 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -89,7 +89,7 @@ bool qmp_is_oob(const QDict *dict)
}
QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
- bool allow_oob)
+ bool allow_oob, Monitor *cur_mon)
{
Error *err = NULL;
bool oob;
@@ -152,7 +152,13 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
args = qdict_get_qdict(dict, "arguments");
qobject_ref(args);
}
+
+ assert(monitor_cur() == NULL);
+ monitor_set_cur(cur_mon);
+
cmd->fn(args, &ret, &err);
+
+ monitor_set_cur(NULL);
qobject_unref(args);
if (err) {
/* or assert(!ret) after reviewing all handlers: */
diff --git a/qga/main.c b/qga/main.c
index 3febf3b0fd..241779a1d6 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -578,7 +578,7 @@ static void process_event(void *opaque, QObject *obj, Error *err)
}
g_debug("processing command");
- rsp = qmp_dispatch(&ga_commands, obj, false);
+ rsp = qmp_dispatch(&ga_commands, obj, false, NULL);
end:
ret = send_response(s, rsp);
diff --git a/stubs/monitor-core.c b/stubs/monitor-core.c
index 0cd2d864b2..dc1748bf13 100644
--- a/stubs/monitor-core.c
+++ b/stubs/monitor-core.c
@@ -8,6 +8,11 @@ Monitor *monitor_cur(void)
return NULL;
}
+Monitor *monitor_set_cur(Monitor *mon)
+{
+ return NULL;
+}
+
void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
{
}
diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index d12ff47e26..5f1b245e19 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -152,7 +152,7 @@ static QObject *do_qmp_dispatch(bool allow_oob, const char *template, ...)
req = qdict_from_vjsonf_nofail(template, ap);
va_end(ap);
- resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob);
+ resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob, NULL);
g_assert(resp);
ret = qdict_get(resp, "return");
g_assert(ret);
@@ -175,7 +175,7 @@ static void do_qmp_dispatch_error(bool allow_oob, ErrorClass cls,
req = qdict_from_vjsonf_nofail(template, ap);
va_end(ap);
- resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob);
+ resp = qmp_dispatch(&qmp_commands, QOBJECT(req), allow_oob, NULL);
g_assert(resp);
error = qdict_get_qdict(resp, "error");
g_assert(error);
@@ -231,7 +231,7 @@ static void test_dispatch_cmd_success_response(void)
QDict *resp;
qdict_put_str(req, "execute", "cmd-success-response");
- resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false);
+ resp = qmp_dispatch(&qmp_commands, QOBJECT(req), false, NULL);
g_assert_null(resp);
qobject_unref(req);
}
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (5 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-14 15:11 ` Markus Armbruster
2020-10-02 7:53 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
` (7 subsequent siblings)
14 siblings, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
This way, a monitor command handler will still be able to access the
current monitor, but when it yields, all other code code will correctly
get NULL from monitor_cur().
This uses a hash table to map the coroutine pointer to the current
monitor of that coroutine. Outside of coroutine context, we associate
the current monitor with the leader coroutine of the current thread.
Approaches to implement some form of coroutine local storage directly in
the coroutine core code have been considered and discarded because they
didn't end up being much more generic than the hash table and their
performance impact on coroutines not using coroutine local storage was
unclear. As the block layer uses a coroutine per I/O request, this is a
fast path and we have to be careful. It's safest to just stay out of
this path with code only used by the monitor.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
include/monitor/monitor.h | 2 +-
monitor/hmp.c | 4 ++--
monitor/monitor.c | 34 +++++++++++++++++++++++++++-------
qapi/qmp-dispatch.c | 4 ++--
stubs/monitor-core.c | 2 +-
5 files changed, 33 insertions(+), 13 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 7b0ad1de12..026f8a31b2 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -13,7 +13,7 @@ typedef struct MonitorOptions MonitorOptions;
extern QemuOptsList qemu_mon_opts;
Monitor *monitor_cur(void);
-Monitor *monitor_set_cur(Monitor *mon);
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon);
bool monitor_cur_is_qmp(void);
void monitor_init_globals(void);
diff --git a/monitor/hmp.c b/monitor/hmp.c
index 896c670183..4b66ca1cd6 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -1081,9 +1081,9 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
}
/* old_mon is non-NULL when called from qmp_human_monitor_command() */
- old_mon = monitor_set_cur(&mon->common);
+ old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
cmd->cmd(&mon->common, qdict);
- monitor_set_cur(old_mon);
+ monitor_set_cur(qemu_coroutine_self(), old_mon);
qobject_unref(qdict);
}
diff --git a/monitor/monitor.c b/monitor/monitor.c
index be3839a7aa..629aa073ee 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -58,29 +58,48 @@ IOThread *mon_iothread;
/* Bottom half to dispatch the requests received from I/O thread */
QEMUBH *qmp_dispatcher_bh;
-/* Protects mon_list, monitor_qapi_event_state, monitor_destroyed. */
+/*
+ * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
+ * monitor_destroyed.
+ */
QemuMutex monitor_lock;
static GHashTable *monitor_qapi_event_state;
+static GHashTable *coroutine_mon; /* Maps Coroutine* to Monitor* */
MonitorList mon_list;
int mon_refcount;
static bool monitor_destroyed;
-static __thread Monitor *cur_monitor;
-
Monitor *monitor_cur(void)
{
- return cur_monitor;
+ Monitor *mon;
+
+ qemu_mutex_lock(&monitor_lock);
+ mon = g_hash_table_lookup(coroutine_mon, qemu_coroutine_self());
+ qemu_mutex_unlock(&monitor_lock);
+
+ return mon;
}
/**
* Sets a new current monitor and returns the old one.
+ *
+ * If a non-NULL monitor is set for a coroutine, another call resetting it to
+ * NULL is required before the coroutine terminates, otherwise a stale entry
+ * would remain in the hash table.
*/
-Monitor *monitor_set_cur(Monitor *mon)
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
{
- Monitor *old_monitor = cur_monitor;
+ Monitor *old_monitor = monitor_cur();
+
+ qemu_mutex_lock(&monitor_lock);
+ if (mon) {
+ g_hash_table_replace(coroutine_mon, co, mon);
+ } else {
+ g_hash_table_remove(coroutine_mon, co);
+ }
+ qemu_mutex_unlock(&monitor_lock);
- cur_monitor = mon;
return old_monitor;
}
@@ -623,6 +642,7 @@ void monitor_init_globals_core(void)
{
monitor_qapi_event_init();
qemu_mutex_init(&monitor_lock);
+ coroutine_mon = g_hash_table_new(NULL, NULL);
/*
* The dispatcher BH must run in the main loop thread, since we
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 2fdbc0fba4..5677ba92ca 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -154,11 +154,11 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
}
assert(monitor_cur() == NULL);
- monitor_set_cur(cur_mon);
+ monitor_set_cur(qemu_coroutine_self(), cur_mon);
cmd->fn(args, &ret, &err);
- monitor_set_cur(NULL);
+ monitor_set_cur(qemu_coroutine_self(), NULL);
qobject_unref(args);
if (err) {
/* or assert(!ret) after reviewing all handlers: */
diff --git a/stubs/monitor-core.c b/stubs/monitor-core.c
index dc1748bf13..d058a2a00d 100644
--- a/stubs/monitor-core.c
+++ b/stubs/monitor-core.c
@@ -8,7 +8,7 @@ Monitor *monitor_cur(void)
return NULL;
}
-Monitor *monitor_set_cur(Monitor *mon)
+Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
{
return NULL;
}
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (6 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-14 15:15 ` Markus Armbruster
` (2 more replies)
2020-09-09 15:11 ` [PATCH v7 09/13] qmp: Move dispatcher to a coroutine Kevin Wolf
` (6 subsequent siblings)
14 siblings, 3 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
This patch adds a new 'coroutine' flag to QMP command definitions that
tells the QMP dispatcher that the command handler is safe to be run in a
coroutine.
The documentation of the new flag pretends that this flag is already
used as intended, which it isn't yet after this patch. We'll implement
this in another patch in this series.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
docs/devel/qapi-code-gen.txt | 12 ++++++++++++
include/qapi/qmp/dispatch.h | 1 +
tests/test-qmp-cmds.c | 4 ++++
scripts/qapi/commands.py | 10 +++++++---
scripts/qapi/doc.py | 2 +-
scripts/qapi/expr.py | 10 ++++++++--
scripts/qapi/introspect.py | 2 +-
scripts/qapi/schema.py | 12 ++++++++----
tests/qapi-schema/test-qapi.py | 7 ++++---
tests/qapi-schema/meson.build | 1 +
tests/qapi-schema/oob-coroutine.err | 2 ++
tests/qapi-schema/oob-coroutine.json | 2 ++
tests/qapi-schema/oob-coroutine.out | 0
tests/qapi-schema/qapi-schema-test.json | 1 +
tests/qapi-schema/qapi-schema-test.out | 2 ++
15 files changed, 54 insertions(+), 14 deletions(-)
create mode 100644 tests/qapi-schema/oob-coroutine.err
create mode 100644 tests/qapi-schema/oob-coroutine.json
create mode 100644 tests/qapi-schema/oob-coroutine.out
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index f3e7ced212..36daa9b5f8 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -472,6 +472,7 @@ Syntax:
'*gen': false,
'*allow-oob': true,
'*allow-preconfig': true,
+ '*coroutine': true,
'*if': COND,
'*features': FEATURES }
@@ -596,6 +597,17 @@ before the machine is built. It defaults to false. For example:
QMP is available before the machine is built only when QEMU was
started with --preconfig.
+Member 'coroutine' tells the QMP dispatcher whether the command handler
+is safe to be run in a coroutine. It defaults to false. If it is true,
+the command handler is called from coroutine context and may yield while
+waiting for an external event (such as I/O completion) in order to avoid
+blocking the guest and other background operations.
+
+It is an error to specify both 'coroutine': true and 'allow-oob': true
+for a command. We don't currently have a use case for both together and
+without a use case, it's not entirely clear what the semantics should
+be.
+
The optional 'if' member specifies a conditional. See "Configuring
the schema" below for more on this.
diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 0c2f467028..9fd2b720a7 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -25,6 +25,7 @@ typedef enum QmpCommandOptions
QCO_NO_SUCCESS_RESP = (1U << 0),
QCO_ALLOW_OOB = (1U << 1),
QCO_ALLOW_PRECONFIG = (1U << 2),
+ QCO_COROUTINE = (1U << 3),
} QmpCommandOptions;
typedef struct QmpCommand
diff --git a/tests/test-qmp-cmds.c b/tests/test-qmp-cmds.c
index 5f1b245e19..d3413bfef0 100644
--- a/tests/test-qmp-cmds.c
+++ b/tests/test-qmp-cmds.c
@@ -36,6 +36,10 @@ void qmp_cmd_success_response(Error **errp)
{
}
+void qmp_coroutine_cmd(Error **errp)
+{
+}
+
Empty2 *qmp_user_def_cmd0(Error **errp)
{
return g_new0(Empty2, 1);
diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
index 3cf9e1110b..6e6fc94a14 100644
--- a/scripts/qapi/commands.py
+++ b/scripts/qapi/commands.py
@@ -176,7 +176,8 @@ out:
return ret
-def gen_register_command(name, success_response, allow_oob, allow_preconfig):
+def gen_register_command(name, success_response, allow_oob, allow_preconfig,
+ coroutine):
options = []
if not success_response:
@@ -185,6 +186,8 @@ def gen_register_command(name, success_response, allow_oob, allow_preconfig):
options += ['QCO_ALLOW_OOB']
if allow_preconfig:
options += ['QCO_ALLOW_PRECONFIG']
+ if coroutine:
+ options += ['QCO_COROUTINE']
if not options:
options = ['QCO_NO_OPTIONS']
@@ -267,7 +270,7 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
def visit_command(self, name, info, ifcond, features,
arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
+ allow_oob, allow_preconfig, coroutine):
if not gen:
return
# FIXME: If T is a user-defined type, the user is responsible
@@ -285,7 +288,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
self._genh.add(gen_marshal_decl(name))
self._genc.add(gen_marshal(name, arg_type, boxed, ret_type))
self._regy.add(gen_register_command(name, success_response,
- allow_oob, allow_preconfig))
+ allow_oob, allow_preconfig,
+ coroutine))
def gen_commands(schema, output_dir, prefix):
diff --git a/scripts/qapi/doc.py b/scripts/qapi/doc.py
index 92f584edcf..11771de923 100644
--- a/scripts/qapi/doc.py
+++ b/scripts/qapi/doc.py
@@ -264,7 +264,7 @@ class QAPISchemaGenDocVisitor(QAPISchemaVisitor):
def visit_command(self, name, info, ifcond, features,
arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
+ allow_oob, allow_preconfig, coroutine):
doc = self.cur_doc
self._gen.add(texi_msg('Command', doc, ifcond,
texi_arguments(doc,
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 2942520399..928cd1eb5c 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -88,10 +88,16 @@ def check_flags(expr, info):
if key in expr and expr[key] is not False:
raise QAPISemError(
info, "flag '%s' may only use false value" % key)
- for key in ['boxed', 'allow-oob', 'allow-preconfig']:
+ for key in ['boxed', 'allow-oob', 'allow-preconfig', 'coroutine']:
if key in expr and expr[key] is not True:
raise QAPISemError(
info, "flag '%s' may only use true value" % key)
+ if 'allow-oob' in expr and 'coroutine' in expr:
+ # This is not necessarily a fundamental incompatibility, but we don't
+ # have a use case and the desired semantics isn't obvious. The simplest
+ # solution is to forbid it until we get a use case for it.
+ raise QAPISemError(info, "flags 'allow-oob' and 'coroutine' "
+ "are incompatible")
def check_if(expr, info, source):
@@ -342,7 +348,7 @@ def check_exprs(exprs):
['command'],
['data', 'returns', 'boxed', 'if', 'features',
'gen', 'success-response', 'allow-oob',
- 'allow-preconfig'])
+ 'allow-preconfig', 'coroutine'])
normalize_members(expr.get('data'))
check_command(expr, info)
elif meta == 'event':
diff --git a/scripts/qapi/introspect.py b/scripts/qapi/introspect.py
index 23652be810..5907b09cd5 100644
--- a/scripts/qapi/introspect.py
+++ b/scripts/qapi/introspect.py
@@ -216,7 +216,7 @@ const QLitObject %(c_name)s = %(c_string)s;
def visit_command(self, name, info, ifcond, features,
arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
+ allow_oob, allow_preconfig, coroutine):
arg_type = arg_type or self._schema.the_empty_object_type
ret_type = ret_type or self._schema.the_empty_object_type
obj = {'arg-type': self._use_type(arg_type),
diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 78309a00f0..c44d391c3f 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -128,7 +128,7 @@ class QAPISchemaVisitor:
def visit_command(self, name, info, ifcond, features,
arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
+ allow_oob, allow_preconfig, coroutine):
pass
def visit_event(self, name, info, ifcond, features, arg_type, boxed):
@@ -713,7 +713,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
def __init__(self, name, info, doc, ifcond, features,
arg_type, ret_type,
- gen, success_response, boxed, allow_oob, allow_preconfig):
+ gen, success_response, boxed, allow_oob, allow_preconfig,
+ coroutine):
super().__init__(name, info, doc, ifcond, features)
assert not arg_type or isinstance(arg_type, str)
assert not ret_type or isinstance(ret_type, str)
@@ -726,6 +727,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
self.boxed = boxed
self.allow_oob = allow_oob
self.allow_preconfig = allow_preconfig
+ self.coroutine = coroutine
def check(self, schema):
super().check(schema)
@@ -768,7 +770,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
visitor.visit_command(
self.name, self.info, self.ifcond, self.features,
self.arg_type, self.ret_type, self.gen, self.success_response,
- self.boxed, self.allow_oob, self.allow_preconfig)
+ self.boxed, self.allow_oob, self.allow_preconfig, self.coroutine)
class QAPISchemaEvent(QAPISchemaEntity):
@@ -1074,6 +1076,7 @@ class QAPISchema:
boxed = expr.get('boxed', False)
allow_oob = expr.get('allow-oob', False)
allow_preconfig = expr.get('allow-preconfig', False)
+ coroutine = expr.get('coroutine', False)
ifcond = expr.get('if')
features = self._make_features(expr.get('features'), info)
if isinstance(data, OrderedDict):
@@ -1086,7 +1089,8 @@ class QAPISchema:
self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, features,
data, rets,
gen, success_response,
- boxed, allow_oob, allow_preconfig))
+ boxed, allow_oob, allow_preconfig,
+ coroutine))
def _def_event(self, expr, info, doc):
name = expr['event']
diff --git a/tests/qapi-schema/test-qapi.py b/tests/qapi-schema/test-qapi.py
index f396b471eb..e8db9d09d9 100755
--- a/tests/qapi-schema/test-qapi.py
+++ b/tests/qapi-schema/test-qapi.py
@@ -68,12 +68,13 @@ class QAPISchemaTestVisitor(QAPISchemaVisitor):
def visit_command(self, name, info, ifcond, features,
arg_type, ret_type, gen, success_response, boxed,
- allow_oob, allow_preconfig):
+ allow_oob, allow_preconfig, coroutine):
print('command %s %s -> %s'
% (name, arg_type and arg_type.name,
ret_type and ret_type.name))
- print(' gen=%s success_response=%s boxed=%s oob=%s preconfig=%s'
- % (gen, success_response, boxed, allow_oob, allow_preconfig))
+ print(' gen=%s success_response=%s boxed=%s oob=%s preconfig=%s%s'
+ % (gen, success_response, boxed, allow_oob, allow_preconfig,
+ " coroutine=True" if coroutine else ""))
self._print_if(ifcond)
self._print_features(features)
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index c87d141417..7a9bc69cc0 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -141,6 +141,7 @@ schemas = [
'nested-struct-data.json',
'nested-struct-data-invalid-dict.json',
'non-objects.json',
+ 'oob-coroutine.json',
'oob-test.json',
'allow-preconfig-test.json',
'pragma-doc-required-crap.json',
diff --git a/tests/qapi-schema/oob-coroutine.err b/tests/qapi-schema/oob-coroutine.err
new file mode 100644
index 0000000000..c01a4992bd
--- /dev/null
+++ b/tests/qapi-schema/oob-coroutine.err
@@ -0,0 +1,2 @@
+oob-coroutine.json: In command 'oob-command-1':
+oob-coroutine.json:2: flags 'allow-oob' and 'coroutine' are incompatible
diff --git a/tests/qapi-schema/oob-coroutine.json b/tests/qapi-schema/oob-coroutine.json
new file mode 100644
index 0000000000..0f67663bcd
--- /dev/null
+++ b/tests/qapi-schema/oob-coroutine.json
@@ -0,0 +1,2 @@
+# Check that incompatible flags allow-oob and coroutine are rejected
+{ 'command': 'oob-command-1', 'allow-oob': true, 'coroutine': true }
diff --git a/tests/qapi-schema/oob-coroutine.out b/tests/qapi-schema/oob-coroutine.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/qapi-schema/qapi-schema-test.json b/tests/qapi-schema/qapi-schema-test.json
index 3a9f2cbb33..63f92adf68 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -148,6 +148,7 @@
'returns': 'UserDefTwo' }
{ 'command': 'cmd-success-response', 'data': {}, 'success-response': false }
+{ 'command': 'coroutine-cmd', 'data': {}, 'coroutine': true }
# Returning a non-dictionary requires a name from the whitelist
{ 'command': 'guest-get-time', 'data': {'a': 'int', '*b': 'int' },
diff --git a/tests/qapi-schema/qapi-schema-test.out b/tests/qapi-schema/qapi-schema-test.out
index 891b4101e0..8868ca0dca 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -203,6 +203,8 @@ command user_def_cmd2 q_obj_user_def_cmd2-arg -> UserDefTwo
gen=True success_response=True boxed=False oob=False preconfig=False
command cmd-success-response None -> None
gen=True success_response=False boxed=False oob=False preconfig=False
+command coroutine-cmd None -> None
+ gen=True success_response=True boxed=False oob=False preconfig=False coroutine=True
object q_obj_guest-get-time-arg
member a: int optional=False
member b: int optional=True
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 09/13] qmp: Move dispatcher to a coroutine
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (7 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-14 15:30 ` Markus Armbruster
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 10/13] hmp: Add support for coroutine command handlers Kevin Wolf
` (5 subsequent siblings)
14 siblings, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
This moves the QMP dispatcher to a coroutine and runs all QMP command
handlers that declare 'coroutine': true in coroutine context so they
can avoid blocking the main loop while doing I/O or waiting for other
events.
For commands that are not declared safe to run in a coroutine, the
dispatcher drops out of coroutine context by calling the QMP command
handler from a bottom half.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
---
include/qapi/qmp/dispatch.h | 1 +
monitor/monitor-internal.h | 6 +-
monitor/monitor.c | 55 +++++++++++++---
monitor/qmp.c | 122 +++++++++++++++++++++++++++---------
qapi/qmp-dispatch.c | 61 ++++++++++++++++--
qapi/qmp-registry.c | 3 +
util/aio-posix.c | 8 ++-
7 files changed, 210 insertions(+), 46 deletions(-)
diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
index 9fd2b720a7..af8d96c570 100644
--- a/include/qapi/qmp/dispatch.h
+++ b/include/qapi/qmp/dispatch.h
@@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
typedef struct QmpCommand
{
const char *name;
+ /* Runs in coroutine context if QCO_COROUTINE is set */
QmpCommandFunc *fn;
QmpCommandOptions options;
QTAILQ_ENTRY(QmpCommand) node;
diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
index b39e03b744..b55d6df07f 100644
--- a/monitor/monitor-internal.h
+++ b/monitor/monitor-internal.h
@@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
extern IOThread *mon_iothread;
-extern QEMUBH *qmp_dispatcher_bh;
+extern Coroutine *qmp_dispatcher_co;
+extern bool qmp_dispatcher_co_shutdown;
+extern bool qmp_dispatcher_co_busy;
extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
extern QemuMutex monitor_lock;
extern MonitorList mon_list;
@@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
void monitor_data_destroy_qmp(MonitorQMP *mon);
-void monitor_qmp_bh_dispatcher(void *data);
+void coroutine_fn monitor_qmp_dispatcher_co(void *data);
int get_monitor_def(int64_t *pval, const char *name);
void help_cmd(Monitor *mon, const char *name);
diff --git a/monitor/monitor.c b/monitor/monitor.c
index 629aa073ee..ac2722bf91 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -55,8 +55,32 @@ typedef struct {
/* Shared monitor I/O thread */
IOThread *mon_iothread;
-/* Bottom half to dispatch the requests received from I/O thread */
-QEMUBH *qmp_dispatcher_bh;
+/* Coroutine to dispatch the requests received from I/O thread */
+Coroutine *qmp_dispatcher_co;
+
+/* Set to true when the dispatcher coroutine should terminate */
+bool qmp_dispatcher_co_shutdown;
+
+/*
+ * qmp_dispatcher_co_busy is used for synchronisation between the
+ * monitor thread and the main thread to ensure that the dispatcher
+ * coroutine never gets scheduled a second time when it's already
+ * scheduled (scheduling the same coroutine twice is forbidden).
+ *
+ * It is true if the coroutine is active and processing requests.
+ * Additional requests may then be pushed onto mon->qmp_requests,
+ * and @qmp_dispatcher_co_shutdown may be set without further ado.
+ * @qmp_dispatcher_co_busy must not be woken up in this case.
+ *
+ * If false, you also have to set @qmp_dispatcher_co_busy to true and
+ * wake up @qmp_dispatcher_co after pushing the new requests.
+ *
+ * The coroutine will automatically change this variable back to false
+ * before it yields. Nobody else may set the variable to false.
+ *
+ * Access must be atomic for thread safety.
+ */
+bool qmp_dispatcher_co_busy;
/*
* Protects mon_list, monitor_qapi_event_state, coroutine_mon,
@@ -623,9 +647,24 @@ void monitor_cleanup(void)
}
qemu_mutex_unlock(&monitor_lock);
- /* QEMUBHs needs to be deleted before destroying the I/O thread */
- qemu_bh_delete(qmp_dispatcher_bh);
- qmp_dispatcher_bh = NULL;
+ /*
+ * The dispatcher needs to stop before destroying the I/O thread.
+ *
+ * We need to poll both qemu_aio_context and iohandler_ctx to make
+ * sure that the dispatcher coroutine keeps making progress and
+ * eventually terminates. qemu_aio_context is automatically
+ * polled by calling AIO_WAIT_WHILE on it, but we must poll
+ * iohandler_ctx manually.
+ */
+ qmp_dispatcher_co_shutdown = true;
+ if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+ aio_co_wake(qmp_dispatcher_co);
+ }
+
+ AIO_WAIT_WHILE(qemu_get_aio_context(),
+ (aio_poll(iohandler_get_aio_context(), false),
+ atomic_mb_read(&qmp_dispatcher_co_busy)));
+
if (mon_iothread) {
iothread_destroy(mon_iothread);
mon_iothread = NULL;
@@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
* have commands assuming that context. It would be nice to get
* rid of those assumptions.
*/
- qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
- monitor_qmp_bh_dispatcher,
- NULL);
+ qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
+ atomic_mb_set(&qmp_dispatcher_co_busy, true);
+ aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
}
int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
diff --git a/monitor/qmp.c b/monitor/qmp.c
index 922fdb5541..69f6e93f38 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -133,6 +133,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
}
}
+/*
+ * Runs outside of coroutine context for OOB commands, but in
+ * coroutine context for everything else.
+ */
static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
{
QDict *rsp;
@@ -205,43 +209,99 @@ static QMPRequest *monitor_qmp_requests_pop_any_with_lock(void)
return req_obj;
}
-void monitor_qmp_bh_dispatcher(void *data)
+void coroutine_fn monitor_qmp_dispatcher_co(void *data)
{
- QMPRequest *req_obj = monitor_qmp_requests_pop_any_with_lock();
+ QMPRequest *req_obj = NULL;
QDict *rsp;
bool need_resume;
MonitorQMP *mon;
- if (!req_obj) {
- return;
- }
+ while (true) {
+ assert(atomic_mb_read(&qmp_dispatcher_co_busy) == true);
- mon = req_obj->mon;
- /* qmp_oob_enabled() might change after "qmp_capabilities" */
- need_resume = !qmp_oob_enabled(mon) ||
- mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
- qemu_mutex_unlock(&mon->qmp_queue_lock);
- if (req_obj->req) {
- QDict *qdict = qobject_to(QDict, req_obj->req);
- QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
- trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
- monitor_qmp_dispatch(mon, req_obj->req);
- } else {
- assert(req_obj->err);
- rsp = qmp_error_response(req_obj->err);
- req_obj->err = NULL;
- monitor_qmp_respond(mon, rsp);
- qobject_unref(rsp);
- }
+ /*
+ * Mark the dispatcher as not busy already here so that we
+ * don't miss any new requests coming in the middle of our
+ * processing.
+ */
+ atomic_mb_set(&qmp_dispatcher_co_busy, false);
+
+ while (!(req_obj = monitor_qmp_requests_pop_any_with_lock())) {
+ /*
+ * No more requests to process. Wait to be reentered from
+ * handle_qmp_command() when it pushes more requests, or
+ * from monitor_cleanup() when it requests shutdown.
+ */
+ if (!qmp_dispatcher_co_shutdown) {
+ qemu_coroutine_yield();
+
+ /*
+ * busy must be set to true again by whoever
+ * rescheduled us to avoid double scheduling
+ */
+ assert(atomic_xchg(&qmp_dispatcher_co_busy, false) == true);
+ }
+
+ /*
+ * qmp_dispatcher_co_shutdown may have changed if we
+ * yielded and were reentered from monitor_cleanup()
+ */
+ if (qmp_dispatcher_co_shutdown) {
+ return;
+ }
+ }
- if (need_resume) {
- /* Pairs with the monitor_suspend() in handle_qmp_command() */
- monitor_resume(&mon->common);
- }
- qmp_request_free(req_obj);
+ if (atomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
+ /*
+ * Someone rescheduled us (probably because a new requests
+ * came in), but we didn't actually yield. Do that now,
+ * only to be immediately reentered and removed from the
+ * list of scheduled coroutines.
+ */
+ qemu_coroutine_yield();
+ }
- /* Reschedule instead of looping so the main loop stays responsive */
- qemu_bh_schedule(qmp_dispatcher_bh);
+ /*
+ * Move the coroutine from iohandler_ctx to qemu_aio_context for
+ * executing the command handler so that it can make progress if it
+ * involves an AIO_WAIT_WHILE().
+ */
+ aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
+ qemu_coroutine_yield();
+
+ mon = req_obj->mon;
+ /* qmp_oob_enabled() might change after "qmp_capabilities" */
+ need_resume = !qmp_oob_enabled(mon) ||
+ mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
+ qemu_mutex_unlock(&mon->qmp_queue_lock);
+ if (req_obj->req) {
+ QDict *qdict = qobject_to(QDict, req_obj->req);
+ QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
+ trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
+ monitor_qmp_dispatch(mon, req_obj->req);
+ } else {
+ assert(req_obj->err);
+ rsp = qmp_error_response(req_obj->err);
+ req_obj->err = NULL;
+ monitor_qmp_respond(mon, rsp);
+ qobject_unref(rsp);
+ }
+
+ if (need_resume) {
+ /* Pairs with the monitor_suspend() in handle_qmp_command() */
+ monitor_resume(&mon->common);
+ }
+ qmp_request_free(req_obj);
+
+ /*
+ * Yield and reschedule so the main loop stays responsive.
+ *
+ * Move back to iohandler_ctx so that nested event loops for
+ * qemu_aio_context don't start new monitor commands.
+ */
+ aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
+ qemu_coroutine_yield();
+ }
}
static void handle_qmp_command(void *opaque, QObject *req, Error *err)
@@ -302,7 +362,9 @@ static void handle_qmp_command(void *opaque, QObject *req, Error *err)
qemu_mutex_unlock(&mon->qmp_queue_lock);
/* Kick the dispatcher routine */
- qemu_bh_schedule(qmp_dispatcher_bh);
+ if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
+ aio_co_wake(qmp_dispatcher_co);
+ }
}
static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
index 5677ba92ca..754f7b854c 100644
--- a/qapi/qmp-dispatch.c
+++ b/qapi/qmp-dispatch.c
@@ -12,12 +12,16 @@
*/
#include "qemu/osdep.h"
+
+#include "block/aio.h"
#include "qapi/error.h"
#include "qapi/qmp/dispatch.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qjson.h"
#include "sysemu/runstate.h"
#include "qapi/qmp/qbool.h"
+#include "qemu/coroutine.h"
+#include "qemu/main-loop.h"
static QDict *qmp_dispatch_check_obj(QDict *dict, bool allow_oob,
Error **errp)
@@ -88,6 +92,30 @@ bool qmp_is_oob(const QDict *dict)
&& !qdict_haskey(dict, "execute");
}
+typedef struct QmpDispatchBH {
+ const QmpCommand *cmd;
+ Monitor *cur_mon;
+ QDict *args;
+ QObject **ret;
+ Error **errp;
+ Coroutine *co;
+} QmpDispatchBH;
+
+static void do_qmp_dispatch_bh(void *opaque)
+{
+ QmpDispatchBH *data = opaque;
+
+ assert(monitor_cur() == NULL);
+ monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
+ data->cmd->fn(data->args, data->ret, data->errp);
+ monitor_set_cur(qemu_coroutine_self(), NULL);
+ aio_co_wake(data->co);
+}
+
+/*
+ * Runs outside of coroutine context for OOB commands, but in coroutine
+ * context for everything else.
+ */
QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
bool allow_oob, Monitor *cur_mon)
{
@@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
qobject_ref(args);
}
+ assert(!(oob && qemu_in_coroutine()));
assert(monitor_cur() == NULL);
- monitor_set_cur(qemu_coroutine_self(), cur_mon);
-
- cmd->fn(args, &ret, &err);
-
- monitor_set_cur(qemu_coroutine_self(), NULL);
+ if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
+ monitor_set_cur(qemu_coroutine_self(), cur_mon);
+ cmd->fn(args, &ret, &err);
+ monitor_set_cur(qemu_coroutine_self(), NULL);
+ } else {
+ /*
+ * Not being in coroutine context implies that we're handling
+ * an OOB command, which must not have QCO_COROUTINE.
+ *
+ * This implies that we are in coroutine context, but the
+ * command doesn't have QCO_COROUTINE. We must drop out of
+ * coroutine context for this one.
+ */
+ assert(!oob && qemu_in_coroutine() && !(cmd->options & QCO_COROUTINE));
+
+ QmpDispatchBH data = {
+ .cur_mon = cur_mon,
+ .cmd = cmd,
+ .args = args,
+ .ret = &ret,
+ .errp = &err,
+ .co = qemu_coroutine_self(),
+ };
+ aio_bh_schedule_oneshot(qemu_get_aio_context(), do_qmp_dispatch_bh,
+ &data);
+ qemu_coroutine_yield();
+ }
qobject_unref(args);
if (err) {
/* or assert(!ret) after reviewing all handlers: */
diff --git a/qapi/qmp-registry.c b/qapi/qmp-registry.c
index d0f9a1d3e3..58c65b5052 100644
--- a/qapi/qmp-registry.c
+++ b/qapi/qmp-registry.c
@@ -20,6 +20,9 @@ void qmp_register_command(QmpCommandList *cmds, const char *name,
{
QmpCommand *cmd = g_malloc0(sizeof(*cmd));
+ /* QCO_COROUTINE and QCO_ALLOW_OOB are incompatible for now */
+ assert(!((options & QCO_COROUTINE) && (options & QCO_ALLOW_OOB)));
+
cmd->name = name;
cmd->fn = fn;
cmd->enabled = true;
diff --git a/util/aio-posix.c b/util/aio-posix.c
index f7f13ebfc2..30bb21d699 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -15,6 +15,7 @@
#include "qemu/osdep.h"
#include "block/block.h"
+#include "qemu/main-loop.h"
#include "qemu/rcu.h"
#include "qemu/rcu_queue.h"
#include "qemu/sockets.h"
@@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
* There cannot be two concurrent aio_poll calls for the same AioContext (or
* an aio_poll concurrent with a GSource prepare/check/dispatch callback).
* We rely on this below to avoid slow locked accesses to ctx->notify_me.
+ *
+ * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
+ * is special in that it runs in the main thread, but that thread's context
+ * is qemu_aio_context.
*/
- assert(in_aio_context_home_thread(ctx));
+ assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
+ qemu_get_aio_context() : ctx));
qemu_lockcnt_inc(&ctx->list_lock);
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 10/13] hmp: Add support for coroutine command handlers
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (8 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 09/13] qmp: Move dispatcher to a coroutine Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-16 9:46 ` Dr. David Alan Gilbert
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 11/13] util/async: Add aio_co_reschedule_self() Kevin Wolf
` (4 subsequent siblings)
14 siblings, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Often, QMP command handlers are not only called to handle QMP commands,
but also from a corresponding HMP command handler. In order to give them
a consistent environment, optionally run HMP command handlers in a
coroutine, too.
The implementation is a lot simpler than in QMP because for HMP, we
still block the VM while the coroutine is running.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
monitor/monitor-internal.h | 1 +
monitor/hmp.c | 37 ++++++++++++++++++++++++++++++++-----
2 files changed, 33 insertions(+), 5 deletions(-)
diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
index b55d6df07f..ad2e64be13 100644
--- a/monitor/monitor-internal.h
+++ b/monitor/monitor-internal.h
@@ -74,6 +74,7 @@ typedef struct HMPCommand {
const char *help;
const char *flags; /* p=preconfig */
void (*cmd)(Monitor *mon, const QDict *qdict);
+ bool coroutine;
/*
* @sub_table is a list of 2nd level of commands. If it does not exist,
* cmd should be used. If it exists, sub_table[?].cmd should be
diff --git a/monitor/hmp.c b/monitor/hmp.c
index 4b66ca1cd6..b858b0dbde 100644
--- a/monitor/hmp.c
+++ b/monitor/hmp.c
@@ -1056,12 +1056,26 @@ fail:
return NULL;
}
+typedef struct HandleHmpCommandCo {
+ Monitor *mon;
+ const HMPCommand *cmd;
+ QDict *qdict;
+ bool done;
+} HandleHmpCommandCo;
+
+static void handle_hmp_command_co(void *opaque)
+{
+ HandleHmpCommandCo *data = opaque;
+ data->cmd->cmd(data->mon, data->qdict);
+ monitor_set_cur(qemu_coroutine_self(), NULL);
+ data->done = true;
+}
+
void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
{
QDict *qdict;
const HMPCommand *cmd;
const char *cmd_start = cmdline;
- Monitor *old_mon;
trace_handle_hmp_command(mon, cmdline);
@@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
return;
}
- /* old_mon is non-NULL when called from qmp_human_monitor_command() */
- old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
- cmd->cmd(&mon->common, qdict);
- monitor_set_cur(qemu_coroutine_self(), old_mon);
+ if (!cmd->coroutine) {
+ /* old_mon is non-NULL when called from qmp_human_monitor_command() */
+ Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
+ cmd->cmd(&mon->common, qdict);
+ monitor_set_cur(qemu_coroutine_self(), old_mon);
+ } else {
+ HandleHmpCommandCo data = {
+ .mon = &mon->common,
+ .cmd = cmd,
+ .qdict = qdict,
+ .done = false,
+ };
+ Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
+ monitor_set_cur(co, &mon->common);
+ aio_co_enter(qemu_get_aio_context(), co);
+ AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
+ }
qobject_unref(qdict);
}
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 11/13] util/async: Add aio_co_reschedule_self()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (9 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 10/13] hmp: Add support for coroutine command handlers Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-15 14:25 ` Stefan Hajnoczi
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context() Kevin Wolf
` (3 subsequent siblings)
14 siblings, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Add a function that can be used to move the currently running coroutine
to a different AioContext (and therefore potentially a different
thread).
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/block/aio.h | 10 ++++++++++
util/async.c | 30 ++++++++++++++++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/include/block/aio.h b/include/block/aio.h
index b2f703fa3f..c37617b404 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -17,6 +17,7 @@
#ifdef CONFIG_LINUX_IO_URING
#include <liburing.h>
#endif
+#include "qemu/coroutine.h"
#include "qemu/queue.h"
#include "qemu/event_notifier.h"
#include "qemu/thread.h"
@@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
*/
void aio_co_schedule(AioContext *ctx, struct Coroutine *co);
+/**
+ * aio_co_reschedule_self:
+ * @new_ctx: the new context
+ *
+ * Move the currently running coroutine to new_ctx. If the coroutine is already
+ * running in new_ctx, do nothing.
+ */
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx);
+
/**
* aio_co_wake:
* @co: the coroutine
diff --git a/util/async.c b/util/async.c
index 4266745dee..a609e18693 100644
--- a/util/async.c
+++ b/util/async.c
@@ -569,6 +569,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
aio_context_unref(ctx);
}
+typedef struct AioCoRescheduleSelf {
+ Coroutine *co;
+ AioContext *new_ctx;
+} AioCoRescheduleSelf;
+
+static void aio_co_reschedule_self_bh(void *opaque)
+{
+ AioCoRescheduleSelf *data = opaque;
+ aio_co_schedule(data->new_ctx, data->co);
+}
+
+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx)
+{
+ AioContext *old_ctx = qemu_get_current_aio_context();
+
+ if (old_ctx != new_ctx) {
+ AioCoRescheduleSelf data = {
+ .co = qemu_coroutine_self(),
+ .new_ctx = new_ctx,
+ };
+ /*
+ * We can't directly schedule the coroutine in the target context
+ * because this would be racy: The other thread could try to enter the
+ * coroutine before it has yielded in this one.
+ */
+ aio_bh_schedule_oneshot(old_ctx, aio_co_reschedule_self_bh, &data);
+ qemu_coroutine_yield();
+ }
+}
+
void aio_co_wake(struct Coroutine *co)
{
AioContext *ctx;
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (10 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 11/13] util/async: Add aio_co_reschedule_self() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-15 14:31 ` Stefan Hajnoczi
2020-09-09 15:11 ` [PATCH v7 13/13] block: Convert 'block_resize' to coroutine Kevin Wolf
` (2 subsequent siblings)
14 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
Add a function to move the current coroutine to the AioContext of a
given BlockDriverState.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
include/block/block.h | 6 ++++++
block.c | 10 ++++++++++
2 files changed, 16 insertions(+)
diff --git a/include/block/block.h b/include/block/block.h
index 981ab5b314..80ab322f11 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
*/
AioContext *bdrv_get_aio_context(BlockDriverState *bs);
+/**
+ * Move the current coroutine to the AioContext of @bs and return the old
+ * AioContext of the coroutine.
+ */
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
+
/**
* Transfer control to @co in the aio context of @bs
*/
diff --git a/block.c b/block.c
index 9538af4884..81403e00d1 100644
--- a/block.c
+++ b/block.c
@@ -6372,6 +6372,16 @@ AioContext *bdrv_get_aio_context(BlockDriverState *bs)
return bs ? bs->aio_context : qemu_get_aio_context();
}
+AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs)
+{
+ Coroutine *self = qemu_coroutine_self();
+ AioContext *old_ctx = qemu_coroutine_get_aio_context(self);
+ AioContext *new_ctx = bdrv_get_aio_context(bs);
+
+ aio_co_reschedule_self(new_ctx);
+ return old_ctx;
+}
+
void bdrv_coroutine_enter(BlockDriverState *bs, Coroutine *co)
{
aio_co_enter(bdrv_get_aio_context(bs), co);
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v7 13/13] block: Convert 'block_resize' to coroutine
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (11 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context() Kevin Wolf
@ 2020-09-09 15:11 ` Kevin Wolf
2020-09-15 14:57 ` Stefan Hajnoczi
2020-09-09 15:24 ` [PATCH v7 00/13] monitor: Optionally run handlers in coroutines no-reply
2020-09-10 13:24 ` Stefan Hajnoczi
14 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-09 15:11 UTC (permalink / raw)
To: qemu-block
Cc: kwolf, qemu-devel, armbru, marcandre.lureau, stefanha, dgilbert
block_resize performs some I/O that could potentially take quite some
time, so use it as an example for the new 'coroutine': true annotation
in the QAPI schema.
bdrv_truncate() requires that we're already in the right AioContext for
the BlockDriverState if called in coroutine context. So instead of just
taking the AioContext lock, move the QMP handler coroutine to the
context.
Call blk_unref() only after switching back because blk_unref() may only
be called in the main thread.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qapi/block-core.json | 3 ++-
blockdev.c | 13 ++++++-------
hmp-commands.hx | 1 +
3 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0345f6f2d2..d3e49c9419 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1302,7 +1302,8 @@
{ 'command': 'block_resize',
'data': { '*device': 'str',
'*node-name': 'str',
- 'size': 'int' } }
+ 'size': 'int' },
+ 'coroutine': true }
##
# @NewImageMode:
diff --git a/blockdev.c b/blockdev.c
index 7f2561081e..064989fc2d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2439,14 +2439,14 @@ BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
return ret;
}
-void qmp_block_resize(bool has_device, const char *device,
- bool has_node_name, const char *node_name,
- int64_t size, Error **errp)
+void coroutine_fn qmp_block_resize(bool has_device, const char *device,
+ bool has_node_name, const char *node_name,
+ int64_t size, Error **errp)
{
Error *local_err = NULL;
BlockBackend *blk = NULL;
BlockDriverState *bs;
- AioContext *aio_context;
+ AioContext *old_ctx;
bs = bdrv_lookup_bs(has_device ? device : NULL,
has_node_name ? node_name : NULL,
@@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
return;
}
- aio_context = bdrv_get_aio_context(bs);
- aio_context_acquire(aio_context);
+ old_ctx = bdrv_co_move_to_aio_context(bs);
if (size < 0) {
error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
@@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
bdrv_drained_end(bs);
out:
+ aio_co_reschedule_self(old_ctx);
blk_unref(blk);
- aio_context_release(aio_context);
}
void qmp_block_stream(bool has_job_id, const char *job_id, const char *device,
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 60f395c276..ac360b73f6 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -76,6 +76,7 @@ ERST
.params = "device size",
.help = "resize a block image",
.cmd = hmp_block_resize,
+ .coroutine = true,
},
SRST
--
2.25.4
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (12 preceding siblings ...)
2020-09-09 15:11 ` [PATCH v7 13/13] block: Convert 'block_resize' to coroutine Kevin Wolf
@ 2020-09-09 15:24 ` no-reply
2020-09-10 13:24 ` Stefan Hajnoczi
14 siblings, 0 replies; 62+ messages in thread
From: no-reply @ 2020-09-09 15:24 UTC (permalink / raw)
To: kwolf
Cc: kwolf, qemu-block, qemu-devel, armbru, marcandre.lureau,
stefanha, dgilbert
Patchew URL: https://patchew.org/QEMU/20200909151149.490589-1-kwolf@redhat.com/
Hi,
This series failed build test on FreeBSD host. Please find the details below.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked under the git checkout with
# HEAD pointing to a commit that has the patches applied on top of "base"
# branch
if qemu-system-x86_64 --help >/dev/null 2>&1; then
QEMU=qemu-system-x86_64
elif /usr/libexec/qemu-kvm --help >/dev/null 2>&1; then
QEMU=/usr/libexec/qemu-kvm
else
exit 1
fi
make vm-build-freebsd J=21 QEMU=$QEMU
exit 0
=== TEST SCRIPT END ===
The full log is available at
http://patchew.org/logs/20200909151149.490589-1-kwolf@redhat.com/testing.FreeBSD/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
` (13 preceding siblings ...)
2020-09-09 15:24 ` [PATCH v7 00/13] monitor: Optionally run handlers in coroutines no-reply
@ 2020-09-10 13:24 ` Stefan Hajnoczi
2020-09-14 15:09 ` Markus Armbruster
2020-09-25 17:15 ` Kevin Wolf
14 siblings, 2 replies; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-10 13:24 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 852 bytes --]
On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
> Some QMP command handlers can block the main loop for a relatively long
> time, for example because they perform some I/O. This is quite nasty.
> Allowing such handlers to run in a coroutine where they can yield (and
> therefore release the BQL) while waiting for an event such as I/O
> completion solves the problem.
>
> This series adds the infrastructure to allow this and switches
> block_resize to run in a coroutine as a first example.
>
> This is an alternative solution to Marc-André's "monitor: add
> asynchronous command type" series.
Please clarify the following in the QAPI documentation:
* Is the QMP monitor suspended while the command is pending?
* Are QMP events reported while the command is pending?
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-10 13:24 ` Stefan Hajnoczi
@ 2020-09-14 15:09 ` Markus Armbruster
2020-09-15 14:58 ` Stefan Hajnoczi
2020-09-25 17:15 ` Kevin Wolf
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-14 15:09 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Stefan Hajnoczi <stefanha@redhat.com> writes:
> On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
>> Some QMP command handlers can block the main loop for a relatively long
>> time, for example because they perform some I/O. This is quite nasty.
>> Allowing such handlers to run in a coroutine where they can yield (and
>> therefore release the BQL) while waiting for an event such as I/O
>> completion solves the problem.
>>
>> This series adds the infrastructure to allow this and switches
>> block_resize to run in a coroutine as a first example.
>>
>> This is an alternative solution to Marc-André's "monitor: add
>> asynchronous command type" series.
>
> Please clarify the following in the QAPI documentation:
> * Is the QMP monitor suspended while the command is pending?
> * Are QMP events reported while the command is pending?
Good points. Kevin, I'd be willing to take this as a follow-up patch,
if that's more convenient for you.
> Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan, I could use your proper review of PATCH 11-13. Pretty-please?
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-09 15:11 ` [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch() Kevin Wolf
@ 2020-09-14 15:10 ` Markus Armbruster
2020-09-25 15:13 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-14 15:10 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> The correct way to set the current monitor for a coroutine handler will
> be different than for a blocking handler, so monitor_set_cur() needs to
> be called in qmp_dispatch().
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> include/qapi/qmp/dispatch.h | 3 ++-
> monitor/qmp.c | 8 +-------
> qapi/qmp-dispatch.c | 8 +++++++-
> qga/main.c | 2 +-
> stubs/monitor-core.c | 5 +++++
> tests/test-qmp-cmds.c | 6 +++---
> 6 files changed, 19 insertions(+), 13 deletions(-)
>
> diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> index 5a9cf82472..0c2f467028 100644
> --- a/include/qapi/qmp/dispatch.h
> +++ b/include/qapi/qmp/dispatch.h
> @@ -14,6 +14,7 @@
> #ifndef QAPI_QMP_DISPATCH_H
> #define QAPI_QMP_DISPATCH_H
>
> +#include "monitor/monitor.h"
> #include "qemu/queue.h"
>
> typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
> @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
> bool qmp_has_success_response(const QmpCommand *cmd);
> QDict *qmp_error_response(Error *err);
> QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> - bool allow_oob);
> + bool allow_oob, Monitor *cur_mon);
> bool qmp_is_oob(const QDict *dict);
>
> typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
> diff --git a/monitor/qmp.c b/monitor/qmp.c
> index 8469970c69..922fdb5541 100644
> --- a/monitor/qmp.c
> +++ b/monitor/qmp.c
> @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>
> static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> {
> - Monitor *old_mon;
> QDict *rsp;
> QDict *error;
>
> - old_mon = monitor_set_cur(&mon->common);
> - assert(old_mon == NULL);
> -
> - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> -
> - monitor_set_cur(NULL);
> + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
>
> if (mon->commands == &qmp_cap_negotiation_commands) {
> error = qdict_get_qdict(rsp, "error");
> diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
> index 79347e0864..2fdbc0fba4 100644
> --- a/qapi/qmp-dispatch.c
> +++ b/qapi/qmp-dispatch.c
> @@ -89,7 +89,7 @@ bool qmp_is_oob(const QDict *dict)
> }
>
> QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> - bool allow_oob)
> + bool allow_oob, Monitor *cur_mon)
> {
> Error *err = NULL;
> bool oob;
> @@ -152,7 +152,13 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> args = qdict_get_qdict(dict, "arguments");
> qobject_ref(args);
> }
> +
> + assert(monitor_cur() == NULL);
> + monitor_set_cur(cur_mon);
> +
> cmd->fn(args, &ret, &err);
> +
> + monitor_set_cur(NULL);
> qobject_unref(args);
> if (err) {
> /* or assert(!ret) after reviewing all handlers: */
> diff --git a/qga/main.c b/qga/main.c
> index 3febf3b0fd..241779a1d6 100644
> --- a/qga/main.c
> +++ b/qga/main.c
> @@ -578,7 +578,7 @@ static void process_event(void *opaque, QObject *obj, Error *err)
> }
>
> g_debug("processing command");
> - rsp = qmp_dispatch(&ga_commands, obj, false);
> + rsp = qmp_dispatch(&ga_commands, obj, false, NULL);
>
> end:
> ret = send_response(s, rsp);
I dislike how this entangles qemu-ga even more with the monitor. But I
want to get this wrapped more than I want it to be spotless. Perhaps I
can clean up some on top.
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-09 15:11 ` [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property Kevin Wolf
@ 2020-09-14 15:11 ` Markus Armbruster
2020-09-25 15:23 ` Kevin Wolf
2020-10-02 7:53 ` Markus Armbruster
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-14 15:11 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> This way, a monitor command handler will still be able to access the
> current monitor, but when it yields, all other code code will correctly
> get NULL from monitor_cur().
>
> This uses a hash table to map the coroutine pointer to the current
> monitor of that coroutine. Outside of coroutine context, we associate
> the current monitor with the leader coroutine of the current thread.
In qemu-system-FOO, the hash table can have only these entries:
* (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while
executing a QMP command out-of-band.
* (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main
thread) to a QMP monitor, while executing a QMP command in-band and in
coroutine context.
* (QMP) One mapping the main thread's leader to a QMP monitor, while
executing a QMP command in-band and out of coroutine context, in a
bottom half.
* (HMP) One mapping the main thread's leader to an HMP monitor, while
executing an HMP command out of coroutine context.
* (HMP-CO) One mapping a transient coroutine in the main thread to an
HMP monitor, while executing an HMP command in coroutine context.
In-band execution is one command after the other.
Therefore, at most one monitor command can be executing in-band at any
time.
Therefore, the hash table has at most *two* entries: one (OOB), and one
of the other four.
Can you shoot any holes into my argument?
I suspect there's a simpler solution struggling to get out. But this
solution works, so in it goes. Should the simpler one succeed at
getting out, it can go in on top. If not, I'll probably add even more
comments to remind myself of these facts.
> Approaches to implement some form of coroutine local storage directly in
> the coroutine core code have been considered and discarded because they
> didn't end up being much more generic than the hash table and their
> performance impact on coroutines not using coroutine local storage was
> unclear. As the block layer uses a coroutine per I/O request, this is a
> fast path and we have to be careful. It's safest to just stay out of
> this path with code only used by the monitor.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
@ 2020-09-14 15:15 ` Markus Armbruster
2020-09-25 15:37 ` Kevin Wolf
2020-10-02 7:53 ` Markus Armbruster
2020-10-02 7:59 ` Markus Armbruster
2 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-14 15:15 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> This patch adds a new 'coroutine' flag to QMP command definitions that
> tells the QMP dispatcher that the command handler is safe to be run in a
> coroutine.
>
> The documentation of the new flag pretends that this flag is already
> used as intended, which it isn't yet after this patch. We'll implement
> this in another patch in this series.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> ---
> docs/devel/qapi-code-gen.txt | 12 ++++++++++++
> include/qapi/qmp/dispatch.h | 1 +
> tests/test-qmp-cmds.c | 4 ++++
> scripts/qapi/commands.py | 10 +++++++---
> scripts/qapi/doc.py | 2 +-
> scripts/qapi/expr.py | 10 ++++++++--
> scripts/qapi/introspect.py | 2 +-
> scripts/qapi/schema.py | 12 ++++++++----
> tests/qapi-schema/test-qapi.py | 7 ++++---
> tests/qapi-schema/meson.build | 1 +
> tests/qapi-schema/oob-coroutine.err | 2 ++
> tests/qapi-schema/oob-coroutine.json | 2 ++
> tests/qapi-schema/oob-coroutine.out | 0
> tests/qapi-schema/qapi-schema-test.json | 1 +
> tests/qapi-schema/qapi-schema-test.out | 2 ++
> 15 files changed, 54 insertions(+), 14 deletions(-)
> create mode 100644 tests/qapi-schema/oob-coroutine.err
> create mode 100644 tests/qapi-schema/oob-coroutine.json
> create mode 100644 tests/qapi-schema/oob-coroutine.out
>
> diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> index f3e7ced212..36daa9b5f8 100644
> --- a/docs/devel/qapi-code-gen.txt
> +++ b/docs/devel/qapi-code-gen.txt
> @@ -472,6 +472,7 @@ Syntax:
> '*gen': false,
> '*allow-oob': true,
> '*allow-preconfig': true,
> + '*coroutine': true,
> '*if': COND,
> '*features': FEATURES }
>
> @@ -596,6 +597,17 @@ before the machine is built. It defaults to false. For example:
> QMP is available before the machine is built only when QEMU was
> started with --preconfig.
>
> +Member 'coroutine' tells the QMP dispatcher whether the command handler
> +is safe to be run in a coroutine.
We need to document what exactly makes a command handler coroutine safe
/ unsafe. We discussed this at some length in review of PATCH v4 1/4:
Message-ID: <874kwnvgad.fsf@dusky.pond.sub.org>
https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html
I'd be willing to accept a follow-up patch, if that's more convenient
for you.
> It defaults to false. If it is true,
> +the command handler is called from coroutine context and may yield while
Is it *always* called from coroutine context, or could it be called
outside coroutine context, too? I guess the former, thanks to PATCH 10,
and as long as we diligently mark HMP commands that such call QMP
handlers, like you do in PATCH 13.
What's the worst than can happen when we neglect to mark such an HMP
command?
> +waiting for an external event (such as I/O completion) in order to avoid
> +blocking the guest and other background operations.
> +
> +It is an error to specify both 'coroutine': true and 'allow-oob': true
> +for a command. We don't currently have a use case for both together and
> +without a use case, it's not entirely clear what the semantics should
> +be.
> +
> The optional 'if' member specifies a conditional. See "Configuring
> the schema" below for more on this.
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine
2020-09-09 15:11 ` [PATCH v7 09/13] qmp: Move dispatcher to a coroutine Kevin Wolf
@ 2020-09-14 15:30 ` Markus Armbruster
2020-09-25 15:38 ` Kevin Wolf
2020-10-02 8:01 ` Markus Armbruster
1 sibling, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-14 15:30 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> This moves the QMP dispatcher to a coroutine and runs all QMP command
> handlers that declare 'coroutine': true in coroutine context so they
> can avoid blocking the main loop while doing I/O or waiting for other
> events.
>
> For commands that are not declared safe to run in a coroutine, the
> dispatcher drops out of coroutine context by calling the QMP command
> handler from a bottom half.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
> ---
> include/qapi/qmp/dispatch.h | 1 +
> monitor/monitor-internal.h | 6 +-
> monitor/monitor.c | 55 +++++++++++++---
> monitor/qmp.c | 122 +++++++++++++++++++++++++++---------
> qapi/qmp-dispatch.c | 61 ++++++++++++++++--
> qapi/qmp-registry.c | 3 +
> util/aio-posix.c | 8 ++-
> 7 files changed, 210 insertions(+), 46 deletions(-)
>
> diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> index 9fd2b720a7..af8d96c570 100644
> --- a/include/qapi/qmp/dispatch.h
> +++ b/include/qapi/qmp/dispatch.h
> @@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
> typedef struct QmpCommand
> {
> const char *name;
> + /* Runs in coroutine context if QCO_COROUTINE is set */
> QmpCommandFunc *fn;
> QmpCommandOptions options;
> QTAILQ_ENTRY(QmpCommand) node;
> diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
> index b39e03b744..b55d6df07f 100644
> --- a/monitor/monitor-internal.h
> +++ b/monitor/monitor-internal.h
> @@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
>
> typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
> extern IOThread *mon_iothread;
> -extern QEMUBH *qmp_dispatcher_bh;
> +extern Coroutine *qmp_dispatcher_co;
> +extern bool qmp_dispatcher_co_shutdown;
> +extern bool qmp_dispatcher_co_busy;
> extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
> extern QemuMutex monitor_lock;
> extern MonitorList mon_list;
> @@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
>
> void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
> void monitor_data_destroy_qmp(MonitorQMP *mon);
> -void monitor_qmp_bh_dispatcher(void *data);
> +void coroutine_fn monitor_qmp_dispatcher_co(void *data);
>
> int get_monitor_def(int64_t *pval, const char *name);
> void help_cmd(Monitor *mon, const char *name);
> diff --git a/monitor/monitor.c b/monitor/monitor.c
> index 629aa073ee..ac2722bf91 100644
> --- a/monitor/monitor.c
> +++ b/monitor/monitor.c
> @@ -55,8 +55,32 @@ typedef struct {
> /* Shared monitor I/O thread */
> IOThread *mon_iothread;
>
> -/* Bottom half to dispatch the requests received from I/O thread */
> -QEMUBH *qmp_dispatcher_bh;
> +/* Coroutine to dispatch the requests received from I/O thread */
> +Coroutine *qmp_dispatcher_co;
> +
> +/* Set to true when the dispatcher coroutine should terminate */
> +bool qmp_dispatcher_co_shutdown;
> +
> +/*
> + * qmp_dispatcher_co_busy is used for synchronisation between the
> + * monitor thread and the main thread to ensure that the dispatcher
> + * coroutine never gets scheduled a second time when it's already
> + * scheduled (scheduling the same coroutine twice is forbidden).
> + *
> + * It is true if the coroutine is active and processing requests.
> + * Additional requests may then be pushed onto mon->qmp_requests,
> + * and @qmp_dispatcher_co_shutdown may be set without further ado.
> + * @qmp_dispatcher_co_busy must not be woken up in this case.
> + *
> + * If false, you also have to set @qmp_dispatcher_co_busy to true and
> + * wake up @qmp_dispatcher_co after pushing the new requests.
> + *
> + * The coroutine will automatically change this variable back to false
> + * before it yields. Nobody else may set the variable to false.
> + *
> + * Access must be atomic for thread safety.
> + */
> +bool qmp_dispatcher_co_busy;
>
> /*
> * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
> @@ -623,9 +647,24 @@ void monitor_cleanup(void)
> }
> qemu_mutex_unlock(&monitor_lock);
>
> - /* QEMUBHs needs to be deleted before destroying the I/O thread */
> - qemu_bh_delete(qmp_dispatcher_bh);
> - qmp_dispatcher_bh = NULL;
> + /*
> + * The dispatcher needs to stop before destroying the I/O thread.
> + *
> + * We need to poll both qemu_aio_context and iohandler_ctx to make
> + * sure that the dispatcher coroutine keeps making progress and
> + * eventually terminates. qemu_aio_context is automatically
> + * polled by calling AIO_WAIT_WHILE on it, but we must poll
> + * iohandler_ctx manually.
> + */
> + qmp_dispatcher_co_shutdown = true;
> + if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
> + aio_co_wake(qmp_dispatcher_co);
> + }
> +
> + AIO_WAIT_WHILE(qemu_get_aio_context(),
> + (aio_poll(iohandler_get_aio_context(), false),
> + atomic_mb_read(&qmp_dispatcher_co_busy)));
> +
> if (mon_iothread) {
> iothread_destroy(mon_iothread);
> mon_iothread = NULL;
> @@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
> * have commands assuming that context. It would be nice to get
> * rid of those assumptions.
> */
> - qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
> - monitor_qmp_bh_dispatcher,
> - NULL);
> + qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
> + atomic_mb_set(&qmp_dispatcher_co_busy, true);
> + aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
> }
>
> int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
> diff --git a/monitor/qmp.c b/monitor/qmp.c
> index 922fdb5541..69f6e93f38 100644
> --- a/monitor/qmp.c
> +++ b/monitor/qmp.c
> @@ -133,6 +133,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> }
> }
>
> +/*
> + * Runs outside of coroutine context for OOB commands, but in
> + * coroutine context for everything else.
> + */
> static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> {
> QDict *rsp;
> @@ -205,43 +209,99 @@ static QMPRequest *monitor_qmp_requests_pop_any_with_lock(void)
> return req_obj;
> }
>
> -void monitor_qmp_bh_dispatcher(void *data)
> +void coroutine_fn monitor_qmp_dispatcher_co(void *data)
> {
> - QMPRequest *req_obj = monitor_qmp_requests_pop_any_with_lock();
> + QMPRequest *req_obj = NULL;
> QDict *rsp;
> bool need_resume;
> MonitorQMP *mon;
>
> - if (!req_obj) {
> - return;
> - }
> + while (true) {
> + assert(atomic_mb_read(&qmp_dispatcher_co_busy) == true);
>
> - mon = req_obj->mon;
> - /* qmp_oob_enabled() might change after "qmp_capabilities" */
> - need_resume = !qmp_oob_enabled(mon) ||
> - mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
> - qemu_mutex_unlock(&mon->qmp_queue_lock);
> - if (req_obj->req) {
> - QDict *qdict = qobject_to(QDict, req_obj->req);
> - QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
> - trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
> - monitor_qmp_dispatch(mon, req_obj->req);
> - } else {
> - assert(req_obj->err);
> - rsp = qmp_error_response(req_obj->err);
> - req_obj->err = NULL;
> - monitor_qmp_respond(mon, rsp);
> - qobject_unref(rsp);
> - }
> + /*
> + * Mark the dispatcher as not busy already here so that we
> + * don't miss any new requests coming in the middle of our
> + * processing.
> + */
> + atomic_mb_set(&qmp_dispatcher_co_busy, false);
> +
> + while (!(req_obj = monitor_qmp_requests_pop_any_with_lock())) {
> + /*
> + * No more requests to process. Wait to be reentered from
> + * handle_qmp_command() when it pushes more requests, or
> + * from monitor_cleanup() when it requests shutdown.
> + */
> + if (!qmp_dispatcher_co_shutdown) {
> + qemu_coroutine_yield();
> +
> + /*
> + * busy must be set to true again by whoever
> + * rescheduled us to avoid double scheduling
> + */
> + assert(atomic_xchg(&qmp_dispatcher_co_busy, false) == true);
> + }
> +
> + /*
> + * qmp_dispatcher_co_shutdown may have changed if we
> + * yielded and were reentered from monitor_cleanup()
> + */
> + if (qmp_dispatcher_co_shutdown) {
> + return;
> + }
> + }
>
> - if (need_resume) {
> - /* Pairs with the monitor_suspend() in handle_qmp_command() */
> - monitor_resume(&mon->common);
> - }
> - qmp_request_free(req_obj);
> + if (atomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
> + /*
> + * Someone rescheduled us (probably because a new requests
> + * came in), but we didn't actually yield. Do that now,
> + * only to be immediately reentered and removed from the
> + * list of scheduled coroutines.
> + */
> + qemu_coroutine_yield();
> + }
>
> - /* Reschedule instead of looping so the main loop stays responsive */
> - qemu_bh_schedule(qmp_dispatcher_bh);
> + /*
> + * Move the coroutine from iohandler_ctx to qemu_aio_context for
> + * executing the command handler so that it can make progress if it
> + * involves an AIO_WAIT_WHILE().
> + */
> + aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
> + qemu_coroutine_yield();
> +
> + mon = req_obj->mon;
> + /* qmp_oob_enabled() might change after "qmp_capabilities" */
> + need_resume = !qmp_oob_enabled(mon) ||
> + mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
> + qemu_mutex_unlock(&mon->qmp_queue_lock);
> + if (req_obj->req) {
> + QDict *qdict = qobject_to(QDict, req_obj->req);
> + QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
> + trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
> + monitor_qmp_dispatch(mon, req_obj->req);
> + } else {
> + assert(req_obj->err);
> + rsp = qmp_error_response(req_obj->err);
> + req_obj->err = NULL;
> + monitor_qmp_respond(mon, rsp);
> + qobject_unref(rsp);
> + }
> +
> + if (need_resume) {
> + /* Pairs with the monitor_suspend() in handle_qmp_command() */
> + monitor_resume(&mon->common);
> + }
> + qmp_request_free(req_obj);
> +
> + /*
> + * Yield and reschedule so the main loop stays responsive.
> + *
> + * Move back to iohandler_ctx so that nested event loops for
> + * qemu_aio_context don't start new monitor commands.
> + */
> + aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
> + qemu_coroutine_yield();
> + }
> }
>
> static void handle_qmp_command(void *opaque, QObject *req, Error *err)
> @@ -302,7 +362,9 @@ static void handle_qmp_command(void *opaque, QObject *req, Error *err)
> qemu_mutex_unlock(&mon->qmp_queue_lock);
>
> /* Kick the dispatcher routine */
> - qemu_bh_schedule(qmp_dispatcher_bh);
> + if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
> + aio_co_wake(qmp_dispatcher_co);
> + }
> }
>
> static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
> diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
> index 5677ba92ca..754f7b854c 100644
> --- a/qapi/qmp-dispatch.c
> +++ b/qapi/qmp-dispatch.c
> @@ -12,12 +12,16 @@
> */
>
> #include "qemu/osdep.h"
> +
> +#include "block/aio.h"
> #include "qapi/error.h"
> #include "qapi/qmp/dispatch.h"
> #include "qapi/qmp/qdict.h"
> #include "qapi/qmp/qjson.h"
> #include "sysemu/runstate.h"
> #include "qapi/qmp/qbool.h"
> +#include "qemu/coroutine.h"
> +#include "qemu/main-loop.h"
>
> static QDict *qmp_dispatch_check_obj(QDict *dict, bool allow_oob,
> Error **errp)
> @@ -88,6 +92,30 @@ bool qmp_is_oob(const QDict *dict)
> && !qdict_haskey(dict, "execute");
> }
>
> +typedef struct QmpDispatchBH {
> + const QmpCommand *cmd;
> + Monitor *cur_mon;
> + QDict *args;
> + QObject **ret;
> + Error **errp;
> + Coroutine *co;
> +} QmpDispatchBH;
> +
> +static void do_qmp_dispatch_bh(void *opaque)
> +{
> + QmpDispatchBH *data = opaque;
> +
> + assert(monitor_cur() == NULL);
> + monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
> + data->cmd->fn(data->args, data->ret, data->errp);
> + monitor_set_cur(qemu_coroutine_self(), NULL);
> + aio_co_wake(data->co);
> +}
> +
> +/*
> + * Runs outside of coroutine context for OOB commands, but in coroutine
> + * context for everything else.
> + */
> QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> bool allow_oob, Monitor *cur_mon)
> {
> @@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> qobject_ref(args);
> }
>
> + assert(!(oob && qemu_in_coroutine()));
> assert(monitor_cur() == NULL);
> - monitor_set_cur(qemu_coroutine_self(), cur_mon);
> -
> - cmd->fn(args, &ret, &err);
> -
> - monitor_set_cur(qemu_coroutine_self(), NULL);
> + if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
> + monitor_set_cur(qemu_coroutine_self(), cur_mon);
> + cmd->fn(args, &ret, &err);
> + monitor_set_cur(qemu_coroutine_self(), NULL);
> + } else {
> + /*
> + * Not being in coroutine context implies that we're handling
> + * an OOB command, which must not have QCO_COROUTINE.
> + *
> + * This implies that we are in coroutine context, but the
> + * command doesn't have QCO_COROUTINE. We must drop out of
> + * coroutine context for this one.
> + */
I had to read this several times to get it. The first sentence leads me
into coroutine context, and then the next sentence tells me the
opposite, throwing me into confusion.
Perhaps something like this:
/*
* Actual context doesn't match the one the command needs.
* Case 1: we are in coroutine context, but command does not
* have QCO_COROUTINE. We need to drop out of coroutine
* context for executing it.
* Case 2: we are outside coroutine context, but command has
* QCO_COROUTINE. Can't actually happen, because we get here
* outside coroutine context only when executing a command
* out of band, and OOB commands never have QCO_COROUTINE.
*/
> + assert(!oob && qemu_in_coroutine() && !(cmd->options & QCO_COROUTINE));
> +
> + QmpDispatchBH data = {
> + .cur_mon = cur_mon,
> + .cmd = cmd,
> + .args = args,
> + .ret = &ret,
> + .errp = &err,
> + .co = qemu_coroutine_self(),
> + };
> + aio_bh_schedule_oneshot(qemu_get_aio_context(), do_qmp_dispatch_bh,
> + &data);
> + qemu_coroutine_yield();
> + }
> qobject_unref(args);
> if (err) {
> /* or assert(!ret) after reviewing all handlers: */
> diff --git a/qapi/qmp-registry.c b/qapi/qmp-registry.c
> index d0f9a1d3e3..58c65b5052 100644
> --- a/qapi/qmp-registry.c
> +++ b/qapi/qmp-registry.c
> @@ -20,6 +20,9 @@ void qmp_register_command(QmpCommandList *cmds, const char *name,
> {
> QmpCommand *cmd = g_malloc0(sizeof(*cmd));
>
> + /* QCO_COROUTINE and QCO_ALLOW_OOB are incompatible for now */
> + assert(!((options & QCO_COROUTINE) && (options & QCO_ALLOW_OOB)));
> +
> cmd->name = name;
> cmd->fn = fn;
> cmd->enabled = true;
> diff --git a/util/aio-posix.c b/util/aio-posix.c
> index f7f13ebfc2..30bb21d699 100644
> --- a/util/aio-posix.c
> +++ b/util/aio-posix.c
> @@ -15,6 +15,7 @@
>
> #include "qemu/osdep.h"
> #include "block/block.h"
> +#include "qemu/main-loop.h"
> #include "qemu/rcu.h"
> #include "qemu/rcu_queue.h"
> #include "qemu/sockets.h"
> @@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
> * There cannot be two concurrent aio_poll calls for the same AioContext (or
> * an aio_poll concurrent with a GSource prepare/check/dispatch callback).
> * We rely on this below to avoid slow locked accesses to ctx->notify_me.
> + *
> + * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
> + * is special in that it runs in the main thread, but that thread's context
> + * is qemu_aio_context.
> */
> - assert(in_aio_context_home_thread(ctx));
> + assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
> + qemu_get_aio_context() : ctx));
>
> qemu_lockcnt_inc(&ctx->list_lock);
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 11/13] util/async: Add aio_co_reschedule_self()
2020-09-09 15:11 ` [PATCH v7 11/13] util/async: Add aio_co_reschedule_self() Kevin Wolf
@ 2020-09-15 14:25 ` Stefan Hajnoczi
2020-10-02 8:01 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-15 14:25 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 467 bytes --]
On Wed, Sep 09, 2020 at 05:11:47PM +0200, Kevin Wolf wrote:
> Add a function that can be used to move the currently running coroutine
> to a different AioContext (and therefore potentially a different
> thread).
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> include/block/aio.h | 10 ++++++++++
> util/async.c | 30 ++++++++++++++++++++++++++++++
> 2 files changed, 40 insertions(+)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()
2020-09-09 15:11 ` [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context() Kevin Wolf
@ 2020-09-15 14:31 ` Stefan Hajnoczi
2020-09-25 16:00 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-15 14:31 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1501 bytes --]
On Wed, Sep 09, 2020 at 05:11:48PM +0200, Kevin Wolf wrote:
> Add a function to move the current coroutine to the AioContext of a
> given BlockDriverState.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> include/block/block.h | 6 ++++++
> block.c | 10 ++++++++++
> 2 files changed, 16 insertions(+)
>
> diff --git a/include/block/block.h b/include/block/block.h
> index 981ab5b314..80ab322f11 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
> */
> AioContext *bdrv_get_aio_context(BlockDriverState *bs);
>
> +/**
> + * Move the current coroutine to the AioContext of @bs and return the old
> + * AioContext of the coroutine.
> + */
> +AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
I'm not sure this function handles all cases:
1. Being called without the BQL (i.e. from an IOThread).
2. Being called while a device stops using its IOThread.
The races that come to mind are fetching the AioContext for bs and then
scheduling a BH. The BH is executed later on by the event loop. There
might be cases where the AioContext for bs is updated before the BH
runs.
I didn't investigate these cases but wanted to mention them in case you
want to add doc comments about when this function can be used or if
you'd like to verify them yourself.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine
2020-09-09 15:11 ` [PATCH v7 13/13] block: Convert 'block_resize' to coroutine Kevin Wolf
@ 2020-09-15 14:57 ` Stefan Hajnoczi
2020-09-25 16:07 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-15 14:57 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1448 bytes --]
On Wed, Sep 09, 2020 at 05:11:49PM +0200, Kevin Wolf wrote:
> @@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
> return;
> }
>
> - aio_context = bdrv_get_aio_context(bs);
> - aio_context_acquire(aio_context);
> + old_ctx = bdrv_co_move_to_aio_context(bs);
>
> if (size < 0) {
> error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
Is it safe to call blk_new() outside the BQL since it mutates global state?
In other words, could another thread race with us?
> @@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
> bdrv_drained_end(bs);
>
> out:
> + aio_co_reschedule_self(old_ctx);
> blk_unref(blk);
> - aio_context_release(aio_context);
The following precondition is violated by the blk_unref -> bdrv_drain ->
AIO_WAIT_WHILE() call if blk->refcnt is 1 here:
* The caller's thread must be the IOThread that owns @ctx or the main loop
* thread (with @ctx acquired exactly once).
blk_unref() is called from the main loop thread without having acquired
blk's AioContext.
Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but
I'm not sure if that can be guaranteed.
The following seems safer although it's uglier:
aio_context = bdrv_get_aio_context(bs);
aio_context_acquire(aio_context);
blk_unref(blk);
aio_context_release(aio_context);
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-14 15:09 ` Markus Armbruster
@ 2020-09-15 14:58 ` Stefan Hajnoczi
0 siblings, 0 replies; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-15 14:58 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, marcandre.lureau, qemu-devel, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]
On Mon, Sep 14, 2020 at 05:09:49PM +0200, Markus Armbruster wrote:
> Stefan Hajnoczi <stefanha@redhat.com> writes:
>
> > On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
> >> Some QMP command handlers can block the main loop for a relatively long
> >> time, for example because they perform some I/O. This is quite nasty.
> >> Allowing such handlers to run in a coroutine where they can yield (and
> >> therefore release the BQL) while waiting for an event such as I/O
> >> completion solves the problem.
> >>
> >> This series adds the infrastructure to allow this and switches
> >> block_resize to run in a coroutine as a first example.
> >>
> >> This is an alternative solution to Marc-André's "monitor: add
> >> asynchronous command type" series.
> >
> > Please clarify the following in the QAPI documentation:
> > * Is the QMP monitor suspended while the command is pending?
> > * Are QMP events reported while the command is pending?
>
> Good points. Kevin, I'd be willing to take this as a follow-up patch,
> if that's more convenient for you.
>
> > Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
>
> Stefan, I could use your proper review of PATCH 11-13. Pretty-please?
Sounds good. I have reviewed the patches 11-13 and left questions for
Kevin.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 10/13] hmp: Add support for coroutine command handlers
2020-09-09 15:11 ` [PATCH v7 10/13] hmp: Add support for coroutine command handlers Kevin Wolf
@ 2020-09-16 9:46 ` Dr. David Alan Gilbert
2020-10-02 8:01 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Dr. David Alan Gilbert @ 2020-09-16 9:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: marcandre.lureau, stefanha, armbru, qemu-block, qemu-devel
* Kevin Wolf (kwolf@redhat.com) wrote:
> Often, QMP command handlers are not only called to handle QMP commands,
> but also from a corresponding HMP command handler. In order to give them
> a consistent environment, optionally run HMP command handlers in a
> coroutine, too.
>
> The implementation is a lot simpler than in QMP because for HMP, we
> still block the VM while the coroutine is running.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> monitor/monitor-internal.h | 1 +
> monitor/hmp.c | 37 ++++++++++++++++++++++++++++++++-----
> 2 files changed, 33 insertions(+), 5 deletions(-)
>
> diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
> index b55d6df07f..ad2e64be13 100644
> --- a/monitor/monitor-internal.h
> +++ b/monitor/monitor-internal.h
> @@ -74,6 +74,7 @@ typedef struct HMPCommand {
> const char *help;
> const char *flags; /* p=preconfig */
> void (*cmd)(Monitor *mon, const QDict *qdict);
> + bool coroutine;
> /*
> * @sub_table is a list of 2nd level of commands. If it does not exist,
> * cmd should be used. If it exists, sub_table[?].cmd should be
> diff --git a/monitor/hmp.c b/monitor/hmp.c
> index 4b66ca1cd6..b858b0dbde 100644
> --- a/monitor/hmp.c
> +++ b/monitor/hmp.c
> @@ -1056,12 +1056,26 @@ fail:
> return NULL;
> }
>
> +typedef struct HandleHmpCommandCo {
> + Monitor *mon;
> + const HMPCommand *cmd;
> + QDict *qdict;
> + bool done;
> +} HandleHmpCommandCo;
> +
> +static void handle_hmp_command_co(void *opaque)
> +{
> + HandleHmpCommandCo *data = opaque;
> + data->cmd->cmd(data->mon, data->qdict);
> + monitor_set_cur(qemu_coroutine_self(), NULL);
> + data->done = true;
> +}
> +
> void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
> {
> QDict *qdict;
> const HMPCommand *cmd;
> const char *cmd_start = cmdline;
> - Monitor *old_mon;
>
> trace_handle_hmp_command(mon, cmdline);
>
> @@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
> return;
> }
>
> - /* old_mon is non-NULL when called from qmp_human_monitor_command() */
> - old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
> - cmd->cmd(&mon->common, qdict);
> - monitor_set_cur(qemu_coroutine_self(), old_mon);
> + if (!cmd->coroutine) {
> + /* old_mon is non-NULL when called from qmp_human_monitor_command() */
> + Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
> + cmd->cmd(&mon->common, qdict);
> + monitor_set_cur(qemu_coroutine_self(), old_mon);
> + } else {
> + HandleHmpCommandCo data = {
> + .mon = &mon->common,
> + .cmd = cmd,
> + .qdict = qdict,
> + .done = false,
> + };
> + Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
> + monitor_set_cur(co, &mon->common);
> + aio_co_enter(qemu_get_aio_context(), co);
> + AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
> + }
>
> qobject_unref(qdict);
> }
> --
> 2.25.4
>
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-14 15:10 ` Markus Armbruster
@ 2020-09-25 15:13 ` Kevin Wolf
2020-09-28 11:42 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 15:13 UTC (permalink / raw)
To: Markus Armbruster
Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > The correct way to set the current monitor for a coroutine handler will
> > be different than for a blocking handler, so monitor_set_cur() needs to
> > be called in qmp_dispatch().
> >
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> > include/qapi/qmp/dispatch.h | 3 ++-
> > monitor/qmp.c | 8 +-------
> > qapi/qmp-dispatch.c | 8 +++++++-
> > qga/main.c | 2 +-
> > stubs/monitor-core.c | 5 +++++
> > tests/test-qmp-cmds.c | 6 +++---
> > 6 files changed, 19 insertions(+), 13 deletions(-)
> >
> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> > index 5a9cf82472..0c2f467028 100644
> > --- a/include/qapi/qmp/dispatch.h
> > +++ b/include/qapi/qmp/dispatch.h
> > @@ -14,6 +14,7 @@
> > #ifndef QAPI_QMP_DISPATCH_H
> > #define QAPI_QMP_DISPATCH_H
> >
> > +#include "monitor/monitor.h"
> > #include "qemu/queue.h"
> >
> > typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
> > @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
> > bool qmp_has_success_response(const QmpCommand *cmd);
> > QDict *qmp_error_response(Error *err);
> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> > - bool allow_oob);
> > + bool allow_oob, Monitor *cur_mon);
> > bool qmp_is_oob(const QDict *dict);
> >
> > typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> > index 8469970c69..922fdb5541 100644
> > --- a/monitor/qmp.c
> > +++ b/monitor/qmp.c
> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> >
> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> > {
> > - Monitor *old_mon;
> > QDict *rsp;
> > QDict *error;
> >
> > - old_mon = monitor_set_cur(&mon->common);
> > - assert(old_mon == NULL);
> > -
> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> > -
> > - monitor_set_cur(NULL);
> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>
> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
It's 79 characters. Should be fine even with your local deviation from
the coding style to require less than that for comments?
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-14 15:11 ` Markus Armbruster
@ 2020-09-25 15:23 ` Kevin Wolf
2020-09-28 7:47 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 15:23 UTC (permalink / raw)
To: Markus Armbruster
Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > This way, a monitor command handler will still be able to access the
> > current monitor, but when it yields, all other code code will correctly
> > get NULL from monitor_cur().
> >
> > This uses a hash table to map the coroutine pointer to the current
> > monitor of that coroutine. Outside of coroutine context, we associate
> > the current monitor with the leader coroutine of the current thread.
>
> In qemu-system-FOO, the hash table can have only these entries:
>
> * (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while
> executing a QMP command out-of-band.
>
> * (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main
> thread) to a QMP monitor, while executing a QMP command in-band and in
> coroutine context.
>
> * (QMP) One mapping the main thread's leader to a QMP monitor, while
> executing a QMP command in-band and out of coroutine context, in a
> bottom half.
>
> * (HMP) One mapping the main thread's leader to an HMP monitor, while
> executing an HMP command out of coroutine context.
>
> * (HMP-CO) One mapping a transient coroutine in the main thread to an
> HMP monitor, while executing an HMP command in coroutine context.
>
> In-band execution is one command after the other.
>
> Therefore, at most one monitor command can be executing in-band at any
> time.
>
> Therefore, the hash table has at most *two* entries: one (OOB), and one
> of the other four.
>
> Can you shoot any holes into my argument?
I think with human-monitor-command, you can have three mappings:
1. The main thread's leader (it is a non-coroutine QMP command) to the
QMP monitor
2. With a coroutine HMP command, one mapping from the transient HMP
coroutine to the transient HMP monitor (with a non-coroutine HMP
command, we'd instead temporarily change the mapping from 1.)
3. The OOB entry
> I suspect there's a simpler solution struggling to get out. But this
> solution works, so in it goes. Should the simpler one succeed at
> getting out, it can go in on top. If not, I'll probably add even more
> comments to remind myself of these facts.
I think the simple approach you had posted could work if you can fill in
the HMP part, but I think it wasn't completely trivial and you told me
not to bother for now, so I didn't. It may still be a viable path
forward if you like it better.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-14 15:15 ` Markus Armbruster
@ 2020-09-25 15:37 ` Kevin Wolf
2020-09-28 8:23 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 15:37 UTC (permalink / raw)
To: Markus Armbruster
Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Am 14.09.2020 um 17:15 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > This patch adds a new 'coroutine' flag to QMP command definitions that
> > tells the QMP dispatcher that the command handler is safe to be run in a
> > coroutine.
> >
> > The documentation of the new flag pretends that this flag is already
> > used as intended, which it isn't yet after this patch. We'll implement
> > this in another patch in this series.
> >
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
> > ---
> > docs/devel/qapi-code-gen.txt | 12 ++++++++++++
> > include/qapi/qmp/dispatch.h | 1 +
> > tests/test-qmp-cmds.c | 4 ++++
> > scripts/qapi/commands.py | 10 +++++++---
> > scripts/qapi/doc.py | 2 +-
> > scripts/qapi/expr.py | 10 ++++++++--
> > scripts/qapi/introspect.py | 2 +-
> > scripts/qapi/schema.py | 12 ++++++++----
> > tests/qapi-schema/test-qapi.py | 7 ++++---
> > tests/qapi-schema/meson.build | 1 +
> > tests/qapi-schema/oob-coroutine.err | 2 ++
> > tests/qapi-schema/oob-coroutine.json | 2 ++
> > tests/qapi-schema/oob-coroutine.out | 0
> > tests/qapi-schema/qapi-schema-test.json | 1 +
> > tests/qapi-schema/qapi-schema-test.out | 2 ++
> > 15 files changed, 54 insertions(+), 14 deletions(-)
> > create mode 100644 tests/qapi-schema/oob-coroutine.err
> > create mode 100644 tests/qapi-schema/oob-coroutine.json
> > create mode 100644 tests/qapi-schema/oob-coroutine.out
> >
> > diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
> > index f3e7ced212..36daa9b5f8 100644
> > --- a/docs/devel/qapi-code-gen.txt
> > +++ b/docs/devel/qapi-code-gen.txt
> > @@ -472,6 +472,7 @@ Syntax:
> > '*gen': false,
> > '*allow-oob': true,
> > '*allow-preconfig': true,
> > + '*coroutine': true,
> > '*if': COND,
> > '*features': FEATURES }
> >
> > @@ -596,6 +597,17 @@ before the machine is built. It defaults to false. For example:
> > QMP is available before the machine is built only when QEMU was
> > started with --preconfig.
> >
> > +Member 'coroutine' tells the QMP dispatcher whether the command handler
> > +is safe to be run in a coroutine.
>
> We need to document what exactly makes a command handler coroutine safe
> / unsafe. We discussed this at some length in review of PATCH v4 1/4:
>
> Message-ID: <874kwnvgad.fsf@dusky.pond.sub.org>
> https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html
>
> I'd be willing to accept a follow-up patch, if that's more convenient
> for you.
Did we ever arrive at a conclusion for a good definition?
I mean I can give a definition like "it's coroutine safe if it results
in the right behaviour even when called in coroutine context", but
that's not really helpful.
FWIW, I would also have a hard time giving a much better definition than
this for thread safety.
> > It defaults to false. If it is true,
> > +the command handler is called from coroutine context and may yield while
>
> Is it *always* called from coroutine context, or could it be called
> outside coroutine context, too? I guess the former, thanks to PATCH 10,
> and as long as we diligently mark HMP commands that such call QMP
> handlers, like you do in PATCH 13.
Yes, it must *always* be called from coroutine context. This is the
reason why I included HMP after all, even though I'm really mostly just
interested in QMP.
> What's the worst than can happen when we neglect to mark such an HMP
> command?
When the command handler tries to yield and it's not in coroutine
context, it will abort(). If it never tries to yield, I think it would
just work - but of course, the ability to yield is the only reason why
you would want to run a command handler in a coroutine.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine
2020-09-14 15:30 ` Markus Armbruster
@ 2020-09-25 15:38 ` Kevin Wolf
2020-09-28 8:24 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 15:38 UTC (permalink / raw)
To: Markus Armbruster
Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Am 14.09.2020 um 17:30 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > This moves the QMP dispatcher to a coroutine and runs all QMP command
> > handlers that declare 'coroutine': true in coroutine context so they
> > can avoid blocking the main loop while doing I/O or waiting for other
> > events.
> >
> > For commands that are not declared safe to run in a coroutine, the
> > dispatcher drops out of coroutine context by calling the QMP command
> > handler from a bottom half.
> >
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
> > ---
> > include/qapi/qmp/dispatch.h | 1 +
> > monitor/monitor-internal.h | 6 +-
> > monitor/monitor.c | 55 +++++++++++++---
> > monitor/qmp.c | 122 +++++++++++++++++++++++++++---------
> > qapi/qmp-dispatch.c | 61 ++++++++++++++++--
> > qapi/qmp-registry.c | 3 +
> > util/aio-posix.c | 8 ++-
> > 7 files changed, 210 insertions(+), 46 deletions(-)
> >
> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> > index 9fd2b720a7..af8d96c570 100644
> > --- a/include/qapi/qmp/dispatch.h
> > +++ b/include/qapi/qmp/dispatch.h
> > @@ -31,6 +31,7 @@ typedef enum QmpCommandOptions
> > typedef struct QmpCommand
> > {
> > const char *name;
> > + /* Runs in coroutine context if QCO_COROUTINE is set */
> > QmpCommandFunc *fn;
> > QmpCommandOptions options;
> > QTAILQ_ENTRY(QmpCommand) node;
> > diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
> > index b39e03b744..b55d6df07f 100644
> > --- a/monitor/monitor-internal.h
> > +++ b/monitor/monitor-internal.h
> > @@ -155,7 +155,9 @@ static inline bool monitor_is_qmp(const Monitor *mon)
> >
> > typedef QTAILQ_HEAD(MonitorList, Monitor) MonitorList;
> > extern IOThread *mon_iothread;
> > -extern QEMUBH *qmp_dispatcher_bh;
> > +extern Coroutine *qmp_dispatcher_co;
> > +extern bool qmp_dispatcher_co_shutdown;
> > +extern bool qmp_dispatcher_co_busy;
> > extern QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
> > extern QemuMutex monitor_lock;
> > extern MonitorList mon_list;
> > @@ -173,7 +175,7 @@ void monitor_fdsets_cleanup(void);
> >
> > void qmp_send_response(MonitorQMP *mon, const QDict *rsp);
> > void monitor_data_destroy_qmp(MonitorQMP *mon);
> > -void monitor_qmp_bh_dispatcher(void *data);
> > +void coroutine_fn monitor_qmp_dispatcher_co(void *data);
> >
> > int get_monitor_def(int64_t *pval, const char *name);
> > void help_cmd(Monitor *mon, const char *name);
> > diff --git a/monitor/monitor.c b/monitor/monitor.c
> > index 629aa073ee..ac2722bf91 100644
> > --- a/monitor/monitor.c
> > +++ b/monitor/monitor.c
> > @@ -55,8 +55,32 @@ typedef struct {
> > /* Shared monitor I/O thread */
> > IOThread *mon_iothread;
> >
> > -/* Bottom half to dispatch the requests received from I/O thread */
> > -QEMUBH *qmp_dispatcher_bh;
> > +/* Coroutine to dispatch the requests received from I/O thread */
> > +Coroutine *qmp_dispatcher_co;
> > +
> > +/* Set to true when the dispatcher coroutine should terminate */
> > +bool qmp_dispatcher_co_shutdown;
> > +
> > +/*
> > + * qmp_dispatcher_co_busy is used for synchronisation between the
> > + * monitor thread and the main thread to ensure that the dispatcher
> > + * coroutine never gets scheduled a second time when it's already
> > + * scheduled (scheduling the same coroutine twice is forbidden).
> > + *
> > + * It is true if the coroutine is active and processing requests.
> > + * Additional requests may then be pushed onto mon->qmp_requests,
> > + * and @qmp_dispatcher_co_shutdown may be set without further ado.
> > + * @qmp_dispatcher_co_busy must not be woken up in this case.
> > + *
> > + * If false, you also have to set @qmp_dispatcher_co_busy to true and
> > + * wake up @qmp_dispatcher_co after pushing the new requests.
> > + *
> > + * The coroutine will automatically change this variable back to false
> > + * before it yields. Nobody else may set the variable to false.
> > + *
> > + * Access must be atomic for thread safety.
> > + */
> > +bool qmp_dispatcher_co_busy;
> >
> > /*
> > * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
> > @@ -623,9 +647,24 @@ void monitor_cleanup(void)
> > }
> > qemu_mutex_unlock(&monitor_lock);
> >
> > - /* QEMUBHs needs to be deleted before destroying the I/O thread */
> > - qemu_bh_delete(qmp_dispatcher_bh);
> > - qmp_dispatcher_bh = NULL;
> > + /*
> > + * The dispatcher needs to stop before destroying the I/O thread.
> > + *
> > + * We need to poll both qemu_aio_context and iohandler_ctx to make
> > + * sure that the dispatcher coroutine keeps making progress and
> > + * eventually terminates. qemu_aio_context is automatically
> > + * polled by calling AIO_WAIT_WHILE on it, but we must poll
> > + * iohandler_ctx manually.
> > + */
> > + qmp_dispatcher_co_shutdown = true;
> > + if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
> > + aio_co_wake(qmp_dispatcher_co);
> > + }
> > +
> > + AIO_WAIT_WHILE(qemu_get_aio_context(),
> > + (aio_poll(iohandler_get_aio_context(), false),
> > + atomic_mb_read(&qmp_dispatcher_co_busy)));
> > +
> > if (mon_iothread) {
> > iothread_destroy(mon_iothread);
> > mon_iothread = NULL;
> > @@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
> > * have commands assuming that context. It would be nice to get
> > * rid of those assumptions.
> > */
> > - qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
> > - monitor_qmp_bh_dispatcher,
> > - NULL);
> > + qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
> > + atomic_mb_set(&qmp_dispatcher_co_busy, true);
> > + aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
> > }
> >
> > int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> > index 922fdb5541..69f6e93f38 100644
> > --- a/monitor/qmp.c
> > +++ b/monitor/qmp.c
> > @@ -133,6 +133,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> > }
> > }
> >
> > +/*
> > + * Runs outside of coroutine context for OOB commands, but in
> > + * coroutine context for everything else.
> > + */
> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> > {
> > QDict *rsp;
> > @@ -205,43 +209,99 @@ static QMPRequest *monitor_qmp_requests_pop_any_with_lock(void)
> > return req_obj;
> > }
> >
> > -void monitor_qmp_bh_dispatcher(void *data)
> > +void coroutine_fn monitor_qmp_dispatcher_co(void *data)
> > {
> > - QMPRequest *req_obj = monitor_qmp_requests_pop_any_with_lock();
> > + QMPRequest *req_obj = NULL;
> > QDict *rsp;
> > bool need_resume;
> > MonitorQMP *mon;
> >
> > - if (!req_obj) {
> > - return;
> > - }
> > + while (true) {
> > + assert(atomic_mb_read(&qmp_dispatcher_co_busy) == true);
> >
> > - mon = req_obj->mon;
> > - /* qmp_oob_enabled() might change after "qmp_capabilities" */
> > - need_resume = !qmp_oob_enabled(mon) ||
> > - mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
> > - qemu_mutex_unlock(&mon->qmp_queue_lock);
> > - if (req_obj->req) {
> > - QDict *qdict = qobject_to(QDict, req_obj->req);
> > - QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
> > - trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
> > - monitor_qmp_dispatch(mon, req_obj->req);
> > - } else {
> > - assert(req_obj->err);
> > - rsp = qmp_error_response(req_obj->err);
> > - req_obj->err = NULL;
> > - monitor_qmp_respond(mon, rsp);
> > - qobject_unref(rsp);
> > - }
> > + /*
> > + * Mark the dispatcher as not busy already here so that we
> > + * don't miss any new requests coming in the middle of our
> > + * processing.
> > + */
> > + atomic_mb_set(&qmp_dispatcher_co_busy, false);
> > +
> > + while (!(req_obj = monitor_qmp_requests_pop_any_with_lock())) {
> > + /*
> > + * No more requests to process. Wait to be reentered from
> > + * handle_qmp_command() when it pushes more requests, or
> > + * from monitor_cleanup() when it requests shutdown.
> > + */
> > + if (!qmp_dispatcher_co_shutdown) {
> > + qemu_coroutine_yield();
> > +
> > + /*
> > + * busy must be set to true again by whoever
> > + * rescheduled us to avoid double scheduling
> > + */
> > + assert(atomic_xchg(&qmp_dispatcher_co_busy, false) == true);
> > + }
> > +
> > + /*
> > + * qmp_dispatcher_co_shutdown may have changed if we
> > + * yielded and were reentered from monitor_cleanup()
> > + */
> > + if (qmp_dispatcher_co_shutdown) {
> > + return;
> > + }
> > + }
> >
> > - if (need_resume) {
> > - /* Pairs with the monitor_suspend() in handle_qmp_command() */
> > - monitor_resume(&mon->common);
> > - }
> > - qmp_request_free(req_obj);
> > + if (atomic_xchg(&qmp_dispatcher_co_busy, true) == true) {
> > + /*
> > + * Someone rescheduled us (probably because a new requests
> > + * came in), but we didn't actually yield. Do that now,
> > + * only to be immediately reentered and removed from the
> > + * list of scheduled coroutines.
> > + */
> > + qemu_coroutine_yield();
> > + }
> >
> > - /* Reschedule instead of looping so the main loop stays responsive */
> > - qemu_bh_schedule(qmp_dispatcher_bh);
> > + /*
> > + * Move the coroutine from iohandler_ctx to qemu_aio_context for
> > + * executing the command handler so that it can make progress if it
> > + * involves an AIO_WAIT_WHILE().
> > + */
> > + aio_co_schedule(qemu_get_aio_context(), qmp_dispatcher_co);
> > + qemu_coroutine_yield();
> > +
> > + mon = req_obj->mon;
> > + /* qmp_oob_enabled() might change after "qmp_capabilities" */
> > + need_resume = !qmp_oob_enabled(mon) ||
> > + mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX - 1;
> > + qemu_mutex_unlock(&mon->qmp_queue_lock);
> > + if (req_obj->req) {
> > + QDict *qdict = qobject_to(QDict, req_obj->req);
> > + QObject *id = qdict ? qdict_get(qdict, "id") : NULL;
> > + trace_monitor_qmp_cmd_in_band(qobject_get_try_str(id) ?: "");
> > + monitor_qmp_dispatch(mon, req_obj->req);
> > + } else {
> > + assert(req_obj->err);
> > + rsp = qmp_error_response(req_obj->err);
> > + req_obj->err = NULL;
> > + monitor_qmp_respond(mon, rsp);
> > + qobject_unref(rsp);
> > + }
> > +
> > + if (need_resume) {
> > + /* Pairs with the monitor_suspend() in handle_qmp_command() */
> > + monitor_resume(&mon->common);
> > + }
> > + qmp_request_free(req_obj);
> > +
> > + /*
> > + * Yield and reschedule so the main loop stays responsive.
> > + *
> > + * Move back to iohandler_ctx so that nested event loops for
> > + * qemu_aio_context don't start new monitor commands.
> > + */
> > + aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
> > + qemu_coroutine_yield();
> > + }
> > }
> >
> > static void handle_qmp_command(void *opaque, QObject *req, Error *err)
> > @@ -302,7 +362,9 @@ static void handle_qmp_command(void *opaque, QObject *req, Error *err)
> > qemu_mutex_unlock(&mon->qmp_queue_lock);
> >
> > /* Kick the dispatcher routine */
> > - qemu_bh_schedule(qmp_dispatcher_bh);
> > + if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
> > + aio_co_wake(qmp_dispatcher_co);
> > + }
> > }
> >
> > static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
> > diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
> > index 5677ba92ca..754f7b854c 100644
> > --- a/qapi/qmp-dispatch.c
> > +++ b/qapi/qmp-dispatch.c
> > @@ -12,12 +12,16 @@
> > */
> >
> > #include "qemu/osdep.h"
> > +
> > +#include "block/aio.h"
> > #include "qapi/error.h"
> > #include "qapi/qmp/dispatch.h"
> > #include "qapi/qmp/qdict.h"
> > #include "qapi/qmp/qjson.h"
> > #include "sysemu/runstate.h"
> > #include "qapi/qmp/qbool.h"
> > +#include "qemu/coroutine.h"
> > +#include "qemu/main-loop.h"
> >
> > static QDict *qmp_dispatch_check_obj(QDict *dict, bool allow_oob,
> > Error **errp)
> > @@ -88,6 +92,30 @@ bool qmp_is_oob(const QDict *dict)
> > && !qdict_haskey(dict, "execute");
> > }
> >
> > +typedef struct QmpDispatchBH {
> > + const QmpCommand *cmd;
> > + Monitor *cur_mon;
> > + QDict *args;
> > + QObject **ret;
> > + Error **errp;
> > + Coroutine *co;
> > +} QmpDispatchBH;
> > +
> > +static void do_qmp_dispatch_bh(void *opaque)
> > +{
> > + QmpDispatchBH *data = opaque;
> > +
> > + assert(monitor_cur() == NULL);
> > + monitor_set_cur(qemu_coroutine_self(), data->cur_mon);
> > + data->cmd->fn(data->args, data->ret, data->errp);
> > + monitor_set_cur(qemu_coroutine_self(), NULL);
> > + aio_co_wake(data->co);
> > +}
> > +
> > +/*
> > + * Runs outside of coroutine context for OOB commands, but in coroutine
> > + * context for everything else.
> > + */
> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> > bool allow_oob, Monitor *cur_mon)
> > {
> > @@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> > qobject_ref(args);
> > }
> >
> > + assert(!(oob && qemu_in_coroutine()));
> > assert(monitor_cur() == NULL);
> > - monitor_set_cur(qemu_coroutine_self(), cur_mon);
> > -
> > - cmd->fn(args, &ret, &err);
> > -
> > - monitor_set_cur(qemu_coroutine_self(), NULL);
> > + if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
> > + monitor_set_cur(qemu_coroutine_self(), cur_mon);
> > + cmd->fn(args, &ret, &err);
> > + monitor_set_cur(qemu_coroutine_self(), NULL);
> > + } else {
> > + /*
> > + * Not being in coroutine context implies that we're handling
> > + * an OOB command, which must not have QCO_COROUTINE.
> > + *
> > + * This implies that we are in coroutine context, but the
> > + * command doesn't have QCO_COROUTINE. We must drop out of
> > + * coroutine context for this one.
> > + */
>
> I had to read this several times to get it. The first sentence leads me
> into coroutine context, and then the next sentence tells me the
> opposite, throwing me into confusion.
>
> Perhaps something like this:
>
> /*
> * Actual context doesn't match the one the command needs.
> * Case 1: we are in coroutine context, but command does not
> * have QCO_COROUTINE. We need to drop out of coroutine
> * context for executing it.
> * Case 2: we are outside coroutine context, but command has
> * QCO_COROUTINE. Can't actually happen, because we get here
> * outside coroutine context only when executing a command
> * out of band, and OOB commands never have QCO_COROUTINE.
> */
Works for me. Can you squash this in while applying?
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()
2020-09-15 14:31 ` Stefan Hajnoczi
@ 2020-09-25 16:00 ` Kevin Wolf
2020-09-28 8:59 ` Stefan Hajnoczi
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 16:00 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 2319 bytes --]
Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:
> On Wed, Sep 09, 2020 at 05:11:48PM +0200, Kevin Wolf wrote:
> > Add a function to move the current coroutine to the AioContext of a
> > given BlockDriverState.
> >
> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > ---
> > include/block/block.h | 6 ++++++
> > block.c | 10 ++++++++++
> > 2 files changed, 16 insertions(+)
> >
> > diff --git a/include/block/block.h b/include/block/block.h
> > index 981ab5b314..80ab322f11 100644
> > --- a/include/block/block.h
> > +++ b/include/block/block.h
> > @@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
> > */
> > AioContext *bdrv_get_aio_context(BlockDriverState *bs);
> >
> > +/**
> > + * Move the current coroutine to the AioContext of @bs and return the old
> > + * AioContext of the coroutine.
> > + */
> > +AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
>
> I'm not sure this function handles all cases:
> 1. Being called without the BQL (i.e. from an IOThread).
> 2. Being called while a device stops using its IOThread.
>
> The races that come to mind are fetching the AioContext for bs and then
> scheduling a BH. The BH is executed later on by the event loop. There
> might be cases where the AioContext for bs is updated before the BH
> runs.
Updating the AioContext of a node drains the BDS first, so it would
execute any BHs still pending in the old AioContext. So this part looks
safe to me.
I'm not sure what you mean with the BQL part. I don't think I'm
accessing anything protected by the BQL?
> I didn't investigate these cases but wanted to mention them in case you
> want to add doc comments about when this function can be used or if
> you'd like to verify them yourself.
One thing that I'm not completely sure about (but that isn't really
related to this specific patch) is AioContext changes later in the
process when the actual command handler has yielded. We may have to be
careful to prevent those for coroutine based QMP commands in the block
layer.
By sheer luck, qmp_block_resize() creates a new BlockBackend that has
blk->allow_aio_context_change = false. So we're actually good in the
only command I'm converting. Phew.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine
2020-09-15 14:57 ` Stefan Hajnoczi
@ 2020-09-25 16:07 ` Kevin Wolf
2020-09-28 9:05 ` Stefan Hajnoczi
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 16:07 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1917 bytes --]
Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:
> On Wed, Sep 09, 2020 at 05:11:49PM +0200, Kevin Wolf wrote:
> > @@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
> > return;
> > }
> >
> > - aio_context = bdrv_get_aio_context(bs);
> > - aio_context_acquire(aio_context);
> > + old_ctx = bdrv_co_move_to_aio_context(bs);
> >
> > if (size < 0) {
> > error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
>
> Is it safe to call blk_new() outside the BQL since it mutates global state?
>
> In other words, could another thread race with us?
Hm, probably not.
Would it be safer to have the bdrv_co_move_to_aio_context() call only
immediately before the drain?
> > @@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
> > bdrv_drained_end(bs);
> >
> > out:
> > + aio_co_reschedule_self(old_ctx);
> > blk_unref(blk);
> > - aio_context_release(aio_context);
>
> The following precondition is violated by the blk_unref -> bdrv_drain ->
> AIO_WAIT_WHILE() call if blk->refcnt is 1 here:
>
> * The caller's thread must be the IOThread that owns @ctx or the main loop
> * thread (with @ctx acquired exactly once).
>
> blk_unref() is called from the main loop thread without having acquired
> blk's AioContext.
>
> Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but
> I'm not sure if that can be guaranteed.
>
> The following seems safer although it's uglier:
>
> aio_context = bdrv_get_aio_context(bs);
> aio_context_acquire(aio_context);
> blk_unref(blk);
> aio_context_release(aio_context);
May we actually acquire aio_context if blk is in the main thread? I
think we must only do this if it's in a different iothread because we'd
end up with a recursive lock and drain would hang.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-10 13:24 ` Stefan Hajnoczi
2020-09-14 15:09 ` Markus Armbruster
@ 2020-09-25 17:15 ` Kevin Wolf
2020-09-28 8:46 ` Stefan Hajnoczi
2020-09-28 9:30 ` Markus Armbruster
1 sibling, 2 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-25 17:15 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]
Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:
> On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
> > Some QMP command handlers can block the main loop for a relatively long
> > time, for example because they perform some I/O. This is quite nasty.
> > Allowing such handlers to run in a coroutine where they can yield (and
> > therefore release the BQL) while waiting for an event such as I/O
> > completion solves the problem.
> >
> > This series adds the infrastructure to allow this and switches
> > block_resize to run in a coroutine as a first example.
> >
> > This is an alternative solution to Marc-André's "monitor: add
> > asynchronous command type" series.
>
> Please clarify the following in the QAPI documentation:
> * Is the QMP monitor suspended while the command is pending?
Suspended as in monitor_suspend()? No.
> * Are QMP events reported while the command is pending?
Hm, I don't know to be honest. But I think so.
Does it matter, though? I don't think events have a defined order
compared to command results, and the client can't respond to the event
anyway until the current command has completed.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-25 15:23 ` Kevin Wolf
@ 2020-09-28 7:47 ` Markus Armbruster
2020-09-28 10:42 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 7:47 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > This way, a monitor command handler will still be able to access the
>> > current monitor, but when it yields, all other code code will correctly
>> > get NULL from monitor_cur().
>> >
>> > This uses a hash table to map the coroutine pointer to the current
>> > monitor of that coroutine. Outside of coroutine context, we associate
>> > the current monitor with the leader coroutine of the current thread.
>>
>> In qemu-system-FOO, the hash table can have only these entries:
>>
>> * (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while
>> executing a QMP command out-of-band.
>>
>> * (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main
>> thread) to a QMP monitor, while executing a QMP command in-band and in
>> coroutine context.
>>
>> * (QMP) One mapping the main thread's leader to a QMP monitor, while
>> executing a QMP command in-band and out of coroutine context, in a
>> bottom half.
>>
>> * (HMP) One mapping the main thread's leader to an HMP monitor, while
>> executing an HMP command out of coroutine context.
>>
>> * (HMP-CO) One mapping a transient coroutine in the main thread to an
>> HMP monitor, while executing an HMP command in coroutine context.
>>
>> In-band execution is one command after the other.
>>
>> Therefore, at most one monitor command can be executing in-band at any
>> time.
>>
>> Therefore, the hash table has at most *two* entries: one (OOB), and one
>> of the other four.
>>
>> Can you shoot any holes into my argument?
>
> I think with human-monitor-command, you can have three mappings:
>
> 1. The main thread's leader (it is a non-coroutine QMP command) to the
> QMP monitor
This is (QMP).
> 2. With a coroutine HMP command, one mapping from the transient HMP
> coroutine to the transient HMP monitor (with a non-coroutine HMP
> command, we'd instead temporarily change the mapping from 1.)
This is (HMP-CO).
> 3. The OOB entry
This is (OOB).
To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band,
non-coroutine QMP command needs to execute interleaved with the in-band,
coroutine HMP command.
Such an interleaving contradicts "In-band execution is one command after
the other", which is a fundamental assumption in-band commands may make.
If the assumption is invalid, we got a problem. Is it?
>> I suspect there's a simpler solution struggling to get out. But this
>> solution works, so in it goes. Should the simpler one succeed at
>> getting out, it can go in on top. If not, I'll probably add even more
>> comments to remind myself of these facts.
>
> I think the simple approach you had posted could work if you can fill in
> the HMP part, but I think it wasn't completely trivial and you told me
> not to bother for now,
Correct. I decided to go with the code you already tested.
> so I didn't. It may still be a viable path
> forward if you like it better.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-25 15:37 ` Kevin Wolf
@ 2020-09-28 8:23 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 8:23 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 14.09.2020 um 17:15 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > This patch adds a new 'coroutine' flag to QMP command definitions that
>> > tells the QMP dispatcher that the command handler is safe to be run in a
>> > coroutine.
>> >
>> > The documentation of the new flag pretends that this flag is already
>> > used as intended, which it isn't yet after this patch. We'll implement
>> > this in another patch in this series.
>> >
>> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> > Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
>> > ---
>> > docs/devel/qapi-code-gen.txt | 12 ++++++++++++
>> > include/qapi/qmp/dispatch.h | 1 +
>> > tests/test-qmp-cmds.c | 4 ++++
>> > scripts/qapi/commands.py | 10 +++++++---
>> > scripts/qapi/doc.py | 2 +-
>> > scripts/qapi/expr.py | 10 ++++++++--
>> > scripts/qapi/introspect.py | 2 +-
>> > scripts/qapi/schema.py | 12 ++++++++----
>> > tests/qapi-schema/test-qapi.py | 7 ++++---
>> > tests/qapi-schema/meson.build | 1 +
>> > tests/qapi-schema/oob-coroutine.err | 2 ++
>> > tests/qapi-schema/oob-coroutine.json | 2 ++
>> > tests/qapi-schema/oob-coroutine.out | 0
>> > tests/qapi-schema/qapi-schema-test.json | 1 +
>> > tests/qapi-schema/qapi-schema-test.out | 2 ++
>> > 15 files changed, 54 insertions(+), 14 deletions(-)
>> > create mode 100644 tests/qapi-schema/oob-coroutine.err
>> > create mode 100644 tests/qapi-schema/oob-coroutine.json
>> > create mode 100644 tests/qapi-schema/oob-coroutine.out
>> >
>> > diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
>> > index f3e7ced212..36daa9b5f8 100644
>> > --- a/docs/devel/qapi-code-gen.txt
>> > +++ b/docs/devel/qapi-code-gen.txt
>> > @@ -472,6 +472,7 @@ Syntax:
>> > '*gen': false,
>> > '*allow-oob': true,
>> > '*allow-preconfig': true,
>> > + '*coroutine': true,
>> > '*if': COND,
>> > '*features': FEATURES }
>> >
>> > @@ -596,6 +597,17 @@ before the machine is built. It defaults to false. For example:
>> > QMP is available before the machine is built only when QEMU was
>> > started with --preconfig.
>> >
>> > +Member 'coroutine' tells the QMP dispatcher whether the command handler
>> > +is safe to be run in a coroutine.
>>
>> We need to document what exactly makes a command handler coroutine safe
>> / unsafe. We discussed this at some length in review of PATCH v4 1/4:
>>
>> Message-ID: <874kwnvgad.fsf@dusky.pond.sub.org>
>> https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05015.html
>>
>> I'd be willing to accept a follow-up patch, if that's more convenient
>> for you.
>
> Did we ever arrive at a conclusion for a good definition?
>
> I mean I can give a definition like "it's coroutine safe if it results
> in the right behaviour even when called in coroutine context", but
> that's not really helpful.
If an actual definition is out of reach, we should still say
*something*. Even a mere "it's complicated" would help, because it
warns the reader he's on thin ice.
Can we give examples for "unsafe"? You mentioned nested event loops.
> FWIW, I would also have a hard time giving a much better definition than
> this for thread safety.
For what it's worth, here's how POSIX.1-2017 defined "thread-safe":
A thread-safe function can be safely invoked concurrently with other
calls to the same function, or with calls to any other thread-safe
functions, by multiple threads. Each function defined in the System
Interfaces volume of POSIX.1-2017 is thread-safe unless explicitly
stated otherwise. Examples are any "pure" function, a function
which holds a mutex locked while it is accessing static storage, or
objects shared among threads.
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_407
>> > It defaults to false. If it is true,
>> > +the command handler is called from coroutine context and may yield while
>>
>> Is it *always* called from coroutine context, or could it be called
>> outside coroutine context, too? I guess the former, thanks to PATCH 10,
>> and as long as we diligently mark HMP commands that such call QMP
>> handlers, like you do in PATCH 13.
>
> Yes, it must *always* be called from coroutine context. This is the
> reason why I included HMP after all, even though I'm really mostly just
> interested in QMP.
>
>> What's the worst than can happen when we neglect to mark such an HMP
>> command?
>
> When the command handler tries to yield and it's not in coroutine
> context, it will abort(). If it never tries to yield, I think it would
> just work - but of course, the ability to yield is the only reason why
> you would want to run a command handler in a coroutine.
Let's spell this out. After your paragraph
Member 'coroutine' tells the QMP dispatcher whether the command handler
is safe to be run in a coroutine. It defaults to false. If it is true,
the command handler is called from coroutine context and may yield while
waiting for an external event (such as I/O completion) in order to avoid
blocking the guest and other background operations.
add something like
Since the command handler may assume coroutine-context, any callers
other than the QMP dispatcher must also call it in coroutine
context. In particular, HMP commands calling such a QMP command
handler must be marked .coroutine = true in hmp-commands.hx.
Patch ordering issue: this becomes possible only in PATCH 10.
Possible solutions:
* Add the last sentence only in PATCH 10.
* Write "HMP commands cannot call such a QMP command handler" in PATCH
8, replace it in PATCH 10.
* Add a "TODO make that possible" line here, drop it in PATCH 10.
* Reorder patches.
I don't like the first one much. Anyway, up to you.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine
2020-09-25 15:38 ` Kevin Wolf
@ 2020-09-28 8:24 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 8:24 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 14.09.2020 um 17:30 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > This moves the QMP dispatcher to a coroutine and runs all QMP command
>> > handlers that declare 'coroutine': true in coroutine context so they
>> > can avoid blocking the main loop while doing I/O or waiting for other
>> > events.
>> >
>> > For commands that are not declared safe to run in a coroutine, the
>> > dispatcher drops out of coroutine context by calling the QMP command
>> > handler from a bottom half.
>> >
>> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> > Reviewed-by: Markus Armbruster <armbru@redhat.com>
>> > ---
[...]
>> > diff --git a/qapi/qmp-dispatch.c b/qapi/qmp-dispatch.c
>> > index 5677ba92ca..754f7b854c 100644
>> > --- a/qapi/qmp-dispatch.c
>> > +++ b/qapi/qmp-dispatch.c
[...]
>> > @@ -153,12 +181,35 @@ QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
>> > qobject_ref(args);
>> > }
>> >
>> > + assert(!(oob && qemu_in_coroutine()));
>> > assert(monitor_cur() == NULL);
>> > - monitor_set_cur(qemu_coroutine_self(), cur_mon);
>> > -
>> > - cmd->fn(args, &ret, &err);
>> > -
>> > - monitor_set_cur(qemu_coroutine_self(), NULL);
>> > + if (!!(cmd->options & QCO_COROUTINE) == qemu_in_coroutine()) {
>> > + monitor_set_cur(qemu_coroutine_self(), cur_mon);
>> > + cmd->fn(args, &ret, &err);
>> > + monitor_set_cur(qemu_coroutine_self(), NULL);
>> > + } else {
>> > + /*
>> > + * Not being in coroutine context implies that we're handling
>> > + * an OOB command, which must not have QCO_COROUTINE.
>> > + *
>> > + * This implies that we are in coroutine context, but the
>> > + * command doesn't have QCO_COROUTINE. We must drop out of
>> > + * coroutine context for this one.
>> > + */
>>
>> I had to read this several times to get it. The first sentence leads me
>> into coroutine context, and then the next sentence tells me the
>> opposite, throwing me into confusion.
>>
>> Perhaps something like this:
>>
>> /*
>> * Actual context doesn't match the one the command needs.
>> * Case 1: we are in coroutine context, but command does not
>> * have QCO_COROUTINE. We need to drop out of coroutine
>> * context for executing it.
>> * Case 2: we are outside coroutine context, but command has
>> * QCO_COROUTINE. Can't actually happen, because we get here
>> * outside coroutine context only when executing a command
>> * out of band, and OOB commands never have QCO_COROUTINE.
>> */
>
> Works for me. Can you squash this in while applying?
Sure!
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-25 17:15 ` Kevin Wolf
@ 2020-09-28 8:46 ` Stefan Hajnoczi
2020-09-28 9:47 ` Kevin Wolf
2020-09-28 9:30 ` Markus Armbruster
1 sibling, 1 reply; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-28 8:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 1848 bytes --]
On Fri, Sep 25, 2020 at 07:15:41PM +0200, Kevin Wolf wrote:
> Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:
> > On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
> > > Some QMP command handlers can block the main loop for a relatively long
> > > time, for example because they perform some I/O. This is quite nasty.
> > > Allowing such handlers to run in a coroutine where they can yield (and
> > > therefore release the BQL) while waiting for an event such as I/O
> > > completion solves the problem.
> > >
> > > This series adds the infrastructure to allow this and switches
> > > block_resize to run in a coroutine as a first example.
> > >
> > > This is an alternative solution to Marc-André's "monitor: add
> > > asynchronous command type" series.
> >
> > Please clarify the following in the QAPI documentation:
> > * Is the QMP monitor suspended while the command is pending?
>
> Suspended as in monitor_suspend()? No.
>
> > * Are QMP events reported while the command is pending?
>
> Hm, I don't know to be honest. But I think so.
>
> Does it matter, though? I don't think events have a defined order
> compared to command results, and the client can't respond to the event
> anyway until the current command has completed.
You're right, I don't think it matters because the client must expect
QMP events at any time.
I was trying to understand the semantics of coroutine monitor commands
from two perspectives:
1. The QMP client - do coroutine commands behave differently from
non-coroutine commands? I think the answer is no. The monitor will
not process more commands until the coroutine finishes?
2. The command implementation - which thread does the coroutine run in?
I guess it runs in the main loop thread with the BQL and the
AioContext acquired?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()
2020-09-25 16:00 ` Kevin Wolf
@ 2020-09-28 8:59 ` Stefan Hajnoczi
2020-09-28 10:21 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-28 8:59 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]
On Fri, Sep 25, 2020 at 06:00:51PM +0200, Kevin Wolf wrote:
> Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:
> > On Wed, Sep 09, 2020 at 05:11:48PM +0200, Kevin Wolf wrote:
> > > Add a function to move the current coroutine to the AioContext of a
> > > given BlockDriverState.
> > >
> > > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > > ---
> > > include/block/block.h | 6 ++++++
> > > block.c | 10 ++++++++++
> > > 2 files changed, 16 insertions(+)
> > >
> > > diff --git a/include/block/block.h b/include/block/block.h
> > > index 981ab5b314..80ab322f11 100644
> > > --- a/include/block/block.h
> > > +++ b/include/block/block.h
> > > @@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
> > > */
> > > AioContext *bdrv_get_aio_context(BlockDriverState *bs);
> > >
> > > +/**
> > > + * Move the current coroutine to the AioContext of @bs and return the old
> > > + * AioContext of the coroutine.
> > > + */
> > > +AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
> >
> > I'm not sure this function handles all cases:
> > 1. Being called without the BQL (i.e. from an IOThread).
> > 2. Being called while a device stops using its IOThread.
> >
> > The races that come to mind are fetching the AioContext for bs and then
> > scheduling a BH. The BH is executed later on by the event loop. There
> > might be cases where the AioContext for bs is updated before the BH
> > runs.
The scenario I'm thinking about is where bs' AioContext changes while we
are trying to move there.
There is a window of time between fetching bs' AioContext, scheduling a
BH in our old AioContext (not in bs' AioContext), and then scheduling
the coroutine into the AioContext we previously fetched for bs.
Is it possible for the AioContext value we stashed to be outdated by the
time we use it?
I think the answer is that it's safe to use this function from the main
loop thread under the BQL. That way nothing else will change bs'
AioContext while we're running. But it's probably not safe to use this
function from an arbitrary IOThread (without the BQL).
I think this limitation is okay but it needs to be documented. Maybe an
assertion can verify that it holds.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine
2020-09-25 16:07 ` Kevin Wolf
@ 2020-09-28 9:05 ` Stefan Hajnoczi
2020-09-28 10:33 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Stefan Hajnoczi @ 2020-09-28 9:05 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 2167 bytes --]
On Fri, Sep 25, 2020 at 06:07:50PM +0200, Kevin Wolf wrote:
> Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:
> > On Wed, Sep 09, 2020 at 05:11:49PM +0200, Kevin Wolf wrote:
> > > @@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
> > > return;
> > > }
> > >
> > > - aio_context = bdrv_get_aio_context(bs);
> > > - aio_context_acquire(aio_context);
> > > + old_ctx = bdrv_co_move_to_aio_context(bs);
> > >
> > > if (size < 0) {
> > > error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
> >
> > Is it safe to call blk_new() outside the BQL since it mutates global state?
> >
> > In other words, could another thread race with us?
>
> Hm, probably not.
>
> Would it be safer to have the bdrv_co_move_to_aio_context() call only
> immediately before the drain?
Yes, sounds good.
> > > @@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
> > > bdrv_drained_end(bs);
> > >
> > > out:
> > > + aio_co_reschedule_self(old_ctx);
> > > blk_unref(blk);
> > > - aio_context_release(aio_context);
> >
> > The following precondition is violated by the blk_unref -> bdrv_drain ->
> > AIO_WAIT_WHILE() call if blk->refcnt is 1 here:
> >
> > * The caller's thread must be the IOThread that owns @ctx or the main loop
> > * thread (with @ctx acquired exactly once).
> >
> > blk_unref() is called from the main loop thread without having acquired
> > blk's AioContext.
> >
> > Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but
> > I'm not sure if that can be guaranteed.
> >
> > The following seems safer although it's uglier:
> >
> > aio_context = bdrv_get_aio_context(bs);
> > aio_context_acquire(aio_context);
> > blk_unref(blk);
> > aio_context_release(aio_context);
>
> May we actually acquire aio_context if blk is in the main thread? I
> think we must only do this if it's in a different iothread because we'd
> end up with a recursive lock and drain would hang.
Right :). Maybe an aio_context_acquire_once() API would help.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-25 17:15 ` Kevin Wolf
2020-09-28 8:46 ` Stefan Hajnoczi
@ 2020-09-28 9:30 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 9:30 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, armbru, dgilbert, qemu-devel, marcandre.lureau,
Stefan Hajnoczi
Kevin Wolf <kwolf@redhat.com> writes:
> Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:
>> On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
>> > Some QMP command handlers can block the main loop for a relatively long
>> > time, for example because they perform some I/O. This is quite nasty.
>> > Allowing such handlers to run in a coroutine where they can yield (and
>> > therefore release the BQL) while waiting for an event such as I/O
>> > completion solves the problem.
>> >
>> > This series adds the infrastructure to allow this and switches
>> > block_resize to run in a coroutine as a first example.
>> >
>> > This is an alternative solution to Marc-André's "monitor: add
>> > asynchronous command type" series.
>>
>> Please clarify the following in the QAPI documentation:
>> * Is the QMP monitor suspended while the command is pending?
>
> Suspended as in monitor_suspend()? No.
A suspended monitor doesn't read monitor input.
We suspend
* a QMP monitor while the request queue is full
* an HMP monitor while it executes a command
* a multiplexed HMP monitor while the "mux-focus" is elsewhere
* an HMP monitor when it executes command "quit", forever
* an HMP monitor while it executes command "migrate" without -d
Let me explain the first item in a bit more detail. Before OOB, a QMP
monitor was also suspended while it executed a command. To make OOB
work, we moved the QMP monitors to an I/O thread and added a request
queue, drained by the main loop. QMP monitors continue to read
commands, execute right away if OOB, else queue, suspend when queue gets
full, resume when it gets non-full.
The "run command in coroutine context" feature does not affect any of
this.
qapi-code-gen.txt does not talk about monitor suspension at all. It's
instead discussed in qmp-spec.txt section 2.3.1 Out-of-band execution.
Stefan, what would you like us to clarify, and where?
>> * Are QMP events reported while the command is pending?
>
> Hm, I don't know to be honest. But I think so.
Yes, events should be reported while a command is being executed.
Sending events takes locks. Their critical sections are all
short-lived. Another possible delay is the underlying character device
failing the send with EAGAIN. That's all.
Fine print: qapi_event_emit() takes @monitor_lock. It sends to each QMP
monitor with qmp_send_response(), which uses monitor_puts(), which takes
the monitor's @mon_lock.
The "run command in coroutine context" feature does not affect any of
this.
> Does it matter, though? I don't think events have a defined order
> compared to command results, and the client can't respond to the event
> anyway until the current command has completed.
Stefan, what would you like us to clarify, and where?
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 00/13] monitor: Optionally run handlers in coroutines
2020-09-28 8:46 ` Stefan Hajnoczi
@ 2020-09-28 9:47 ` Kevin Wolf
0 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-28 9:47 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 2495 bytes --]
Am 28.09.2020 um 10:46 hat Stefan Hajnoczi geschrieben:
> On Fri, Sep 25, 2020 at 07:15:41PM +0200, Kevin Wolf wrote:
> > Am 10.09.2020 um 15:24 hat Stefan Hajnoczi geschrieben:
> > > On Wed, Sep 09, 2020 at 05:11:36PM +0200, Kevin Wolf wrote:
> > > > Some QMP command handlers can block the main loop for a relatively long
> > > > time, for example because they perform some I/O. This is quite nasty.
> > > > Allowing such handlers to run in a coroutine where they can yield (and
> > > > therefore release the BQL) while waiting for an event such as I/O
> > > > completion solves the problem.
> > > >
> > > > This series adds the infrastructure to allow this and switches
> > > > block_resize to run in a coroutine as a first example.
> > > >
> > > > This is an alternative solution to Marc-André's "monitor: add
> > > > asynchronous command type" series.
> > >
> > > Please clarify the following in the QAPI documentation:
> > > * Is the QMP monitor suspended while the command is pending?
> >
> > Suspended as in monitor_suspend()? No.
> >
> > > * Are QMP events reported while the command is pending?
> >
> > Hm, I don't know to be honest. But I think so.
> >
> > Does it matter, though? I don't think events have a defined order
> > compared to command results, and the client can't respond to the event
> > anyway until the current command has completed.
>
> You're right, I don't think it matters because the client must expect
> QMP events at any time.
>
> I was trying to understand the semantics of coroutine monitor commands
> from two perspectives:
>
> 1. The QMP client - do coroutine commands behave differently from
> non-coroutine commands? I think the answer is no. The monitor will
> not process more commands until the coroutine finishes?
No, on the wire, things should look exactly the same. If you consider
more than the QMP traffic and the client communicates with the guest, it
might see that the guest isn't blocked any more, of course.
> 2. The command implementation - which thread does the coroutine run in?
> I guess it runs in the main loop thread with the BQL and the
> AioContext acquired?
By default, yes. But the coroutine can reschedule itself to another
thread. Block-related handlers will want to reschedule themselves to
bs->aio_context because you can't just hold the AioContext lock from
another thread across yields. This is what block_resize does after this
series.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context()
2020-09-28 8:59 ` Stefan Hajnoczi
@ 2020-09-28 10:21 ` Kevin Wolf
0 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-28 10:21 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 3406 bytes --]
Am 28.09.2020 um 10:59 hat Stefan Hajnoczi geschrieben:
> On Fri, Sep 25, 2020 at 06:00:51PM +0200, Kevin Wolf wrote:
> > Am 15.09.2020 um 16:31 hat Stefan Hajnoczi geschrieben:
> > > On Wed, Sep 09, 2020 at 05:11:48PM +0200, Kevin Wolf wrote:
> > > > Add a function to move the current coroutine to the AioContext of a
> > > > given BlockDriverState.
> > > >
> > > > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> > > > ---
> > > > include/block/block.h | 6 ++++++
> > > > block.c | 10 ++++++++++
> > > > 2 files changed, 16 insertions(+)
> > > >
> > > > diff --git a/include/block/block.h b/include/block/block.h
> > > > index 981ab5b314..80ab322f11 100644
> > > > --- a/include/block/block.h
> > > > +++ b/include/block/block.h
> > > > @@ -626,6 +626,12 @@ bool bdrv_debug_is_suspended(BlockDriverState *bs, const char *tag);
> > > > */
> > > > AioContext *bdrv_get_aio_context(BlockDriverState *bs);
> > > >
> > > > +/**
> > > > + * Move the current coroutine to the AioContext of @bs and return the old
> > > > + * AioContext of the coroutine.
> > > > + */
> > > > +AioContext *coroutine_fn bdrv_co_move_to_aio_context(BlockDriverState *bs);
> > >
> > > I'm not sure this function handles all cases:
> > > 1. Being called without the BQL (i.e. from an IOThread).
> > > 2. Being called while a device stops using its IOThread.
> > >
> > > The races that come to mind are fetching the AioContext for bs and then
> > > scheduling a BH. The BH is executed later on by the event loop. There
> > > might be cases where the AioContext for bs is updated before the BH
> > > runs.
>
> The scenario I'm thinking about is where bs' AioContext changes while we
> are trying to move there.
>
> There is a window of time between fetching bs' AioContext, scheduling a
> BH in our old AioContext (not in bs' AioContext), and then scheduling
> the coroutine into the AioContext we previously fetched for bs.
>
> Is it possible for the AioContext value we stashed to be outdated by the
> time we use it?
>
> I think the answer is that it's safe to use this function from the main
> loop thread under the BQL. That way nothing else will change bs'
> AioContext while we're running. But it's probably not safe to use this
> function from an arbitrary IOThread (without the BQL).
It's probably the safest to treat it as such. The first part of it (the
window between fetching bs->aio_context and using it) is actually also
true for this ubiquitous sequence:
AioContext *ctx = bdrv_get_aio_context(bs);
aio_context_acquire(ctx);
I never really thought about this, but this is only safe in the main
thread. Most of its users are of course monitor command handlers, which
always run in the main thread.
> I think this limitation is okay but it needs to be documented. Maybe an
> assertion can verify that it holds.
Yes, why not.
Maybe we should actually change the interface into a pair of
bdrv_co_enter/leave() functions that also increase bs->in_flight so that
the whole section will complete before the AioContext of bs changes
(changing the AioContext while the handle coroutine has yielded and will
continue to run in the old context would be bad).
block_resize is safe without it, but it might be better to introduce
patterns that will be safe without being extra careful in each command.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 13/13] block: Convert 'block_resize' to coroutine
2020-09-28 9:05 ` Stefan Hajnoczi
@ 2020-09-28 10:33 ` Kevin Wolf
0 siblings, 0 replies; 62+ messages in thread
From: Kevin Wolf @ 2020-09-28 10:33 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: qemu-devel, marcandre.lureau, armbru, qemu-block, dgilbert
[-- Attachment #1: Type: text/plain, Size: 2580 bytes --]
Am 28.09.2020 um 11:05 hat Stefan Hajnoczi geschrieben:
> On Fri, Sep 25, 2020 at 06:07:50PM +0200, Kevin Wolf wrote:
> > Am 15.09.2020 um 16:57 hat Stefan Hajnoczi geschrieben:
> > > On Wed, Sep 09, 2020 at 05:11:49PM +0200, Kevin Wolf wrote:
> > > > @@ -2456,8 +2456,7 @@ void qmp_block_resize(bool has_device, const char *device,
> > > > return;
> > > > }
> > > >
> > > > - aio_context = bdrv_get_aio_context(bs);
> > > > - aio_context_acquire(aio_context);
> > > > + old_ctx = bdrv_co_move_to_aio_context(bs);
> > > >
> > > > if (size < 0) {
> > > > error_setg(errp, QERR_INVALID_PARAMETER_VALUE, "size", "a >0 size");
> > >
> > > Is it safe to call blk_new() outside the BQL since it mutates global state?
> > >
> > > In other words, could another thread race with us?
> >
> > Hm, probably not.
> >
> > Would it be safer to have the bdrv_co_move_to_aio_context() call only
> > immediately before the drain?
>
> Yes, sounds good.
>
> > > > @@ -2479,8 +2478,8 @@ void qmp_block_resize(bool has_device, const char *device,
> > > > bdrv_drained_end(bs);
> > > >
> > > > out:
> > > > + aio_co_reschedule_self(old_ctx);
> > > > blk_unref(blk);
> > > > - aio_context_release(aio_context);
> > >
> > > The following precondition is violated by the blk_unref -> bdrv_drain ->
> > > AIO_WAIT_WHILE() call if blk->refcnt is 1 here:
> > >
> > > * The caller's thread must be the IOThread that owns @ctx or the main loop
> > > * thread (with @ctx acquired exactly once).
> > >
> > > blk_unref() is called from the main loop thread without having acquired
> > > blk's AioContext.
> > >
> > > Normally blk->refcnt will be > 1 so bdrv_drain() won't be called, but
> > > I'm not sure if that can be guaranteed.
> > >
> > > The following seems safer although it's uglier:
> > >
> > > aio_context = bdrv_get_aio_context(bs);
> > > aio_context_acquire(aio_context);
> > > blk_unref(blk);
> > > aio_context_release(aio_context);
> >
> > May we actually acquire aio_context if blk is in the main thread? I
> > think we must only do this if it's in a different iothread because we'd
> > end up with a recursive lock and drain would hang.
>
> Right :). Maybe an aio_context_acquire_once() API would help.
If you want it to work in the general case, how would you implement
this? As far as I know there is no way to tell whether we already own
the lock or not.
Something like aio_context_acquire_unless_self() might be easier to
implement.
Kevin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-28 7:47 ` Markus Armbruster
@ 2020-09-28 10:42 ` Kevin Wolf
2020-09-28 12:21 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-28 10:42 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Am 28.09.2020 um 09:47 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > This way, a monitor command handler will still be able to access the
> >> > current monitor, but when it yields, all other code code will correctly
> >> > get NULL from monitor_cur().
> >> >
> >> > This uses a hash table to map the coroutine pointer to the current
> >> > monitor of that coroutine. Outside of coroutine context, we associate
> >> > the current monitor with the leader coroutine of the current thread.
> >>
> >> In qemu-system-FOO, the hash table can have only these entries:
> >>
> >> * (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while
> >> executing a QMP command out-of-band.
> >>
> >> * (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main
> >> thread) to a QMP monitor, while executing a QMP command in-band and in
> >> coroutine context.
> >>
> >> * (QMP) One mapping the main thread's leader to a QMP monitor, while
> >> executing a QMP command in-band and out of coroutine context, in a
> >> bottom half.
> >>
> >> * (HMP) One mapping the main thread's leader to an HMP monitor, while
> >> executing an HMP command out of coroutine context.
> >>
> >> * (HMP-CO) One mapping a transient coroutine in the main thread to an
> >> HMP monitor, while executing an HMP command in coroutine context.
> >>
> >> In-band execution is one command after the other.
> >>
> >> Therefore, at most one monitor command can be executing in-band at any
> >> time.
> >>
> >> Therefore, the hash table has at most *two* entries: one (OOB), and one
> >> of the other four.
> >>
> >> Can you shoot any holes into my argument?
> >
> > I think with human-monitor-command, you can have three mappings:
> >
> > 1. The main thread's leader (it is a non-coroutine QMP command) to the
> > QMP monitor
>
> This is (QMP).
>
> > 2. With a coroutine HMP command, one mapping from the transient HMP
> > coroutine to the transient HMP monitor (with a non-coroutine HMP
> > command, we'd instead temporarily change the mapping from 1.)
>
> This is (HMP-CO).
>
> > 3. The OOB entry
>
> This is (OOB).
>
> To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band,
> non-coroutine QMP command needs to execute interleaved with the in-band,
> coroutine HMP command.
>
> Such an interleaving contradicts "In-band execution is one command after
> the other", which is a fundamental assumption in-band commands may make.
> If the assumption is invalid, we got a problem. Is it?
Interleaving, or rather executing another command in the middle of its
implementation is the very purpose of human-monitor-command (which is
what I was talking about, so "the in-band non-coroutine QMP command" is
a very specific one).
It's the only command I can think of that is exceptional in this way
and would lead to three mappings.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-25 15:13 ` Kevin Wolf
@ 2020-09-28 11:42 ` Markus Armbruster
2020-09-28 14:30 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 11:42 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > The correct way to set the current monitor for a coroutine handler will
>> > be different than for a blocking handler, so monitor_set_cur() needs to
>> > be called in qmp_dispatch().
>> >
>> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> > ---
>> > include/qapi/qmp/dispatch.h | 3 ++-
>> > monitor/qmp.c | 8 +-------
>> > qapi/qmp-dispatch.c | 8 +++++++-
>> > qga/main.c | 2 +-
>> > stubs/monitor-core.c | 5 +++++
>> > tests/test-qmp-cmds.c | 6 +++---
>> > 6 files changed, 19 insertions(+), 13 deletions(-)
>> >
>> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
>> > index 5a9cf82472..0c2f467028 100644
>> > --- a/include/qapi/qmp/dispatch.h
>> > +++ b/include/qapi/qmp/dispatch.h
>> > @@ -14,6 +14,7 @@
>> > #ifndef QAPI_QMP_DISPATCH_H
>> > #define QAPI_QMP_DISPATCH_H
>> >
>> > +#include "monitor/monitor.h"
>> > #include "qemu/queue.h"
>> >
>> > typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
>> > @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
>> > bool qmp_has_success_response(const QmpCommand *cmd);
>> > QDict *qmp_error_response(Error *err);
>> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
>> > - bool allow_oob);
>> > + bool allow_oob, Monitor *cur_mon);
>> > bool qmp_is_oob(const QDict *dict);
>> >
>> > typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
>> > diff --git a/monitor/qmp.c b/monitor/qmp.c
>> > index 8469970c69..922fdb5541 100644
>> > --- a/monitor/qmp.c
>> > +++ b/monitor/qmp.c
>> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>> >
>> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>> > {
>> > - Monitor *old_mon;
>> > QDict *rsp;
>> > QDict *error;
>> >
>> > - old_mon = monitor_set_cur(&mon->common);
>> > - assert(old_mon == NULL);
>> > -
>> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
>> > -
>> > - monitor_set_cur(NULL);
>> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>>
>> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
>
> It's 79 characters. Should be fine even with your local deviation from
> the coding style to require less than that for comments?
Let me rephrase my remark.
For me,
rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
&mon->common);
is significantly easier to read than
rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
Would you mind me wrapping this line in my tree?
A few more in PATCH 08-11.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-28 10:42 ` Kevin Wolf
@ 2020-09-28 12:21 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-09-28 12:21 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 28.09.2020 um 09:47 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 14.09.2020 um 17:11 hat Markus Armbruster geschrieben:
>> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >>
>> >> > This way, a monitor command handler will still be able to access the
>> >> > current monitor, but when it yields, all other code code will correctly
>> >> > get NULL from monitor_cur().
>> >> >
>> >> > This uses a hash table to map the coroutine pointer to the current
>> >> > monitor of that coroutine. Outside of coroutine context, we associate
>> >> > the current monitor with the leader coroutine of the current thread.
>> >>
>> >> In qemu-system-FOO, the hash table can have only these entries:
>> >>
>> >> * (OOB) One mapping @mon_iothread's thread leader to a QMP monitor, while
>> >> executing a QMP command out-of-band.
>> >>
>> >> * (QMP-CO) One mapping @qmp_dispatcher_co (a coroutine in the main
>> >> thread) to a QMP monitor, while executing a QMP command in-band and in
>> >> coroutine context.
>> >>
>> >> * (QMP) One mapping the main thread's leader to a QMP monitor, while
>> >> executing a QMP command in-band and out of coroutine context, in a
>> >> bottom half.
>> >>
>> >> * (HMP) One mapping the main thread's leader to an HMP monitor, while
>> >> executing an HMP command out of coroutine context.
>> >>
>> >> * (HMP-CO) One mapping a transient coroutine in the main thread to an
>> >> HMP monitor, while executing an HMP command in coroutine context.
>> >>
>> >> In-band execution is one command after the other.
>> >>
>> >> Therefore, at most one monitor command can be executing in-band at any
>> >> time.
>> >>
>> >> Therefore, the hash table has at most *two* entries: one (OOB), and one
>> >> of the other four.
>> >>
>> >> Can you shoot any holes into my argument?
>> >
>> > I think with human-monitor-command, you can have three mappings:
>> >
>> > 1. The main thread's leader (it is a non-coroutine QMP command) to the
>> > QMP monitor
>>
>> This is (QMP).
>>
>> > 2. With a coroutine HMP command, one mapping from the transient HMP
>> > coroutine to the transient HMP monitor (with a non-coroutine HMP
>> > command, we'd instead temporarily change the mapping from 1.)
>>
>> This is (HMP-CO).
>>
>> > 3. The OOB entry
>>
>> This is (OOB).
>>
>> To get 1. (QMP) and 2, (HMP-CO) at the same time, the in-band,
>> non-coroutine QMP command needs to execute interleaved with the in-band,
>> coroutine HMP command.
>>
>> Such an interleaving contradicts "In-band execution is one command after
>> the other", which is a fundamental assumption in-band commands may make.
>> If the assumption is invalid, we got a problem. Is it?
>
> Interleaving, or rather executing another command in the middle of its
> implementation is the very purpose of human-monitor-command (which is
> what I was talking about, so "the in-band non-coroutine QMP command" is
> a very specific one).
Got it now, thanks.
> It's the only command I can think of that is exceptional in this way
> and would lead to three mappings.
>
> Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-28 11:42 ` Markus Armbruster
@ 2020-09-28 14:30 ` Kevin Wolf
2020-09-30 9:26 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-28 14:30 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > The correct way to set the current monitor for a coroutine handler will
> >> > be different than for a blocking handler, so monitor_set_cur() needs to
> >> > be called in qmp_dispatch().
> >> >
> >> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> >> > ---
> >> > include/qapi/qmp/dispatch.h | 3 ++-
> >> > monitor/qmp.c | 8 +-------
> >> > qapi/qmp-dispatch.c | 8 +++++++-
> >> > qga/main.c | 2 +-
> >> > stubs/monitor-core.c | 5 +++++
> >> > tests/test-qmp-cmds.c | 6 +++---
> >> > 6 files changed, 19 insertions(+), 13 deletions(-)
> >> >
> >> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> >> > index 5a9cf82472..0c2f467028 100644
> >> > --- a/include/qapi/qmp/dispatch.h
> >> > +++ b/include/qapi/qmp/dispatch.h
> >> > @@ -14,6 +14,7 @@
> >> > #ifndef QAPI_QMP_DISPATCH_H
> >> > #define QAPI_QMP_DISPATCH_H
> >> >
> >> > +#include "monitor/monitor.h"
> >> > #include "qemu/queue.h"
> >> >
> >> > typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
> >> > @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
> >> > bool qmp_has_success_response(const QmpCommand *cmd);
> >> > QDict *qmp_error_response(Error *err);
> >> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> >> > - bool allow_oob);
> >> > + bool allow_oob, Monitor *cur_mon);
> >> > bool qmp_is_oob(const QDict *dict);
> >> >
> >> > typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> >> > index 8469970c69..922fdb5541 100644
> >> > --- a/monitor/qmp.c
> >> > +++ b/monitor/qmp.c
> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> >> >
> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> >> > {
> >> > - Monitor *old_mon;
> >> > QDict *rsp;
> >> > QDict *error;
> >> >
> >> > - old_mon = monitor_set_cur(&mon->common);
> >> > - assert(old_mon == NULL);
> >> > -
> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> >> > -
> >> > - monitor_set_cur(NULL);
> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> >>
> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
> >
> > It's 79 characters. Should be fine even with your local deviation from
> > the coding style to require less than that for comments?
>
> Let me rephrase my remark.
>
> For me,
>
> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
> &mon->common);
>
> is significantly easier to read than
>
> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
I guess this is highly subjective. I find wrapped lines harder to read.
For answering subjective questions like this, we generally use the
coding style document.
Anyway, I guess following an idiosyncratic coding style that is
different from every other subsystem in QEMU is possible (if
inconvenient) if I know what it is.
My problem is more that I don't know what the exact rules are. Can they
only be figured out experimentally by submitting patches and seeing
whether you like them or not?
> Would you mind me wrapping this line in my tree?
I have no say in this subsystem and I take it that you want all code to
look as if you had written it yourself, so do as you wish.
But I understand that I'll have to respin anyway, so if you could
explain what you're after, I might be able to apply the rules for the
next version of the series.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-28 14:30 ` Kevin Wolf
@ 2020-09-30 9:26 ` Markus Armbruster
2020-09-30 11:29 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-30 9:26 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
>> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >>
>> >> > The correct way to set the current monitor for a coroutine handler will
>> >> > be different than for a blocking handler, so monitor_set_cur() needs to
>> >> > be called in qmp_dispatch().
>> >> >
>> >> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
>> >> > ---
>> >> > include/qapi/qmp/dispatch.h | 3 ++-
>> >> > monitor/qmp.c | 8 +-------
>> >> > qapi/qmp-dispatch.c | 8 +++++++-
>> >> > qga/main.c | 2 +-
>> >> > stubs/monitor-core.c | 5 +++++
>> >> > tests/test-qmp-cmds.c | 6 +++---
>> >> > 6 files changed, 19 insertions(+), 13 deletions(-)
>> >> >
>> >> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
>> >> > index 5a9cf82472..0c2f467028 100644
>> >> > --- a/include/qapi/qmp/dispatch.h
>> >> > +++ b/include/qapi/qmp/dispatch.h
>> >> > @@ -14,6 +14,7 @@
>> >> > #ifndef QAPI_QMP_DISPATCH_H
>> >> > #define QAPI_QMP_DISPATCH_H
>> >> >
>> >> > +#include "monitor/monitor.h"
>> >> > #include "qemu/queue.h"
>> >> >
>> >> > typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
>> >> > @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
>> >> > bool qmp_has_success_response(const QmpCommand *cmd);
>> >> > QDict *qmp_error_response(Error *err);
>> >> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
>> >> > - bool allow_oob);
>> >> > + bool allow_oob, Monitor *cur_mon);
>> >> > bool qmp_is_oob(const QDict *dict);
>> >> >
>> >> > typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
>> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
>> >> > index 8469970c69..922fdb5541 100644
>> >> > --- a/monitor/qmp.c
>> >> > +++ b/monitor/qmp.c
>> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>> >> >
>> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>> >> > {
>> >> > - Monitor *old_mon;
>> >> > QDict *rsp;
>> >> > QDict *error;
>> >> >
>> >> > - old_mon = monitor_set_cur(&mon->common);
>> >> > - assert(old_mon == NULL);
>> >> > -
>> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
>> >> > -
>> >> > - monitor_set_cur(NULL);
>> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>> >>
>> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
>> >
>> > It's 79 characters. Should be fine even with your local deviation from
>> > the coding style to require less than that for comments?
>>
>> Let me rephrase my remark.
>>
>> For me,
>>
>> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
>> &mon->common);
>>
>> is significantly easier to read than
>>
>> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>
> I guess this is highly subjective. I find wrapped lines harder to read.
> For answering subjective questions like this, we generally use the
> coding style document.
>
> Anyway, I guess following an idiosyncratic coding style that is
> different from every other subsystem in QEMU is possible (if
> inconvenient) if I know what it is.
The applicable coding style document is PEP 8.
> My problem is more that I don't know what the exact rules are. Can they
> only be figured out experimentally by submitting patches and seeing
> whether you like them or not?
PEP 8:
A style guide is about consistency. Consistency with this style
guide is important. Consistency within a project is more important.
Consistency within one module or function is the most important.
In other words, you should make a reasonable effort to blend in.
>> Would you mind me wrapping this line in my tree?
>
> I have no say in this subsystem and I take it that you want all code to
> look as if you had written it yourself, so do as you wish.
I'm refusing the bait.
> But I understand that I'll have to respin anyway, so if you could
> explain what you're after, I might be able to apply the rules for the
> next version of the series.
First, PEP 8 again:
Limit all lines to a maximum of 79 characters.
For flowing long blocks of text with fewer structural restrictions
(docstrings or comments), the line length should be limited to 72
characters.
Second, an argument we two had on this list, during review of a prior
version of this patch series, talking about C:
Legibility. Humans tend to have trouble following long lines with
their eyes (I sure do). Typographic manuals suggest to limit
columns to roughly 60 characters for exactly that reason[*].
Code is special. It's typically indented, and long identifiers push
it further to the right, function arguments in particular. We
compromised at 80 columns.
[...]
[*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
The width of the line not counting indentation matters for legibility.
The line I flagged as long is 75 characters wide not counting
indentation. That's needlessly hard to read for me.
PEP 8's line length limit is a *limit*, not a sacred right to push right
to the limit.
Since I get to read this code a lot, I've taken care to avoid illegibly
wide lines, and I've guided contributors to blend in.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-30 9:26 ` Markus Armbruster
@ 2020-09-30 11:29 ` Kevin Wolf
2020-09-30 13:14 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-30 11:29 UTC (permalink / raw)
To: Markus Armbruster
Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> >> >>
> >> >> > The correct way to set the current monitor for a coroutine handler will
> >> >> > be different than for a blocking handler, so monitor_set_cur() needs to
> >> >> > be called in qmp_dispatch().
> >> >> >
> >> >> > Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> >> >> > ---
> >> >> > include/qapi/qmp/dispatch.h | 3 ++-
> >> >> > monitor/qmp.c | 8 +-------
> >> >> > qapi/qmp-dispatch.c | 8 +++++++-
> >> >> > qga/main.c | 2 +-
> >> >> > stubs/monitor-core.c | 5 +++++
> >> >> > tests/test-qmp-cmds.c | 6 +++---
> >> >> > 6 files changed, 19 insertions(+), 13 deletions(-)
> >> >> >
> >> >> > diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> >> >> > index 5a9cf82472..0c2f467028 100644
> >> >> > --- a/include/qapi/qmp/dispatch.h
> >> >> > +++ b/include/qapi/qmp/dispatch.h
> >> >> > @@ -14,6 +14,7 @@
> >> >> > #ifndef QAPI_QMP_DISPATCH_H
> >> >> > #define QAPI_QMP_DISPATCH_H
> >> >> >
> >> >> > +#include "monitor/monitor.h"
> >> >> > #include "qemu/queue.h"
> >> >> >
> >> >> > typedef void (QmpCommandFunc)(QDict *, QObject **, Error **);
> >> >> > @@ -49,7 +50,7 @@ const char *qmp_command_name(const QmpCommand *cmd);
> >> >> > bool qmp_has_success_response(const QmpCommand *cmd);
> >> >> > QDict *qmp_error_response(Error *err);
> >> >> > QDict *qmp_dispatch(const QmpCommandList *cmds, QObject *request,
> >> >> > - bool allow_oob);
> >> >> > + bool allow_oob, Monitor *cur_mon);
> >> >> > bool qmp_is_oob(const QDict *dict);
> >> >> >
> >> >> > typedef void (*qmp_cmd_callback_fn)(const QmpCommand *cmd, void *opaque);
> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> >> >> > index 8469970c69..922fdb5541 100644
> >> >> > --- a/monitor/qmp.c
> >> >> > +++ b/monitor/qmp.c
> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> >> >> >
> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> >> >> > {
> >> >> > - Monitor *old_mon;
> >> >> > QDict *rsp;
> >> >> > QDict *error;
> >> >> >
> >> >> > - old_mon = monitor_set_cur(&mon->common);
> >> >> > - assert(old_mon == NULL);
> >> >> > -
> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> >> >> > -
> >> >> > - monitor_set_cur(NULL);
> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> >> >>
> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
> >> >
> >> > It's 79 characters. Should be fine even with your local deviation from
> >> > the coding style to require less than that for comments?
> >>
> >> Let me rephrase my remark.
> >>
> >> For me,
> >>
> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
> >> &mon->common);
> >>
> >> is significantly easier to read than
> >>
> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> >
> > I guess this is highly subjective. I find wrapped lines harder to read.
> > For answering subjective questions like this, we generally use the
> > coding style document.
> >
> > Anyway, I guess following an idiosyncratic coding style that is
> > different from every other subsystem in QEMU is possible (if
> > inconvenient) if I know what it is.
>
> The applicable coding style document is PEP 8.
I'll happily apply PEP 8 to Python code, but this is C. I don't think
PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
guide, but we're not writing C code for the Python project here...)
> > My problem is more that I don't know what the exact rules are. Can they
> > only be figured out experimentally by submitting patches and seeing
> > whether you like them or not?
>
> PEP 8:
>
> A style guide is about consistency. Consistency with this style
> guide is important. Consistency within a project is more important.
> Consistency within one module or function is the most important.
>
> In other words, you should make a reasonable effort to blend in.
The project style guide for C is defined in CODING_STYLE.rst. Missing
consistency with it is what I'm complaining about.
I also agree that consistency within one module or function is most
important, which is why I allow you to reformat my code. But I don't
think it means that local coding style rules shouldn't be documented,
especially if you can't just look at the code and see immediately how
it's supposed to be.
> >> Would you mind me wrapping this line in my tree?
> >
> > I have no say in this subsystem and I take it that you want all code to
> > look as if you had written it yourself, so do as you wish.
>
> I'm refusing the bait.
>
> > But I understand that I'll have to respin anyway, so if you could
> > explain what you're after, I might be able to apply the rules for the
> > next version of the series.
>
> First, PEP 8 again:
>
> Limit all lines to a maximum of 79 characters.
>
> For flowing long blocks of text with fewer structural restrictions
> (docstrings or comments), the line length should be limited to 72
> characters.
Ok, that's finally clear limits at least.
Any other rules from PEP 8 that you want to see applied to C code?
Would you mind documenting this somewhere?
> Second, an argument we two had on this list, during review of a prior
> version of this patch series, talking about C:
>
> Legibility. Humans tend to have trouble following long lines with
> their eyes (I sure do). Typographic manuals suggest to limit
> columns to roughly 60 characters for exactly that reason[*].
>
> Code is special. It's typically indented, and long identifiers push
> it further to the right, function arguments in particular. We
> compromised at 80 columns.
>
> [...]
>
> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
>
> The width of the line not counting indentation matters for legibility.
>
> The line I flagged as long is 75 characters wide not counting
> indentation. That's needlessly hard to read for me.
>
> PEP 8's line length limit is a *limit*, not a sacred right to push right
> to the limit.
>
> Since I get to read this code a lot, I've taken care to avoid illegibly
> wide lines, and I've guided contributors to blend in.
As I said, I don't mind the exact number much. I do mind predictability,
though. (And ideally also consistency across the project because
otherwise I need to change my editor settings for individual files.)
So if you don't like 79 columns, give me any other number. But
please, do give me something specific I can work with. "illegibly wide"
is not something I can work with because it's highly subjective.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-30 11:29 ` Kevin Wolf
@ 2020-09-30 13:14 ` Markus Armbruster
2020-09-30 14:00 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-09-30 13:14 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Kevin Wolf <kwolf@redhat.com> writes:
> Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
>> Kevin Wolf <kwolf@redhat.com> writes:
>>
>> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
>> >> Kevin Wolf <kwolf@redhat.com> writes:
>> >>
>> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
>> >> >> Kevin Wolf <kwolf@redhat.com> writes:
[...]
>> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
>> >> >> > index 8469970c69..922fdb5541 100644
>> >> >> > --- a/monitor/qmp.c
>> >> >> > +++ b/monitor/qmp.c
>> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>> >> >> >
>> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>> >> >> > {
>> >> >> > - Monitor *old_mon;
>> >> >> > QDict *rsp;
>> >> >> > QDict *error;
>> >> >> >
>> >> >> > - old_mon = monitor_set_cur(&mon->common);
>> >> >> > - assert(old_mon == NULL);
>> >> >> > -
>> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
>> >> >> > -
>> >> >> > - monitor_set_cur(NULL);
>> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>> >> >>
>> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
>> >> >
>> >> > It's 79 characters. Should be fine even with your local deviation from
>> >> > the coding style to require less than that for comments?
>> >>
>> >> Let me rephrase my remark.
>> >>
>> >> For me,
>> >>
>> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
>> >> &mon->common);
>> >>
>> >> is significantly easier to read than
>> >>
>> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>> >
>> > I guess this is highly subjective. I find wrapped lines harder to read.
>> > For answering subjective questions like this, we generally use the
>> > coding style document.
>> >
>> > Anyway, I guess following an idiosyncratic coding style that is
>> > different from every other subsystem in QEMU is possible (if
>> > inconvenient) if I know what it is.
>>
>> The applicable coding style document is PEP 8.
>
> I'll happily apply PEP 8 to Python code, but this is C. I don't think
> PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
> guide, but we're not writing C code for the Python project here...)
I got confused (too much Python code review), my apologies.
>> > My problem is more that I don't know what the exact rules are. Can they
>> > only be figured out experimentally by submitting patches and seeing
>> > whether you like them or not?
>>
>> PEP 8:
>>
>> A style guide is about consistency. Consistency with this style
>> guide is important. Consistency within a project is more important.
>> Consistency within one module or function is the most important.
>>
>> In other words, you should make a reasonable effort to blend in.
>
> The project style guide for C is defined in CODING_STYLE.rst. Missing
> consistency with it is what I'm complaining about.
>
> I also agree that consistency within one module or function is most
> important, which is why I allow you to reformat my code. But I don't
> think it means that local coding style rules shouldn't be documented,
> especially if you can't just look at the code and see immediately how
> it's supposed to be.
>
>> >> Would you mind me wrapping this line in my tree?
>> >
>> > I have no say in this subsystem and I take it that you want all code to
>> > look as if you had written it yourself, so do as you wish.
>>
>> I'm refusing the bait.
>>
>> > But I understand that I'll have to respin anyway, so if you could
>> > explain what you're after, I might be able to apply the rules for the
>> > next version of the series.
>>
>> First, PEP 8 again:
>>
>> Limit all lines to a maximum of 79 characters.
>>
>> For flowing long blocks of text with fewer structural restrictions
>> (docstrings or comments), the line length should be limited to 72
>> characters.
>
> Ok, that's finally clear limits at least.
>
> Any other rules from PEP 8 that you want to see applied to C code?
PEP 8 does not apply to C.
> Would you mind documenting this somewhere?
>
>> Second, an argument we two had on this list, during review of a prior
>> version of this patch series, talking about C:
>>
>> Legibility. Humans tend to have trouble following long lines with
>> their eyes (I sure do). Typographic manuals suggest to limit
>> columns to roughly 60 characters for exactly that reason[*].
>>
>> Code is special. It's typically indented, and long identifiers push
>> it further to the right, function arguments in particular. We
>> compromised at 80 columns.
>>
>> [...]
>>
>> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
>>
>> The width of the line not counting indentation matters for legibility.
>>
>> The line I flagged as long is 75 characters wide not counting
>> indentation. That's needlessly hard to read for me.
>>
>> PEP 8's line length limit is a *limit*, not a sacred right to push right
>> to the limit.
>>
>> Since I get to read this code a lot, I've taken care to avoid illegibly
>> wide lines, and I've guided contributors to blend in.
>
> As I said, I don't mind the exact number much. I do mind predictability,
> though. (And ideally also consistency across the project because
> otherwise I need to change my editor settings for individual files.)
>
> So if you don't like 79 columns, give me any other number. But
> please, do give me something specific I can work with. "illegibly wide"
> is not something I can work with because it's highly subjective.
Taste is subjective.
We can always make CODING_STYLE.rst more detailed. I view that as a
last resort when we waste too much time arguing.
Back to line length.
CODING_STYLE.rst sets a *limit*.
Going over the limit violates CODING_STYLE.rst. There are (rare) cases
where that is justified.
CODING_STYLE.rst neither demands nor prohibits breaking lines before the
limit is reached.
Until CODING_STYLE.rst prohibits breaking lines unless they exceed the
limit, I will continue to ask for breaking lines when that makes the
code easier to read and more consistent with the code around it, for
code I maintain, and admittedly in my opinion.
These requests appear to irk you a great deal. I don't understand, but
I'm sorry about it all the same. By arguing about it repeatedly, you've
irked some back. Brought it on myself, I guess. However, if that's
what it takes to keep the code I maintain legible and consistent, I'll
pay the price.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-30 13:14 ` Markus Armbruster
@ 2020-09-30 14:00 ` Kevin Wolf
2020-09-30 17:20 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-09-30 14:00 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-block, marcandre.lureau, qemu-devel, stefanha, dgilbert
Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:
> Kevin Wolf <kwolf@redhat.com> writes:
>
> > Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
> >> Kevin Wolf <kwolf@redhat.com> writes:
> >>
> >> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> >> >>
> >> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> >> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> [...]
> >> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> >> >> >> > index 8469970c69..922fdb5541 100644
> >> >> >> > --- a/monitor/qmp.c
> >> >> >> > +++ b/monitor/qmp.c
> >> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> >> >> >> >
> >> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> >> >> >> > {
> >> >> >> > - Monitor *old_mon;
> >> >> >> > QDict *rsp;
> >> >> >> > QDict *error;
> >> >> >> >
> >> >> >> > - old_mon = monitor_set_cur(&mon->common);
> >> >> >> > - assert(old_mon == NULL);
> >> >> >> > -
> >> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> >> >> >> > -
> >> >> >> > - monitor_set_cur(NULL);
> >> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> >> >> >>
> >> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
> >> >> >
> >> >> > It's 79 characters. Should be fine even with your local deviation from
> >> >> > the coding style to require less than that for comments?
> >> >>
> >> >> Let me rephrase my remark.
> >> >>
> >> >> For me,
> >> >>
> >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
> >> >> &mon->common);
> >> >>
> >> >> is significantly easier to read than
> >> >>
> >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> >> >
> >> > I guess this is highly subjective. I find wrapped lines harder to read.
> >> > For answering subjective questions like this, we generally use the
> >> > coding style document.
> >> >
> >> > Anyway, I guess following an idiosyncratic coding style that is
> >> > different from every other subsystem in QEMU is possible (if
> >> > inconvenient) if I know what it is.
> >>
> >> The applicable coding style document is PEP 8.
> >
> > I'll happily apply PEP 8 to Python code, but this is C. I don't think
> > PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
> > guide, but we're not writing C code for the Python project here...)
>
> I got confused (too much Python code review), my apologies.
>
> >> > My problem is more that I don't know what the exact rules are. Can they
> >> > only be figured out experimentally by submitting patches and seeing
> >> > whether you like them or not?
> >>
> >> PEP 8:
> >>
> >> A style guide is about consistency. Consistency with this style
> >> guide is important. Consistency within a project is more important.
> >> Consistency within one module or function is the most important.
> >>
> >> In other words, you should make a reasonable effort to blend in.
> >
> > The project style guide for C is defined in CODING_STYLE.rst. Missing
> > consistency with it is what I'm complaining about.
> >
> > I also agree that consistency within one module or function is most
> > important, which is why I allow you to reformat my code. But I don't
> > think it means that local coding style rules shouldn't be documented,
> > especially if you can't just look at the code and see immediately how
> > it's supposed to be.
> >
> >> >> Would you mind me wrapping this line in my tree?
> >> >
> >> > I have no say in this subsystem and I take it that you want all code to
> >> > look as if you had written it yourself, so do as you wish.
> >>
> >> I'm refusing the bait.
> >>
> >> > But I understand that I'll have to respin anyway, so if you could
> >> > explain what you're after, I might be able to apply the rules for the
> >> > next version of the series.
> >>
> >> First, PEP 8 again:
> >>
> >> Limit all lines to a maximum of 79 characters.
> >>
> >> For flowing long blocks of text with fewer structural restrictions
> >> (docstrings or comments), the line length should be limited to 72
> >> characters.
> >
> > Ok, that's finally clear limits at least.
> >
> > Any other rules from PEP 8 that you want to see applied to C code?
>
> PEP 8 does not apply to C.
>
> > Would you mind documenting this somewhere?
> >
> >> Second, an argument we two had on this list, during review of a prior
> >> version of this patch series, talking about C:
> >>
> >> Legibility. Humans tend to have trouble following long lines with
> >> their eyes (I sure do). Typographic manuals suggest to limit
> >> columns to roughly 60 characters for exactly that reason[*].
> >>
> >> Code is special. It's typically indented, and long identifiers push
> >> it further to the right, function arguments in particular. We
> >> compromised at 80 columns.
> >>
> >> [...]
> >>
> >> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
> >>
> >> The width of the line not counting indentation matters for legibility.
> >>
> >> The line I flagged as long is 75 characters wide not counting
> >> indentation. That's needlessly hard to read for me.
> >>
> >> PEP 8's line length limit is a *limit*, not a sacred right to push right
> >> to the limit.
> >>
> >> Since I get to read this code a lot, I've taken care to avoid illegibly
> >> wide lines, and I've guided contributors to blend in.
> >
> > As I said, I don't mind the exact number much. I do mind predictability,
> > though. (And ideally also consistency across the project because
> > otherwise I need to change my editor settings for individual files.)
> >
> > So if you don't like 79 columns, give me any other number. But
> > please, do give me something specific I can work with. "illegibly wide"
> > is not something I can work with because it's highly subjective.
>
> Taste is subjective.
>
> We can always make CODING_STYLE.rst more detailed. I view that as a
> last resort when we waste too much time arguing.
>
> Back to line length.
>
> CODING_STYLE.rst sets a *limit*.
>
> Going over the limit violates CODING_STYLE.rst. There are (rare) cases
> where that is justified.
>
> CODING_STYLE.rst neither demands nor prohibits breaking lines before the
> limit is reached.
>
> Until CODING_STYLE.rst prohibits breaking lines unless they exceed the
> limit, I will continue to ask for breaking lines when that makes the
> code easier to read and more consistent with the code around it, for
> code I maintain, and admittedly in my opinion.
>
> These requests appear to irk you a great deal. I don't understand, but
> I'm sorry about it all the same. By arguing about it repeatedly, you've
> irked some back. Brought it on myself, I guess. However, if that's
> what it takes to keep the code I maintain legible and consistent, I'll
> pay the price.
I conclude that I'll never be able to submit code that passes your
review in the first attempt because I don't know the specific criteria
(and you don't seem to know them either before you see the patch).
Fine, I'll live with it. It's just one of the things that makes working
in your subsystems more frustrating than in others.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-30 14:00 ` Kevin Wolf
@ 2020-09-30 17:20 ` Dr. David Alan Gilbert
2020-10-01 10:14 ` Kevin Wolf
0 siblings, 1 reply; 62+ messages in thread
From: Dr. David Alan Gilbert @ 2020-09-30 17:20 UTC (permalink / raw)
To: Kevin Wolf
Cc: qemu-block, marcandre.lureau, Markus Armbruster, stefanha, qemu-devel
* Kevin Wolf (kwolf@redhat.com) wrote:
> Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:
> > Kevin Wolf <kwolf@redhat.com> writes:
> >
> > > Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
> > >> Kevin Wolf <kwolf@redhat.com> writes:
> > >>
> > >> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
> > >> >> Kevin Wolf <kwolf@redhat.com> writes:
> > >> >>
> > >> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> > >> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> > [...]
> > >> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> > >> >> >> > index 8469970c69..922fdb5541 100644
> > >> >> >> > --- a/monitor/qmp.c
> > >> >> >> > +++ b/monitor/qmp.c
> > >> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> > >> >> >> >
> > >> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> > >> >> >> > {
> > >> >> >> > - Monitor *old_mon;
> > >> >> >> > QDict *rsp;
> > >> >> >> > QDict *error;
> > >> >> >> >
> > >> >> >> > - old_mon = monitor_set_cur(&mon->common);
> > >> >> >> > - assert(old_mon == NULL);
> > >> >> >> > -
> > >> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> > >> >> >> > -
> > >> >> >> > - monitor_set_cur(NULL);
> > >> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> > >> >> >>
> > >> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
> > >> >> >
> > >> >> > It's 79 characters. Should be fine even with your local deviation from
> > >> >> > the coding style to require less than that for comments?
> > >> >>
> > >> >> Let me rephrase my remark.
> > >> >>
> > >> >> For me,
> > >> >>
> > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
> > >> >> &mon->common);
> > >> >>
> > >> >> is significantly easier to read than
> > >> >>
> > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> > >> >
> > >> > I guess this is highly subjective. I find wrapped lines harder to read.
> > >> > For answering subjective questions like this, we generally use the
> > >> > coding style document.
> > >> >
> > >> > Anyway, I guess following an idiosyncratic coding style that is
> > >> > different from every other subsystem in QEMU is possible (if
> > >> > inconvenient) if I know what it is.
> > >>
> > >> The applicable coding style document is PEP 8.
> > >
> > > I'll happily apply PEP 8 to Python code, but this is C. I don't think
> > > PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
> > > guide, but we're not writing C code for the Python project here...)
> >
> > I got confused (too much Python code review), my apologies.
> >
> > >> > My problem is more that I don't know what the exact rules are. Can they
> > >> > only be figured out experimentally by submitting patches and seeing
> > >> > whether you like them or not?
> > >>
> > >> PEP 8:
> > >>
> > >> A style guide is about consistency. Consistency with this style
> > >> guide is important. Consistency within a project is more important.
> > >> Consistency within one module or function is the most important.
> > >>
> > >> In other words, you should make a reasonable effort to blend in.
> > >
> > > The project style guide for C is defined in CODING_STYLE.rst. Missing
> > > consistency with it is what I'm complaining about.
> > >
> > > I also agree that consistency within one module or function is most
> > > important, which is why I allow you to reformat my code. But I don't
> > > think it means that local coding style rules shouldn't be documented,
> > > especially if you can't just look at the code and see immediately how
> > > it's supposed to be.
> > >
> > >> >> Would you mind me wrapping this line in my tree?
> > >> >
> > >> > I have no say in this subsystem and I take it that you want all code to
> > >> > look as if you had written it yourself, so do as you wish.
> > >>
> > >> I'm refusing the bait.
> > >>
> > >> > But I understand that I'll have to respin anyway, so if you could
> > >> > explain what you're after, I might be able to apply the rules for the
> > >> > next version of the series.
> > >>
> > >> First, PEP 8 again:
> > >>
> > >> Limit all lines to a maximum of 79 characters.
> > >>
> > >> For flowing long blocks of text with fewer structural restrictions
> > >> (docstrings or comments), the line length should be limited to 72
> > >> characters.
> > >
> > > Ok, that's finally clear limits at least.
> > >
> > > Any other rules from PEP 8 that you want to see applied to C code?
> >
> > PEP 8 does not apply to C.
> >
> > > Would you mind documenting this somewhere?
> > >
> > >> Second, an argument we two had on this list, during review of a prior
> > >> version of this patch series, talking about C:
> > >>
> > >> Legibility. Humans tend to have trouble following long lines with
> > >> their eyes (I sure do). Typographic manuals suggest to limit
> > >> columns to roughly 60 characters for exactly that reason[*].
> > >>
> > >> Code is special. It's typically indented, and long identifiers push
> > >> it further to the right, function arguments in particular. We
> > >> compromised at 80 columns.
> > >>
> > >> [...]
> > >>
> > >> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
> > >>
> > >> The width of the line not counting indentation matters for legibility.
> > >>
> > >> The line I flagged as long is 75 characters wide not counting
> > >> indentation. That's needlessly hard to read for me.
> > >>
> > >> PEP 8's line length limit is a *limit*, not a sacred right to push right
> > >> to the limit.
> > >>
> > >> Since I get to read this code a lot, I've taken care to avoid illegibly
> > >> wide lines, and I've guided contributors to blend in.
> > >
> > > As I said, I don't mind the exact number much. I do mind predictability,
> > > though. (And ideally also consistency across the project because
> > > otherwise I need to change my editor settings for individual files.)
> > >
> > > So if you don't like 79 columns, give me any other number. But
> > > please, do give me something specific I can work with. "illegibly wide"
> > > is not something I can work with because it's highly subjective.
> >
> > Taste is subjective.
> >
> > We can always make CODING_STYLE.rst more detailed. I view that as a
> > last resort when we waste too much time arguing.
> >
> > Back to line length.
> >
> > CODING_STYLE.rst sets a *limit*.
> >
> > Going over the limit violates CODING_STYLE.rst. There are (rare) cases
> > where that is justified.
> >
> > CODING_STYLE.rst neither demands nor prohibits breaking lines before the
> > limit is reached.
> >
> > Until CODING_STYLE.rst prohibits breaking lines unless they exceed the
> > limit, I will continue to ask for breaking lines when that makes the
> > code easier to read and more consistent with the code around it, for
> > code I maintain, and admittedly in my opinion.
> >
> > These requests appear to irk you a great deal. I don't understand, but
> > I'm sorry about it all the same. By arguing about it repeatedly, you've
> > irked some back. Brought it on myself, I guess. However, if that's
> > what it takes to keep the code I maintain legible and consistent, I'll
> > pay the price.
>
> I conclude that I'll never be able to submit code that passes your
> review in the first attempt because I don't know the specific criteria
> (and you don't seem to know them either before you see the patch).
>
> Fine, I'll live with it. It's just one of the things that makes working
> in your subsystems more frustrating than in others.
Hmm,
IMHO the thing here is that there's two different things here:
a) A CODING_STYLE limit - and personally I use every last character
of that when appropriate
b) For this particular case, Markus is saying he prefers the wrap
there.
I don't think I see (b) as incompatible as a preference, but lets be
sensible; if it's something you want to change in merge that seems
reasonable, if it's something that you ask to change in a respin that's
kind of reasonable, just don't hold up a big patch series for an
argument over something that's legal in the coding style and isn't
particularly offensive!
Dave
> Kevin
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-09-30 17:20 ` Dr. David Alan Gilbert
@ 2020-10-01 10:14 ` Kevin Wolf
2020-10-01 16:00 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Kevin Wolf @ 2020-10-01 10:14 UTC (permalink / raw)
To: Dr. David Alan Gilbert
Cc: qemu-block, marcandre.lureau, Markus Armbruster, stefanha, qemu-devel
Am 30.09.2020 um 19:20 hat Dr. David Alan Gilbert geschrieben:
> * Kevin Wolf (kwolf@redhat.com) wrote:
> > Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:
> > > Kevin Wolf <kwolf@redhat.com> writes:
> > >
> > > > Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
> > > >> Kevin Wolf <kwolf@redhat.com> writes:
> > > >>
> > > >> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
> > > >> >> Kevin Wolf <kwolf@redhat.com> writes:
> > > >> >>
> > > >> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
> > > >> >> >> Kevin Wolf <kwolf@redhat.com> writes:
> > > [...]
> > > >> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
> > > >> >> >> > index 8469970c69..922fdb5541 100644
> > > >> >> >> > --- a/monitor/qmp.c
> > > >> >> >> > +++ b/monitor/qmp.c
> > > >> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
> > > >> >> >> >
> > > >> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
> > > >> >> >> > {
> > > >> >> >> > - Monitor *old_mon;
> > > >> >> >> > QDict *rsp;
> > > >> >> >> > QDict *error;
> > > >> >> >> >
> > > >> >> >> > - old_mon = monitor_set_cur(&mon->common);
> > > >> >> >> > - assert(old_mon == NULL);
> > > >> >> >> > -
> > > >> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
> > > >> >> >> > -
> > > >> >> >> > - monitor_set_cur(NULL);
> > > >> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> > > >> >> >>
> > > >> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
> > > >> >> >
> > > >> >> > It's 79 characters. Should be fine even with your local deviation from
> > > >> >> > the coding style to require less than that for comments?
> > > >> >>
> > > >> >> Let me rephrase my remark.
> > > >> >>
> > > >> >> For me,
> > > >> >>
> > > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
> > > >> >> &mon->common);
> > > >> >>
> > > >> >> is significantly easier to read than
> > > >> >>
> > > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
> > > >> >
> > > >> > I guess this is highly subjective. I find wrapped lines harder to read.
> > > >> > For answering subjective questions like this, we generally use the
> > > >> > coding style document.
> > > >> >
> > > >> > Anyway, I guess following an idiosyncratic coding style that is
> > > >> > different from every other subsystem in QEMU is possible (if
> > > >> > inconvenient) if I know what it is.
> > > >>
> > > >> The applicable coding style document is PEP 8.
> > > >
> > > > I'll happily apply PEP 8 to Python code, but this is C. I don't think
> > > > PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
> > > > guide, but we're not writing C code for the Python project here...)
> > >
> > > I got confused (too much Python code review), my apologies.
> > >
> > > >> > My problem is more that I don't know what the exact rules are. Can they
> > > >> > only be figured out experimentally by submitting patches and seeing
> > > >> > whether you like them or not?
> > > >>
> > > >> PEP 8:
> > > >>
> > > >> A style guide is about consistency. Consistency with this style
> > > >> guide is important. Consistency within a project is more important.
> > > >> Consistency within one module or function is the most important.
> > > >>
> > > >> In other words, you should make a reasonable effort to blend in.
> > > >
> > > > The project style guide for C is defined in CODING_STYLE.rst. Missing
> > > > consistency with it is what I'm complaining about.
> > > >
> > > > I also agree that consistency within one module or function is most
> > > > important, which is why I allow you to reformat my code. But I don't
> > > > think it means that local coding style rules shouldn't be documented,
> > > > especially if you can't just look at the code and see immediately how
> > > > it's supposed to be.
> > > >
> > > >> >> Would you mind me wrapping this line in my tree?
> > > >> >
> > > >> > I have no say in this subsystem and I take it that you want all code to
> > > >> > look as if you had written it yourself, so do as you wish.
> > > >>
> > > >> I'm refusing the bait.
> > > >>
> > > >> > But I understand that I'll have to respin anyway, so if you could
> > > >> > explain what you're after, I might be able to apply the rules for the
> > > >> > next version of the series.
> > > >>
> > > >> First, PEP 8 again:
> > > >>
> > > >> Limit all lines to a maximum of 79 characters.
> > > >>
> > > >> For flowing long blocks of text with fewer structural restrictions
> > > >> (docstrings or comments), the line length should be limited to 72
> > > >> characters.
> > > >
> > > > Ok, that's finally clear limits at least.
> > > >
> > > > Any other rules from PEP 8 that you want to see applied to C code?
> > >
> > > PEP 8 does not apply to C.
> > >
> > > > Would you mind documenting this somewhere?
> > > >
> > > >> Second, an argument we two had on this list, during review of a prior
> > > >> version of this patch series, talking about C:
> > > >>
> > > >> Legibility. Humans tend to have trouble following long lines with
> > > >> their eyes (I sure do). Typographic manuals suggest to limit
> > > >> columns to roughly 60 characters for exactly that reason[*].
> > > >>
> > > >> Code is special. It's typically indented, and long identifiers push
> > > >> it further to the right, function arguments in particular. We
> > > >> compromised at 80 columns.
> > > >>
> > > >> [...]
> > > >>
> > > >> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
> > > >>
> > > >> The width of the line not counting indentation matters for legibility.
> > > >>
> > > >> The line I flagged as long is 75 characters wide not counting
> > > >> indentation. That's needlessly hard to read for me.
> > > >>
> > > >> PEP 8's line length limit is a *limit*, not a sacred right to push right
> > > >> to the limit.
> > > >>
> > > >> Since I get to read this code a lot, I've taken care to avoid illegibly
> > > >> wide lines, and I've guided contributors to blend in.
> > > >
> > > > As I said, I don't mind the exact number much. I do mind predictability,
> > > > though. (And ideally also consistency across the project because
> > > > otherwise I need to change my editor settings for individual files.)
> > > >
> > > > So if you don't like 79 columns, give me any other number. But
> > > > please, do give me something specific I can work with. "illegibly wide"
> > > > is not something I can work with because it's highly subjective.
> > >
> > > Taste is subjective.
> > >
> > > We can always make CODING_STYLE.rst more detailed. I view that as a
> > > last resort when we waste too much time arguing.
> > >
> > > Back to line length.
> > >
> > > CODING_STYLE.rst sets a *limit*.
> > >
> > > Going over the limit violates CODING_STYLE.rst. There are (rare) cases
> > > where that is justified.
> > >
> > > CODING_STYLE.rst neither demands nor prohibits breaking lines before the
> > > limit is reached.
> > >
> > > Until CODING_STYLE.rst prohibits breaking lines unless they exceed the
> > > limit, I will continue to ask for breaking lines when that makes the
> > > code easier to read and more consistent with the code around it, for
> > > code I maintain, and admittedly in my opinion.
> > >
> > > These requests appear to irk you a great deal. I don't understand, but
> > > I'm sorry about it all the same. By arguing about it repeatedly, you've
> > > irked some back. Brought it on myself, I guess. However, if that's
> > > what it takes to keep the code I maintain legible and consistent, I'll
> > > pay the price.
> >
> > I conclude that I'll never be able to submit code that passes your
> > review in the first attempt because I don't know the specific criteria
> > (and you don't seem to know them either before you see the patch).
> >
> > Fine, I'll live with it. It's just one of the things that makes working
> > in your subsystems more frustrating than in others.
>
> Hmm,
> IMHO the thing here is that there's two different things here:
>
> a) A CODING_STYLE limit - and personally I use every last character
> of that when appropriate
> b) For this particular case, Markus is saying he prefers the wrap
> there.
>
> I don't think I see (b) as incompatible as a preference, but lets be
> sensible; if it's something you want to change in merge that seems
> reasonable, if it's something that you ask to change in a respin that's
> kind of reasonable, just don't hold up a big patch series for an
> argument over something that's legal in the coding style and isn't
> particularly offensive!
I'll just change this one in the next version. Changing a single
well-known instance not a big problem. It's just unfortunate that there
are "A few more in PATCH 08-11" and I don't know how to identify them.
So Markus will have to comment again in the next version (as he did in
other places in previous versions of this series) and potentially modify
my patches while applying to match his taste. The latter is something I
try hard to avoid as a maintainer, but I admit this means compromising
instead of perfectionism. And the former is just a bit tiring when with
every version of a series you get additional comments about style
preferences.
Kevin
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-10-01 10:14 ` Kevin Wolf
@ 2020-10-01 16:00 ` Markus Armbruster
2020-10-02 8:04 ` Markus Armbruster
0 siblings, 1 reply; 62+ messages in thread
From: Markus Armbruster @ 2020-10-01 16:00 UTC (permalink / raw)
To: Kevin Wolf
Cc: stefanha, marcandre.lureau, Dr. David Alan Gilbert, qemu-block,
qemu-devel
Kevin Wolf <kwolf@redhat.com> writes:
> Am 30.09.2020 um 19:20 hat Dr. David Alan Gilbert geschrieben:
>> * Kevin Wolf (kwolf@redhat.com) wrote:
>> > Am 30.09.2020 um 15:14 hat Markus Armbruster geschrieben:
>> > > Kevin Wolf <kwolf@redhat.com> writes:
>> > >
>> > > > Am 30.09.2020 um 11:26 hat Markus Armbruster geschrieben:
>> > > >> Kevin Wolf <kwolf@redhat.com> writes:
>> > > >>
>> > > >> > Am 28.09.2020 um 13:42 hat Markus Armbruster geschrieben:
>> > > >> >> Kevin Wolf <kwolf@redhat.com> writes:
>> > > >> >>
>> > > >> >> > Am 14.09.2020 um 17:10 hat Markus Armbruster geschrieben:
>> > > >> >> >> Kevin Wolf <kwolf@redhat.com> writes:
>> > > [...]
>> > > >> >> >> > diff --git a/monitor/qmp.c b/monitor/qmp.c
>> > > >> >> >> > index 8469970c69..922fdb5541 100644
>> > > >> >> >> > --- a/monitor/qmp.c
>> > > >> >> >> > +++ b/monitor/qmp.c
>> > > >> >> >> > @@ -135,16 +135,10 @@ static void monitor_qmp_respond(MonitorQMP *mon, QDict *rsp)
>> > > >> >> >> >
>> > > >> >> >> > static void monitor_qmp_dispatch(MonitorQMP *mon, QObject *req)
>> > > >> >> >> > {
>> > > >> >> >> > - Monitor *old_mon;
>> > > >> >> >> > QDict *rsp;
>> > > >> >> >> > QDict *error;
>> > > >> >> >> >
>> > > >> >> >> > - old_mon = monitor_set_cur(&mon->common);
>> > > >> >> >> > - assert(old_mon == NULL);
>> > > >> >> >> > -
>> > > >> >> >> > - rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon));
>> > > >> >> >> > -
>> > > >> >> >> > - monitor_set_cur(NULL);
>> > > >> >> >> > + rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>> > > >> >> >>
>> > > >> >> >> Long line. Happy to wrap it in my tree. A few more in PATCH 08-11.
>> > > >> >> >
>> > > >> >> > It's 79 characters. Should be fine even with your local deviation from
>> > > >> >> > the coding style to require less than that for comments?
>> > > >> >>
>> > > >> >> Let me rephrase my remark.
>> > > >> >>
>> > > >> >> For me,
>> > > >> >>
>> > > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon),
>> > > >> >> &mon->common);
>> > > >> >>
>> > > >> >> is significantly easier to read than
>> > > >> >>
>> > > >> >> rsp = qmp_dispatch(mon->commands, req, qmp_oob_enabled(mon), &mon->common);
>> > > >> >
>> > > >> > I guess this is highly subjective. I find wrapped lines harder to read.
>> > > >> > For answering subjective questions like this, we generally use the
>> > > >> > coding style document.
>> > > >> >
>> > > >> > Anyway, I guess following an idiosyncratic coding style that is
>> > > >> > different from every other subsystem in QEMU is possible (if
>> > > >> > inconvenient) if I know what it is.
>> > > >>
>> > > >> The applicable coding style document is PEP 8.
>> > > >
>> > > > I'll happily apply PEP 8 to Python code, but this is C. I don't think
>> > > > PEP 8 applies very well to C code. (In fact, PEP 7 exists as a C style
>> > > > guide, but we're not writing C code for the Python project here...)
>> > >
>> > > I got confused (too much Python code review), my apologies.
>> > >
>> > > >> > My problem is more that I don't know what the exact rules are. Can they
>> > > >> > only be figured out experimentally by submitting patches and seeing
>> > > >> > whether you like them or not?
>> > > >>
>> > > >> PEP 8:
>> > > >>
>> > > >> A style guide is about consistency. Consistency with this style
>> > > >> guide is important. Consistency within a project is more important.
>> > > >> Consistency within one module or function is the most important.
>> > > >>
>> > > >> In other words, you should make a reasonable effort to blend in.
>> > > >
>> > > > The project style guide for C is defined in CODING_STYLE.rst. Missing
>> > > > consistency with it is what I'm complaining about.
>> > > >
>> > > > I also agree that consistency within one module or function is most
>> > > > important, which is why I allow you to reformat my code. But I don't
>> > > > think it means that local coding style rules shouldn't be documented,
>> > > > especially if you can't just look at the code and see immediately how
>> > > > it's supposed to be.
>> > > >
>> > > >> >> Would you mind me wrapping this line in my tree?
>> > > >> >
>> > > >> > I have no say in this subsystem and I take it that you want all code to
>> > > >> > look as if you had written it yourself, so do as you wish.
>> > > >>
>> > > >> I'm refusing the bait.
>> > > >>
>> > > >> > But I understand that I'll have to respin anyway, so if you could
>> > > >> > explain what you're after, I might be able to apply the rules for the
>> > > >> > next version of the series.
>> > > >>
>> > > >> First, PEP 8 again:
>> > > >>
>> > > >> Limit all lines to a maximum of 79 characters.
>> > > >>
>> > > >> For flowing long blocks of text with fewer structural restrictions
>> > > >> (docstrings or comments), the line length should be limited to 72
>> > > >> characters.
>> > > >
>> > > > Ok, that's finally clear limits at least.
>> > > >
>> > > > Any other rules from PEP 8 that you want to see applied to C code?
>> > >
>> > > PEP 8 does not apply to C.
>> > >
>> > > > Would you mind documenting this somewhere?
>> > > >
>> > > >> Second, an argument we two had on this list, during review of a prior
>> > > >> version of this patch series, talking about C:
>> > > >>
>> > > >> Legibility. Humans tend to have trouble following long lines with
>> > > >> their eyes (I sure do). Typographic manuals suggest to limit
>> > > >> columns to roughly 60 characters for exactly that reason[*].
>> > > >>
>> > > >> Code is special. It's typically indented, and long identifiers push
>> > > >> it further to the right, function arguments in particular. We
>> > > >> compromised at 80 columns.
>> > > >>
>> > > >> [...]
>> > > >>
>> > > >> [*] https://en.wikipedia.org/wiki/Column_(typography)#Typographic_style
>> > > >>
>> > > >> The width of the line not counting indentation matters for legibility.
>> > > >>
>> > > >> The line I flagged as long is 75 characters wide not counting
>> > > >> indentation. That's needlessly hard to read for me.
>> > > >>
>> > > >> PEP 8's line length limit is a *limit*, not a sacred right to push right
>> > > >> to the limit.
>> > > >>
>> > > >> Since I get to read this code a lot, I've taken care to avoid illegibly
>> > > >> wide lines, and I've guided contributors to blend in.
>> > > >
>> > > > As I said, I don't mind the exact number much. I do mind predictability,
>> > > > though. (And ideally also consistency across the project because
>> > > > otherwise I need to change my editor settings for individual files.)
>> > > >
>> > > > So if you don't like 79 columns, give me any other number. But
>> > > > please, do give me something specific I can work with. "illegibly wide"
>> > > > is not something I can work with because it's highly subjective.
>> > >
>> > > Taste is subjective.
>> > >
>> > > We can always make CODING_STYLE.rst more detailed. I view that as a
>> > > last resort when we waste too much time arguing.
>> > >
>> > > Back to line length.
>> > >
>> > > CODING_STYLE.rst sets a *limit*.
>> > >
>> > > Going over the limit violates CODING_STYLE.rst. There are (rare) cases
>> > > where that is justified.
>> > >
>> > > CODING_STYLE.rst neither demands nor prohibits breaking lines before the
>> > > limit is reached.
>> > >
>> > > Until CODING_STYLE.rst prohibits breaking lines unless they exceed the
>> > > limit, I will continue to ask for breaking lines when that makes the
>> > > code easier to read and more consistent with the code around it, for
>> > > code I maintain, and admittedly in my opinion.
>> > >
>> > > These requests appear to irk you a great deal. I don't understand, but
>> > > I'm sorry about it all the same. By arguing about it repeatedly, you've
>> > > irked some back. Brought it on myself, I guess. However, if that's
>> > > what it takes to keep the code I maintain legible and consistent, I'll
>> > > pay the price.
>> >
>> > I conclude that I'll never be able to submit code that passes your
>> > review in the first attempt because I don't know the specific criteria
>> > (and you don't seem to know them either before you see the patch).
>> >
>> > Fine, I'll live with it. It's just one of the things that makes working
>> > in your subsystems more frustrating than in others.
>>
>> Hmm,
>> IMHO the thing here is that there's two different things here:
>>
>> a) A CODING_STYLE limit - and personally I use every last character
>> of that when appropriate
>> b) For this particular case, Markus is saying he prefers the wrap
>> there.
>>
>> I don't think I see (b) as incompatible as a preference, but lets be
>> sensible; if it's something you want to change in merge that seems
>> reasonable, if it's something that you ask to change in a respin that's
>> kind of reasonable, just don't hold up a big patch series for an
>> argument over something that's legal in the coding style and isn't
>> particularly offensive!
I don't think I ever asked for a respin just to adjust style. I always
offer to adjust style myself in my tree. If a respin is needed for some
other reason, also making the style adjustments I requested is courteous
and appreciated.
I don't think I ever rejected a patch just due to differences over
style. If a patch submitter refused to make the style adjustments I
want, and refused to permit me to make them, I'd commit as is, then maye
adjust on top. This is hypothetical.
> I'll just change this one in the next version. Changing a single
> well-known instance not a big problem. It's just unfortunate that there
> are "A few more in PATCH 08-11" and I don't know how to identify them.
When I do that, and you'd rather have a complete list, just ask. Out of
time for today, but I can get it for you first thing tomorrow.
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon
2020-09-09 15:11 ` [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon Kevin Wolf
@ 2020-10-02 7:51 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 7:51 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> cur_mon really needs to be coroutine-local as soon as we move monitor
> command handlers to coroutines and let them yield. As a first step, just
> remove all direct accesses to cur_mon so that we can implement this in
> the getter function later.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
[...]
> diff --git a/tests/test-util-sockets.c b/tests/test-util-sockets.c
> index af9f5c0c70..6c50dbf051 100644
> --- a/tests/test-util-sockets.c
> +++ b/tests/test-util-sockets.c
> @@ -52,6 +52,7 @@ static void test_fd_is_socket_good(void)
>
> static int mon_fd = -1;
> static const char *mon_fdname;
> +__thread Monitor *cur_mon;
>
> int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
> {
> @@ -66,15 +67,12 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error **errp)
>
> /*
> * Syms of stubs in libqemuutil.a are discarded at .o file granularity.
> - * To replace monitor_get_fd() we must ensure everything in
> - * stubs/monitor.c is defined, to make sure monitor.o is discarded
> - * otherwise we get duplicate syms at link time.
> + * To replace monitor_get_fd() and monitor_cur(), we must ensure that we also
> + * replace any other symbol that is used in the binary and would be taken from
> + * the same stub object file, otherwise we get duplicate syms at link time.
Wrapping the comment around column 70 or so would make it easier to
read. File has no maintainers. Up to you.
> */
> -__thread Monitor *cur_mon;
> +Monitor *monitor_cur(void) { return cur_mon; }
> int monitor_vprintf(Monitor *mon, const char *fmt, va_list ap) { abort(); }
> -void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp) {}
> -void monitor_init_hmp(Chardev *chr, bool use_readline, Error **errp) {}
> -
>
> static void test_socket_fd_pass_name_good(void)
> {
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property
2020-09-09 15:11 ` [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property Kevin Wolf
2020-09-14 15:11 ` Markus Armbruster
@ 2020-10-02 7:53 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 7:53 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> This way, a monitor command handler will still be able to access the
> current monitor, but when it yields, all other code code will correctly
> get NULL from monitor_cur().
>
> This uses a hash table to map the coroutine pointer to the current
> monitor of that coroutine. Outside of coroutine context, we associate
> the current monitor with the leader coroutine of the current thread.
>
> Approaches to implement some form of coroutine local storage directly in
> the coroutine core code have been considered and discarded because they
> didn't end up being much more generic than the hash table and their
> performance impact on coroutines not using coroutine local storage was
> unclear. As the block layer uses a coroutine per I/O request, this is a
> fast path and we have to be careful. It's safest to just stay out of
> this path with code only used by the monitor.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> include/monitor/monitor.h | 2 +-
> monitor/hmp.c | 4 ++--
> monitor/monitor.c | 34 +++++++++++++++++++++++++++-------
> qapi/qmp-dispatch.c | 4 ++--
> stubs/monitor-core.c | 2 +-
> 5 files changed, 33 insertions(+), 13 deletions(-)
>
> diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
> index 7b0ad1de12..026f8a31b2 100644
> --- a/include/monitor/monitor.h
> +++ b/include/monitor/monitor.h
> @@ -13,7 +13,7 @@ typedef struct MonitorOptions MonitorOptions;
> extern QemuOptsList qemu_mon_opts;
>
> Monitor *monitor_cur(void);
> -Monitor *monitor_set_cur(Monitor *mon);
> +Monitor *monitor_set_cur(Coroutine *co, Monitor *mon);
> bool monitor_cur_is_qmp(void);
>
> void monitor_init_globals(void);
> diff --git a/monitor/hmp.c b/monitor/hmp.c
> index 896c670183..4b66ca1cd6 100644
> --- a/monitor/hmp.c
> +++ b/monitor/hmp.c
> @@ -1081,9 +1081,9 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
> }
>
> /* old_mon is non-NULL when called from qmp_human_monitor_command() */
> - old_mon = monitor_set_cur(&mon->common);
> + old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
> cmd->cmd(&mon->common, qdict);
> - monitor_set_cur(old_mon);
> + monitor_set_cur(qemu_coroutine_self(), old_mon);
>
> qobject_unref(qdict);
> }
> diff --git a/monitor/monitor.c b/monitor/monitor.c
> index be3839a7aa..629aa073ee 100644
> --- a/monitor/monitor.c
> +++ b/monitor/monitor.c
> @@ -58,29 +58,48 @@ IOThread *mon_iothread;
> /* Bottom half to dispatch the requests received from I/O thread */
> QEMUBH *qmp_dispatcher_bh;
>
> -/* Protects mon_list, monitor_qapi_event_state, monitor_destroyed. */
> +/*
> + * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
> + * monitor_destroyed.
> + */
> QemuMutex monitor_lock;
> static GHashTable *monitor_qapi_event_state;
> +static GHashTable *coroutine_mon; /* Maps Coroutine* to Monitor* */
>
> MonitorList mon_list;
> int mon_refcount;
> static bool monitor_destroyed;
>
> -static __thread Monitor *cur_monitor;
> -
> Monitor *monitor_cur(void)
> {
> - return cur_monitor;
> + Monitor *mon;
> +
> + qemu_mutex_lock(&monitor_lock);
> + mon = g_hash_table_lookup(coroutine_mon, qemu_coroutine_self());
> + qemu_mutex_unlock(&monitor_lock);
> +
> + return mon;
> }
>
> /**
> * Sets a new current monitor and returns the old one.
> + *
> + * If a non-NULL monitor is set for a coroutine, another call resetting it to
> + * NULL is required before the coroutine terminates, otherwise a stale entry
> + * would remain in the hash table.
Wrapping the comment around column 70 or so would make it easier to
read.
> */
> -Monitor *monitor_set_cur(Monitor *mon)
> +Monitor *monitor_set_cur(Coroutine *co, Monitor *mon)
> {
> - Monitor *old_monitor = cur_monitor;
> + Monitor *old_monitor = monitor_cur();
> +
> + qemu_mutex_lock(&monitor_lock);
> + if (mon) {
> + g_hash_table_replace(coroutine_mon, co, mon);
> + } else {
> + g_hash_table_remove(coroutine_mon, co);
> + }
> + qemu_mutex_unlock(&monitor_lock);
>
> - cur_monitor = mon;
> return old_monitor;
> }
>
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
2020-09-14 15:15 ` Markus Armbruster
@ 2020-10-02 7:53 ` Markus Armbruster
2020-10-02 7:59 ` Markus Armbruster
2 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 7:53 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> This patch adds a new 'coroutine' flag to QMP command definitions that
> tells the QMP dispatcher that the command handler is safe to be run in a
> coroutine.
>
> The documentation of the new flag pretends that this flag is already
> used as intended, which it isn't yet after this patch. We'll implement
> this in another patch in this series.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
[...]
> diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
> index 2942520399..928cd1eb5c 100644
> --- a/scripts/qapi/expr.py
> +++ b/scripts/qapi/expr.py
> @@ -88,10 +88,16 @@ def check_flags(expr, info):
> if key in expr and expr[key] is not False:
> raise QAPISemError(
> info, "flag '%s' may only use false value" % key)
> - for key in ['boxed', 'allow-oob', 'allow-preconfig']:
> + for key in ['boxed', 'allow-oob', 'allow-preconfig', 'coroutine']:
> if key in expr and expr[key] is not True:
> raise QAPISemError(
> info, "flag '%s' may only use true value" % key)
> + if 'allow-oob' in expr and 'coroutine' in expr:
> + # This is not necessarily a fundamental incompatibility, but we don't
> + # have a use case and the desired semantics isn't obvious. The simplest
> + # solution is to forbid it until we get a use case for it.
> + raise QAPISemError(info, "flags 'allow-oob' and 'coroutine' "
> + "are incompatible")
PEP 8: For flowing long blocks of text with fewer structural
restrictions (docstrings or comments), the line length should be limited
to 72 characters.
PEP 8: You should use two spaces after a sentence-ending period in
multi- sentence comments, except after the final sentence.
>
>
> def check_if(expr, info, source):
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
2020-09-14 15:15 ` Markus Armbruster
2020-10-02 7:53 ` Markus Armbruster
@ 2020-10-02 7:59 ` Markus Armbruster
2 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 7:59 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Hit send too fast.
Kevin Wolf <kwolf@redhat.com> writes:
> This patch adds a new 'coroutine' flag to QMP command definitions that
> tells the QMP dispatcher that the command handler is safe to be run in a
> coroutine.
>
> The documentation of the new flag pretends that this flag is already
> used as intended, which it isn't yet after this patch. We'll implement
> this in another patch in this series.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
[...]
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 78309a00f0..c44d391c3f 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -128,7 +128,7 @@ class QAPISchemaVisitor:
>
> def visit_command(self, name, info, ifcond, features,
> arg_type, ret_type, gen, success_response, boxed,
> - allow_oob, allow_preconfig):
> + allow_oob, allow_preconfig, coroutine):
> pass
>
> def visit_event(self, name, info, ifcond, features, arg_type, boxed):
> @@ -713,7 +713,8 @@ class QAPISchemaCommand(QAPISchemaEntity):
>
> def __init__(self, name, info, doc, ifcond, features,
> arg_type, ret_type,
> - gen, success_response, boxed, allow_oob, allow_preconfig):
> + gen, success_response, boxed, allow_oob, allow_preconfig,
> + coroutine):
> super().__init__(name, info, doc, ifcond, features)
> assert not arg_type or isinstance(arg_type, str)
> assert not ret_type or isinstance(ret_type, str)
> @@ -726,6 +727,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
> self.boxed = boxed
> self.allow_oob = allow_oob
> self.allow_preconfig = allow_preconfig
> + self.coroutine = coroutine
>
> def check(self, schema):
> super().check(schema)
> @@ -768,7 +770,7 @@ class QAPISchemaCommand(QAPISchemaEntity):
> visitor.visit_command(
> self.name, self.info, self.ifcond, self.features,
> self.arg_type, self.ret_type, self.gen, self.success_response,
> - self.boxed, self.allow_oob, self.allow_preconfig)
> + self.boxed, self.allow_oob, self.allow_preconfig, self.coroutine)
Recommend to break the line after preconfig, like you do elsewhere.
>
>
> class QAPISchemaEvent(QAPISchemaEntity):
> @@ -1074,6 +1076,7 @@ class QAPISchema:
> boxed = expr.get('boxed', False)
> allow_oob = expr.get('allow-oob', False)
> allow_preconfig = expr.get('allow-preconfig', False)
> + coroutine = expr.get('coroutine', False)
> ifcond = expr.get('if')
> features = self._make_features(expr.get('features'), info)
> if isinstance(data, OrderedDict):
> @@ -1086,7 +1089,8 @@ class QAPISchema:
> self._def_entity(QAPISchemaCommand(name, info, doc, ifcond, features,
> data, rets,
> gen, success_response,
> - boxed, allow_oob, allow_preconfig))
> + boxed, allow_oob, allow_preconfig,
> + coroutine))
Preexisting: the arguments are kind of squeezed onto the right margin.
Hanging indent would avoid that. Feel free to ignore.
>
> def _def_event(self, expr, info, doc):
> name = expr['event']
[...]
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 09/13] qmp: Move dispatcher to a coroutine
2020-09-09 15:11 ` [PATCH v7 09/13] qmp: Move dispatcher to a coroutine Kevin Wolf
2020-09-14 15:30 ` Markus Armbruster
@ 2020-10-02 8:01 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 8:01 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> This moves the QMP dispatcher to a coroutine and runs all QMP command
> handlers that declare 'coroutine': true in coroutine context so they
> can avoid blocking the main loop while doing I/O or waiting for other
> events.
>
> For commands that are not declared safe to run in a coroutine, the
> dispatcher drops out of coroutine context by calling the QMP command
> handler from a bottom half.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> Reviewed-by: Markus Armbruster <armbru@redhat.com>
[...]
> diff --git a/monitor/monitor.c b/monitor/monitor.c
> index 629aa073ee..ac2722bf91 100644
> --- a/monitor/monitor.c
> +++ b/monitor/monitor.c
> @@ -55,8 +55,32 @@ typedef struct {
> /* Shared monitor I/O thread */
> IOThread *mon_iothread;
>
> -/* Bottom half to dispatch the requests received from I/O thread */
> -QEMUBH *qmp_dispatcher_bh;
> +/* Coroutine to dispatch the requests received from I/O thread */
> +Coroutine *qmp_dispatcher_co;
> +
> +/* Set to true when the dispatcher coroutine should terminate */
> +bool qmp_dispatcher_co_shutdown;
> +
> +/*
> + * qmp_dispatcher_co_busy is used for synchronisation between the
> + * monitor thread and the main thread to ensure that the dispatcher
> + * coroutine never gets scheduled a second time when it's already
> + * scheduled (scheduling the same coroutine twice is forbidden).
> + *
> + * It is true if the coroutine is active and processing requests.
> + * Additional requests may then be pushed onto mon->qmp_requests,
> + * and @qmp_dispatcher_co_shutdown may be set without further ado.
> + * @qmp_dispatcher_co_busy must not be woken up in this case.
> + *
> + * If false, you also have to set @qmp_dispatcher_co_busy to true and
> + * wake up @qmp_dispatcher_co after pushing the new requests.
> + *
> + * The coroutine will automatically change this variable back to false
> + * before it yields. Nobody else may set the variable to false.
> + *
> + * Access must be atomic for thread safety.
> + */
> +bool qmp_dispatcher_co_busy;
>
> /*
> * Protects mon_list, monitor_qapi_event_state, coroutine_mon,
> @@ -623,9 +647,24 @@ void monitor_cleanup(void)
> }
> qemu_mutex_unlock(&monitor_lock);
>
> - /* QEMUBHs needs to be deleted before destroying the I/O thread */
> - qemu_bh_delete(qmp_dispatcher_bh);
> - qmp_dispatcher_bh = NULL;
> + /*
> + * The dispatcher needs to stop before destroying the I/O thread.
> + *
> + * We need to poll both qemu_aio_context and iohandler_ctx to make
> + * sure that the dispatcher coroutine keeps making progress and
> + * eventually terminates. qemu_aio_context is automatically
> + * polled by calling AIO_WAIT_WHILE on it, but we must poll
> + * iohandler_ctx manually.
> + */
> + qmp_dispatcher_co_shutdown = true;
> + if (!atomic_xchg(&qmp_dispatcher_co_busy, true)) {
> + aio_co_wake(qmp_dispatcher_co);
> + }
> +
> + AIO_WAIT_WHILE(qemu_get_aio_context(),
> + (aio_poll(iohandler_get_aio_context(), false),
> + atomic_mb_read(&qmp_dispatcher_co_busy)));
> +
> if (mon_iothread) {
> iothread_destroy(mon_iothread);
> mon_iothread = NULL;
> @@ -649,9 +688,9 @@ void monitor_init_globals_core(void)
> * have commands assuming that context. It would be nice to get
> * rid of those assumptions.
> */
> - qmp_dispatcher_bh = aio_bh_new(iohandler_get_aio_context(),
> - monitor_qmp_bh_dispatcher,
> - NULL);
> + qmp_dispatcher_co = qemu_coroutine_create(monitor_qmp_dispatcher_co, NULL);
Rather long line, caused by rather long identifiers.
Not your fault; you imitated the existing pattern static variable
qmp_dispatcher_bh / extern function monitor_qmp_bh_dispatcher(). But
the prefix monitor_qmp_ is awfully long, and not just here. Let's leave
this for another day.
> + atomic_mb_set(&qmp_dispatcher_co_busy, true);
> + aio_co_schedule(iohandler_get_aio_context(), qmp_dispatcher_co);
> }
>
> int monitor_init(MonitorOptions *opts, bool allow_hmp, Error **errp)
[...]
> diff --git a/util/aio-posix.c b/util/aio-posix.c
> index f7f13ebfc2..30bb21d699 100644
> --- a/util/aio-posix.c
> +++ b/util/aio-posix.c
> @@ -15,6 +15,7 @@
>
> #include "qemu/osdep.h"
> #include "block/block.h"
> +#include "qemu/main-loop.h"
> #include "qemu/rcu.h"
> #include "qemu/rcu_queue.h"
> #include "qemu/sockets.h"
> @@ -558,8 +559,13 @@ bool aio_poll(AioContext *ctx, bool blocking)
> * There cannot be two concurrent aio_poll calls for the same AioContext (or
> * an aio_poll concurrent with a GSource prepare/check/dispatch callback).
> * We rely on this below to avoid slow locked accesses to ctx->notify_me.
> + *
> + * aio_poll() may only be called in the AioContext's thread. iohandler_ctx
> + * is special in that it runs in the main thread, but that thread's context
> + * is qemu_aio_context.
Wrapping the comment around column 70 or so would make it easier to
read. Up to you.
> */
> - assert(in_aio_context_home_thread(ctx));
> + assert(in_aio_context_home_thread(ctx == iohandler_get_aio_context() ?
> + qemu_get_aio_context() : ctx));
>
> qemu_lockcnt_inc(&ctx->list_lock);
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 10/13] hmp: Add support for coroutine command handlers
2020-09-09 15:11 ` [PATCH v7 10/13] hmp: Add support for coroutine command handlers Kevin Wolf
2020-09-16 9:46 ` Dr. David Alan Gilbert
@ 2020-10-02 8:01 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 8:01 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> Often, QMP command handlers are not only called to handle QMP commands,
> but also from a corresponding HMP command handler. In order to give them
> a consistent environment, optionally run HMP command handlers in a
> coroutine, too.
>
> The implementation is a lot simpler than in QMP because for HMP, we
> still block the VM while the coroutine is running.
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> monitor/monitor-internal.h | 1 +
> monitor/hmp.c | 37 ++++++++++++++++++++++++++++++++-----
> 2 files changed, 33 insertions(+), 5 deletions(-)
>
> diff --git a/monitor/monitor-internal.h b/monitor/monitor-internal.h
> index b55d6df07f..ad2e64be13 100644
> --- a/monitor/monitor-internal.h
> +++ b/monitor/monitor-internal.h
> @@ -74,6 +74,7 @@ typedef struct HMPCommand {
> const char *help;
> const char *flags; /* p=preconfig */
> void (*cmd)(Monitor *mon, const QDict *qdict);
> + bool coroutine;
> /*
> * @sub_table is a list of 2nd level of commands. If it does not exist,
> * cmd should be used. If it exists, sub_table[?].cmd should be
> diff --git a/monitor/hmp.c b/monitor/hmp.c
> index 4b66ca1cd6..b858b0dbde 100644
> --- a/monitor/hmp.c
> +++ b/monitor/hmp.c
> @@ -1056,12 +1056,26 @@ fail:
> return NULL;
> }
>
> +typedef struct HandleHmpCommandCo {
> + Monitor *mon;
> + const HMPCommand *cmd;
> + QDict *qdict;
> + bool done;
> +} HandleHmpCommandCo;
> +
> +static void handle_hmp_command_co(void *opaque)
> +{
> + HandleHmpCommandCo *data = opaque;
> + data->cmd->cmd(data->mon, data->qdict);
> + monitor_set_cur(qemu_coroutine_self(), NULL);
> + data->done = true;
> +}
> +
> void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
> {
> QDict *qdict;
> const HMPCommand *cmd;
> const char *cmd_start = cmdline;
> - Monitor *old_mon;
>
> trace_handle_hmp_command(mon, cmdline);
>
> @@ -1080,10 +1094,23 @@ void handle_hmp_command(MonitorHMP *mon, const char *cmdline)
> return;
> }
>
> - /* old_mon is non-NULL when called from qmp_human_monitor_command() */
> - old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
> - cmd->cmd(&mon->common, qdict);
> - monitor_set_cur(qemu_coroutine_self(), old_mon);
> + if (!cmd->coroutine) {
> + /* old_mon is non-NULL when called from qmp_human_monitor_command() */
> + Monitor *old_mon = monitor_set_cur(qemu_coroutine_self(), &mon->common);
Long line. David seems fine with it, and he's the maintainer.
> + cmd->cmd(&mon->common, qdict);
> + monitor_set_cur(qemu_coroutine_self(), old_mon);
> + } else {
> + HandleHmpCommandCo data = {
> + .mon = &mon->common,
> + .cmd = cmd,
> + .qdict = qdict,
> + .done = false,
> + };
> + Coroutine *co = qemu_coroutine_create(handle_hmp_command_co, &data);
> + monitor_set_cur(co, &mon->common);
> + aio_co_enter(qemu_get_aio_context(), co);
> + AIO_WAIT_WHILE(qemu_get_aio_context(), !data.done);
> + }
>
> qobject_unref(qdict);
> }
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 11/13] util/async: Add aio_co_reschedule_self()
2020-09-09 15:11 ` [PATCH v7 11/13] util/async: Add aio_co_reschedule_self() Kevin Wolf
2020-09-15 14:25 ` Stefan Hajnoczi
@ 2020-10-02 8:01 ` Markus Armbruster
1 sibling, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 8:01 UTC (permalink / raw)
To: Kevin Wolf; +Cc: stefanha, marcandre.lureau, qemu-devel, qemu-block, dgilbert
Additional nitpick detail on Kevin's request.
Kevin Wolf <kwolf@redhat.com> writes:
> Add a function that can be used to move the currently running coroutine
> to a different AioContext (and therefore potentially a different
> thread).
>
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
> include/block/aio.h | 10 ++++++++++
> util/async.c | 30 ++++++++++++++++++++++++++++++
> 2 files changed, 40 insertions(+)
>
> diff --git a/include/block/aio.h b/include/block/aio.h
> index b2f703fa3f..c37617b404 100644
> --- a/include/block/aio.h
> +++ b/include/block/aio.h
> @@ -17,6 +17,7 @@
> #ifdef CONFIG_LINUX_IO_URING
> #include <liburing.h>
> #endif
> +#include "qemu/coroutine.h"
> #include "qemu/queue.h"
> #include "qemu/event_notifier.h"
> #include "qemu/thread.h"
> @@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
> */
> void aio_co_schedule(AioContext *ctx, struct Coroutine *co);
>
> +/**
> + * aio_co_reschedule_self:
> + * @new_ctx: the new context
> + *
> + * Move the currently running coroutine to new_ctx. If the coroutine is already
> + * running in new_ctx, do nothing.
Wrapping the comment around column 70 or so would make it easier to
read. Up to you.
> + */
> +void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx);
> +
> /**
> * aio_co_wake:
> * @co: the coroutine
> diff --git a/util/async.c b/util/async.c
> index 4266745dee..a609e18693 100644
> --- a/util/async.c
> +++ b/util/async.c
> @@ -569,6 +569,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
> aio_context_unref(ctx);
> }
>
> +typedef struct AioCoRescheduleSelf {
> + Coroutine *co;
> + AioContext *new_ctx;
> +} AioCoRescheduleSelf;
> +
> +static void aio_co_reschedule_self_bh(void *opaque)
> +{
> + AioCoRescheduleSelf *data = opaque;
> + aio_co_schedule(data->new_ctx, data->co);
> +}
> +
> +void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx)
> +{
> + AioContext *old_ctx = qemu_get_current_aio_context();
> +
> + if (old_ctx != new_ctx) {
> + AioCoRescheduleSelf data = {
> + .co = qemu_coroutine_self(),
> + .new_ctx = new_ctx,
> + };
> + /*
> + * We can't directly schedule the coroutine in the target context
> + * because this would be racy: The other thread could try to enter the
> + * coroutine before it has yielded in this one.
> + */
Likewise.
> + aio_bh_schedule_oneshot(old_ctx, aio_co_reschedule_self_bh, &data);
> + qemu_coroutine_yield();
> + }
> +}
> +
> void aio_co_wake(struct Coroutine *co)
> {
> AioContext *ctx;
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch()
2020-10-01 16:00 ` Markus Armbruster
@ 2020-10-02 8:04 ` Markus Armbruster
0 siblings, 0 replies; 62+ messages in thread
From: Markus Armbruster @ 2020-10-02 8:04 UTC (permalink / raw)
To: Kevin Wolf
Cc: stefanha, marcandre.lureau, Dr. David Alan Gilbert, qemu-block,
qemu-devel
Markus Armbruster <armbru@redhat.com> writes:
> Kevin Wolf <kwolf@redhat.com> writes:
[...]
>> I'll just change this one in the next version. Changing a single
>> well-known instance not a big problem. It's just unfortunate that there
>> are "A few more in PATCH 08-11" and I don't know how to identify them.
>
> When I do that, and you'd rather have a complete list, just ask. Out of
> time for today, but I can get it for you first thing tomorrow.
>
> [...]
Done, with the detail level cranked up to "lots" ;)
^ permalink raw reply [flat|nested] 62+ messages in thread
end of thread, other threads:[~2020-10-02 8:06 UTC | newest]
Thread overview: 62+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-09 15:11 [PATCH v7 00/13] monitor: Optionally run handlers in coroutines Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 01/13] monitor: Add Monitor parameter to monitor_set_cpu() Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 02/13] monitor: Add Monitor parameter to monitor_get_cpu_index() Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 03/13] monitor: Use getter/setter functions for cur_mon Kevin Wolf
2020-10-02 7:51 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 04/13] hmp: Update current monitor only in handle_hmp_command() Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 05/13] qmp: Assert that no other monitor is active Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 06/13] qmp: Call monitor_set_cur() only in qmp_dispatch() Kevin Wolf
2020-09-14 15:10 ` Markus Armbruster
2020-09-25 15:13 ` Kevin Wolf
2020-09-28 11:42 ` Markus Armbruster
2020-09-28 14:30 ` Kevin Wolf
2020-09-30 9:26 ` Markus Armbruster
2020-09-30 11:29 ` Kevin Wolf
2020-09-30 13:14 ` Markus Armbruster
2020-09-30 14:00 ` Kevin Wolf
2020-09-30 17:20 ` Dr. David Alan Gilbert
2020-10-01 10:14 ` Kevin Wolf
2020-10-01 16:00 ` Markus Armbruster
2020-10-02 8:04 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 07/13] monitor: Make current monitor a per-coroutine property Kevin Wolf
2020-09-14 15:11 ` Markus Armbruster
2020-09-25 15:23 ` Kevin Wolf
2020-09-28 7:47 ` Markus Armbruster
2020-09-28 10:42 ` Kevin Wolf
2020-09-28 12:21 ` Markus Armbruster
2020-10-02 7:53 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 08/13] qapi: Add a 'coroutine' flag for commands Kevin Wolf
2020-09-14 15:15 ` Markus Armbruster
2020-09-25 15:37 ` Kevin Wolf
2020-09-28 8:23 ` Markus Armbruster
2020-10-02 7:53 ` Markus Armbruster
2020-10-02 7:59 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 09/13] qmp: Move dispatcher to a coroutine Kevin Wolf
2020-09-14 15:30 ` Markus Armbruster
2020-09-25 15:38 ` Kevin Wolf
2020-09-28 8:24 ` Markus Armbruster
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 10/13] hmp: Add support for coroutine command handlers Kevin Wolf
2020-09-16 9:46 ` Dr. David Alan Gilbert
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 11/13] util/async: Add aio_co_reschedule_self() Kevin Wolf
2020-09-15 14:25 ` Stefan Hajnoczi
2020-10-02 8:01 ` Markus Armbruster
2020-09-09 15:11 ` [PATCH v7 12/13] block: Add bdrv_co_move_to_aio_context() Kevin Wolf
2020-09-15 14:31 ` Stefan Hajnoczi
2020-09-25 16:00 ` Kevin Wolf
2020-09-28 8:59 ` Stefan Hajnoczi
2020-09-28 10:21 ` Kevin Wolf
2020-09-09 15:11 ` [PATCH v7 13/13] block: Convert 'block_resize' to coroutine Kevin Wolf
2020-09-15 14:57 ` Stefan Hajnoczi
2020-09-25 16:07 ` Kevin Wolf
2020-09-28 9:05 ` Stefan Hajnoczi
2020-09-28 10:33 ` Kevin Wolf
2020-09-09 15:24 ` [PATCH v7 00/13] monitor: Optionally run handlers in coroutines no-reply
2020-09-10 13:24 ` Stefan Hajnoczi
2020-09-14 15:09 ` Markus Armbruster
2020-09-15 14:58 ` Stefan Hajnoczi
2020-09-25 17:15 ` Kevin Wolf
2020-09-28 8:46 ` Stefan Hajnoczi
2020-09-28 9:47 ` Kevin Wolf
2020-09-28 9:30 ` Markus Armbruster
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.