* [PULL 0/8] Block patches
@ 2020-05-19 8:00 Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 1/8] tests/fuzz/Makefile: Do not link code using unavailable devices Stefan Hajnoczi
` (8 more replies)
0 siblings, 9 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf
The following changes since commit 013a18edbbc59cdad019100c7d03c0494642b74c:
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-2020051=
4' into staging (2020-05-14 16:17:55 +0100)
are available in the Git repository at:
https://github.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to ba607ca8bff4d2c2062902f8355657c865ac7c29:
aio-posix: disable fdmon-io_uring when GSource is used (2020-05-18 18:16:00=
+0100)
----------------------------------------------------------------
Pull request
----------------------------------------------------------------
Philippe Mathieu-Daud=C3=A9 (6):
tests/fuzz/Makefile: Do not link code using unavailable devices
Makefile: List fuzz targets in 'make help'
tests/fuzz: Add missing space in test description
tests/fuzz: Remove unuseful/unused typedefs
tests/fuzz: Extract pciconfig_fuzz_qos() method
tests/fuzz: Extract ioport_fuzz_qtest() method
Stefan Hajnoczi (2):
aio-posix: don't duplicate fd handler deletion in
fdmon_io_uring_destroy()
aio-posix: disable fdmon-io_uring when GSource is used
Makefile | 6 +++-
tests/qtest/fuzz/Makefile.include | 6 ++--
include/block/aio.h | 3 ++
tests/qtest/fuzz/i440fx_fuzz.c | 47 ++++++++++++++++++++-----------
util/aio-posix.c | 13 +++++++++
util/aio-win32.c | 4 +++
util/async.c | 1 +
util/fdmon-io_uring.c | 13 +++++++--
8 files changed, 69 insertions(+), 24 deletions(-)
--=20
2.25.3
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PULL 1/8] tests/fuzz/Makefile: Do not link code using unavailable devices
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 2/8] Makefile: List fuzz targets in 'make help' Stefan Hajnoczi
` (7 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Some devices availability depends on CONFIG options.
Use these options to only link tests when requested device
is available.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-2-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qtest/fuzz/Makefile.include | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qtest/fuzz/Makefile.include b/tests/qtest/fuzz/Makefile.include
index cde3e9636c..f259d866c9 100644
--- a/tests/qtest/fuzz/Makefile.include
+++ b/tests/qtest/fuzz/Makefile.include
@@ -7,9 +7,9 @@ fuzz-obj-y += tests/qtest/fuzz/fork_fuzz.o
fuzz-obj-y += tests/qtest/fuzz/qos_fuzz.o
# Targets
-fuzz-obj-y += tests/qtest/fuzz/i440fx_fuzz.o
-fuzz-obj-y += tests/qtest/fuzz/virtio_net_fuzz.o
-fuzz-obj-y += tests/qtest/fuzz/virtio_scsi_fuzz.o
+fuzz-obj-$(CONFIG_PCI_I440FX) += tests/qtest/fuzz/i440fx_fuzz.o
+fuzz-obj-$(CONFIG_VIRTIO_NET) += tests/qtest/fuzz/virtio_net_fuzz.o
+fuzz-obj-$(CONFIG_SCSI) += tests/qtest/fuzz/virtio_scsi_fuzz.o
FUZZ_CFLAGS += -I$(SRC_PATH)/tests -I$(SRC_PATH)/tests/qtest
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 2/8] Makefile: List fuzz targets in 'make help'
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 1/8] tests/fuzz/Makefile: Do not link code using unavailable devices Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 3/8] tests/fuzz: Add missing space in test description Stefan Hajnoczi
` (6 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
List softmmu fuzz targets in 'make help' output:
$ make help
...
Architecture specific targets:
aarch64-softmmu/all - Build for aarch64-softmmu
aarch64-softmmu/fuzz - Build fuzzer for aarch64-softmmu
alpha-softmmu/all - Build for alpha-softmmu
alpha-softmmu/fuzz - Build fuzzer for alpha-softmmu
arm-softmmu/all - Build for arm-softmmu
arm-softmmu/fuzz - Build fuzzer for arm-softmmu
...
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-3-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
Makefile | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index 34275f57c9..40e4f7677b 100644
--- a/Makefile
+++ b/Makefile
@@ -1252,7 +1252,11 @@ endif
@$(if $(TARGET_DIRS), \
echo 'Architecture specific targets:'; \
$(foreach t, $(TARGET_DIRS), \
- $(call print-help-run,$(t)/all,Build for $(t));) \
+ $(call print-help-run,$(t)/all,Build for $(t)); \
+ $(if $(CONFIG_FUZZ), \
+ $(if $(findstring softmmu,$(t)), \
+ $(call print-help-run,$(t)/fuzz,Build fuzzer for $(t)); \
+ ))) \
echo '')
@$(if $(TOOLS), \
echo 'Tools targets:'; \
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 3/8] tests/fuzz: Add missing space in test description
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 1/8] tests/fuzz/Makefile: Do not link code using unavailable devices Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 2/8] Makefile: List fuzz targets in 'make help' Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 4/8] tests/fuzz: Remove unuseful/unused typedefs Stefan Hajnoczi
` (5 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-4-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qtest/fuzz/i440fx_fuzz.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qtest/fuzz/i440fx_fuzz.c b/tests/qtest/fuzz/i440fx_fuzz.c
index ab5f112584..96fed9ff12 100644
--- a/tests/qtest/fuzz/i440fx_fuzz.c
+++ b/tests/qtest/fuzz/i440fx_fuzz.c
@@ -159,7 +159,7 @@ static void register_pci_fuzz_targets(void)
/* Uses simple qtest commands and reboots to reset state */
fuzz_add_target(&(FuzzTarget){
.name = "i440fx-qtest-reboot-fuzz",
- .description = "Fuzz the i440fx using raw qtest commands and"
+ .description = "Fuzz the i440fx using raw qtest commands and "
"rebooting after each run",
.get_init_cmdline = i440fx_argv,
.fuzz = i440fx_fuzz_qtest});
@@ -167,7 +167,7 @@ static void register_pci_fuzz_targets(void)
/* Uses libqos and forks to prevent state leakage */
fuzz_add_qos_target(&(FuzzTarget){
.name = "i440fx-qos-fork-fuzz",
- .description = "Fuzz the i440fx using raw qtest commands and"
+ .description = "Fuzz the i440fx using raw qtest commands and "
"rebooting after each run",
.pre_vm_init = &fork_init,
.fuzz = i440fx_fuzz_qos_fork,},
@@ -182,7 +182,7 @@ static void register_pci_fuzz_targets(void)
*/
fuzz_add_qos_target(&(FuzzTarget){
.name = "i440fx-qos-noreset-fuzz",
- .description = "Fuzz the i440fx using raw qtest commands and"
+ .description = "Fuzz the i440fx using raw qtest commands and "
"rebooting after each run",
.fuzz = i440fx_fuzz_qos,},
"i440FX-pcihost",
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 4/8] tests/fuzz: Remove unuseful/unused typedefs
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (2 preceding siblings ...)
2020-05-19 8:00 ` [PULL 3/8] tests/fuzz: Add missing space in test description Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 5/8] tests/fuzz: Extract pciconfig_fuzz_qos() method Stefan Hajnoczi
` (4 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
These typedefs are not used. Use a simple structure,
remote the typedefs.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-5-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qtest/fuzz/i440fx_fuzz.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/tests/qtest/fuzz/i440fx_fuzz.c b/tests/qtest/fuzz/i440fx_fuzz.c
index 96fed9ff12..c197b026db 100644
--- a/tests/qtest/fuzz/i440fx_fuzz.c
+++ b/tests/qtest/fuzz/i440fx_fuzz.c
@@ -45,12 +45,11 @@ static void i440fx_fuzz_qtest(QTestState *s,
* loop over the Data, breaking it up into actions. each action has an
* opcode, address offset and value
*/
- typedef struct QTestFuzzAction {
+ struct {
uint8_t opcode;
uint8_t addr;
uint32_t value;
- } QTestFuzzAction;
- QTestFuzzAction a;
+ } a;
while (Size >= sizeof(a)) {
/* make a copy of the action so we can normalize the values in-place */
@@ -91,19 +90,18 @@ static void i440fx_fuzz_qos(QTestState *s,
* Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
* value written over Port IO
*/
- typedef struct QOSFuzzAction {
+ struct {
uint8_t opcode;
uint8_t offset;
int devfn;
uint32_t value;
- } QOSFuzzAction;
+ } a;
static QPCIBus *bus;
if (!bus) {
bus = qpci_new_pc(s, fuzz_qos_alloc);
}
- QOSFuzzAction a;
while (Size >= sizeof(a)) {
memcpy(&a, Data, sizeof(a));
switch (a.opcode % ACTION_MAX) {
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 5/8] tests/fuzz: Extract pciconfig_fuzz_qos() method
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (3 preceding siblings ...)
2020-05-19 8:00 ` [PULL 4/8] tests/fuzz: Remove unuseful/unused typedefs Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 6/8] tests/fuzz: Extract ioport_fuzz_qtest() method Stefan Hajnoczi
` (3 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Extract the generic pciconfig_fuzz_qos() method from
i440fx_fuzz_qos(). This will help to write tests not
specific to the i440FX controller.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-6-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qtest/fuzz/i440fx_fuzz.c | 20 ++++++++++++++------
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/tests/qtest/fuzz/i440fx_fuzz.c b/tests/qtest/fuzz/i440fx_fuzz.c
index c197b026db..558fa17c93 100644
--- a/tests/qtest/fuzz/i440fx_fuzz.c
+++ b/tests/qtest/fuzz/i440fx_fuzz.c
@@ -84,7 +84,7 @@ static void i440fx_fuzz_qtest(QTestState *s,
flush_events(s);
}
-static void i440fx_fuzz_qos(QTestState *s,
+static void pciconfig_fuzz_qos(QTestState *s, QPCIBus *bus,
const unsigned char *Data, size_t Size) {
/*
* Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
@@ -97,11 +97,6 @@ static void i440fx_fuzz_qos(QTestState *s,
uint32_t value;
} a;
- static QPCIBus *bus;
- if (!bus) {
- bus = qpci_new_pc(s, fuzz_qos_alloc);
- }
-
while (Size >= sizeof(a)) {
memcpy(&a, Data, sizeof(a));
switch (a.opcode % ACTION_MAX) {
@@ -130,6 +125,19 @@ static void i440fx_fuzz_qos(QTestState *s,
flush_events(s);
}
+static void i440fx_fuzz_qos(QTestState *s,
+ const unsigned char *Data,
+ size_t Size)
+{
+ static QPCIBus *bus;
+
+ if (!bus) {
+ bus = qpci_new_pc(s, fuzz_qos_alloc);
+ }
+
+ pciconfig_fuzz_qos(s, bus, Data, Size);
+}
+
static void i440fx_fuzz_qos_fork(QTestState *s,
const unsigned char *Data, size_t Size) {
if (fork() == 0) {
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 6/8] tests/fuzz: Extract ioport_fuzz_qtest() method
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (4 preceding siblings ...)
2020-05-19 8:00 ` [PULL 5/8] tests/fuzz: Extract pciconfig_fuzz_qos() method Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 7/8] aio-posix: don't duplicate fd handler deletion in fdmon_io_uring_destroy() Stefan Hajnoczi
` (2 subsequent siblings)
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Stefan Weil, Max Reitz, Alexander Bulekov, Bandan Das,
Stefan Hajnoczi, Paolo Bonzini, Fam Zheng, Kevin Wolf,
Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Extract generic ioport_fuzz_qtest() method from
i440fx_fuzz_qtest(). This will help to write tests
not specific to the i440FX controller.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200514143433.18569-7-philmd@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
tests/qtest/fuzz/i440fx_fuzz.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/tests/qtest/fuzz/i440fx_fuzz.c b/tests/qtest/fuzz/i440fx_fuzz.c
index 558fa17c93..bcd6769b4c 100644
--- a/tests/qtest/fuzz/i440fx_fuzz.c
+++ b/tests/qtest/fuzz/i440fx_fuzz.c
@@ -39,7 +39,7 @@ enum action_id {
ACTION_MAX
};
-static void i440fx_fuzz_qtest(QTestState *s,
+static void ioport_fuzz_qtest(QTestState *s,
const unsigned char *Data, size_t Size) {
/*
* loop over the Data, breaking it up into actions. each action has an
@@ -84,10 +84,17 @@ static void i440fx_fuzz_qtest(QTestState *s,
flush_events(s);
}
+static void i440fx_fuzz_qtest(QTestState *s,
+ const unsigned char *Data,
+ size_t Size)
+{
+ ioport_fuzz_qtest(s, Data, Size);
+}
+
static void pciconfig_fuzz_qos(QTestState *s, QPCIBus *bus,
const unsigned char *Data, size_t Size) {
/*
- * Same as i440fx_fuzz_qtest, but using QOS. devfn is incorporated into the
+ * Same as ioport_fuzz_qtest, but using QOS. devfn is incorporated into the
* value written over Port IO
*/
struct {
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 7/8] aio-posix: don't duplicate fd handler deletion in fdmon_io_uring_destroy()
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (5 preceding siblings ...)
2020-05-19 8:00 ` [PULL 6/8] tests/fuzz: Extract ioport_fuzz_qtest() method Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 8/8] aio-posix: disable fdmon-io_uring when GSource is used Stefan Hajnoczi
2020-05-19 15:15 ` [PULL 0/8] Block patches Peter Maydell
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Oleksandr Natalenko, Stefan Weil, Max Reitz, Alexander Bulekov,
Bandan Das, Stefan Hajnoczi, Paolo Bonzini, Fam Zheng,
Kevin Wolf
The io_uring file descriptor monitoring implementation has an internal
list of fd handlers that are pending submission to io_uring.
fdmon_io_uring_destroy() deletes all fd handlers on the list.
Don't delete fd handlers directly in fdmon_io_uring_destroy() for two
reasons:
1. This duplicates the aio-posix.c AioHandler deletion code and could
become outdated if the struct changes.
2. Only handlers with the FDMON_IO_URING_REMOVE flag set are safe to
remove. If the flag is not set then something still has a pointer to
the fd handler. Let aio-posix.c and its user worry about that. In
practice this isn't an issue because fdmon_io_uring_destroy() is only
called when shutting down so all users have removed their fd
handlers, but the next patch will need this!
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Oleksandr Natalenko <oleksandr@redhat.com>
Message-id: 20200511183630.279750-2-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
util/aio-posix.c | 1 +
util/fdmon-io_uring.c | 13 ++++++++++---
2 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/util/aio-posix.c b/util/aio-posix.c
index c3613d299e..8af334ab19 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -679,6 +679,7 @@ void aio_context_destroy(AioContext *ctx)
{
fdmon_io_uring_destroy(ctx);
fdmon_epoll_disable(ctx);
+ aio_free_deleted_handlers(ctx);
}
void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
diff --git a/util/fdmon-io_uring.c b/util/fdmon-io_uring.c
index d5a80ed6fb..1d14177df0 100644
--- a/util/fdmon-io_uring.c
+++ b/util/fdmon-io_uring.c
@@ -342,11 +342,18 @@ void fdmon_io_uring_destroy(AioContext *ctx)
io_uring_queue_exit(&ctx->fdmon_io_uring);
- /* No need to submit these anymore, just free them. */
+ /* Move handlers due to be removed onto the deleted list */
while ((node = QSLIST_FIRST_RCU(&ctx->submit_list))) {
+ unsigned flags = atomic_fetch_and(&node->flags,
+ ~(FDMON_IO_URING_PENDING |
+ FDMON_IO_URING_ADD |
+ FDMON_IO_URING_REMOVE));
+
+ if (flags & FDMON_IO_URING_REMOVE) {
+ QLIST_INSERT_HEAD_RCU(&ctx->deleted_aio_handlers, node, node_deleted);
+ }
+
QSLIST_REMOVE_HEAD_RCU(&ctx->submit_list, node_submitted);
- QLIST_REMOVE(node, node);
- g_free(node);
}
ctx->fdmon_ops = &fdmon_poll_ops;
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [PULL 8/8] aio-posix: disable fdmon-io_uring when GSource is used
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (6 preceding siblings ...)
2020-05-19 8:00 ` [PULL 7/8] aio-posix: don't duplicate fd handler deletion in fdmon_io_uring_destroy() Stefan Hajnoczi
@ 2020-05-19 8:00 ` Stefan Hajnoczi
2020-05-19 15:15 ` [PULL 0/8] Block patches Peter Maydell
8 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2020-05-19 8:00 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Peter Maydell, Thomas Huth, qemu-block,
Oleksandr Natalenko, Stefan Weil, Max Reitz, Alexander Bulekov,
Bandan Das, Stefan Hajnoczi, Paolo Bonzini, Fam Zheng,
Kevin Wolf
The glib event loop does not call fdmon_io_uring_wait() so fd handlers
waiting to be submitted build up in the list. There is no benefit is
using io_uring when the glib GSource is being used, so disable it
instead of implementing a more complex fix.
This fixes a memory leak where AioHandlers would build up and increasing
amounts of CPU time were spent iterating them in aio_pending(). The
symptom is that guests become slow when QEMU is built with io_uring
support.
Buglink: https://bugs.launchpad.net/qemu/+bug/1877716
Fixes: 73fd282e7b6dd4e4ea1c3bbb3d302c8db51e4ccf ("aio-posix: add io_uring fd monitoring implementation")
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Oleksandr Natalenko <oleksandr@redhat.com>
Message-id: 20200511183630.279750-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
include/block/aio.h | 3 +++
util/aio-posix.c | 12 ++++++++++++
util/aio-win32.c | 4 ++++
util/async.c | 1 +
4 files changed, 20 insertions(+)
diff --git a/include/block/aio.h b/include/block/aio.h
index 62ed954344..b2f703fa3f 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -701,6 +701,9 @@ void aio_context_setup(AioContext *ctx);
*/
void aio_context_destroy(AioContext *ctx);
+/* Used internally, do not call outside AioContext code */
+void aio_context_use_g_source(AioContext *ctx);
+
/**
* aio_context_set_poll_params:
* @ctx: the aio context
diff --git a/util/aio-posix.c b/util/aio-posix.c
index 8af334ab19..1b2a3af65b 100644
--- a/util/aio-posix.c
+++ b/util/aio-posix.c
@@ -682,6 +682,18 @@ void aio_context_destroy(AioContext *ctx)
aio_free_deleted_handlers(ctx);
}
+void aio_context_use_g_source(AioContext *ctx)
+{
+ /*
+ * Disable io_uring when the glib main loop is used because it doesn't
+ * support mixed glib/aio_poll() usage. It relies on aio_poll() being
+ * called regularly so that changes to the monitored file descriptors are
+ * submitted, otherwise a list of pending fd handlers builds up.
+ */
+ fdmon_io_uring_destroy(ctx);
+ aio_free_deleted_handlers(ctx);
+}
+
void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
int64_t grow, int64_t shrink, Error **errp)
{
diff --git a/util/aio-win32.c b/util/aio-win32.c
index 729d533faf..953c56ab48 100644
--- a/util/aio-win32.c
+++ b/util/aio-win32.c
@@ -414,6 +414,10 @@ void aio_context_destroy(AioContext *ctx)
{
}
+void aio_context_use_g_source(AioContext *ctx)
+{
+}
+
void aio_context_set_poll_params(AioContext *ctx, int64_t max_ns,
int64_t grow, int64_t shrink, Error **errp)
{
diff --git a/util/async.c b/util/async.c
index 3165a28f2f..1319eee3bc 100644
--- a/util/async.c
+++ b/util/async.c
@@ -362,6 +362,7 @@ static GSourceFuncs aio_source_funcs = {
GSource *aio_get_g_source(AioContext *ctx)
{
+ aio_context_use_g_source(ctx);
g_source_ref(&ctx->source);
return &ctx->source;
}
--
2.25.3
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [PULL 0/8] Block patches
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
` (7 preceding siblings ...)
2020-05-19 8:00 ` [PULL 8/8] aio-posix: disable fdmon-io_uring when GSource is used Stefan Hajnoczi
@ 2020-05-19 15:15 ` Peter Maydell
8 siblings, 0 replies; 17+ messages in thread
From: Peter Maydell @ 2020-05-19 15:15 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Laurent Vivier, Kevin Wolf, Thomas Huth, Qemu-block, Stefan Weil,
QEMU Developers, Max Reitz, Alexander Bulekov, Bandan Das,
Paolo Bonzini, Fam Zheng
On Tue, 19 May 2020 at 09:01, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit 013a18edbbc59cdad019100c7d03c0494642b74c:
>
> Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-2020051=
> 4' into staging (2020-05-14 16:17:55 +0100)
>
> are available in the Git repository at:
>
> https://github.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to ba607ca8bff4d2c2062902f8355657c865ac7c29:
>
> aio-posix: disable fdmon-io_uring when GSource is used (2020-05-18 18:16:00=
> +0100)
>
> ----------------------------------------------------------------
> Pull request
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PULL 0/8] Block patches
@ 2023-08-24 16:58 Stefan Hajnoczi
0 siblings, 0 replies; 17+ messages in thread
From: Stefan Hajnoczi @ 2023-08-24 16:58 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Xu, Thomas Huth, Marc-André Lureau, Kevin Wolf,
Juan Quintela, Fam Zheng, Paolo Bonzini, Laurent Vivier,
Stefan Hajnoczi, Marcel Apfelbaum, John Snow, Jeuk Kim,
Hanna Reitz, Leonardo Bras, qemu-block,
Philippe Mathieu-Daudé,
Vladimir Sementsov-Ogievskiy, Daniel P. Berrangé,
Michael S. Tsirkin
The following changes since commit b0dd9a7d6dd15a6898e9c585b521e6bec79b25aa:
Open 8.2 development tree (2023-08-22 07:14:07 -0700)
are available in the Git repository at:
https://gitlab.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 892470a8de848a685acb67ba739485424efc3aed:
tests/qemu-iotests/197: add testcase for CoR with subclusters (2023-08-24 10:28:50 -0400)
----------------------------------------------------------------
Pull request
First block pull request for the QEMU 8.2 release cycle.
----------------------------------------------------------------
Andrey Drobyshev via (3):
block: add subcluster_size field to BlockDriverInfo
block/io: align requests to subcluster_size
tests/qemu-iotests/197: add testcase for CoR with subclusters
Fabiano Rosas (1):
block-migration: Ensure we don't crash during migration cleanup
Jeuk Kim (4):
hw/ufs: Initial commit for emulated Universal-Flash-Storage
hw/ufs: Support for Query Transfer Requests
hw/ufs: Support for UFS logical unit
tests/qtest: Introduce tests for UFS
MAINTAINERS | 7 +
docs/specs/pci-ids.rst | 2 +
meson.build | 1 +
hw/ufs/trace.h | 1 +
hw/ufs/ufs.h | 131 +++
include/block/block-common.h | 5 +
include/block/block-io.h | 8 +-
include/block/ufs.h | 1090 +++++++++++++++++++++++++
include/hw/pci/pci.h | 1 +
include/hw/pci/pci_ids.h | 1 +
include/scsi/constants.h | 1 +
block.c | 7 +
block/io.c | 50 +-
block/mirror.c | 8 +-
block/qcow2.c | 1 +
hw/ufs/lu.c | 1445 ++++++++++++++++++++++++++++++++
hw/ufs/ufs.c | 1494 ++++++++++++++++++++++++++++++++++
migration/block.c | 11 +-
tests/qtest/ufs-test.c | 584 +++++++++++++
hw/Kconfig | 1 +
hw/meson.build | 1 +
hw/ufs/Kconfig | 4 +
hw/ufs/meson.build | 1 +
hw/ufs/trace-events | 58 ++
tests/qemu-iotests/197 | 29 +
tests/qemu-iotests/197.out | 24 +
tests/qtest/meson.build | 1 +
27 files changed, 4932 insertions(+), 35 deletions(-)
create mode 100644 hw/ufs/trace.h
create mode 100644 hw/ufs/ufs.h
create mode 100644 include/block/ufs.h
create mode 100644 hw/ufs/lu.c
create mode 100644 hw/ufs/ufs.c
create mode 100644 tests/qtest/ufs-test.c
create mode 100644 hw/ufs/Kconfig
create mode 100644 hw/ufs/meson.build
create mode 100644 hw/ufs/trace-events
--
2.41.0
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PULL 0/8] Block patches
2023-06-01 15:25 Stefan Hajnoczi
@ 2023-06-01 21:29 ` Richard Henderson
0 siblings, 0 replies; 17+ messages in thread
From: Richard Henderson @ 2023-06-01 21:29 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel
On 6/1/23 08:25, Stefan Hajnoczi wrote:
> The following changes since commit c6a5fc2ac76c5ab709896ee1b0edd33685a67ed1:
>
> decodetree: Add --output-null for meson testing (2023-05-31 19:56:42 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 98b126f5e3228a346c774e569e26689943b401dd:
>
> qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa (2023-06-01 11:08:21 -0400)
>
> ----------------------------------------------------------------
> Pull request
>
> - Stefano Garzarella's blkio block driver 'fd' parameter
> - My thread-local blk_io_plug() series
Applied, thanks. Please update https://wiki.qemu.org/ChangeLog/8.1 as appropriate.
r~
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PULL 0/8] Block patches
@ 2023-06-01 15:25 Stefan Hajnoczi
2023-06-01 21:29 ` Richard Henderson
0 siblings, 1 reply; 17+ messages in thread
From: Stefan Hajnoczi @ 2023-06-01 15:25 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-block, Stefano Stabellini, Aarushi Mehta, Anthony Perard,
Thomas Huth, Julia Suvorova, Paolo Bonzini, Fam Zheng,
Hanna Reitz, Philippe Mathieu-Daudé,
Stefano Garzarella, Michael S. Tsirkin, Daniel P. Berrangé,
Markus Armbruster, Cornelia Huck, Marc-André Lureau,
xen-devel, Paul Durrant, Kevin Wolf, Richard Henderson,
Eric Blake, Stefan Hajnoczi, Raphael Norwitz, kvm
The following changes since commit c6a5fc2ac76c5ab709896ee1b0edd33685a67ed1:
decodetree: Add --output-null for meson testing (2023-05-31 19:56:42 -0700)
are available in the Git repository at:
https://gitlab.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 98b126f5e3228a346c774e569e26689943b401dd:
qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa (2023-06-01 11:08:21 -0400)
----------------------------------------------------------------
Pull request
- Stefano Garzarella's blkio block driver 'fd' parameter
- My thread-local blk_io_plug() series
----------------------------------------------------------------
Stefan Hajnoczi (6):
block: add blk_io_plug_call() API
block/nvme: convert to blk_io_plug_call() API
block/blkio: convert to blk_io_plug_call() API
block/io_uring: convert to blk_io_plug_call() API
block/linux-aio: convert to blk_io_plug_call() API
block: remove bdrv_co_io_plug() API
Stefano Garzarella (2):
block/blkio: use qemu_open() to support fd passing for virtio-blk
qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa
MAINTAINERS | 1 +
qapi/block-core.json | 6 ++
meson.build | 4 +
include/block/block-io.h | 3 -
include/block/block_int-common.h | 11 ---
include/block/raw-aio.h | 14 ---
include/sysemu/block-backend-io.h | 13 +--
block/blkio.c | 96 ++++++++++++------
block/block-backend.c | 22 -----
block/file-posix.c | 38 -------
block/io.c | 37 -------
block/io_uring.c | 44 ++++-----
block/linux-aio.c | 41 +++-----
block/nvme.c | 44 +++------
block/plug.c | 159 ++++++++++++++++++++++++++++++
hw/block/dataplane/xen-block.c | 8 +-
hw/block/virtio-blk.c | 4 +-
hw/scsi/virtio-scsi.c | 6 +-
block/meson.build | 1 +
block/trace-events | 6 +-
20 files changed, 293 insertions(+), 265 deletions(-)
create mode 100644 block/plug.c
--
2.40.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PULL 0/8] Block patches
2022-04-20 12:40 Hanna Reitz
@ 2022-04-20 18:12 ` Richard Henderson
0 siblings, 0 replies; 17+ messages in thread
From: Richard Henderson @ 2022-04-20 18:12 UTC (permalink / raw)
To: Hanna Reitz, qemu-block; +Cc: Peter Maydell, qemu-devel
On 4/20/22 05:40, Hanna Reitz wrote:
> The following changes since commit 1be5a765c08cee3a9587c8a8d3fc2ea247b13f9c:
>
> Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging (2022-04-19 18:22:16 -0700)
>
> are available in the Git repository at:
>
> https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-04-20
>
> for you to fetch changes up to 0423f75351ab83b844a31349218b0eadd830e07a:
>
> qcow2: Add errp to rebuild_refcount_structure() (2022-04-20 12:09:17 +0200)
>
> ----------------------------------------------------------------
> Block patches:
> - Some changes for qcow2's refcount repair algorithm to make it work for
> qcow2 images stored on block devices
> - Skip test cases that require zstd when support for it is missing
> - Some refactoring in the iotests' meson.build
Applied, thanks. Please update the wiki changelog for 7.1 as appropriate.
r~
>
> ----------------------------------------------------------------
> Hanna Reitz (6):
> iotests.py: Add supports_qcow2_zstd_compression()
> iotests/065: Check for zstd support
> iotests/303: Check for zstd support
> qcow2: Improve refcount structure rebuilding
> iotests/108: Test new refcount rebuild algorithm
> qcow2: Add errp to rebuild_refcount_structure()
>
> Thomas Huth (2):
> tests/qemu-iotests/meson.build: Improve the indentation
> tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
>
> block/qcow2-refcount.c | 353 +++++++++++++++++++++++----------
> tests/check-block.sh | 26 ---
> tests/qemu-iotests/065 | 24 ++-
> tests/qemu-iotests/108 | 259 +++++++++++++++++++++++-
> tests/qemu-iotests/108.out | 81 ++++++++
> tests/qemu-iotests/303 | 4 +-
> tests/qemu-iotests/iotests.py | 20 ++
> tests/qemu-iotests/meson.build | 73 ++++---
> 8 files changed, 673 insertions(+), 167 deletions(-)
>
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PULL 0/8] Block patches
@ 2022-04-20 12:40 Hanna Reitz
2022-04-20 18:12 ` Richard Henderson
0 siblings, 1 reply; 17+ messages in thread
From: Hanna Reitz @ 2022-04-20 12:40 UTC (permalink / raw)
To: qemu-block; +Cc: Peter Maydell, Hanna Reitz, Richard Henderson, qemu-devel
The following changes since commit 1be5a765c08cee3a9587c8a8d3fc2ea247b13f9c:
Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging (2022-04-19 18:22:16 -0700)
are available in the Git repository at:
https://gitlab.com/hreitz/qemu.git tags/pull-block-2022-04-20
for you to fetch changes up to 0423f75351ab83b844a31349218b0eadd830e07a:
qcow2: Add errp to rebuild_refcount_structure() (2022-04-20 12:09:17 +0200)
----------------------------------------------------------------
Block patches:
- Some changes for qcow2's refcount repair algorithm to make it work for
qcow2 images stored on block devices
- Skip test cases that require zstd when support for it is missing
- Some refactoring in the iotests' meson.build
----------------------------------------------------------------
Hanna Reitz (6):
iotests.py: Add supports_qcow2_zstd_compression()
iotests/065: Check for zstd support
iotests/303: Check for zstd support
qcow2: Improve refcount structure rebuilding
iotests/108: Test new refcount rebuild algorithm
qcow2: Add errp to rebuild_refcount_structure()
Thomas Huth (2):
tests/qemu-iotests/meson.build: Improve the indentation
tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
block/qcow2-refcount.c | 353 +++++++++++++++++++++++----------
tests/check-block.sh | 26 ---
tests/qemu-iotests/065 | 24 ++-
tests/qemu-iotests/108 | 259 +++++++++++++++++++++++-
tests/qemu-iotests/108.out | 81 ++++++++
tests/qemu-iotests/303 | 4 +-
tests/qemu-iotests/iotests.py | 20 ++
tests/qemu-iotests/meson.build | 73 ++++---
8 files changed, 673 insertions(+), 167 deletions(-)
--
2.35.1
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [PULL 0/8] Block patches
2021-05-24 13:01 Stefan Hajnoczi
@ 2021-05-24 18:01 ` Peter Maydell
0 siblings, 0 replies; 17+ messages in thread
From: Peter Maydell @ 2021-05-24 18:01 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Kevin Wolf, Elena Ufimtseva, Vladimir Sementsov-Ogievskiy,
Jagannathan Raman, Qemu-block, John G Johnson, John Snow,
QEMU Developers, Max Reitz
On Mon, 24 May 2021 at 14:02, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> The following changes since commit 6c769690ac845fa62642a5f93b4e4bd906adab95:
>
> Merge remote-tracking branch 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging (2021-05-21 12:02:34 +0100)
>
> are available in the Git repository at:
>
> https://gitlab.com/stefanha/qemu.git tags/block-pull-request
>
> for you to fetch changes up to 0a6f0c76a030710780ce10d6347a70f098024d21:
>
> coroutine-sleep: introduce qemu_co_sleep (2021-05-21 18:22:33 +0100)
>
> ----------------------------------------------------------------
> Pull request
>
> (Resent due to an email preparation mistake.)
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/6.1
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 17+ messages in thread
* [PULL 0/8] Block patches
@ 2021-05-24 13:01 Stefan Hajnoczi
2021-05-24 18:01 ` Peter Maydell
0 siblings, 1 reply; 17+ messages in thread
From: Stefan Hajnoczi @ 2021-05-24 13:01 UTC (permalink / raw)
To: Peter Maydell, qemu-devel
Cc: Kevin Wolf, Elena Ufimtseva, Vladimir Sementsov-Ogievskiy,
Jagannathan Raman, qemu-block, John G Johnson, John Snow,
Max Reitz, Stefan Hajnoczi
The following changes since commit 6c769690ac845fa62642a5f93b4e4bd906adab95:
Merge remote-tracking branch 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging (2021-05-21 12:02:34 +0100)
are available in the Git repository at:
https://gitlab.com/stefanha/qemu.git tags/block-pull-request
for you to fetch changes up to 0a6f0c76a030710780ce10d6347a70f098024d21:
coroutine-sleep: introduce qemu_co_sleep (2021-05-21 18:22:33 +0100)
----------------------------------------------------------------
Pull request
(Resent due to an email preparation mistake.)
----------------------------------------------------------------
Paolo Bonzini (6):
coroutine-sleep: use a stack-allocated timer
coroutine-sleep: disallow NULL QemuCoSleepState** argument
coroutine-sleep: allow qemu_co_sleep_wake that wakes nothing
coroutine-sleep: move timer out of QemuCoSleepState
coroutine-sleep: replace QemuCoSleepState pointer with struct in the
API
coroutine-sleep: introduce qemu_co_sleep
Philippe Mathieu-Daudé (1):
bitops.h: Improve find_xxx_bit() documentation
Zenghui Yu (1):
multi-process: Initialize variables declared with g_auto*
include/qemu/bitops.h | 15 ++++++--
include/qemu/coroutine.h | 27 ++++++++-----
block/block-copy.c | 10 ++---
block/nbd.c | 14 +++----
hw/remote/memory.c | 5 +--
hw/remote/proxy.c | 3 +-
util/qemu-coroutine-sleep.c | 75 +++++++++++++++++++------------------
7 files changed, 79 insertions(+), 70 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-08-24 16:58 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-19 8:00 [PULL 0/8] Block patches Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 1/8] tests/fuzz/Makefile: Do not link code using unavailable devices Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 2/8] Makefile: List fuzz targets in 'make help' Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 3/8] tests/fuzz: Add missing space in test description Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 4/8] tests/fuzz: Remove unuseful/unused typedefs Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 5/8] tests/fuzz: Extract pciconfig_fuzz_qos() method Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 6/8] tests/fuzz: Extract ioport_fuzz_qtest() method Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 7/8] aio-posix: don't duplicate fd handler deletion in fdmon_io_uring_destroy() Stefan Hajnoczi
2020-05-19 8:00 ` [PULL 8/8] aio-posix: disable fdmon-io_uring when GSource is used Stefan Hajnoczi
2020-05-19 15:15 ` [PULL 0/8] Block patches Peter Maydell
2021-05-24 13:01 Stefan Hajnoczi
2021-05-24 18:01 ` Peter Maydell
2022-04-20 12:40 Hanna Reitz
2022-04-20 18:12 ` Richard Henderson
2023-06-01 15:25 Stefan Hajnoczi
2023-06-01 21:29 ` Richard Henderson
2023-08-24 16:58 Stefan Hajnoczi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.