qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 00/15] io_uring: add Linux io_uring AIO engine
@ 2019-10-25 16:04 Stefan Hajnoczi
  2019-10-25 16:04 ` [PATCH v2 01/15] configure: permit use of io_uring Stefan Hajnoczi
                   ` (15 more replies)
  0 siblings, 16 replies; 27+ messages in thread
From: Stefan Hajnoczi @ 2019-10-25 16:04 UTC (permalink / raw)
  To: qemu-devel
  Cc: Fam Zheng, oleksandr, qemu-block, Julia Suvorova,
	Markus Armbruster, Max Reitz, Stefan Hajnoczi, Paolo Bonzini,
	Kevin Wolf, Aarushi Mehta

v11:
 * Drop fd registration because it breaks QEMU's file locking and will need to
   be resolved in a separate patch series
 * Drop line-wrapping changes that accidentally broke several qemu-iotests

v10:
 * Dropped kernel submission queue polling, it requires root and has additional
   limitations.  It should be benchmarked and considered for inclusion later,
   maybe even together with kernel side changes.
 * Add io_uring_register_files() return value to trace_luring_fd_register()
 * Fix indentation in luring_fd_unregister()
 * Set s->fd_reg.fd_array to NULL after g_free() to avoid dangling pointers
 * Simplify fd registration code
 * Add luring_fd_unregister() and call it from file-posix.c to prevent
   fd leaks
 * Add trace_luring_fd_unregister() trace event
 * Add missing space to qemu-img command-line documentation
 * Update MAINTAINERS file [Julia]
 * Rename MAX_EVENTS to MAX_ENTRIES [Julia]
 * Define ioq_submit() before callers so the prototype isn't necessary [Julia]
 * Declare variables at the beginning of the block in luring_init() [Julia]

This patch series is based on Aarushi Mehta's v9 patch series written for
Google Summer of Code 2019:

  https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg00179.html

It adds a new AIO engine that uses the new Linux io_uring API.  This is the
successor to Linux AIO with a number of improvements:
1. Both O_DIRECT and buffered I/O work
2. fdatasync(2) is supported (no need for a separate thread pool!)
3. True async behavior so the syscall doesn't block (Linux AIO got there to some degree...)
4. Advanced performance optimizations are available (file registration, memory
   buffer registration, completion polling, submission polling).

Since Aarushi has been busy, I have taken up this patch series.  Booting a
guest works with -drive aio=io_uring and -drive aio=io_uring,cache=none with a
raw file on XFS.

I currently recommend using -drive aio=io_uring only with host block devices
(like NVMe devices).  As of Linux v5.4-rc1 I still hit kernel bugs when using
image files on ext4 or XFS.

Aarushi Mehta (15):
  configure: permit use of io_uring
  qapi/block-core: add option for io_uring
  block/block: add BDRV flag for io_uring
  block/io_uring: implements interfaces for io_uring
  stubs: add stubs for io_uring interface
  util/async: add aio interfaces for io_uring
  blockdev: adds bdrv_parse_aio to use io_uring
  block/file-posix.c: extend to use io_uring
  block: add trace events for io_uring
  block/io_uring: adds userspace completion polling
  qemu-io: adds option to use aio engine
  qemu-img: adds option to use aio engine for benchmarking
  qemu-nbd: adds option for aio engines
  tests/qemu-iotests: enable testing with aio options
  tests/qemu-iotests: use AIOMODE with various tests

 MAINTAINERS                   |   9 +
 qapi/block-core.json          |   4 +-
 configure                     |  27 +++
 block/Makefile.objs           |   3 +
 stubs/Makefile.objs           |   1 +
 include/block/aio.h           |  16 +-
 include/block/block.h         |   2 +
 include/block/raw-aio.h       |  12 +
 block.c                       |  22 ++
 block/file-posix.c            |  99 ++++++--
 block/io_uring.c              | 433 ++++++++++++++++++++++++++++++++++
 blockdev.c                    |  12 +-
 qemu-img.c                    |  11 +-
 qemu-io.c                     |  25 +-
 qemu-nbd.c                    |  12 +-
 stubs/io_uring.c              |  32 +++
 util/async.c                  |  36 +++
 block/trace-events            |  12 +
 qemu-img-cmds.hx              |   4 +-
 qemu-img.texi                 |   5 +-
 qemu-nbd.texi                 |   4 +-
 tests/qemu-iotests/028        |   2 +-
 tests/qemu-iotests/058        |   2 +-
 tests/qemu-iotests/089        |   4 +-
 tests/qemu-iotests/091        |   4 +-
 tests/qemu-iotests/109        |   2 +-
 tests/qemu-iotests/147        |   5 +-
 tests/qemu-iotests/181        |   8 +-
 tests/qemu-iotests/183        |   4 +-
 tests/qemu-iotests/185        |  10 +-
 tests/qemu-iotests/200        |   2 +-
 tests/qemu-iotests/201        |   8 +-
 tests/qemu-iotests/check      |  15 +-
 tests/qemu-iotests/common.rc  |  14 ++
 tests/qemu-iotests/iotests.py |  12 +-
 35 files changed, 797 insertions(+), 76 deletions(-)
 create mode 100644 block/io_uring.c
 create mode 100644 stubs/io_uring.c

-- 
2.21.0



^ permalink raw reply	[flat|nested] 27+ messages in thread

end of thread, other threads:[~2019-12-03 16:05 UTC | newest]

Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-10-25 16:04 [PATCH v2 00/15] io_uring: add Linux io_uring AIO engine Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 01/15] configure: permit use of io_uring Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 02/15] qapi/block-core: add option for io_uring Stefan Hajnoczi
2019-10-25 19:09   ` Markus Armbruster
2019-10-25 16:04 ` [PATCH v2 03/15] block/block: add BDRV flag " Stefan Hajnoczi
2019-11-11 10:57   ` Kevin Wolf
2019-11-11 16:25     ` Max Reitz
2019-11-13 11:42       ` Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 04/15] block/io_uring: implements interfaces " Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 05/15] stubs: add stubs for io_uring interface Stefan Hajnoczi
2019-11-11 11:13   ` Kevin Wolf
2019-11-13 11:42     ` Stefan Hajnoczi
2019-12-03 11:16     ` Stefan Hajnoczi
2019-12-03 14:28       ` Kevin Wolf
2019-12-03 15:55         ` Paolo Bonzini
2019-10-25 16:04 ` [PATCH v2 06/15] util/async: add aio interfaces for io_uring Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 07/15] blockdev: adds bdrv_parse_aio to use io_uring Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 08/15] block/file-posix.c: extend " Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 09/15] block: add trace events for io_uring Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 10/15] block/io_uring: adds userspace completion polling Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 11/15] qemu-io: adds option to use aio engine Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 12/15] qemu-img: adds option to use aio engine for benchmarking Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 13/15] qemu-nbd: adds option for aio engines Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 14/15] tests/qemu-iotests: enable testing with aio options Stefan Hajnoczi
2019-10-25 16:04 ` [PATCH v2 15/15] tests/qemu-iotests: use AIOMODE with various tests Stefan Hajnoczi
2019-11-04 10:32 ` [PATCH v2 00/15] io_uring: add Linux io_uring AIO engine Stefan Hajnoczi
2019-11-13 11:43   ` Stefan Hajnoczi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).