[PATCHSET v2] Support for polled aio

[Jens Axboe <axboe@kernel.dk>]

For the grand introduction to this feature, see my original posting
here:

https://lore.kernel.org/linux-block/20181117235317.7366-1-axboe@kernel.dk/

Since last time, a number of significant changes has been made:

- Addition of the io_setup2() system call. I didn't like tracking the
  fact that an io_context was polled or not internally, I think it's
  much better to make this explicit. The system call is like io_setup(),
  except we pass in a flags argument. In the flags, the application can
  set IOCTX_FLAG_IOPOLL to ask for this context to be polled. The syscall
  is only wired up for x86-64 for now.

- Addition of fops->iopoll() to properly handle polling on files.

- Various optimizations to how we track and handle poll requests
  internally.

- Split some patches into prep patches, to reduce the size of the
  final poll support patch.

- Various cleanups, tweaks, improvements.

In terms of efficiency, I've got one device that maxes out at 845K
4k IOPS, and the polled IO can reach this with just a single thread/core.
For comparison, non-polled on the same device reaches 600K IOPS. On
a slightly faster device, I can reach 910K IOPS on a single thread. All
of this is on a 2.2GHz server box, faster clock rate will go faster.

In terms of how to use polling, I'd advise you to check out the basic
changes made to fio to support it. It's simple:

1) Ensure that you call io_setup2() with IOCTX_FLAG_IOPOLL, instead
   of io_queue_init() or io_setup().
2) Set iocb->u.c.flags to IOCB_FLAG_HIPRI.

That's it, everything else works like before.

Patches are against my mq-perf branch, and can also be found in my
aio poll branch.

 arch/x86/entry/syscalls/syscall_64.tbl |   1 +
 fs/aio.c                               | 469 +++++++++++++++++++++----
 fs/block_dev.c                         |  25 +-
 fs/iomap.c                             |  50 ++-
 fs/xfs/xfs_file.c                      |   1 +
 include/linux/fs.h                     |   1 +
 include/linux/iomap.h                  |   1 +
 include/linux/syscalls.h               |   2 +
 include/uapi/asm-generic/unistd.h      |   4 +-
 include/uapi/linux/aio_abi.h           |   4 +
 kernel/sys_ni.c                        |   1 +
 11 files changed, 461 insertions(+), 98 deletions(-)

-- 
Jens Axboe