From: Kanchan Joshi <joshi.k@samsung.com>
To: axboe@kernel.dk, hch@lst.de
Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org,
linux-block@vger.kernel.org, ming.lei@redhat.com,
joshiiitr@gmail.com, gost.dev@samsung.com,
Kanchan Joshi <joshi.k@samsung.com>
Subject: [PATCH 0/4] iopoll support for io_uring/nvme passthrough
Date: Fri, 5 Aug 2022 21:12:22 +0530 [thread overview]
Message-ID: <20220805154226.155008-1-joshi.k@samsung.com> (raw)
In-Reply-To: CGME20220805155300epcas5p1b98722e20990d0095238964e2be9db34@epcas5p1.samsung.com
Hi,
Series enables async polling on io_uring command, and nvme passthrough
(for io-commands) is wired up to leverage that.
512b randread performance (KIOP) below:
QD_batch block passthru passthru-poll block-poll
1_1 80 81 158 157
8_2 406 470 680 700
16_4 620 656 931 920
128_32 879 1056 1120 1132
Polling is giving the clear win here.
Upstream fio is used for testing.
passthru command line:
fio -iodepth=64 -rw=randread -ioengine=io_uring_cmd -bs=512 -numjobs=1
-runtime=60 -group_reporting -iodepth_batch_submit=16
-iodepth_batch_complete_min=1 -iodepth_batch_complete_max=16
-cmd_type=nvme -hipri=0 -filename=/dev/ng1n1 -name=io_uring_cmd_64
block command line:
fio -direct=1 -iodepth=64 -rw=randread -ioengine=io_uring -bs=512
-numjobs=1 -runtime=60 -group_reporting -iodepth_batch_submit=16
-iodepth_batch_complete_min=1 -iodepth_batch_complete_max=16
-hipri=0 -filename=/dev/nvme1n1 name=io_uring_64
Bit of code went into non-passthrough path for io_uring (patch 2) but I
do not see that causing any performance regression.
peak-perf test showed 2.3M IOPS with or without this series for
block-io.
io_uring: Running taskset -c 0,12 t/io_uring -b512 -d128 -c32 -s32 -p1
-F1 -B1 -n2 /dev/nvme0n1
submitter=0, tid=3089, file=/dev/nvme0n1, node=-1
submitter=1, tid=3090, file=/dev/nvme0n1, node=-1
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128
Engine=io_uring, sq_ring=128, cq_ring=128
polled=1, fixedbufs=1/0, register_files=1, buffered=0, QD=128
Engine=io_uring, sq_ring=128, cq_ring=128
IOPS=2.31M, BW=1126MiB/s, IOS/call=31/31
IOPS=2.30M, BW=1124MiB/s, IOS/call=32/31
IOPS=2.30M, BW=1123MiB/s, IOS/call=32/32
Kanchan Joshi (4):
fs: add file_operations->uring_cmd_iopoll
io_uring: add iopoll infrastructure for io_uring_cmd
block: export blk_rq_is_poll
nvme: wire up async polling for io passthrough commands
block/blk-mq.c | 3 +-
drivers/nvme/host/core.c | 1 +
drivers/nvme/host/ioctl.c | 73 ++++++++++++++++++++++++++++++++---
drivers/nvme/host/multipath.c | 1 +
drivers/nvme/host/nvme.h | 2 +
include/linux/blk-mq.h | 1 +
include/linux/fs.h | 1 +
include/linux/io_uring.h | 8 +++-
io_uring/io_uring.c | 6 +++
io_uring/opdef.c | 1 +
io_uring/rw.c | 8 +++-
io_uring/uring_cmd.c | 11 +++++-
12 files changed, 105 insertions(+), 11 deletions(-)
--
2.25.1
next parent reply other threads:[~2022-08-05 15:55 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20220805155300epcas5p1b98722e20990d0095238964e2be9db34@epcas5p1.samsung.com>
2022-08-05 15:42 ` Kanchan Joshi [this message]
[not found] ` <CGME20220805155304epcas5p1bb687a8f9b25317af39def01696626e8@epcas5p1.samsung.com>
2022-08-05 15:42 ` [PATCH 1/4] fs: add file_operations->uring_cmd_iopoll Kanchan Joshi
[not found] ` <CGME20220805155307epcas5p4bab3f05dc13d8fc2f03c7a26e9bd8c7c@epcas5p4.samsung.com>
2022-08-05 15:42 ` [PATCH 2/4] io_uring: add iopoll infrastructure for io_uring_cmd Kanchan Joshi
[not found] ` <CGME20220805155310epcas5p2bd7ec5b9bee73893958f4bc84038eca0@epcas5p2.samsung.com>
2022-08-05 15:42 ` [PATCH 3/4] block: export blk_rq_is_poll Kanchan Joshi
[not found] ` <CGME20220805155313epcas5p2d35d22831bd07ef33fbdc28bd99ae1d0@epcas5p2.samsung.com>
2022-08-05 15:42 ` [PATCH 4/4] nvme: wire up async polling for io passthrough commands Kanchan Joshi
2022-08-05 17:03 ` Jens Axboe
2022-08-05 17:07 ` Kanchan Joshi
2022-08-05 21:22 ` kernel test robot
2022-08-06 0:06 ` kernel test robot
2022-08-06 1:38 ` kernel test robot
2022-08-07 12:25 ` kernel test robot
2022-08-05 17:04 ` [PATCH 0/4] iopoll support for io_uring/nvme passthrough Jens Axboe
2022-08-05 17:13 ` Kanchan Joshi
2022-08-05 17:27 ` Jens Axboe
2022-08-05 17:18 ` Jens Axboe
2022-08-05 18:11 ` Keith Busch
2022-08-05 18:15 ` Jens Axboe
2022-08-07 17:58 ` Kanchan Joshi
2022-08-07 18:46 ` Jens Axboe
2022-08-05 18:21 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220805154226.155008-1-joshi.k@samsung.com \
--to=joshi.k@samsung.com \
--cc=axboe@kernel.dk \
--cc=gost.dev@samsung.com \
--cc=hch@lst.de \
--cc=io-uring@vger.kernel.org \
--cc=joshiiitr@gmail.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=ming.lei@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.