archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 0/2] nvmet: add polling support
@ 2019-12-10  6:25 Chaitanya Kulkarni
  2019-12-10  6:25 ` [RFC PATCH 1/2] nvmet: add bdev-ns " Chaitanya Kulkarni
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Chaitanya Kulkarni @ 2019-12-10  6:25 UTC (permalink / raw)
  To: sagi, hch; +Cc: Chaitanya Kulkarni, linux-nvme

Hi Christoph/Sagi/Keith,

This patch series implements the polling support for the NVMeOF target
with maintaining separate kthread for polling.

By implementing the polling for the file and bdev I was able to create
stubs for the target polling which will be useful for passthru code.

1. General approach:-

After submitting the request in (nvmet_file|bdev_execute_rw()) code path
we issue request on the available polling thread's list (poll_list)
which is shared between submission thread and the polling thread. We
create per CPU two polling threads to improve the scalability for I/O

When polling thread is woken up by the submission thread it moves the
request to its private list (done list) for processing if it finds
completed request (by the backend bdev/file). This allows outstanding
submission requests to make forward progress under pressure.

When there are no completed requests on the poll_list then polling
thread removes the request from the poll_list and calls backend
specific poll function on this request until it gets the completion.
On completion, it finishes the request in the same context over

2. Testing and verification:-

I did fio verification tests for a couple of days and didn't find
any issues with data verification. Also, for bdev and file I could see
the io_poll stats under debugfs with considered/invoked/success numbers
being increased.

3. Performance improvements:-

There are still some performance improvements which can be done.

4. User interface:-

Users can optionally enable/disable polling with newly added configfs
attribute use_poll. This only enables polling in case backend
(bdev/file) supports it otherwise we just ignore this value.

It will be great if you can provide some feedback.

I'm aware of the io-wq code for io_using posted on the mailing list.
I was just wondering should we explore that to implement the polling ?

Patchset info :-

Repo:-          git://
Bramch :-       nvme/for-5.5

commit 7e4c6b9a5d22485acf009b3c3510a370f096dd54 (origin/nvme/for-5.5)
Author: Keith Busch <>
Date:   Fri Dec 6 08:11:17 2019 +0900

    nvme/pci: Fix read queue count


Chaitanya Kulkarni (2):
  nvmet: add bdev-ns polling support
  nvmet: add file-ns polling support

 drivers/nvme/target/Makefile      |   3 +-
 drivers/nvme/target/configfs.c    |  29 ++++++
 drivers/nvme/target/core.c        |  17 +++
 drivers/nvme/target/io-cmd-bdev.c |  61 +++++++++--
 drivers/nvme/target/io-cmd-file.c |  60 +++++++++--
 drivers/nvme/target/io-poll.c     | 165 ++++++++++++++++++++++++++++++
 drivers/nvme/target/nvmet.h       |  40 +++++++-
 7 files changed, 354 insertions(+), 21 deletions(-)
 create mode 100644 drivers/nvme/target/io-poll.c


linux-nvme mailing list

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2020-01-30 18:20 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-12-10  6:25 [RFC PATCH 0/2] nvmet: add polling support Chaitanya Kulkarni
2019-12-10  6:25 ` [RFC PATCH 1/2] nvmet: add bdev-ns " Chaitanya Kulkarni
2020-01-20 12:52   ` Max Gurtovoy
2020-01-21 19:22     ` Chaitanya Kulkarni
2020-01-23 14:23       ` Max Gurtovoy
2020-01-30 18:19         ` Chaitanya Kulkarni
2019-12-10  6:25 ` [RFC PATCH 2/2] nvmet: add file-ns " Chaitanya Kulkarni
2019-12-12  1:01 ` [RFC PATCH 0/2] nvmet: add " Sagi Grimberg
2019-12-12  5:44   ` Chaitanya Kulkarni
2019-12-12 20:32     ` Sagi Grimberg
2020-01-20  5:13       ` Chaitanya Kulkarni
2020-01-20  4:48   ` Chaitanya Kulkarni

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).