[PATCH] nvme-core: introduce the preferential local retry mechanism

[PATCH] nvme-core: introduce the preferential local retry mechanism

[Chao Leng]

We need a mechanism:the host retry io in the current path first after
a defined time in some scenarios. Because if work with dm-multipath or
other multipath software, the multipath software will set cmd_flags
as REQ_FAILFAST_TRANSPORT. If io return with any error, nvme will return
io error to block layer now, the multipath will set the path fault and
retry in other path, but realy need retry io after a defined time
in the current path in some scenarios.

So we need introduce nvme_req_local_retry to check if need retry
in the current path, if needed, do not need checking the cmd_flags
through blk_noretry_request. According the protocol define,
the error code NVME_SC_CMD_INTERRUPTED is a good choice.

Scenarios which need retry io after a defined time in the current path:

Scenario 1: user configures a QoS policy on the storage system.
Storage system need to tell host to retry io after a defined time
when need control QoS. Host should retry the io in the current path
instead of return io to block layer.

Scenario 2:many hosts(more than 100) access the same storage device.
I/Os are not delivered at the same time in most cases,however a large
number of I/Os may be burst delivered at the same time sometimes.
In this case, the storage device can not treat all I/Os, needs host
retry in the current path after a defined time.

Scenario 3: storage software upgrade inline. The storage system may need
host retry in the current path after software upgrade completed.

Signed-off-by: Chao Leng <lengchao@huawei.com>
---
 drivers/nvme/host/core.c | 2 +-
 drivers/nvme/host/nvme.h | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 1a1ad5e5212c..3d39528051ad 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -242,7 +242,7 @@ static blk_status_t nvme_error_status(u16 status)
 
 static inline bool nvme_req_needs_retry(struct request *req)
 {
-	if (blk_noretry_request(req))
+	if (!nvme_req_local_retry(req) && blk_noretry_request(req))
 		return false;
 	if (nvme_req(req)->status & NVME_SC_DNR)
 		return false;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index c0f4226d3299..17064455f2db 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -710,4 +710,12 @@ void nvme_hwmon_init(struct nvme_ctrl *ctrl);
 static inline void nvme_hwmon_init(struct nvme_ctrl *ctrl) { }
 #endif
 
+static inline bool nvme_req_local_retry(struct request *req)
+{
+	if (nvme_req(req)->status == NVME_SC_CMD_INTERRUPTED)
+		return true;
+	return false;
+}
+
+
 #endif /* _NVME_H */
-- 
2.16.4


_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [PATCH] nvme-core: introduce the preferential local retry mechanism

[Christoph Hellwig]

If the callers asks for failfast, the caller will get failfast,
period.

FYI, I would absolutely recommend against ever using dm-multipath
with nvme.  This is not a supported setup by the nvme maintainers.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [PATCH] nvme-core: introduce the preferential local retry mechanism

[Chao Leng]

On 2020/7/28 19:20, Christoph Hellwig wrote:
> If the callers asks for failfast, the caller will get failfast,
> period.If user issue cmd through ioctl, nvme core will set the cmd_flags:
REQ_FAILFAST_DRIVER, and then in some scenarios which need retry io
after a defined time, we will return io error, this is not what we
expected. So we need introduce the preferential local retry mechanism.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [PATCH] nvme-core: introduce the preferential local retry mechanism

[Chao Leng]

On 2020/7/28 19:20, Christoph Hellwig wrote:
> If the callers asks for failfast, the caller will get failfast,
> period.
> 
If user issue cmd through ioctrl, nvme core will set cmd_flags:
REQ_FAILFAST_DRIVER, and then in some scenarios which need retry io
after a defined time, we will return io error, this is not what we
expected. So we need introduce the preferential local retry mechanism.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [PATCH] nvme-core: introduce the preferential local retry mechanism

[Keith Busch]

On Wed, Jul 29, 2020 at 10:06:43AM +0800, Chao Leng wrote:
> 
> 
> On 2020/7/28 19:20, Christoph Hellwig wrote:
> > If the callers asks for failfast, the caller will get failfast,
> > period.
> > 
> If user issue cmd through ioctrl, nvme core will set cmd_flags:
> REQ_FAILFAST_DRIVER, and then in some scenarios which need retry io
> after a defined time, we will return io error, this is not what we
> expected. So we need introduce the preferential local retry mechanism.

Userspace owns error handling with ioctl passthrough commands. The
interface exists for unfiltered access to the device; the driver will
return status exactly as the device provided it. If userspace wants to
retry an error'ed passthrough, it is free to do so.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme