* [PATCH] nvme-core: mark passthru requests RQF_QUIET flag @ 2022-04-06 16:41 Chaitanya Kulkarni 2022-04-06 16:52 ` Keith Busch 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-06 16:41 UTC (permalink / raw) To: linux-nvme; +Cc: hch, kbusch, alan.adamson, Chaitanya Kulkarni From: Christoph Hellwig <hch@lst.de> Since addition of the nvme_log_error() we are getting error message when running block tests framework due to internal passthru commands :- [ 612.754938] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1" [ 616.361730] run blktests nvme/012 at 2022-04-06 09:26:43 [ 616.382902] loop0: detected capacity change from 0 to 2097152 [ 616.392680] nvmet: adding nsid 1 to subsystem blktests-subsystem-1 [ 616.400913] nvmet: creating nvm controller 1 for subsystem blktests-subsystem-1 for NQN testhostnqn. [ 616.401001] nvme1: Identify(0x6), Invalid Field in Command (sct 0x0 / sc 0x2) MORE DNR [ 627.427947] nvmet: adding nsid 1 to subsystem blktests-subsystem-1 [ 627.437084] nvmet: creating nvm controller 1 for subsystem blktests-subsystem-1 for NQN testhostnqn. [ 627.437161] nvme1: Identify(0x6), Invalid Field in Command (sct 0x0 / sc 0x2) MORE DNR [ 627.438984] nvme nvme1: creating 48 I/O queues. [ 627.442620] nvme nvme1: new ctrl: "blktests-subsystem-1" [ 628.506885] XFS (nvme1n1): Mounting V5 Filesystem [ 628.516895] XFS (nvme1n1): Ending clean mount [ 628.519966] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff) [ 704.852721] XFS (nvme1n1): Unmounting Filesystem [ 704.864724] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1" This patch adds the passthru request with RQF_QUIET flag and ignores the error reporting for nvme_log_error() if RQF_QUIET flag is set when request is failed. With this patch we don't get the above error message. Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> [kch: make a formal patch & test with blktests] Signed-off-by: Christoph Hellwig <hch@lst.de> --- drivers/nvme/host/core.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f204c6f78b5b..b913a89c743e 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) { blk_status_t status = nvme_error_status(nvme_req(req)->status); - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && + !(req->rq_flags & RQF_QUIET))) nvme_log_error(req); nvme_end_req_zoned(req); nvme_trace_bio_complete(req); @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) cmd->common.flags &= ~NVME_CMD_SGL_ALL; req->cmd_flags |= REQ_FAILFAST_DRIVER; + req->rq_flags |= RQF_QUIET; if (req->mq_hctx->type == HCTX_TYPE_POLL) req->cmd_flags |= REQ_POLLED; nvme_clear_nvme_request(req); -- 2.29.0 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 16:41 [PATCH] nvme-core: mark passthru requests RQF_QUIET flag Chaitanya Kulkarni @ 2022-04-06 16:52 ` Keith Busch 2022-04-06 17:01 ` Chaitanya Kulkarni 2022-04-06 17:03 ` Chaitanya Kulkarni 0 siblings, 2 replies; 23+ messages in thread From: Keith Busch @ 2022-04-06 16:52 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: linux-nvme, hch, alan.adamson On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: > @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) > { > blk_status_t status = nvme_error_status(nvme_req(req)->status); > > - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) > + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && > + !(req->rq_flags & RQF_QUIET))) > nvme_log_error(req); > nvme_end_req_zoned(req); > nvme_trace_bio_complete(req); > @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) > cmd->common.flags &= ~NVME_CMD_SGL_ALL; > > req->cmd_flags |= REQ_FAILFAST_DRIVER; > + req->rq_flags |= RQF_QUIET; This defeats the admin error logging logic since every admin command comes through here. If you're sure we should do this, then I suppose you can remove that unreachable code. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 16:52 ` Keith Busch @ 2022-04-06 17:01 ` Chaitanya Kulkarni 2022-04-06 17:21 ` Keith Busch 2022-04-06 17:03 ` Chaitanya Kulkarni 1 sibling, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-06 17:01 UTC (permalink / raw) To: Keith Busch; +Cc: linux-nvme, hch, alan.adamson, Chaitanya Kulkarni On 4/6/22 09:52, Keith Busch wrote: > On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: >> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) >> { >> blk_status_t status = nvme_error_status(nvme_req(req)->status); >> >> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && >> + !(req->rq_flags & RQF_QUIET))) >> nvme_log_error(req); >> nvme_end_req_zoned(req); >> nvme_trace_bio_complete(req); >> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) >> cmd->common.flags &= ~NVME_CMD_SGL_ALL; >> >> req->cmd_flags |= REQ_FAILFAST_DRIVER; >> + req->rq_flags |= RQF_QUIET; > > This defeats the admin error logging logic since every admin command comes > through here. If you're sure we should do this, then I suppose you can remove > that unreachable code. If you point out the unreachable code that will be great, I'll keep looking meanwhile... -ck ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 17:01 ` Chaitanya Kulkarni @ 2022-04-06 17:21 ` Keith Busch 2022-04-06 22:06 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Keith Busch @ 2022-04-06 17:21 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: linux-nvme, hch, alan.adamson On Wed, Apr 06, 2022 at 05:01:46PM +0000, Chaitanya Kulkarni wrote: > On 4/6/22 09:52, Keith Busch wrote: > > On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: > >> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) > >> { > >> blk_status_t status = nvme_error_status(nvme_req(req)->status); > >> > >> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) > >> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && > >> + !(req->rq_flags & RQF_QUIET))) > >> nvme_log_error(req); > >> nvme_end_req_zoned(req); > >> nvme_trace_bio_complete(req); > >> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) > >> cmd->common.flags &= ~NVME_CMD_SGL_ALL; > >> > >> req->cmd_flags |= REQ_FAILFAST_DRIVER; > >> + req->rq_flags |= RQF_QUIET; > > > > This defeats the admin error logging logic since every admin command comes > > through here. If you're sure we should do this, then I suppose you can remove > > that unreachable code. > > If you point out the unreachable code that will be great, > I'll keep looking meanwhile... The second half of nvme_log_error(), plus nvme_get_admin_opcode_str() and the array it defines are unreachable since all admin commands don't log errors with this change. You could skip the RQF_QUIET setting and check blk_rq_is_passthrough() instead. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 17:21 ` Keith Busch @ 2022-04-06 22:06 ` Alan Adamson 2022-04-06 22:16 ` Chaitanya Kulkarni 2022-04-07 8:51 ` hch 0 siblings, 2 replies; 23+ messages in thread From: Alan Adamson @ 2022-04-06 22:06 UTC (permalink / raw) To: Keith Busch; +Cc: Chaitanya Kulkarni, linux-nvme, hch > On Apr 6, 2022, at 10:21 AM, Keith Busch <kbusch@kernel.org> wrote: > > On Wed, Apr 06, 2022 at 05:01:46PM +0000, Chaitanya Kulkarni wrote: >> On 4/6/22 09:52, Keith Busch wrote: >>> On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: >>>> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) >>>> { >>>> blk_status_t status = nvme_error_status(nvme_req(req)->status); >>>> >>>> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >>>> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && >>>> + !(req->rq_flags & RQF_QUIET))) >>>> nvme_log_error(req); >>>> nvme_end_req_zoned(req); >>>> nvme_trace_bio_complete(req); >>>> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) >>>> cmd->common.flags &= ~NVME_CMD_SGL_ALL; >>>> >>>> req->cmd_flags |= REQ_FAILFAST_DRIVER; >>>> + req->rq_flags |= RQF_QUIET; >>> >>> This defeats the admin error logging logic since every admin command comes >>> through here. If you're sure we should do this, then I suppose you can remove >>> that unreachable code. >> >> If you point out the unreachable code that will be great, >> I'll keep looking meanwhile... > > The second half of nvme_log_error(), plus nvme_get_admin_opcode_str() and the > array it defines are unreachable since all admin commands don't log errors with > this change. > > You could skip the RQF_QUIET setting and check blk_rq_is_passthrough() instead. Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? Alan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 22:06 ` Alan Adamson @ 2022-04-06 22:16 ` Chaitanya Kulkarni 2022-04-06 23:29 ` Alan Adamson 2022-04-07 8:48 ` Christoph Hellwig 2022-04-07 8:51 ` hch 1 sibling, 2 replies; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-06 22:16 UTC (permalink / raw) To: Alan Adamson, Keith Busch, Christoph Hellwig, Hannes Reinecke, Sagi Grimberg Cc: linux-nvme, hch On 4/6/22 15:06, Alan Adamson wrote: > > >> On Apr 6, 2022, at 10:21 AM, Keith Busch <kbusch@kernel.org> wrote: >> >> On Wed, Apr 06, 2022 at 05:01:46PM +0000, Chaitanya Kulkarni wrote: >>> On 4/6/22 09:52, Keith Busch wrote: >>>> On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: >>>>> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) >>>>> { >>>>> blk_status_t status = nvme_error_status(nvme_req(req)->status); >>>>> >>>>> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >>>>> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && >>>>> + !(req->rq_flags & RQF_QUIET))) >>>>> nvme_log_error(req); >>>>> nvme_end_req_zoned(req); >>>>> nvme_trace_bio_complete(req); >>>>> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) >>>>> cmd->common.flags &= ~NVME_CMD_SGL_ALL; >>>>> >>>>> req->cmd_flags |= REQ_FAILFAST_DRIVER; >>>>> + req->rq_flags |= RQF_QUIET; >>>> >>>> This defeats the admin error logging logic since every admin command comes >>>> through here. If you're sure we should do this, then I suppose you can remove >>>> that unreachable code. >>> >>> If you point out the unreachable code that will be great, >>> I'll keep looking meanwhile... >> >> The second half of nvme_log_error(), plus nvme_get_admin_opcode_str() and the >> array it defines are unreachable since all admin commands don't log errors with >> this change. >> >> You could skip the RQF_QUIET setting and check blk_rq_is_passthrough() instead. > > Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. > I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? > > Alan > Sagi/Christoph/Hannes/Keith, After debugging the issue following patch [1] makes errors disappear. But I'm not sure if this behavior aligns with protocol or not. I'll keep digging, meanwhile if anyone has an idea please provide a review on patch in [1] if make sense or not. -ck [1] mask invalid NVME_ID_CNS_CS_CTRL errors. diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 397daaf51f1b..e5eea2f0ac9c 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -718,9 +718,16 @@ static void nvmet_execute_identify(struct nvmet_req *req) switch (req->cmd->identify.csi) { case NVME_CSI_ZNS: return nvmet_execute_identify_cns_cs_ctrl(req); + case NVME_CSI_NVM: + return nvmet_execute_identify_ctrl(req); default: break; } + } else { + switch (req->cmd->identify.csi) { + case NVME_CSI_NVM: + return nvmet_execute_identify_ctrl(req); + } } break; case NVME_ID_CNS_NS_ACTIVE_LIST: diff --git a/drivers/nvme/target/discovery.c b/drivers/nvme/target/discovery.c index c2162eef8ce1..34c7ed055674 100644 --- a/drivers/nvme/target/discovery.c +++ b/drivers/nvme/target/discovery.c @@ -254,7 +254,11 @@ static void nvmet_execute_disc_identify(struct nvmet_req *req) if (!nvmet_check_transfer_len(req, NVME_IDENTIFY_DATA_SIZE)) return; - if (req->cmd->identify.cns != NVME_ID_CNS_CTRL) { + switch (req->cmd->identify.cns) { + case NVME_ID_CNS_CTRL: + case NVME_ID_CNS_CS_CTRL: + break; + default: req->error_loc = offsetof(struct nvme_identify, cns); status = NVME_SC_INVALID_FIELD | NVME_SC_DNR; goto out; ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 22:16 ` Chaitanya Kulkarni @ 2022-04-06 23:29 ` Alan Adamson 2022-04-07 19:40 ` Chaitanya Kulkarni 2022-04-07 8:48 ` Christoph Hellwig 1 sibling, 1 reply; 23+ messages in thread From: Alan Adamson @ 2022-04-06 23:29 UTC (permalink / raw) To: Chaitanya Kulkarni Cc: Keith Busch, Christoph Hellwig, Hannes Reinecke, Sagi Grimberg, linux-nvme I tried the patch with my tests. It resolves the error logging with nvme/012. My verbose errors tests (nvme/039) now passes, but we still see the error when the system boots. root@localhost blktests]# ./check nvme/012 nvme/012 (run mkfs and data verification fio job on NVMeOF block device-backed ns) [passed] runtime 39.180s ... 41.432s [root@localhost blktests]# [ 77.946505] run blktests nvme/012 at 2022-04-06 19:22:23 [ 78.013092] loop: module loaded [ 78.014344] loop0: detected capacity change from 0 to 2097152 [ 78.024875] nvmet: adding nsid 1 to subsystem blktests-subsystem-1 [ 78.052874] nvmet: creating nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-000000000000. [ 78.053107] nvme nvme1: creating 1 I/O queues. [ 78.053182] nvme nvme1: new ctrl: "blktests-subsystem-1" [ 79.125929] XFS (nvme1n1): Mounting V5 Filesystem [ 79.128321] XFS (nvme1n1): Ending clean mount [ 79.128480] xfs filesystem being mounted at /mnt/blktests supports timestamps until 2038 (0x7fffffff) [ 119.253246] XFS (nvme1n1): Unmounting Filesystem [ 119.264005] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1" Verbose Errors blktests ======================= [root@localhost blktests]# ./check nvme/039 nvme/039 => nvme0n1 (test error logging) [passed] runtime 0.060s ... 0.058s [root@localhost blktests]# Boot ==== [ 3.244056] nvme0: Identify(0x6), Invalid Field in Command (sct 0x0 / sc 0x2) DNR Alan ~ > On Apr 6, 2022, at 3:16 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > On 4/6/22 15:06, Alan Adamson wrote: >> >> >>> On Apr 6, 2022, at 10:21 AM, Keith Busch <kbusch@kernel.org> wrote: >>> >>> On Wed, Apr 06, 2022 at 05:01:46PM +0000, Chaitanya Kulkarni wrote: >>>> On 4/6/22 09:52, Keith Busch wrote: >>>>> On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: >>>>>> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) >>>>>> { >>>>>> blk_status_t status = nvme_error_status(nvme_req(req)->status); >>>>>> >>>>>> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >>>>>> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && >>>>>> + !(req->rq_flags & RQF_QUIET))) >>>>>> nvme_log_error(req); >>>>>> nvme_end_req_zoned(req); >>>>>> nvme_trace_bio_complete(req); >>>>>> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) >>>>>> cmd->common.flags &= ~NVME_CMD_SGL_ALL; >>>>>> >>>>>> req->cmd_flags |= REQ_FAILFAST_DRIVER; >>>>>> + req->rq_flags |= RQF_QUIET; >>>>> >>>>> This defeats the admin error logging logic since every admin command comes >>>>> through here. If you're sure we should do this, then I suppose you can remove >>>>> that unreachable code. >>>> >>>> If you point out the unreachable code that will be great, >>>> I'll keep looking meanwhile... >>> >>> The second half of nvme_log_error(), plus nvme_get_admin_opcode_str() and the >>> array it defines are unreachable since all admin commands don't log errors with >>> this change. >>> >>> You could skip the RQF_QUIET setting and check blk_rq_is_passthrough() instead. >> >> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. >> I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? >> >> Alan >> > > Sagi/Christoph/Hannes/Keith, > > After debugging the issue following patch [1] makes errors disappear. > > But I'm not sure if this behavior aligns with protocol or not. > > I'll keep digging, meanwhile if anyone has an idea please provide a > review on patch in [1] if make sense or not. > > -ck > > [1] mask invalid NVME_ID_CNS_CS_CTRL errors. > > diff --git a/drivers/nvme/target/admin-cmd.c > b/drivers/nvme/target/admin-cmd.c > index 397daaf51f1b..e5eea2f0ac9c 100644 > --- a/drivers/nvme/target/admin-cmd.c > +++ b/drivers/nvme/target/admin-cmd.c > @@ -718,9 +718,16 @@ static void nvmet_execute_identify(struct nvmet_req > *req) > switch (req->cmd->identify.csi) { > case NVME_CSI_ZNS: > return > nvmet_execute_identify_cns_cs_ctrl(req); > + case NVME_CSI_NVM: > + return nvmet_execute_identify_ctrl(req); > default: > break; > } > + } else { > + switch (req->cmd->identify.csi) { > + case NVME_CSI_NVM: > + return nvmet_execute_identify_ctrl(req); > + } > } > break; > case NVME_ID_CNS_NS_ACTIVE_LIST: > diff --git a/drivers/nvme/target/discovery.c > b/drivers/nvme/target/discovery.c > index c2162eef8ce1..34c7ed055674 100644 > --- a/drivers/nvme/target/discovery.c > +++ b/drivers/nvme/target/discovery.c > @@ -254,7 +254,11 @@ static void nvmet_execute_disc_identify(struct > nvmet_req *req) > if (!nvmet_check_transfer_len(req, NVME_IDENTIFY_DATA_SIZE)) > return; > > - if (req->cmd->identify.cns != NVME_ID_CNS_CTRL) { > + switch (req->cmd->identify.cns) { > + case NVME_ID_CNS_CTRL: > + case NVME_ID_CNS_CS_CTRL: > + break; > + default: > req->error_loc = offsetof(struct nvme_identify, cns); > status = NVME_SC_INVALID_FIELD | NVME_SC_DNR; > goto out; > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 23:29 ` Alan Adamson @ 2022-04-07 19:40 ` Chaitanya Kulkarni 0 siblings, 0 replies; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-07 19:40 UTC (permalink / raw) To: Alan Adamson Cc: Keith Busch, Christoph Hellwig, Hannes Reinecke, Sagi Grimberg, linux-nvme On 4/6/22 16:29, Alan Adamson wrote: > I tried the patch with my tests. It resolves the error logging with nvme/012. My verbose errors tests (nvme/039) now passes, > but we still see the error when the system boots. > yes, that is something we need to fix, also it'll be great if we can avoid top posting? To fix this I've suggested fixing the call sites see [1]. -ck [1] http://lists.infradead.org/pipermail/linux-nvme/2022-April/031160.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 22:16 ` Chaitanya Kulkarni 2022-04-06 23:29 ` Alan Adamson @ 2022-04-07 8:48 ` Christoph Hellwig 1 sibling, 0 replies; 23+ messages in thread From: Christoph Hellwig @ 2022-04-07 8:48 UTC (permalink / raw) To: Chaitanya Kulkarni Cc: Alan Adamson, Keith Busch, Christoph Hellwig, Hannes Reinecke, Sagi Grimberg, linux-nvme This won't help with other controllers. But it still is a useful patch, so please submit it separately. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 22:06 ` Alan Adamson 2022-04-06 22:16 ` Chaitanya Kulkarni @ 2022-04-07 8:51 ` hch 2022-04-07 11:52 ` Sagi Grimberg 2022-04-07 16:15 ` Alan Adamson 1 sibling, 2 replies; 23+ messages in thread From: hch @ 2022-04-07 8:51 UTC (permalink / raw) To: Alan Adamson; +Cc: Keith Busch, Chaitanya Kulkarni, linux-nvme, hch On Wed, Apr 06, 2022 at 10:06:06PM +0000, Alan Adamson wrote: > Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. > I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? Well, you submitted the logging so we're curious about your use case. SCSI skips logging errors for internally submitted commands, but not for userspace passthrough. So we could move the RQF_QUIET into __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 8:51 ` hch @ 2022-04-07 11:52 ` Sagi Grimberg 2022-04-07 20:10 ` Chaitanya Kulkarni 2022-04-07 16:15 ` Alan Adamson 1 sibling, 1 reply; 23+ messages in thread From: Sagi Grimberg @ 2022-04-07 11:52 UTC (permalink / raw) To: hch, Alan Adamson; +Cc: Keith Busch, Chaitanya Kulkarni, linux-nvme >> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. >> I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? > > Well, you submitted the logging so we're curious about your use case. > SCSI skips logging errors for internally submitted commands, but not for > userspace passthrough. So we could move the RQF_QUIET into > __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. > Makes sense to me ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 11:52 ` Sagi Grimberg @ 2022-04-07 20:10 ` Chaitanya Kulkarni 2022-04-07 20:35 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-07 20:10 UTC (permalink / raw) To: Sagi Grimberg, hch; +Cc: Keith Busch, Alan Adamson, linux-nvme On 4/7/22 04:52, Sagi Grimberg wrote: > >>> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme >>> admin-passthru command will log an error. >>> I ran into this using the blktests I’m coding up for verbose errors. >>> Is this the behavior we want? >> >> Well, you submitted the logging so we're curious about your use case. >> SCSI skips logging errors for internally submitted commands, but not for >> userspace passthrough. So we could move the RQF_QUIET into >> __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. >> > > Makes sense to me This was my initial proposal to fix the call sites with internal passthru commands [1], so this should work. -ck [1] http://lists.infradead.org/pipermail/linux-nvme/2022-April/031160.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 20:10 ` Chaitanya Kulkarni @ 2022-04-07 20:35 ` Alan Adamson 2022-04-07 21:00 ` Chaitanya Kulkarni 0 siblings, 1 reply; 23+ messages in thread From: Alan Adamson @ 2022-04-07 20:35 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme > On Apr 7, 2022, at 1:10 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > On 4/7/22 04:52, Sagi Grimberg wrote: >> >>>> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme >>>> admin-passthru command will log an error. >>>> I ran into this using the blktests I’m coding up for verbose errors. >>>> Is this the behavior we want? >>> >>> Well, you submitted the logging so we're curious about your use case. >>> SCSI skips logging errors for internally submitted commands, but not for >>> userspace passthrough. So we could move the RQF_QUIET into >>> __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. >>> >> >> Makes sense to me > > This was my initial proposal to fix the call sites with internal > passthru commands [1], so this should work. > > -ck > > [1] > http://lists.infradead.org/pipermail/linux-nvme/2022-April/031160.html > > It works (suppresses logging) for driver initiated admin cmds which is good, but it also suppresses logging when running "nvme admin-passthru” which I don’t think we want. Alan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 20:35 ` Alan Adamson @ 2022-04-07 21:00 ` Chaitanya Kulkarni 2022-04-07 21:13 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-07 21:00 UTC (permalink / raw) To: Alan Adamson; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme On 4/7/22 13:35, Alan Adamson wrote: > > >> On Apr 7, 2022, at 1:10 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: >> >> On 4/7/22 04:52, Sagi Grimberg wrote: >>> >>>>> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme >>>>> admin-passthru command will log an error. >>>>> I ran into this using the blktests I’m coding up for verbose errors. >>>>> Is this the behavior we want? >>>> >>>> Well, you submitted the logging so we're curious about your use case. >>>> SCSI skips logging errors for internally submitted commands, but not for >>>> userspace passthrough. So we could move the RQF_QUIET into >>>> __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. >>>> >>> >>> Makes sense to me >> >> This was my initial proposal to fix the call sites with internal >> passthru commands [1], so this should work. >> >> -ck >> >> [1] >> http://lists.infradead.org/pipermail/linux-nvme/2022-April/031160.html >> >> > > It works (suppresses logging) for driver initiated admin cmds which is good, but it also suppresses logging > when running "nvme admin-passthru” which I don’t think we want. > > Alan > I think nvme admin-passthru and io-passthru uses different interface than internal passthru commnds i.e. nvme admin-passthru :- NVME_IOCTL_ADMIN nvme_submit_user_cmd() req allocation and submission no internal passthru cmds. nvme io-passthru:- NVME_IOCTL_IO_CMD nvme_user_cmd() nvme_submit_user_cmd() req allocation and submission, no internal passthru cmds. The internal passthru interface uses it's own request allocation and submission, so if we change that it should not affect the nvme admin-io passthru interface... unless we are using something else for internal passthru which is not going through __nvme_submit_sync_cmd() ultimately. In short what I meant is following patch, this should help us remove the boot time messages you have reported and blktest warnings also, can you please confirm ? if not then I'll keep digging.. nvme (nvme-5.18) # git diff diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f204c6f78b5b..a1ea2f736d42 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) { blk_status_t status = nvme_error_status(nvme_req(req)->status); - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) nvme_log_error(req); nvme_end_req_zoned(req); nvme_trace_bio_complete(req); @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd, else req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, qid ? qid - 1 : 0); - if (IS_ERR(req)) return PTR_ERR(req); + + req->rq_flags |= RQF_QUIET; + nvme_init_request(req, cmd); if (timeout) if not then I'll keep digging.. -ck ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 21:00 ` Chaitanya Kulkarni @ 2022-04-07 21:13 ` Alan Adamson 2022-04-08 2:13 ` Chaitanya Kulkarni 0 siblings, 1 reply; 23+ messages in thread From: Alan Adamson @ 2022-04-07 21:13 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme > On Apr 7, 2022, at 2:00 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > On 4/7/22 13:35, Alan Adamson wrote: >> >> >>> On Apr 7, 2022, at 1:10 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: >>> >>> On 4/7/22 04:52, Sagi Grimberg wrote: >>>> >>>>>> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme >>>>>> admin-passthru command will log an error. >>>>>> I ran into this using the blktests I’m coding up for verbose errors. >>>>>> Is this the behavior we want? >>>>> >>>>> Well, you submitted the logging so we're curious about your use case. >>>>> SCSI skips logging errors for internally submitted commands, but not for >>>>> userspace passthrough. So we could move the RQF_QUIET into >>>>> __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. >>>>> >>>> >>>> Makes sense to me >>> >>> This was my initial proposal to fix the call sites with internal >>> passthru commands [1], so this should work. >>> >>> -ck >>> >>> [1] >>> http://lists.infradead.org/pipermail/linux-nvme/2022-April/031160.html >>> >>> >> >> It works (suppresses logging) for driver initiated admin cmds which is good, but it also suppresses logging >> when running "nvme admin-passthru” which I don’t think we want. >> >> Alan >> > > I think nvme admin-passthru and io-passthru uses different > interface than internal passthru commnds i.e. > > nvme admin-passthru :- NVME_IOCTL_ADMIN > > nvme_submit_user_cmd() > req allocation and submission no internal passthru cmds. > > nvme io-passthru:- NVME_IOCTL_IO_CMD > > nvme_user_cmd() > nvme_submit_user_cmd() > req allocation and submission, no internal passthru cmds. > > The internal passthru interface uses it's own request allocation > and submission, so if we change that it should not affect the > nvme admin-io passthru interface... unless we are using something > else for internal passthru which is not going through > __nvme_submit_sync_cmd() ultimately. > > In short what I meant is following patch, this should help > us remove the boot time messages you have reported and > blktest warnings also, can you please confirm ? > > if not then I'll keep digging.. > > nvme (nvme-5.18) # git diff > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > index f204c6f78b5b..a1ea2f736d42 100644 > --- a/drivers/nvme/host/core.c > +++ b/drivers/nvme/host/core.c > @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) > { > blk_status_t status = nvme_error_status(nvme_req(req)->status); > > - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) > + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) > nvme_log_error(req); > nvme_end_req_zoned(req); > nvme_trace_bio_complete(req); > @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue > *q, struct nvme_command *cmd, > else > req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, > qid ? qid - 1 : 0); > - > if (IS_ERR(req)) > return PTR_ERR(req); > + > + req->rq_flags |= RQF_QUIET; > + > nvme_init_request(req, cmd); > > if (timeout) > > > if not then I'll keep digging.. > > -ck I just tried the patch on my config, it properly suppresses the bootup message, but it also supresses messages from "nvme admin-passthru”. Alan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 21:13 ` Alan Adamson @ 2022-04-08 2:13 ` Chaitanya Kulkarni 2022-04-08 16:24 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-08 2:13 UTC (permalink / raw) To: Alan Adamson; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme >> nvme (nvme-5.18) # git diff >> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c >> index f204c6f78b5b..a1ea2f736d42 100644 >> --- a/drivers/nvme/host/core.c >> +++ b/drivers/nvme/host/core.c >> @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) >> { >> blk_status_t status = nvme_error_status(nvme_req(req)->status); >> >> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >> + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) >> nvme_log_error(req); >> nvme_end_req_zoned(req); >> nvme_trace_bio_complete(req); >> @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue >> *q, struct nvme_command *cmd, >> else >> req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, >> qid ? qid - 1 : 0); >> - >> if (IS_ERR(req)) >> return PTR_ERR(req); >> + >> + req->rq_flags |= RQF_QUIET; >> + >> nvme_init_request(req, cmd); >> >> if (timeout) >> >> >> if not then I'll keep digging.. >> >> -ck > > I just tried the patch on my config, it properly suppresses the bootup message, but it also supresses messages from "nvme admin-passthru”. > > Alan > Can you please share a command line for "nvme admin-passthru" where this patch supresses messages ? -ck ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-08 2:13 ` Chaitanya Kulkarni @ 2022-04-08 16:24 ` Alan Adamson 2022-04-09 0:10 ` Chaitanya Kulkarni 0 siblings, 1 reply; 23+ messages in thread From: Alan Adamson @ 2022-04-08 16:24 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme > On Apr 7, 2022, at 7:13 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > >>> nvme (nvme-5.18) # git diff >>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c >>> index f204c6f78b5b..a1ea2f736d42 100644 >>> --- a/drivers/nvme/host/core.c >>> +++ b/drivers/nvme/host/core.c >>> @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) >>> { >>> blk_status_t status = nvme_error_status(nvme_req(req)->status); >>> >>> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >>> + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) >>> nvme_log_error(req); >>> nvme_end_req_zoned(req); >>> nvme_trace_bio_complete(req); >>> @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue >>> *q, struct nvme_command *cmd, >>> else >>> req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, >>> qid ? qid - 1 : 0); >>> - >>> if (IS_ERR(req)) >>> return PTR_ERR(req); >>> + >>> + req->rq_flags |= RQF_QUIET; >>> + >>> nvme_init_request(req, cmd); >>> >>> if (timeout) >>> >>> >>> if not then I'll keep digging.. >>> >>> -ck >> >> I just tried the patch on my config, it properly suppresses the bootup message, but it also supresses messages from "nvme admin-passthru”. >> >> Alan >> > > Can you please share a command line for "nvme admin-passthru" > where this patch supresses messages ? I have the NVME Fault Injector configured: echo 0x286 > /sys/kernel/debug/${ctrl_dev}/fault_inject/status echo 1000 > /sys/kernel/debug/${ctrl_dev}/fault_inject/times echo 100 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability nvme admin-passthru /dev/${ctrl_dev} --opcode=06 --data-len=4096 --cdw10=1 -r echo 0 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability > > -ck > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-08 16:24 ` Alan Adamson @ 2022-04-09 0:10 ` Chaitanya Kulkarni 2022-04-11 18:31 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-09 0:10 UTC (permalink / raw) To: Alan Adamson; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme >> Can you please share a command line for "nvme admin-passthru" >> where this patch supresses messages ? > > I have the NVME Fault Injector configured: > > echo 0x286 > /sys/kernel/debug/${ctrl_dev}/fault_inject/status > echo 1000 > /sys/kernel/debug/${ctrl_dev}/fault_inject/times > echo 100 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability > > nvme admin-passthru /dev/${ctrl_dev} --opcode=06 --data-len=4096 --cdw10=1 -r > > echo 0 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability > I was able to produce same admin-passthru error messages with my patch. See below detailed execution with the script and the log, can you please tell me what is missing ? >> -ck >> >> > * Without this patch I get 3 Error message from fault injection enabled :- 1. [ 1743.353266] nvme1: Identify(0x6), Invalid Field in Command (sct 0x0 / sc 0x2) MORE DNR 2. [ 1744.370690] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 1000 [ 1744.370698] CPU: 41 PID: 389 Comm: kworker/41:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1744.370702] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1744.370704] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1744.370712] Call Trace: [ 1744.370715] <TASK> [ 1744.370717] dump_stack_lvl+0x48/0x5e [ 1744.370724] should_fail.cold+0x32/0x37 [ 1744.370729] nvme_should_fail+0x38/0x90 [nvme_core] [ 1744.370741] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1744.370745] nvmet_req_complete+0x11/0x50 [nvmet] [ 1744.370754] process_one_work+0x1af/0x380 [ 1744.370758] worker_thread+0x50/0x3a0 [ 1744.370761] ? rescuer_thread+0x370/0x370 [ 1744.370763] kthread+0xe7/0x110 [ 1744.370767] ? kthread_complete_and_exit+0x20/0x20 [ 1744.370770] ret_from_fork+0x22/0x30 [ 1744.370776] </TASK> [ 1744.370783] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR 3. [1744.375045] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 999 [ 1744.375052] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1744.375056] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1744.375058] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1744.375066] Call Trace: [ 1744.375070] <TASK> [ 1744.375072] dump_stack_lvl+0x48/0x5e [ 1744.375079] should_fail.cold+0x32/0x37 [ 1744.375084] nvme_should_fail+0x38/0x90 [nvme_core] [ 1744.375096] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1744.375100] nvmet_req_complete+0x11/0x50 [nvmet] [ 1744.375108] process_one_work+0x1af/0x380 [ 1744.375113] worker_thread+0x50/0x3a0 [ 1744.375115] ? rescuer_thread+0x370/0x370 [ 1744.375118] kthread+0xe7/0x110 [ 1744.375121] ? kthread_complete_and_exit+0x20/0x20 [ 1744.375124] ret_from_fork+0x22/0x30 [ 1744.375130] </TASK> [ 1744.375148] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR * With this patch I only get two from fault injection which are coming from nvme-admin-passthru and no from the internal passthru:- 1. [ 1765.175570] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 1000 [ 1765.175579] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1765.175583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1765.175585] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1765.175593] Call Trace: [ 1765.175596] <TASK> [ 1765.175599] dump_stack_lvl+0x48/0x5e [ 1765.175605] should_fail.cold+0x32/0x37 [ 1765.175610] nvme_should_fail+0x38/0x90 [nvme_core] [ 1765.175623] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1765.175627] nvmet_req_complete+0x11/0x50 [nvmet] [ 1765.175636] process_one_work+0x1af/0x380 [ 1765.175640] worker_thread+0x50/0x3a0 [ 1765.175643] ? rescuer_thread+0x370/0x370 [ 1765.175645] kthread+0xe7/0x110 [ 1765.175648] ? kthread_complete_and_exit+0x20/0x20 [ 1765.175652] ret_from_fork+0x22/0x30 [ 1765.175658] </TASK> [ 1765.175664] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR 2. [ 1765.179829] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 999 [ 1765.179835] CPU: 44 PID: 9897 Comm: kworker/44:0 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1765.179839] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1765.179841] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1765.179850] Call Trace: [ 1765.179853] <TASK> [ 1765.179855] dump_stack_lvl+0x48/0x5e [ 1765.179862] should_fail.cold+0x32/0x37 [ 1765.179867] nvme_should_fail+0x38/0x90 [nvme_core] [ 1765.179879] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1765.179884] nvmet_req_complete+0x11/0x50 [nvmet] [ 1765.179892] process_one_work+0x1af/0x380 [ 1765.179897] ? rescuer_thread+0x370/0x370 [ 1765.179899] worker_thread+0x50/0x3a0 [ 1765.179902] ? rescuer_thread+0x370/0x370 [ 1765.179903] kthread+0xe7/0x110 [ 1765.179907] ? kthread_complete_and_exit+0x20/0x20 [ 1765.179911] ret_from_fork+0x22/0x30 [ 1765.179917] </TASK> [ 1765.179923] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR * Detailed test log with nvmeof nvme-loop controller configured and fault injection enabled with and without this patch to mask internal passthru. It shows only internal passthru error message is masked with this patch and internal passthru error message is present without this patch, nvme-admin-passthru error message present in both cases :- nvme (nvme-5.18) # sh error_inject.sh commit 7bec02cef3d11f3d3a80bfa8739f790377bac8d6 (HEAD -> nvme-5.18) Merge: 226d991feef9 a4a6f3c8f61c Author: Chaitanya Kulkarni <kch@nvidia.com> Date: Fri Apr 8 16:57:45 2022 -0700 Merge branch 'nvme-5.18' of git://git.infradead.org/nvme into nvme-5.18 + umount /mnt/nvme0n1 + clear_dmesg ./compile_nvme.sh: line 3: clear_dmesg: command not found umount: /mnt/nvme0n1: no mount point specified. + ./delete.sh + NQN=testnqn + nvme disconnect -n testnqn Failed to scan topoplogy: No such file or directory real 0m0.002s user 0m0.000s sys 0m0.002s + rm -fr '/sys/kernel/config/nvmet/ports/1/subsystems/*' + rmdir /sys/kernel/config/nvmet/ports/1 rmdir: failed to remove '/sys/kernel/config/nvmet/ports/1': No such file or directory + for subsys in /sys/kernel/config/nvmet/subsystems/* + for ns in ${subsys}/namespaces/* + echo 0 ./delete.sh: line 14: /sys/kernel/config/nvmet/subsystems/*/namespaces/*/enable: No such file or directory + rmdir '/sys/kernel/config/nvmet/subsystems/*/namespaces/*' rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*/namespaces/*': No such file or directory + rmdir '/sys/kernel/config/nvmet/subsystems/*' rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*': No such file or directory + rmdir 'config/nullb/nullb*' rmdir: failed to remove 'config/nullb/nullb*': No such file or directory + umount /mnt/nvme0n1 umount: /mnt/nvme0n1: no mount point specified. + umount /mnt/backend umount: /mnt/backend: not mounted. + modprobe -r nvme_loop + modprobe -r nvme_fabrics + modprobe -r nvmet + modprobe -r nvme + modprobe -r null_blk + tree /sys/kernel/config /sys/kernel/config 0 directories, 0 files + modprobe -r nvme-fabrics + modprobe -r nvme_loop + modprobe -r nvmet + modprobe -r nvme + sleep 1 + modprobe -r nvme-core + lsmod + grep nvme + sleep 1 + git diff + sleep 1 ++ nproc + make -j 48 M=drivers/nvme/target/ clean ++ nproc + make -j 48 M=drivers/nvme/ modules CC [M] drivers/nvme/target/core.o CC [M] drivers/nvme/target/configfs.o CC [M] drivers/nvme/target/admin-cmd.o CC [M] drivers/nvme/target/fabrics-cmd.o CC [M] drivers/nvme/target/discovery.o CC [M] drivers/nvme/target/io-cmd-file.o CC [M] drivers/nvme/target/io-cmd-bdev.o CC [M] drivers/nvme/target/passthru.o CC [M] drivers/nvme/target/zns.o CC [M] drivers/nvme/target/trace.o CC [M] drivers/nvme/target/loop.o CC [M] drivers/nvme/target/rdma.o CC [M] drivers/nvme/target/fc.o CC [M] drivers/nvme/target/fcloop.o CC [M] drivers/nvme/target/tcp.o CC [M] drivers/nvme/host/core.o LD [M] drivers/nvme/target/nvme-loop.o LD [M] drivers/nvme/target/nvme-fcloop.o LD [M] drivers/nvme/target/nvmet.o LD [M] drivers/nvme/target/nvmet-tcp.o LD [M] drivers/nvme/target/nvmet-fc.o LD [M] drivers/nvme/target/nvmet-rdma.o LD [M] drivers/nvme/host/nvme-core.o MODPOST drivers/nvme/Module.symvers LD [M] drivers/nvme/host/nvme-core.ko CC [M] drivers/nvme/target/nvme-fcloop.mod.o CC [M] drivers/nvme/target/nvme-loop.mod.o CC [M] drivers/nvme/target/nvmet-fc.mod.o CC [M] drivers/nvme/target/nvmet-rdma.mod.o CC [M] drivers/nvme/target/nvmet-tcp.mod.o CC [M] drivers/nvme/target/nvmet.mod.o LD [M] drivers/nvme/target/nvme-loop.ko LD [M] drivers/nvme/target/nvme-fcloop.ko LD [M] drivers/nvme/target/nvmet-rdma.ko LD [M] drivers/nvme/target/nvmet-fc.ko LD [M] drivers/nvme/target/nvmet-tcp.ko LD [M] drivers/nvme/target/nvmet.ko + HOST=drivers/nvme/host + TARGET=drivers/nvme/target ++ uname -r + HOST_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ ++ uname -r + TARGET_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target/ + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host// + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// + ls -lrth /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/: total 6.3M -rw-r--r--. 1 root root 2.7M Apr 8 16:58 nvme-core.ko -rw-r--r--. 1 root root 426K Apr 8 16:58 nvme-fabrics.ko -rw-r--r--. 1 root root 925K Apr 8 16:58 nvme-fc.ko -rw-r--r--. 1 root root 714K Apr 8 16:58 nvme.ko -rw-r--r--. 1 root root 856K Apr 8 16:58 nvme-rdma.ko -rw-r--r--. 1 root root 799K Apr 8 16:58 nvme-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target//: total 6.3M -rw-r--r--. 1 root root 475K Apr 8 16:58 nvme-fcloop.ko -rw-r--r--. 1 root root 419K Apr 8 16:58 nvme-loop.ko -rw-r--r--. 1 root root 734K Apr 8 16:58 nvmet-fc.ko -rw-r--r--. 1 root root 3.2M Apr 8 16:58 nvmet.ko -rw-r--r--. 1 root root 822K Apr 8 16:58 nvmet-rdma.ko -rw-r--r--. 1 root root 671K Apr 8 16:58 nvmet-tcp.ko + sync + sync + sync + modprobe nvme + echo 'Press enter to continue ...' Press enter to continue ... + read next ++ NN=1 ++ NQN=testnqn ++ let NR_DEVICES=NN+1 ++ modprobe -r null_blk ++ modprobe -r nvme ++ modprobe null_blk nr_devices=0 ++ modprobe nvme ++ modprobe nvme-fabrics ++ modprobe nvmet ++ modprobe nvme-loop ++ dmesg -c ++ sleep 2 ++ tree /sys/kernel/config /sys/kernel/config ├── nullb │ └── features └── nvmet ├── hosts ├── ports └── subsystems 5 directories, 1 file ++ sleep 1 ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn +++ shuf -i 1-1 -n 1 ++ for i in `shuf -i 1-$NN -n $NN` ++ mkdir config/nullb/nullb1 ++ echo 1 ++ echo 4096 ++ echo 2048 ++ echo 1 +++ cat config/nullb/nullb1/index ++ IDX=0 ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 ++ echo ' ####### /dev/nullb0' ####### /dev/nullb0 ++ echo -n /dev/nullb0 ++ cat /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1/device_path /dev/nullb0 ++ echo 1 ++ dmesg -c [ 1740.356540] nvme nvme0: 48/0/0 default/read/poll queues [ 1743.345785] nvmet: adding nsid 1 to subsystem testnqn ++ mkdir /sys/kernel/config/nvmet/ports/1/ ++ echo -n loop ++ echo -n 1 ++ ln -s /sys/kernel/config/nvmet/subsystems/testnqn /sys/kernel/config/nvmet/ports/1/subsystems/ ++ echo transport=loop,nqn=testnqn ++ sleep 1 ++ mount ++ column -t ++ grep nvme ++ dmesg -c [ 1743.353177] nvmet: creating nvm controller 1 for subsystem testnqn for NQN nqn.2014-08.org.nvmexpress:uuid:510d0435-0ad7-49d4-ae4a-f1c1552b0f0c. [ 1743.353266] nvme1: Identify(0x6), Invalid Field in Command (sct 0x0 / sc 0x2) MORE DNR [ 1743.355380] nvme nvme1: creating 48 I/O queues. [ 1743.359206] nvme nvme1: new ctrl: "testnqn" Node SN Model Namespace Usage Format FW Rev nvme1n1 8bdab4b79aca987d0eba Linux 1 2.15 GB / 2.15 GB 4 KiB + 0 B 5.17.0-r nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 NVMe status: Access Denied: Access to the namespace and/or LBA range is denied due to lack of access rights(0x4286) Node SN Model Namespace Usage Format FW Rev nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 [ 1744.370690] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 1000 [ 1744.370698] CPU: 41 PID: 389 Comm: kworker/41:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1744.370702] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1744.370704] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1744.370712] Call Trace: [ 1744.370715] <TASK> [ 1744.370717] dump_stack_lvl+0x48/0x5e [ 1744.370724] should_fail.cold+0x32/0x37 [ 1744.370729] nvme_should_fail+0x38/0x90 [nvme_core] [ 1744.370741] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1744.370745] nvmet_req_complete+0x11/0x50 [nvmet] [ 1744.370754] process_one_work+0x1af/0x380 [ 1744.370758] worker_thread+0x50/0x3a0 [ 1744.370761] ? rescuer_thread+0x370/0x370 [ 1744.370763] kthread+0xe7/0x110 [ 1744.370767] ? kthread_complete_and_exit+0x20/0x20 [ 1744.370770] ret_from_fork+0x22/0x30 [ 1744.370776] </TASK> [ 1744.370783] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR [ 1744.375045] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 999 [ 1744.375052] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1744.375056] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1744.375058] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1744.375066] Call Trace: [ 1744.375070] <TASK> [ 1744.375072] dump_stack_lvl+0x48/0x5e [ 1744.375079] should_fail.cold+0x32/0x37 [ 1744.375084] nvme_should_fail+0x38/0x90 [nvme_core] [ 1744.375096] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1744.375100] nvmet_req_complete+0x11/0x50 [nvmet] [ 1744.375108] process_one_work+0x1af/0x380 [ 1744.375113] worker_thread+0x50/0x3a0 [ 1744.375115] ? rescuer_thread+0x370/0x370 [ 1744.375118] kthread+0xe7/0x110 [ 1744.375121] ? kthread_complete_and_exit+0x20/0x20 [ 1744.375124] ret_from_fork+0x22/0x30 [ 1744.375130] </TASK> [ 1744.375148] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR + NQN=testnqn + nvme disconnect -n testnqn NQN:testnqn disconnected 1 controller(s) real 0m0.370s user 0m0.001s sys 0m0.004s + rm -fr /sys/kernel/config/nvmet/ports/1/subsystems/testnqn + rmdir /sys/kernel/config/nvmet/ports/1 + for subsys in /sys/kernel/config/nvmet/subsystems/* + for ns in ${subsys}/namespaces/* + echo 0 + rmdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 + rmdir /sys/kernel/config/nvmet/subsystems/testnqn + rmdir config/nullb/nullb1 + umount /mnt/nvme0n1 umount: /mnt/nvme0n1: no mount point specified. + umount /mnt/backend umount: /mnt/backend: not mounted. + modprobe -r nvme_loop + modprobe -r nvme_fabrics + modprobe -r nvmet + modprobe -r nvme + modprobe -r null_blk + tree /sys/kernel/config /sys/kernel/config 0 directories, 0 files From 2d552b2c756fce48f53d66c5c58ffb6b3e3cac6e Mon Sep 17 00:00:00 2001 From: Chaitanya Kulkarni <kch@nvidia.com> Date: Fri, 8 Apr 2022 16:45:18 -0700 Subject: [PATCH] nvme-core: mark internal passthru req REQ_QUIET Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> --- drivers/nvme/host/core.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index f204c6f78b5b..a1ea2f736d42 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) { blk_status_t status = nvme_error_status(nvme_req(req)->status); - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) nvme_log_error(req); nvme_end_req_zoned(req); nvme_trace_bio_complete(req); @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue *q, struct nvme_command *cmd, else req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, qid ? qid - 1 : 0); - if (IS_ERR(req)) return PTR_ERR(req); + + req->rq_flags |= RQF_QUIET; + nvme_init_request(req, cmd); if (timeout) -- 2.29.0 Applying: nvme-core: mark internal passthru req REQ_QUIET commit 3c7952e0f8fcc9affdfa0249ab8211c18a513338 (HEAD -> nvme-5.18) Author: Chaitanya Kulkarni <kch@nvidia.com> Date: Fri Apr 8 16:45:18 2022 -0700 nvme-core: mark internal passthru req REQ_QUIET Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> + umount /mnt/nvme0n1 + clear_dmesg ./compile_nvme.sh: line 3: clear_dmesg: command not found umount: /mnt/nvme0n1: no mount point specified. + ./delete.sh + NQN=testnqn + nvme disconnect -n testnqn Failed to scan topoplogy: No such file or directory real 0m0.002s user 0m0.002s sys 0m0.000s + rm -fr '/sys/kernel/config/nvmet/ports/1/subsystems/*' + rmdir /sys/kernel/config/nvmet/ports/1 rmdir: failed to remove '/sys/kernel/config/nvmet/ports/1': No such file or directory + for subsys in /sys/kernel/config/nvmet/subsystems/* + for ns in ${subsys}/namespaces/* + echo 0 ./delete.sh: line 14: /sys/kernel/config/nvmet/subsystems/*/namespaces/*/enable: No such file or directory + rmdir '/sys/kernel/config/nvmet/subsystems/*/namespaces/*' rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*/namespaces/*': No such file or directory + rmdir '/sys/kernel/config/nvmet/subsystems/*' rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*': No such file or directory + rmdir 'config/nullb/nullb*' rmdir: failed to remove 'config/nullb/nullb*': No such file or directory + umount /mnt/nvme0n1 umount: /mnt/nvme0n1: no mount point specified. + umount /mnt/backend umount: /mnt/backend: not mounted. + modprobe -r nvme_loop + modprobe -r nvme_fabrics + modprobe -r nvmet + modprobe -r nvme + modprobe -r null_blk + tree /sys/kernel/config /sys/kernel/config 0 directories, 0 files + modprobe -r nvme-fabrics + modprobe -r nvme_loop + modprobe -r nvmet + modprobe -r nvme + sleep 1 + modprobe -r nvme-core + lsmod + grep nvme + sleep 1 + git diff + sleep 1 ++ nproc + make -j 48 M=drivers/nvme/target/ clean ++ nproc + make -j 48 M=drivers/nvme/ modules CC [M] drivers/nvme/target/core.o CC [M] drivers/nvme/target/configfs.o CC [M] drivers/nvme/target/admin-cmd.o CC [M] drivers/nvme/target/fabrics-cmd.o CC [M] drivers/nvme/target/discovery.o CC [M] drivers/nvme/target/io-cmd-file.o CC [M] drivers/nvme/target/io-cmd-bdev.o CC [M] drivers/nvme/target/passthru.o CC [M] drivers/nvme/target/zns.o CC [M] drivers/nvme/target/trace.o CC [M] drivers/nvme/target/loop.o CC [M] drivers/nvme/target/rdma.o CC [M] drivers/nvme/target/fc.o CC [M] drivers/nvme/target/fcloop.o CC [M] drivers/nvme/target/tcp.o CC [M] drivers/nvme/host/core.o LD [M] drivers/nvme/target/nvme-loop.o LD [M] drivers/nvme/target/nvme-fcloop.o LD [M] drivers/nvme/target/nvmet.o LD [M] drivers/nvme/target/nvmet-fc.o LD [M] drivers/nvme/target/nvmet-tcp.o LD [M] drivers/nvme/target/nvmet-rdma.o LD [M] drivers/nvme/host/nvme-core.o MODPOST drivers/nvme/Module.symvers CC [M] drivers/nvme/host/nvme-core.mod.o CC [M] drivers/nvme/target/nvme-fcloop.mod.o CC [M] drivers/nvme/target/nvme-loop.mod.o CC [M] drivers/nvme/target/nvmet-fc.mod.o CC [M] drivers/nvme/target/nvmet-rdma.mod.o CC [M] drivers/nvme/target/nvmet-tcp.mod.o CC [M] drivers/nvme/target/nvmet.mod.o LD [M] drivers/nvme/host/nvme-core.ko LD [M] drivers/nvme/target/nvmet-fc.ko LD [M] drivers/nvme/target/nvmet-rdma.ko LD [M] drivers/nvme/target/nvme-fcloop.ko LD [M] drivers/nvme/target/nvmet.ko LD [M] drivers/nvme/target/nvme-loop.ko LD [M] drivers/nvme/target/nvmet-tcp.ko + HOST=drivers/nvme/host + TARGET=drivers/nvme/target ++ uname -r + HOST_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ ++ uname -r + TARGET_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target/ + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host// + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// + ls -lrth /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/: total 6.3M -rw-r--r--. 1 root root 2.7M Apr 8 16:59 nvme-core.ko -rw-r--r--. 1 root root 426K Apr 8 16:59 nvme-fabrics.ko -rw-r--r--. 1 root root 925K Apr 8 16:59 nvme-fc.ko -rw-r--r--. 1 root root 714K Apr 8 16:59 nvme.ko -rw-r--r--. 1 root root 856K Apr 8 16:59 nvme-rdma.ko -rw-r--r--. 1 root root 799K Apr 8 16:59 nvme-tcp.ko /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target//: total 6.3M -rw-r--r--. 1 root root 475K Apr 8 16:59 nvme-fcloop.ko -rw-r--r--. 1 root root 419K Apr 8 16:59 nvme-loop.ko -rw-r--r--. 1 root root 734K Apr 8 16:59 nvmet-fc.ko -rw-r--r--. 1 root root 3.2M Apr 8 16:59 nvmet.ko -rw-r--r--. 1 root root 822K Apr 8 16:59 nvmet-rdma.ko -rw-r--r--. 1 root root 671K Apr 8 16:59 nvmet-tcp.ko + sync + sync + sync + modprobe nvme + echo 'Press enter to continue ...' Press enter to continue ... + read next ++ NN=1 ++ NQN=testnqn ++ let NR_DEVICES=NN+1 ++ modprobe -r null_blk ++ modprobe -r nvme ++ modprobe null_blk nr_devices=0 ++ modprobe nvme ++ modprobe nvme-fabrics ++ modprobe nvmet ++ modprobe nvme-loop ++ dmesg -c ++ sleep 2 ++ tree /sys/kernel/config /sys/kernel/config ├── nullb │ └── features └── nvmet ├── hosts ├── ports └── subsystems 5 directories, 1 file ++ sleep 1 ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn +++ shuf -i 1-1 -n 1 ++ for i in `shuf -i 1-$NN -n $NN` ++ mkdir config/nullb/nullb1 ++ echo 1 ++ echo 4096 ++ echo 2048 ++ echo 1 +++ cat config/nullb/nullb1/index ++ IDX=0 ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 ++ echo ' ####### /dev/nullb0' ####### /dev/nullb0 ++ echo -n /dev/nullb0 ++ cat /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1/device_path /dev/nullb0 ++ echo 1 ++ dmesg -c [ 1761.160765] nvme nvme0: 48/0/0 default/read/poll queues [ 1764.151446] nvmet: adding nsid 1 to subsystem testnqn ++ mkdir /sys/kernel/config/nvmet/ports/1/ ++ echo -n loop ++ echo -n 1 ++ ln -s /sys/kernel/config/nvmet/subsystems/testnqn /sys/kernel/config/nvmet/ports/1/subsystems/ ++ echo transport=loop,nqn=testnqn ++ sleep 1 ++ mount ++ column -t ++ grep nvme ++ dmesg -c [ 1764.158721] nvmet: creating nvm controller 1 for subsystem testnqn for NQN nqn.2014-08.org.nvmexpress:uuid:33bca6cd-e82c-4de0-bba2-f70070f69097. [ 1764.158827] nvme nvme1: creating 48 I/O queues. [ 1764.162745] nvme nvme1: new ctrl: "testnqn" Node SN Model Namespace Usage Format FW Rev nvme1n1 bc09a2ee2829a09471a3 Linux 1 2.15 GB / 2.15 GB 4 KiB + 0 B 5.17.0-r nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 NVMe status: Access Denied: Access to the namespace and/or LBA range is denied due to lack of access rights(0x4286) Node SN Model Namespace Usage Format FW Rev nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 [ 1765.175570] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 1000 [ 1765.175579] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1765.175583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1765.175585] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1765.175593] Call Trace: [ 1765.175596] <TASK> [ 1765.175599] dump_stack_lvl+0x48/0x5e [ 1765.175605] should_fail.cold+0x32/0x37 [ 1765.175610] nvme_should_fail+0x38/0x90 [nvme_core] [ 1765.175623] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1765.175627] nvmet_req_complete+0x11/0x50 [nvmet] [ 1765.175636] process_one_work+0x1af/0x380 [ 1765.175640] worker_thread+0x50/0x3a0 [ 1765.175643] ? rescuer_thread+0x370/0x370 [ 1765.175645] kthread+0xe7/0x110 [ 1765.175648] ? kthread_complete_and_exit+0x20/0x20 [ 1765.175652] ret_from_fork+0x22/0x30 [ 1765.175658] </TASK> [ 1765.175664] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR [ 1765.179829] FAULT_INJECTION: forcing a failure. name fault_inject, interval 1, probability 100, space 0, times 999 [ 1765.179835] CPU: 44 PID: 9897 Comm: kworker/44:0 Tainted: G OE 5.17.0-rc2nvme+ #68 [ 1765.179839] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 [ 1765.179841] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] [ 1765.179850] Call Trace: [ 1765.179853] <TASK> [ 1765.179855] dump_stack_lvl+0x48/0x5e [ 1765.179862] should_fail.cold+0x32/0x37 [ 1765.179867] nvme_should_fail+0x38/0x90 [nvme_core] [ 1765.179879] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] [ 1765.179884] nvmet_req_complete+0x11/0x50 [nvmet] [ 1765.179892] process_one_work+0x1af/0x380 [ 1765.179897] ? rescuer_thread+0x370/0x370 [ 1765.179899] worker_thread+0x50/0x3a0 [ 1765.179902] ? rescuer_thread+0x370/0x370 [ 1765.179903] kthread+0xe7/0x110 [ 1765.179907] ? kthread_complete_and_exit+0x20/0x20 [ 1765.179911] ret_from_fork+0x22/0x30 [ 1765.179917] </TASK> [ 1765.179923] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR + NQN=testnqn + nvme disconnect -n testnqn NQN:testnqn disconnected 1 controller(s) real 0m0.347s user 0m0.002s sys 0m0.004s + rm -fr /sys/kernel/config/nvmet/ports/1/subsystems/testnqn + rmdir /sys/kernel/config/nvmet/ports/1 + for subsys in /sys/kernel/config/nvmet/subsystems/* + for ns in ${subsys}/namespaces/* + echo 0 + rmdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 + rmdir /sys/kernel/config/nvmet/subsystems/testnqn + rmdir config/nullb/nullb1 + umount /mnt/nvme0n1 umount: /mnt/nvme0n1: no mount point specified. + umount /mnt/backend umount: /mnt/backend: not mounted. + modprobe -r nvme_loop + modprobe -r nvme_fabrics + modprobe -r nvmet + modprobe -r nvme + modprobe -r null_blk + tree /sys/kernel/config /sys/kernel/config 0 directories, 0 files HEAD is now at 7bec02cef3d1 Merge branch 'nvme-5.18' of git://git.infradead.org/nvme into nvme-5.18 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-09 0:10 ` Chaitanya Kulkarni @ 2022-04-11 18:31 ` Alan Adamson 2022-04-11 19:53 ` Chaitanya Kulkarni 0 siblings, 1 reply; 23+ messages in thread From: Alan Adamson @ 2022-04-11 18:31 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme > On Apr 8, 2022, at 5:10 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > >>> Can you please share a command line for "nvme admin-passthru" >>> where this patch supresses messages ? >> >> I have the NVME Fault Injector configured: >> >> echo 0x286 > /sys/kernel/debug/${ctrl_dev}/fault_inject/status >> echo 1000 > /sys/kernel/debug/${ctrl_dev}/fault_inject/times >> echo 100 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability >> >> nvme admin-passthru /dev/${ctrl_dev} --opcode=06 --data-len=4096 --cdw10=1 -r >> >> echo 0 > /sys/kernel/debug/${ctrl_dev}/fault_inject/probability >> > > I was able to produce same admin-passthru error messages with my patch. > See below detailed execution with the script and the log, can you please > tell me what is missing ? I reapplied your patch and reran my test and all looks good. I must of applied the incorrect patch last week. Alan > >>> -ck >>> >>> >> > > * Without this patch I get 3 Error message from fault injection > enabled :- > 1. [ 1743.353266] nvme1: Identify(0x6), Invalid Field in > Command (sct 0x0 / sc 0x2) MORE DNR > > 2. [ 1744.370690] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 1000 > [ 1744.370698] CPU: 41 PID: 389 Comm: kworker/41:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1744.370702] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1744.370704] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1744.370712] Call Trace: > [ 1744.370715] <TASK> > [ 1744.370717] dump_stack_lvl+0x48/0x5e > [ 1744.370724] should_fail.cold+0x32/0x37 > [ 1744.370729] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1744.370741] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1744.370745] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1744.370754] process_one_work+0x1af/0x380 > [ 1744.370758] worker_thread+0x50/0x3a0 > [ 1744.370761] ? rescuer_thread+0x370/0x370 > [ 1744.370763] kthread+0xe7/0x110 > [ 1744.370767] ? kthread_complete_and_exit+0x20/0x20 > [ 1744.370770] ret_from_fork+0x22/0x30 > [ 1744.370776] </TASK> > [ 1744.370783] nvme1: Identify(0x6), Access Denied > (sct 0x2 / sc 0x86) DNR > > 3. [1744.375045] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 999 > [ 1744.375052] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1744.375056] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1744.375058] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1744.375066] Call Trace: > [ 1744.375070] <TASK> > [ 1744.375072] dump_stack_lvl+0x48/0x5e > [ 1744.375079] should_fail.cold+0x32/0x37 > [ 1744.375084] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1744.375096] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1744.375100] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1744.375108] process_one_work+0x1af/0x380 > [ 1744.375113] worker_thread+0x50/0x3a0 > [ 1744.375115] ? rescuer_thread+0x370/0x370 > [ 1744.375118] kthread+0xe7/0x110 > [ 1744.375121] ? kthread_complete_and_exit+0x20/0x20 > [ 1744.375124] ret_from_fork+0x22/0x30 > [ 1744.375130] </TASK> > [ 1744.375148] nvme1: Identify(0x6), Access Denied > (sct 0x2 / sc 0x86) DNR > > > * With this patch I only get two from fault injection which are coming > from nvme-admin-passthru and no from the internal passthru:- > > 1. [ 1765.175570] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 1000 > [ 1765.175579] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1765.175583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1765.175585] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1765.175593] Call Trace: > [ 1765.175596] <TASK> > [ 1765.175599] dump_stack_lvl+0x48/0x5e > [ 1765.175605] should_fail.cold+0x32/0x37 > [ 1765.175610] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1765.175623] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1765.175627] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1765.175636] process_one_work+0x1af/0x380 > [ 1765.175640] worker_thread+0x50/0x3a0 > [ 1765.175643] ? rescuer_thread+0x370/0x370 > [ 1765.175645] kthread+0xe7/0x110 > [ 1765.175648] ? kthread_complete_and_exit+0x20/0x20 > [ 1765.175652] ret_from_fork+0x22/0x30 > [ 1765.175658] </TASK> > [ 1765.175664] nvme1: Identify(0x6), Access Denied > (sct 0x2 / sc 0x86) DNR > 2. [ 1765.179829] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 999 > [ 1765.179835] CPU: 44 PID: 9897 Comm: kworker/44:0 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1765.179839] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1765.179841] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1765.179850] Call Trace: > [ 1765.179853] <TASK> > [ 1765.179855] dump_stack_lvl+0x48/0x5e > [ 1765.179862] should_fail.cold+0x32/0x37 > [ 1765.179867] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1765.179879] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1765.179884] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1765.179892] process_one_work+0x1af/0x380 > [ 1765.179897] ? rescuer_thread+0x370/0x370 > [ 1765.179899] worker_thread+0x50/0x3a0 > [ 1765.179902] ? rescuer_thread+0x370/0x370 > [ 1765.179903] kthread+0xe7/0x110 > [ 1765.179907] ? kthread_complete_and_exit+0x20/0x20 > [ 1765.179911] ret_from_fork+0x22/0x30 > [ 1765.179917] </TASK> > [ 1765.179923] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR > > > * Detailed test log with nvmeof nvme-loop controller configured > and fault injection enabled with and without this patch to mask > internal passthru. It shows only internal passthru error message > is masked with this patch and internal passthru error message > is present without this patch, nvme-admin-passthru error message > present in both cases :- > > nvme (nvme-5.18) # sh error_inject.sh > commit 7bec02cef3d11f3d3a80bfa8739f790377bac8d6 (HEAD -> nvme-5.18) > Merge: 226d991feef9 a4a6f3c8f61c > Author: Chaitanya Kulkarni <kch@nvidia.com> > Date: Fri Apr 8 16:57:45 2022 -0700 > > Merge branch 'nvme-5.18' of git://git.infradead.org/nvme into nvme-5.18 > + umount /mnt/nvme0n1 > + clear_dmesg > ./compile_nvme.sh: line 3: clear_dmesg: command not found > umount: /mnt/nvme0n1: no mount point specified. > + ./delete.sh > + NQN=testnqn > + nvme disconnect -n testnqn > Failed to scan topoplogy: No such file or directory > > real 0m0.002s > user 0m0.000s > sys 0m0.002s > + rm -fr '/sys/kernel/config/nvmet/ports/1/subsystems/*' > + rmdir /sys/kernel/config/nvmet/ports/1 > rmdir: failed to remove '/sys/kernel/config/nvmet/ports/1': No such file > or directory > + for subsys in /sys/kernel/config/nvmet/subsystems/* > + for ns in ${subsys}/namespaces/* > + echo 0 > ./delete.sh: line 14: > /sys/kernel/config/nvmet/subsystems/*/namespaces/*/enable: No such file > or directory > + rmdir '/sys/kernel/config/nvmet/subsystems/*/namespaces/*' > rmdir: failed to remove > '/sys/kernel/config/nvmet/subsystems/*/namespaces/*': No such file or > directory > + rmdir '/sys/kernel/config/nvmet/subsystems/*' > rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*': No such > file or directory > + rmdir 'config/nullb/nullb*' > rmdir: failed to remove 'config/nullb/nullb*': No such file or directory > + umount /mnt/nvme0n1 > umount: /mnt/nvme0n1: no mount point specified. > + umount /mnt/backend > umount: /mnt/backend: not mounted. > + modprobe -r nvme_loop > + modprobe -r nvme_fabrics > + modprobe -r nvmet > + modprobe -r nvme > + modprobe -r null_blk > + tree /sys/kernel/config > /sys/kernel/config > > 0 directories, 0 files > + modprobe -r nvme-fabrics > + modprobe -r nvme_loop > + modprobe -r nvmet > + modprobe -r nvme > + sleep 1 > + modprobe -r nvme-core > + lsmod > + grep nvme > + sleep 1 > + git diff > + sleep 1 > ++ nproc > + make -j 48 M=drivers/nvme/target/ clean > ++ nproc > + make -j 48 M=drivers/nvme/ modules > CC [M] drivers/nvme/target/core.o > CC [M] drivers/nvme/target/configfs.o > CC [M] drivers/nvme/target/admin-cmd.o > CC [M] drivers/nvme/target/fabrics-cmd.o > CC [M] drivers/nvme/target/discovery.o > CC [M] drivers/nvme/target/io-cmd-file.o > CC [M] drivers/nvme/target/io-cmd-bdev.o > CC [M] drivers/nvme/target/passthru.o > CC [M] drivers/nvme/target/zns.o > CC [M] drivers/nvme/target/trace.o > CC [M] drivers/nvme/target/loop.o > CC [M] drivers/nvme/target/rdma.o > CC [M] drivers/nvme/target/fc.o > CC [M] drivers/nvme/target/fcloop.o > CC [M] drivers/nvme/target/tcp.o > CC [M] drivers/nvme/host/core.o > LD [M] drivers/nvme/target/nvme-loop.o > LD [M] drivers/nvme/target/nvme-fcloop.o > LD [M] drivers/nvme/target/nvmet.o > LD [M] drivers/nvme/target/nvmet-tcp.o > LD [M] drivers/nvme/target/nvmet-fc.o > LD [M] drivers/nvme/target/nvmet-rdma.o > LD [M] drivers/nvme/host/nvme-core.o > MODPOST drivers/nvme/Module.symvers > LD [M] drivers/nvme/host/nvme-core.ko > CC [M] drivers/nvme/target/nvme-fcloop.mod.o > CC [M] drivers/nvme/target/nvme-loop.mod.o > CC [M] drivers/nvme/target/nvmet-fc.mod.o > CC [M] drivers/nvme/target/nvmet-rdma.mod.o > CC [M] drivers/nvme/target/nvmet-tcp.mod.o > CC [M] drivers/nvme/target/nvmet.mod.o > LD [M] drivers/nvme/target/nvme-loop.ko > LD [M] drivers/nvme/target/nvme-fcloop.ko > LD [M] drivers/nvme/target/nvmet-rdma.ko > LD [M] drivers/nvme/target/nvmet-fc.ko > LD [M] drivers/nvme/target/nvmet-tcp.ko > LD [M] drivers/nvme/target/nvmet.ko > + HOST=drivers/nvme/host > + TARGET=drivers/nvme/target > ++ uname -r > + HOST_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ > ++ uname -r > + TARGET_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target/ > + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko > drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko > drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host// > + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko > drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko > drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// > + ls -lrth /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/: > total 6.3M > -rw-r--r--. 1 root root 2.7M Apr 8 16:58 nvme-core.ko > -rw-r--r--. 1 root root 426K Apr 8 16:58 nvme-fabrics.ko > -rw-r--r--. 1 root root 925K Apr 8 16:58 nvme-fc.ko > -rw-r--r--. 1 root root 714K Apr 8 16:58 nvme.ko > -rw-r--r--. 1 root root 856K Apr 8 16:58 nvme-rdma.ko > -rw-r--r--. 1 root root 799K Apr 8 16:58 nvme-tcp.ko > > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target//: > total 6.3M > -rw-r--r--. 1 root root 475K Apr 8 16:58 nvme-fcloop.ko > -rw-r--r--. 1 root root 419K Apr 8 16:58 nvme-loop.ko > -rw-r--r--. 1 root root 734K Apr 8 16:58 nvmet-fc.ko > -rw-r--r--. 1 root root 3.2M Apr 8 16:58 nvmet.ko > -rw-r--r--. 1 root root 822K Apr 8 16:58 nvmet-rdma.ko > -rw-r--r--. 1 root root 671K Apr 8 16:58 nvmet-tcp.ko > + sync > + sync > + sync > + modprobe nvme > + echo 'Press enter to continue ...' > Press enter to continue ... > + read next > > ++ NN=1 > ++ NQN=testnqn > ++ let NR_DEVICES=NN+1 > ++ modprobe -r null_blk > ++ modprobe -r nvme > ++ modprobe null_blk nr_devices=0 > ++ modprobe nvme > ++ modprobe nvme-fabrics > ++ modprobe nvmet > ++ modprobe nvme-loop > ++ dmesg -c > ++ sleep 2 > ++ tree /sys/kernel/config > /sys/kernel/config > ├── nullb > │ └── features > └── nvmet > ├── hosts > ├── ports > └── subsystems > > 5 directories, 1 file > ++ sleep 1 > ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn > +++ shuf -i 1-1 -n 1 > ++ for i in `shuf -i 1-$NN -n $NN` > ++ mkdir config/nullb/nullb1 > ++ echo 1 > ++ echo 4096 > ++ echo 2048 > ++ echo 1 > +++ cat config/nullb/nullb1/index > ++ IDX=0 > ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 > ++ echo ' ####### /dev/nullb0' > ####### /dev/nullb0 > ++ echo -n /dev/nullb0 > ++ cat /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1/device_path > /dev/nullb0 > ++ echo 1 > ++ dmesg -c > [ 1740.356540] nvme nvme0: 48/0/0 default/read/poll queues > [ 1743.345785] nvmet: adding nsid 1 to subsystem testnqn > ++ mkdir /sys/kernel/config/nvmet/ports/1/ > ++ echo -n loop > ++ echo -n 1 > ++ ln -s /sys/kernel/config/nvmet/subsystems/testnqn > /sys/kernel/config/nvmet/ports/1/subsystems/ > ++ echo transport=loop,nqn=testnqn > ++ sleep 1 > ++ mount > ++ column -t > ++ grep nvme > ++ dmesg -c > [ 1743.353177] nvmet: creating nvm controller 1 for subsystem testnqn > for NQN > nqn.2014-08.org.nvmexpress:uuid:510d0435-0ad7-49d4-ae4a-f1c1552b0f0c. > [ 1743.353266] nvme1: Identify(0x6), Invalid Field in Command (sct 0x0 / > sc 0x2) MORE DNR > [ 1743.355380] nvme nvme1: creating 48 I/O queues. > [ 1743.359206] nvme nvme1: new ctrl: "testnqn" > Node SN Model Namespace Usage Format FW Rev > nvme1n1 8bdab4b79aca987d0eba Linux 1 2.15 GB / 2.15 GB 4 KiB + 0 B 5.17.0-r > nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 > NVMe status: Access Denied: Access to the namespace and/or LBA range is > denied due to lack of access rights(0x4286) > Node SN Model Namespace Usage Format FW Rev > nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 > [ 1744.370690] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 1000 > [ 1744.370698] CPU: 41 PID: 389 Comm: kworker/41:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1744.370702] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1744.370704] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1744.370712] Call Trace: > [ 1744.370715] <TASK> > [ 1744.370717] dump_stack_lvl+0x48/0x5e > [ 1744.370724] should_fail.cold+0x32/0x37 > [ 1744.370729] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1744.370741] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1744.370745] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1744.370754] process_one_work+0x1af/0x380 > [ 1744.370758] worker_thread+0x50/0x3a0 > [ 1744.370761] ? rescuer_thread+0x370/0x370 > [ 1744.370763] kthread+0xe7/0x110 > [ 1744.370767] ? kthread_complete_and_exit+0x20/0x20 > [ 1744.370770] ret_from_fork+0x22/0x30 > [ 1744.370776] </TASK> > [ 1744.370783] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR > [ 1744.375045] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 999 > [ 1744.375052] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1744.375056] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1744.375058] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1744.375066] Call Trace: > [ 1744.375070] <TASK> > [ 1744.375072] dump_stack_lvl+0x48/0x5e > [ 1744.375079] should_fail.cold+0x32/0x37 > [ 1744.375084] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1744.375096] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1744.375100] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1744.375108] process_one_work+0x1af/0x380 > [ 1744.375113] worker_thread+0x50/0x3a0 > [ 1744.375115] ? rescuer_thread+0x370/0x370 > [ 1744.375118] kthread+0xe7/0x110 > [ 1744.375121] ? kthread_complete_and_exit+0x20/0x20 > [ 1744.375124] ret_from_fork+0x22/0x30 > [ 1744.375130] </TASK> > [ 1744.375148] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR > + NQN=testnqn > + nvme disconnect -n testnqn > NQN:testnqn disconnected 1 controller(s) > > real 0m0.370s > user 0m0.001s > sys 0m0.004s > + rm -fr /sys/kernel/config/nvmet/ports/1/subsystems/testnqn > + rmdir /sys/kernel/config/nvmet/ports/1 > + for subsys in /sys/kernel/config/nvmet/subsystems/* > + for ns in ${subsys}/namespaces/* > + echo 0 > + rmdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 > + rmdir /sys/kernel/config/nvmet/subsystems/testnqn > + rmdir config/nullb/nullb1 > + umount /mnt/nvme0n1 > umount: /mnt/nvme0n1: no mount point specified. > + umount /mnt/backend > umount: /mnt/backend: not mounted. > + modprobe -r nvme_loop > + modprobe -r nvme_fabrics > + modprobe -r nvmet > + modprobe -r nvme > + modprobe -r null_blk > + tree /sys/kernel/config > /sys/kernel/config > > 0 directories, 0 files > From 2d552b2c756fce48f53d66c5c58ffb6b3e3cac6e Mon Sep 17 00:00:00 2001 > From: Chaitanya Kulkarni <kch@nvidia.com> > Date: Fri, 8 Apr 2022 16:45:18 -0700 > Subject: [PATCH] nvme-core: mark internal passthru req REQ_QUIET > > Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> > --- > drivers/nvme/host/core.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > index f204c6f78b5b..a1ea2f736d42 100644 > --- a/drivers/nvme/host/core.c > +++ b/drivers/nvme/host/core.c > @@ -370,7 +370,7 @@ static inline void nvme_end_req(struct request *req) > { > blk_status_t status = nvme_error_status(nvme_req(req)->status); > > - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) > + if (unlikely(nvme_req(req)->status && !(req->rq_flags & RQF_QUIET))) > nvme_log_error(req); > nvme_end_req_zoned(req); > nvme_trace_bio_complete(req); > @@ -1086,9 +1086,11 @@ int __nvme_submit_sync_cmd(struct request_queue > *q, struct nvme_command *cmd, > else > req = blk_mq_alloc_request_hctx(q, nvme_req_op(cmd), flags, > qid ? qid - 1 : 0); > - > if (IS_ERR(req)) > return PTR_ERR(req); > + > + req->rq_flags |= RQF_QUIET; > + > nvme_init_request(req, cmd); > > if (timeout) > -- > 2.29.0 > > Applying: nvme-core: mark internal passthru req REQ_QUIET > > commit 3c7952e0f8fcc9affdfa0249ab8211c18a513338 (HEAD -> nvme-5.18) > Author: Chaitanya Kulkarni <kch@nvidia.com> > Date: Fri Apr 8 16:45:18 2022 -0700 > > nvme-core: mark internal passthru req REQ_QUIET > > Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com> > + umount /mnt/nvme0n1 > + clear_dmesg > ./compile_nvme.sh: line 3: clear_dmesg: command not found > umount: /mnt/nvme0n1: no mount point specified. > + ./delete.sh > + NQN=testnqn > + nvme disconnect -n testnqn > Failed to scan topoplogy: No such file or directory > > real 0m0.002s > user 0m0.002s > sys 0m0.000s > + rm -fr '/sys/kernel/config/nvmet/ports/1/subsystems/*' > + rmdir /sys/kernel/config/nvmet/ports/1 > rmdir: failed to remove '/sys/kernel/config/nvmet/ports/1': No such file > or directory > + for subsys in /sys/kernel/config/nvmet/subsystems/* > + for ns in ${subsys}/namespaces/* > + echo 0 > ./delete.sh: line 14: > /sys/kernel/config/nvmet/subsystems/*/namespaces/*/enable: No such file > or directory > + rmdir '/sys/kernel/config/nvmet/subsystems/*/namespaces/*' > rmdir: failed to remove > '/sys/kernel/config/nvmet/subsystems/*/namespaces/*': No such file or > directory > + rmdir '/sys/kernel/config/nvmet/subsystems/*' > rmdir: failed to remove '/sys/kernel/config/nvmet/subsystems/*': No such > file or directory > + rmdir 'config/nullb/nullb*' > rmdir: failed to remove 'config/nullb/nullb*': No such file or directory > + umount /mnt/nvme0n1 > umount: /mnt/nvme0n1: no mount point specified. > + umount /mnt/backend > umount: /mnt/backend: not mounted. > + modprobe -r nvme_loop > + modprobe -r nvme_fabrics > + modprobe -r nvmet > + modprobe -r nvme > + modprobe -r null_blk > + tree /sys/kernel/config > /sys/kernel/config > > 0 directories, 0 files > + modprobe -r nvme-fabrics > + modprobe -r nvme_loop > + modprobe -r nvmet > + modprobe -r nvme > + sleep 1 > + modprobe -r nvme-core > + lsmod > + grep nvme > + sleep 1 > + git diff > + sleep 1 > ++ nproc > + make -j 48 M=drivers/nvme/target/ clean > ++ nproc > + make -j 48 M=drivers/nvme/ modules > CC [M] drivers/nvme/target/core.o > CC [M] drivers/nvme/target/configfs.o > CC [M] drivers/nvme/target/admin-cmd.o > CC [M] drivers/nvme/target/fabrics-cmd.o > CC [M] drivers/nvme/target/discovery.o > CC [M] drivers/nvme/target/io-cmd-file.o > CC [M] drivers/nvme/target/io-cmd-bdev.o > CC [M] drivers/nvme/target/passthru.o > CC [M] drivers/nvme/target/zns.o > CC [M] drivers/nvme/target/trace.o > CC [M] drivers/nvme/target/loop.o > CC [M] drivers/nvme/target/rdma.o > CC [M] drivers/nvme/target/fc.o > CC [M] drivers/nvme/target/fcloop.o > CC [M] drivers/nvme/target/tcp.o > CC [M] drivers/nvme/host/core.o > LD [M] drivers/nvme/target/nvme-loop.o > LD [M] drivers/nvme/target/nvme-fcloop.o > LD [M] drivers/nvme/target/nvmet.o > LD [M] drivers/nvme/target/nvmet-fc.o > LD [M] drivers/nvme/target/nvmet-tcp.o > LD [M] drivers/nvme/target/nvmet-rdma.o > LD [M] drivers/nvme/host/nvme-core.o > MODPOST drivers/nvme/Module.symvers > CC [M] drivers/nvme/host/nvme-core.mod.o > CC [M] drivers/nvme/target/nvme-fcloop.mod.o > CC [M] drivers/nvme/target/nvme-loop.mod.o > CC [M] drivers/nvme/target/nvmet-fc.mod.o > CC [M] drivers/nvme/target/nvmet-rdma.mod.o > CC [M] drivers/nvme/target/nvmet-tcp.mod.o > CC [M] drivers/nvme/target/nvmet.mod.o > LD [M] drivers/nvme/host/nvme-core.ko > LD [M] drivers/nvme/target/nvmet-fc.ko > LD [M] drivers/nvme/target/nvmet-rdma.ko > LD [M] drivers/nvme/target/nvme-fcloop.ko > LD [M] drivers/nvme/target/nvmet.ko > LD [M] drivers/nvme/target/nvme-loop.ko > LD [M] drivers/nvme/target/nvmet-tcp.ko > + HOST=drivers/nvme/host > + TARGET=drivers/nvme/target > ++ uname -r > + HOST_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ > ++ uname -r > + TARGET_DEST=/lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target/ > + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko > drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko > drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host// > + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko > drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko > drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// > + ls -lrth /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/ > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target// > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/host/: > total 6.3M > -rw-r--r--. 1 root root 2.7M Apr 8 16:59 nvme-core.ko > -rw-r--r--. 1 root root 426K Apr 8 16:59 nvme-fabrics.ko > -rw-r--r--. 1 root root 925K Apr 8 16:59 nvme-fc.ko > -rw-r--r--. 1 root root 714K Apr 8 16:59 nvme.ko > -rw-r--r--. 1 root root 856K Apr 8 16:59 nvme-rdma.ko > -rw-r--r--. 1 root root 799K Apr 8 16:59 nvme-tcp.ko > > /lib/modules/5.17.0-rc2nvme+/kernel/drivers/nvme/target//: > total 6.3M > -rw-r--r--. 1 root root 475K Apr 8 16:59 nvme-fcloop.ko > -rw-r--r--. 1 root root 419K Apr 8 16:59 nvme-loop.ko > -rw-r--r--. 1 root root 734K Apr 8 16:59 nvmet-fc.ko > -rw-r--r--. 1 root root 3.2M Apr 8 16:59 nvmet.ko > -rw-r--r--. 1 root root 822K Apr 8 16:59 nvmet-rdma.ko > -rw-r--r--. 1 root root 671K Apr 8 16:59 nvmet-tcp.ko > + sync > + sync > + sync > + modprobe nvme > + echo 'Press enter to continue ...' > Press enter to continue ... > + read next > > ++ NN=1 > ++ NQN=testnqn > ++ let NR_DEVICES=NN+1 > ++ modprobe -r null_blk > ++ modprobe -r nvme > ++ modprobe null_blk nr_devices=0 > ++ modprobe nvme > ++ modprobe nvme-fabrics > ++ modprobe nvmet > ++ modprobe nvme-loop > ++ dmesg -c > ++ sleep 2 > ++ tree /sys/kernel/config > /sys/kernel/config > ├── nullb > │ └── features > └── nvmet > ├── hosts > ├── ports > └── subsystems > > 5 directories, 1 file > ++ sleep 1 > ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn > +++ shuf -i 1-1 -n 1 > ++ for i in `shuf -i 1-$NN -n $NN` > ++ mkdir config/nullb/nullb1 > ++ echo 1 > ++ echo 4096 > ++ echo 2048 > ++ echo 1 > +++ cat config/nullb/nullb1/index > ++ IDX=0 > ++ mkdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 > ++ echo ' ####### /dev/nullb0' > ####### /dev/nullb0 > ++ echo -n /dev/nullb0 > ++ cat /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1/device_path > /dev/nullb0 > ++ echo 1 > ++ dmesg -c > [ 1761.160765] nvme nvme0: 48/0/0 default/read/poll queues > [ 1764.151446] nvmet: adding nsid 1 to subsystem testnqn > ++ mkdir /sys/kernel/config/nvmet/ports/1/ > ++ echo -n loop > ++ echo -n 1 > ++ ln -s /sys/kernel/config/nvmet/subsystems/testnqn > /sys/kernel/config/nvmet/ports/1/subsystems/ > ++ echo transport=loop,nqn=testnqn > ++ sleep 1 > ++ mount > ++ column -t > ++ grep nvme > ++ dmesg -c > [ 1764.158721] nvmet: creating nvm controller 1 for subsystem testnqn > for NQN > nqn.2014-08.org.nvmexpress:uuid:33bca6cd-e82c-4de0-bba2-f70070f69097. > [ 1764.158827] nvme nvme1: creating 48 I/O queues. > [ 1764.162745] nvme nvme1: new ctrl: "testnqn" > Node SN Model Namespace Usage Format FW Rev > nvme1n1 bc09a2ee2829a09471a3 Linux 1 2.15 GB / 2.15 GB 4 KiB + 0 B 5.17.0-r > nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 > NVMe status: Access Denied: Access to the namespace and/or LBA range is > denied due to lack of access rights(0x4286) > Node SN Model Namespace Usage Format FW Rev > nvme0n1 foo QEMU NVMe Ctrl 1 1.07 GB / 1.07 GB 512 B + 0 B 1.0 > [ 1765.175570] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 1000 > [ 1765.175579] CPU: 42 PID: 391 Comm: kworker/42:1 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1765.175583] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1765.175585] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1765.175593] Call Trace: > [ 1765.175596] <TASK> > [ 1765.175599] dump_stack_lvl+0x48/0x5e > [ 1765.175605] should_fail.cold+0x32/0x37 > [ 1765.175610] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1765.175623] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1765.175627] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1765.175636] process_one_work+0x1af/0x380 > [ 1765.175640] worker_thread+0x50/0x3a0 > [ 1765.175643] ? rescuer_thread+0x370/0x370 > [ 1765.175645] kthread+0xe7/0x110 > [ 1765.175648] ? kthread_complete_and_exit+0x20/0x20 > [ 1765.175652] ret_from_fork+0x22/0x30 > [ 1765.175658] </TASK> > [ 1765.175664] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR > [ 1765.179829] FAULT_INJECTION: forcing a failure. > name fault_inject, interval 1, probability 100, space 0, > times 999 > [ 1765.179835] CPU: 44 PID: 9897 Comm: kworker/44:0 Tainted: G > OE 5.17.0-rc2nvme+ #68 > [ 1765.179839] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), > BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014 > [ 1765.179841] Workqueue: nvmet-wq nvme_loop_execute_work [nvme_loop] > [ 1765.179850] Call Trace: > [ 1765.179853] <TASK> > [ 1765.179855] dump_stack_lvl+0x48/0x5e > [ 1765.179862] should_fail.cold+0x32/0x37 > [ 1765.179867] nvme_should_fail+0x38/0x90 [nvme_core] > [ 1765.179879] nvme_loop_queue_response+0xc9/0x143 [nvme_loop] > [ 1765.179884] nvmet_req_complete+0x11/0x50 [nvmet] > [ 1765.179892] process_one_work+0x1af/0x380 > [ 1765.179897] ? rescuer_thread+0x370/0x370 > [ 1765.179899] worker_thread+0x50/0x3a0 > [ 1765.179902] ? rescuer_thread+0x370/0x370 > [ 1765.179903] kthread+0xe7/0x110 > [ 1765.179907] ? kthread_complete_and_exit+0x20/0x20 > [ 1765.179911] ret_from_fork+0x22/0x30 > [ 1765.179917] </TASK> > [ 1765.179923] nvme1: Identify(0x6), Access Denied (sct 0x2 / sc 0x86) DNR > + NQN=testnqn > + nvme disconnect -n testnqn > NQN:testnqn disconnected 1 controller(s) > > real 0m0.347s > user 0m0.002s > sys 0m0.004s > + rm -fr /sys/kernel/config/nvmet/ports/1/subsystems/testnqn > + rmdir /sys/kernel/config/nvmet/ports/1 > + for subsys in /sys/kernel/config/nvmet/subsystems/* > + for ns in ${subsys}/namespaces/* > + echo 0 > + rmdir /sys/kernel/config/nvmet/subsystems/testnqn/namespaces/1 > + rmdir /sys/kernel/config/nvmet/subsystems/testnqn > + rmdir config/nullb/nullb1 > + umount /mnt/nvme0n1 > umount: /mnt/nvme0n1: no mount point specified. > + umount /mnt/backend > umount: /mnt/backend: not mounted. > + modprobe -r nvme_loop > + modprobe -r nvme_fabrics > + modprobe -r nvmet > + modprobe -r nvme > + modprobe -r null_blk > + tree /sys/kernel/config > /sys/kernel/config > > 0 directories, 0 files > HEAD is now at 7bec02cef3d1 Merge branch 'nvme-5.18' of > git://git.infradead.org/nvme into nvme-5.18 > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-11 18:31 ` Alan Adamson @ 2022-04-11 19:53 ` Chaitanya Kulkarni 2022-04-11 21:39 ` Alan Adamson 0 siblings, 1 reply; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-11 19:53 UTC (permalink / raw) To: Alan Adamson; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme >> I was able to produce same admin-passthru error messages with my patch. >> See below detailed execution with the script and the log, can you please >> tell me what is missing ? > > I reapplied your patch and reran my test and all looks good. I must of applied the > incorrect patch last week. > That's what I thought, it will be great if you can send tested by tag this so we can send out later this week :- http://lists.infradead.org/pipermail/linux-nvme/2022-April/031349.html > Alan > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-11 19:53 ` Chaitanya Kulkarni @ 2022-04-11 21:39 ` Alan Adamson 0 siblings, 0 replies; 23+ messages in thread From: Alan Adamson @ 2022-04-11 21:39 UTC (permalink / raw) To: Chaitanya Kulkarni; +Cc: Sagi Grimberg, hch, Keith Busch, linux-nvme - Verified the error log is no longer being seen at boot up. - Verified errors can still be logged while using 'nvme admin-passthru’. Tested-by: Alan Adamson <alan.adamson@oracle.com> > On Apr 11, 2022, at 12:53 PM, Chaitanya Kulkarni <chaitanyak@nvidia.com> wrote: > > >>> I was able to produce same admin-passthru error messages with my patch. >>> See below detailed execution with the script and the log, can you please >>> tell me what is missing ? >> >> I reapplied your patch and reran my test and all looks good. I must of applied the >> incorrect patch last week. >> > > That's what I thought, it will be great if you can send tested by tag > this so we can send out later this week :- > > http://lists.infradead.org/pipermail/linux-nvme/2022-April/031349.html > > >> Alan >> >> > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-07 8:51 ` hch 2022-04-07 11:52 ` Sagi Grimberg @ 2022-04-07 16:15 ` Alan Adamson 1 sibling, 0 replies; 23+ messages in thread From: Alan Adamson @ 2022-04-07 16:15 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Keith Busch, Chaitanya Kulkarni, linux-nvme > On Apr 7, 2022, at 1:51 AM, hch@lst.de wrote: > > On Wed, Apr 06, 2022 at 10:06:06PM +0000, Alan Adamson wrote: >> Using RQF_QUIET or blk_rq_is_passrhrough() will mean no nvme admin-passthru command will log an error. >> I ran into this using the blktests I’m coding up for verbose errors. Is this the behavior we want? > > Well, you submitted the logging so we're curious about your use case. > SCSI skips logging errors for internally submitted commands, but not for > userspace passthrough. So we could move the RQF_QUIET into > __nvme_submit_sync_cmd / nvme_keep_alive_work / nvme_timeout / etc. I think that's what we want, skip logging for nvme driver submitted commands, and allow logging for passthrough commands. Alan ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] nvme-core: mark passthru requests RQF_QUIET flag 2022-04-06 16:52 ` Keith Busch 2022-04-06 17:01 ` Chaitanya Kulkarni @ 2022-04-06 17:03 ` Chaitanya Kulkarni 1 sibling, 0 replies; 23+ messages in thread From: Chaitanya Kulkarni @ 2022-04-06 17:03 UTC (permalink / raw) To: Keith Busch, Chaitanya Kulkarni; +Cc: linux-nvme, hch, alan.adamson On 4/6/22 09:52, Keith Busch wrote: > On Wed, Apr 06, 2022 at 09:41:09AM -0700, Chaitanya Kulkarni wrote: >> @@ -370,7 +370,8 @@ static inline void nvme_end_req(struct request *req) >> { >> blk_status_t status = nvme_error_status(nvme_req(req)->status); >> >> - if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS)) >> + if (unlikely(nvme_req(req)->status != NVME_SC_SUCCESS && >> + !(req->rq_flags & RQF_QUIET))) >> nvme_log_error(req); >> nvme_end_req_zoned(req); >> nvme_trace_bio_complete(req); >> @@ -651,6 +652,7 @@ void nvme_init_request(struct request *req, struct nvme_command *cmd) >> cmd->common.flags &= ~NVME_CMD_SGL_ALL; >> >> req->cmd_flags |= REQ_FAILFAST_DRIVER; >> + req->rq_flags |= RQF_QUIET; > > This defeats the admin error logging logic since every admin command comes > through here. If you're sure we should do this, then I suppose you can remove > that unreachable code. Perhaps we should do the req->rq_flags |= RQF_QUIET in the respective callers ? just thinking out loudly.. -ck ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2022-04-11 21:39 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-04-06 16:41 [PATCH] nvme-core: mark passthru requests RQF_QUIET flag Chaitanya Kulkarni 2022-04-06 16:52 ` Keith Busch 2022-04-06 17:01 ` Chaitanya Kulkarni 2022-04-06 17:21 ` Keith Busch 2022-04-06 22:06 ` Alan Adamson 2022-04-06 22:16 ` Chaitanya Kulkarni 2022-04-06 23:29 ` Alan Adamson 2022-04-07 19:40 ` Chaitanya Kulkarni 2022-04-07 8:48 ` Christoph Hellwig 2022-04-07 8:51 ` hch 2022-04-07 11:52 ` Sagi Grimberg 2022-04-07 20:10 ` Chaitanya Kulkarni 2022-04-07 20:35 ` Alan Adamson 2022-04-07 21:00 ` Chaitanya Kulkarni 2022-04-07 21:13 ` Alan Adamson 2022-04-08 2:13 ` Chaitanya Kulkarni 2022-04-08 16:24 ` Alan Adamson 2022-04-09 0:10 ` Chaitanya Kulkarni 2022-04-11 18:31 ` Alan Adamson 2022-04-11 19:53 ` Chaitanya Kulkarni 2022-04-11 21:39 ` Alan Adamson 2022-04-07 16:15 ` Alan Adamson 2022-04-06 17:03 ` Chaitanya Kulkarni
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.