From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D37C433E0 for ; Thu, 7 Jan 2021 03:32:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 11F3E23100 for ; Thu, 7 Jan 2021 03:32:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726773AbhAGDcn (ORCPT ); Wed, 6 Jan 2021 22:32:43 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:9962 "EHLO szxga06-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726774AbhAGDcj (ORCPT ); Wed, 6 Jan 2021 22:32:39 -0500 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4DBBbM09nrzj3RW; Thu, 7 Jan 2021 11:31:11 +0800 (CST) Received: from huawei.com (10.29.88.127) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Thu, 7 Jan 2021 11:31:52 +0800 From: Chao Leng To: CC: , , , , , Subject: [PATCH v2 4/6] nvme-rdma: avoid IO error and repeated request completion Date: Thu, 7 Jan 2021 11:31:47 +0800 Message-ID: <20210107033149.15701-5-lengchao@huawei.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20210107033149.15701-1-lengchao@huawei.com> References: <20210107033149.15701-1-lengchao@huawei.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.29.88.127] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org When a request is queued failed, blk_status_t is directly returned to the blk-mq. If blk_status_t is not BLK_STS_RESOURCE, BLK_STS_DEV_RESOURCE, BLK_STS_ZONE_RESOURCE, blk-mq call blk_mq_end_request to complete the request with BLK_STS_IOERR. In two scenarios, the request should be retried and may succeed. First, if work with nvme multipath, the request may be retried successfully in another path, because the error is probably related to the path. Second, if work without multipath software, the request may be retried successfully after error recovery. If the request is complete with BLK_STS_IOERR in blk_mq_dispatch_rq_list. The state of request may be changed to MQ_RQ_IN_FLIGHT. If free the request asynchronously such as in nvme_submit_user_cmd, in extreme scenario the request will be repeated freed in tear down. If a non-resource error occurs in queue_rq, should directly call nvme_complete_rq to complete request and set the state of request to MQ_RQ_COMPLETE. nvme_complete_rq will decide to retry, fail over or end the request. Signed-off-by: Chao Leng --- drivers/nvme/host/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index df9f6f4549f1..4a89bf44ecdc 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -2093,7 +2093,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, unmap_qe: ib_dma_unmap_single(dev, req->sqe.dma, sizeof(struct nvme_command), DMA_TO_DEVICE); - return ret; + return nvme_try_complete_failed_req(rq, ret); } static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx) -- 2.16.4 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76E19C433DB for ; Thu, 7 Jan 2021 03:32:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C4CA22D01 for ; Thu, 7 Jan 2021 03:32:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C4CA22D01 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OYjAg8/LToukUrbhLmscXqMwLhVp5QWO9j6ckLoToZs=; b=e1Wofg6hR0cIfLQCuj4QjsHSW sr9WPvvCQl5bLJlrfghWDbC7JY37j2pVLYgX1URT21wP4MCJu8tZWA36SmrTq1/COB9kz0l86d8mk 7/dKt1CuZZPVFFv/1gq5zsGL+MkUluog8am/PuZhIcYDHOB1fZFYq6DwICRpizKTHguaGjR4WlKwH 2h3NWI8pwXF3NAR/zA5CeLJTHFoXcI8en/f26zXi2zSXYQ5nNseXRBqnV+VBeQ1sfN1oxCi2ttrHR 3KW3OFxWi0heScu++2n26VdyHDX5vWtjq69z2FTYOstU0e1ax5KVUORpfYqUCTRoRUh41DwQ+I8H+ Eauwwosvg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxM1y-0001M0-Tk; Thu, 07 Jan 2021 03:32:10 +0000 Received: from szxga06-in.huawei.com ([45.249.212.32]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxM1u-0001JX-1e for linux-nvme@lists.infradead.org; Thu, 07 Jan 2021 03:32:07 +0000 Received: from DGGEMS403-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4DBBbM09nrzj3RW; Thu, 7 Jan 2021 11:31:11 +0800 (CST) Received: from huawei.com (10.29.88.127) by DGGEMS403-HUB.china.huawei.com (10.3.19.203) with Microsoft SMTP Server id 14.3.498.0; Thu, 7 Jan 2021 11:31:52 +0800 From: Chao Leng To: Subject: [PATCH v2 4/6] nvme-rdma: avoid IO error and repeated request completion Date: Thu, 7 Jan 2021 11:31:47 +0800 Message-ID: <20210107033149.15701-5-lengchao@huawei.com> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20210107033149.15701-1-lengchao@huawei.com> References: <20210107033149.15701-1-lengchao@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.29.88.127] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210106_223206_357611_0B9B5F4D X-CRM114-Status: GOOD ( 14.16 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: axboe@kernel.dk, linux-block@vger.kernel.org, sagi@grimberg.me, axboe@fb.com, kbusch@kernel.org, hch@lst.de Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org When a request is queued failed, blk_status_t is directly returned to the blk-mq. If blk_status_t is not BLK_STS_RESOURCE, BLK_STS_DEV_RESOURCE, BLK_STS_ZONE_RESOURCE, blk-mq call blk_mq_end_request to complete the request with BLK_STS_IOERR. In two scenarios, the request should be retried and may succeed. First, if work with nvme multipath, the request may be retried successfully in another path, because the error is probably related to the path. Second, if work without multipath software, the request may be retried successfully after error recovery. If the request is complete with BLK_STS_IOERR in blk_mq_dispatch_rq_list. The state of request may be changed to MQ_RQ_IN_FLIGHT. If free the request asynchronously such as in nvme_submit_user_cmd, in extreme scenario the request will be repeated freed in tear down. If a non-resource error occurs in queue_rq, should directly call nvme_complete_rq to complete request and set the state of request to MQ_RQ_COMPLETE. nvme_complete_rq will decide to retry, fail over or end the request. Signed-off-by: Chao Leng --- drivers/nvme/host/rdma.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index df9f6f4549f1..4a89bf44ecdc 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -2093,7 +2093,7 @@ static blk_status_t nvme_rdma_queue_rq(struct blk_mq_hw_ctx *hctx, unmap_qe: ib_dma_unmap_single(dev, req->sqe.dma, sizeof(struct nvme_command), DMA_TO_DEVICE); - return ret; + return nvme_try_complete_failed_req(rq, ret); } static int nvme_rdma_poll(struct blk_mq_hw_ctx *hctx) -- 2.16.4 _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme