From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78253C4338F for ; Mon, 2 Aug 2021 11:27:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6161160F36 for ; Mon, 2 Aug 2021 11:27:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233585AbhHBL1h (ORCPT ); Mon, 2 Aug 2021 07:27:37 -0400 Received: from smtp-out2.suse.de ([195.135.220.29]:51302 "EHLO smtp-out2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233413AbhHBL1S (ORCPT ); Mon, 2 Aug 2021 07:27:18 -0400 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 1528F1FF7C; Mon, 2 Aug 2021 11:27:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1627903628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kKKLeMvUSjTl9xhk4gs1WHWOoquSNjLGEcBM3zENZtY=; b=FYcAg6x7ObqbbCLc0lYhaAE+gjOO8KOnfUbk8loC5OujR8asbSqtjnmy22/ntdHkU6evLT 2Q4THZ/j6gFTWKZM4OgRgkuhWS+o9+wCHZVlV+K2+ku6vISlViWOv8bdSB4RHNmVeYX3ba gVV3EFA3Hjnj84TgV/5UpiumpJlSPTE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1627903628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kKKLeMvUSjTl9xhk4gs1WHWOoquSNjLGEcBM3zENZtY=; b=5jhoDsS3eb8jhFdAPgL/xPhDtYNueixTSpYFeGWUWGLjmq4v2FnyR3zvXy2ycHO8mxAFxb JmJpWwoo2KsauaAg== Received: from adalid.arch.suse.de (adalid.arch.suse.de [10.161.8.13]) by relay2.suse.de (Postfix) with ESMTP id 002BFA3BC4; Mon, 2 Aug 2021 11:27:07 +0000 (UTC) Received: by adalid.arch.suse.de (Postfix, from userid 17828) id E69F4518C0AE; Mon, 2 Aug 2021 13:27:07 +0200 (CEST) From: Daniel Wagner To: linux-nvme@lists.infradead.org Cc: linux-kernel@vger.kernel.org, James Smart , Keith Busch , Ming Lei , Sagi Grimberg , Hannes Reinecke , Wen Xiong , James Smart , Chao Leng , Daniel Wagner Subject: [PATCH v4 5/8] nvme-fc: avoid race between time out and tear down Date: Mon, 2 Aug 2021 13:26:55 +0200 Message-Id: <20210802112658.75875-6-dwagner@suse.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210802112658.75875-1-dwagner@suse.de> References: <20210802112658.75875-1-dwagner@suse.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: James Smart To avoid race between time out and tear down, in tear down process, first we quiesce the queue, and then delete the timer and cancel the time out work for the queue. This patch merges the admin and io sync ops into the queue teardown logic as shown in the RDMA patch 3017013dcc "nvme-rdma: avoid race between time out and tear down". There is no teardown_lock in nvme-fc. Signed-off-by: James Smart CC: Chao Leng Tested-by: Daniel Wagner [dwagner: updated commit id referenced in commit message] Reviewed-by: Daniel Wagner Reviewed-by: Hannes Reinecke --- drivers/nvme/host/fc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index dbb8ad816df8..133b87db4f1d 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2487,6 +2487,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues) */ if (ctrl->ctrl.queue_count > 1) { nvme_stop_queues(&ctrl->ctrl); + nvme_sync_io_queues(&ctrl->ctrl); blk_mq_tagset_busy_iter(&ctrl->tag_set, nvme_fc_terminate_exchange, &ctrl->ctrl); blk_mq_tagset_wait_completed_request(&ctrl->tag_set); @@ -2510,6 +2511,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues) * clean up the admin queue. Same thing as above. */ blk_mq_quiesce_queue(ctrl->ctrl.admin_q); + blk_sync_queue(ctrl->ctrl.admin_q); blk_mq_tagset_busy_iter(&ctrl->admin_tag_set, nvme_fc_terminate_exchange, &ctrl->ctrl); blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set); -- 2.29.2 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFFF8C4338F for ; Mon, 2 Aug 2021 11:28:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 822F460FC2 for ; Mon, 2 Aug 2021 11:28:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 822F460FC2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DBLnp8l745GGdIhk0aYMeOE5ELZjpzbOTHUXrVsJpKI=; b=dmKpevUQdYisVW hSk97VWBbixLxqOOgX9NEVWG8HbriaNXzFhMIDs3P4NO+rjuI7UzZ5Qk7CvRTnKWaDGR3OTkHwxjt JwYd1D6Mjqhs/sVE6yjaMPXH6xazKMgeEnUL0ycELIPzCBafdXo/PcrB8aof2Bd7W8WMi6nTPVpoZ qKw5H+Rk4mF8gJ+1CDrDWDNFHNRnLmyCto/kQPEWSnkhwQEZbQp/D6bWfgUVeGUL0PslFhPVMLWKr Umb3z/7njxPhGETZXVquhGKk3YaLq2UtR8cDtDTp7gI0vLLcbxVSgT29WyBleq/AYimfF+5dB10u8 sY3airTXV/W/5POZq3Eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAW6e-00FvtE-JC; Mon, 02 Aug 2021 11:27:40 +0000 Received: from smtp-out2.suse.de ([195.135.220.29]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mAW6B-00FvfP-MM for linux-nvme@lists.infradead.org; Mon, 02 Aug 2021 11:27:15 +0000 Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out2.suse.de (Postfix) with ESMTP id 1528F1FF7C; Mon, 2 Aug 2021 11:27:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1627903628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kKKLeMvUSjTl9xhk4gs1WHWOoquSNjLGEcBM3zENZtY=; b=FYcAg6x7ObqbbCLc0lYhaAE+gjOO8KOnfUbk8loC5OujR8asbSqtjnmy22/ntdHkU6evLT 2Q4THZ/j6gFTWKZM4OgRgkuhWS+o9+wCHZVlV+K2+ku6vISlViWOv8bdSB4RHNmVeYX3ba gVV3EFA3Hjnj84TgV/5UpiumpJlSPTE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1627903628; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kKKLeMvUSjTl9xhk4gs1WHWOoquSNjLGEcBM3zENZtY=; b=5jhoDsS3eb8jhFdAPgL/xPhDtYNueixTSpYFeGWUWGLjmq4v2FnyR3zvXy2ycHO8mxAFxb JmJpWwoo2KsauaAg== Received: from adalid.arch.suse.de (adalid.arch.suse.de [10.161.8.13]) by relay2.suse.de (Postfix) with ESMTP id 002BFA3BC4; Mon, 2 Aug 2021 11:27:07 +0000 (UTC) Received: by adalid.arch.suse.de (Postfix, from userid 17828) id E69F4518C0AE; Mon, 2 Aug 2021 13:27:07 +0200 (CEST) From: Daniel Wagner To: linux-nvme@lists.infradead.org Cc: linux-kernel@vger.kernel.org, James Smart , Keith Busch , Ming Lei , Sagi Grimberg , Hannes Reinecke , Wen Xiong , James Smart , Chao Leng , Daniel Wagner Subject: [PATCH v4 5/8] nvme-fc: avoid race between time out and tear down Date: Mon, 2 Aug 2021 13:26:55 +0200 Message-Id: <20210802112658.75875-6-dwagner@suse.de> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210802112658.75875-1-dwagner@suse.de> References: <20210802112658.75875-1-dwagner@suse.de> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210802_042711_982952_023A2B28 X-CRM114-Status: GOOD ( 11.76 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org From: James Smart To avoid race between time out and tear down, in tear down process, first we quiesce the queue, and then delete the timer and cancel the time out work for the queue. This patch merges the admin and io sync ops into the queue teardown logic as shown in the RDMA patch 3017013dcc "nvme-rdma: avoid race between time out and tear down". There is no teardown_lock in nvme-fc. Signed-off-by: James Smart CC: Chao Leng Tested-by: Daniel Wagner [dwagner: updated commit id referenced in commit message] Reviewed-by: Daniel Wagner Reviewed-by: Hannes Reinecke --- drivers/nvme/host/fc.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index dbb8ad816df8..133b87db4f1d 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2487,6 +2487,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues) */ if (ctrl->ctrl.queue_count > 1) { nvme_stop_queues(&ctrl->ctrl); + nvme_sync_io_queues(&ctrl->ctrl); blk_mq_tagset_busy_iter(&ctrl->tag_set, nvme_fc_terminate_exchange, &ctrl->ctrl); blk_mq_tagset_wait_completed_request(&ctrl->tag_set); @@ -2510,6 +2511,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl *ctrl, bool start_queues) * clean up the admin queue. Same thing as above. */ blk_mq_quiesce_queue(ctrl->ctrl.admin_q); + blk_sync_queue(ctrl->ctrl.admin_q); blk_mq_tagset_busy_iter(&ctrl->admin_tag_set, nvme_fc_terminate_exchange, &ctrl->ctrl); blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set); -- 2.29.2 _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme