From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07933C433DB for ; Fri, 19 Mar 2021 19:08:00 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9145C6197A for ; Fri, 19 Mar 2021 19:07:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9145C6197A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yN8OVBs/LRThzHKNeyXBeZ11CuhWKMstKGpnC6RC/1s=; b=OxH87aCG13DpLg6//3of9Vvuj Q9DxYUKhiPRFLTVPbCFMLtWf+fuFWUumnHr3TSzLPfcwPa1y5bRTrjTiA/JovYZcK5VnwCgQUu078 M3WFFlp/XT9L+aYBfWX+YYy/yrP4sB4HwH5FZJTr3QDT7Z9gzs5ZuNDrnQzV5mnuqWc5cHybWplIy kYtWM5unXh62cOXmuj42KicDVqv6yBVKd0By2VmfvgnaICQkW/Rg4fhveDn936zEB2S0cMPOt60wV bR/M9eUh348odKqqUVSAi7NZywkbt0Z8u0jSVeArsjIAkPXYJJpPs79KyTgm0fJeK2I60FZ5b4uxt GOc9TQplA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lNKTF-0081kk-Lj; Fri, 19 Mar 2021 19:07:41 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lNKTA-0081jv-VC for linux-nvme@lists.infradead.org; Fri, 19 Mar 2021 19:07:39 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id CFA1661941; Fri, 19 Mar 2021 19:07:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616180855; bh=18OYojqfd9fzkL0BT8SER6Al3RCFfp8o2DX4PwAIrRY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ueNsl7opBkuhRJN7t4nhHgbaX2rA1p8mdswD5N5upCIeWqlT+rANhAGvcSqAAMuAv Xg3vgeWWHXqoWq3O4K+8PrsFlytpgxBDyY/ohVNhNZWNI8RrlbKDqqJCgRXVmLW+Zz 9NWrKOgX/eSrW24aSOkWlGIhat8hkpLa+765Go0Olfo5DruCEh6ImhBtTB0ZFd3lgP RBQL5dIfae83+AtFS7tkPDfi8QOyIVKsgd5xFtfai/8XE0MxddozMZ/wW7Q3eWPZoV yaigsaUiAC3Oj5i6etunpmY0Z+PwsTf/0kdu2EvTcUCm8KNTTC0BmBUYqFpRUH2bJj AKjgmT5HyEnwA== Date: Sat, 20 Mar 2021 04:07:28 +0900 From: Keith Busch To: Christoph Hellwig Cc: Sagi Grimberg , Chao Leng , linux-nvme@lists.infradead.org, Chaitanya Kulkarni Subject: Re: [PATCH 0/3 rfc] Fix nvme-tcp and nvme-rdma controller reset hangs Message-ID: <20210319190728.GB4198@redsun51.ssa.fujisawa.hgst.com> References: <20210317065910.GC14498@lst.de> <2e391aae-58c7-b8f7-1a9e-d7ad5bb3f8f3@huawei.com> <6c085430-cc10-a2fd-56ee-a360109c940a@grimberg.me> <55142c25-9a70-08a0-d46a-cad21da59d19@huawei.com> <7b7d5223-ddaf-eb88-f112-02834f8c8f93@grimberg.me> <20210318191613.GB31675@redsun51.ssa.fujisawa.hgst.com> <20210318215256.GC31675@redsun51.ssa.fujisawa.hgst.com> <20210319140532.GA7019@lst.de> <20210319172817.GA23660@lst.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210319172817.GA23660@lst.de> User-Agent: Mutt/1.12.1 (2019-06-15) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210319_190737_418733_D99E64B4 X-CRM114-Status: GOOD ( 20.99 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On Fri, Mar 19, 2021 at 06:28:17PM +0100, Christoph Hellwig wrote: > What about something like this? Yes, this looks good. > diff --git a/block/blk-core.c b/block/blk-core.c > index fc60ff20849738..4344f3c9058282 100644 > --- a/block/blk-core.c > +++ b/block/blk-core.c > @@ -792,7 +792,7 @@ static inline blk_status_t blk_check_zone_append(struct request_queue *q, > return BLK_STS_OK; > } > > -static noinline_for_stack bool submit_bio_checks(struct bio *bio) > +noinline_for_stack bool submit_bio_checks(struct bio *bio) > { > struct block_device *bdev = bio->bi_bdev; > struct request_queue *q = bdev->bd_disk->queue; > diff --git a/block/blk-mq.c b/block/blk-mq.c > index d4d7c1caa43966..4ff85692843b49 100644 > --- a/block/blk-mq.c > +++ b/block/blk-mq.c > @@ -2286,6 +2286,43 @@ blk_qc_t blk_mq_submit_bio(struct bio *bio) > return BLK_QC_T_NONE; > } > > +/** > + * blk_mq_submit_bio_direct - hand a bio directly to the driver for I/O > + * @bio: The bio describing the location in memory and on the device. > + * > + * This function behaves similar to submit_bio_noacct(), but does never waits > + * for the queue to be unfreozen, instead it return false and lets the caller > + * deal with the fallout. It also does not protect against recursion and thus > + * must only be used if the called driver is known to be blk-mq based. > + */ > +bool blk_mq_submit_bio_direct(struct bio *bio, blk_qc_t *qc) > +{ > + struct gendisk *disk = bio->bi_bdev->bd_disk; > + struct request_queue *q = disk->queue; > + > + if (WARN_ON_ONCE(!current->bio_list) || > + WARN_ON_ONCE(disk->fops->submit_bio)) { > + bio_io_error(bio); > + goto fail; > + } > + if (!submit_bio_checks(bio)) > + goto fail; > + > + if (unlikely(blk_queue_enter(q, BLK_MQ_REQ_NOWAIT))) > + return false; > + if (!blk_crypto_bio_prep(&bio)) > + goto fail_queue_exit; > + *qc = blk_mq_submit_bio(bio); > + return true; > + > +fail_queue_exit: > + blk_queue_exit(disk->queue); > +fail: > + *qc = BLK_QC_T_NONE; > + return true; > +} > +EXPORT_SYMBOL_GPL(blk_mq_submit_bio_direct); > + > void blk_mq_free_rqs(struct blk_mq_tag_set *set, struct blk_mq_tags *tags, > unsigned int hctx_idx) > { > diff --git a/block/blk.h b/block/blk.h > index 3b53e44b967e4e..c4c66b2a9ffb19 100644 > --- a/block/blk.h > +++ b/block/blk.h > @@ -221,6 +221,7 @@ ssize_t part_timeout_show(struct device *, struct device_attribute *, char *); > ssize_t part_timeout_store(struct device *, struct device_attribute *, > const char *, size_t); > > +bool submit_bio_checks(struct bio *bio); > void __blk_queue_split(struct bio **bio, unsigned int *nr_segs); > int ll_back_merge_fn(struct request *req, struct bio *bio, > unsigned int nr_segs); > diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c > index a1d476e1ac020f..92adebfaf86fd1 100644 > --- a/drivers/nvme/host/multipath.c > +++ b/drivers/nvme/host/multipath.c > @@ -309,6 +309,7 @@ blk_qc_t nvme_ns_head_submit_bio(struct bio *bio) > */ > blk_queue_split(&bio); > > +retry: > srcu_idx = srcu_read_lock(&head->srcu); > ns = nvme_find_path(head); > if (likely(ns)) { > @@ -316,7 +317,12 @@ blk_qc_t nvme_ns_head_submit_bio(struct bio *bio) > bio->bi_opf |= REQ_NVME_MPATH; > trace_block_bio_remap(bio, disk_devt(ns->head->disk), > bio->bi_iter.bi_sector); > - ret = submit_bio_noacct(bio); > + > + if (!blk_mq_submit_bio_direct(bio, &ret)) { > + nvme_mpath_clear_current_path(ns); > + srcu_read_unlock(&head->srcu, srcu_idx); > + goto retry; > + } > } else if (nvme_available_path(head)) { > dev_warn_ratelimited(dev, "no usable path - requeuing I/O\n"); > > diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h > index 2c473c9b899089..6804f397106ada 100644 > --- a/include/linux/blk-mq.h > +++ b/include/linux/blk-mq.h > @@ -615,6 +615,7 @@ static inline void blk_rq_bio_prep(struct request *rq, struct bio *bio, > } > > blk_qc_t blk_mq_submit_bio(struct bio *bio); > +bool blk_mq_submit_bio_direct(struct bio *bio, blk_qc_t *qc); > void blk_mq_hctx_set_fq_lock_class(struct blk_mq_hw_ctx *hctx, > struct lock_class_key *key); > _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme