From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81774C48BE5 for ; Mon, 21 Jun 2021 07:20:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AECE61026 for ; Mon, 21 Jun 2021 07:20:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229789AbhFUHW0 (ORCPT ); Mon, 21 Jun 2021 03:22:26 -0400 Received: from verein.lst.de ([213.95.11.211]:40846 "EHLO verein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229597AbhFUHWZ (ORCPT ); Mon, 21 Jun 2021 03:22:25 -0400 Received: by verein.lst.de (Postfix, from userid 2407) id 11AB568BFE; Mon, 21 Jun 2021 09:20:06 +0200 (CEST) Date: Mon, 21 Jun 2021 09:20:05 +0200 From: Christoph Hellwig To: Ming Lei Cc: Christoph Hellwig , Jens Axboe , Jeffle Xu , Damien Le Moal , Keith Busch , Sagi Grimberg , "Wunderlich, Mark" , "Vasudevan, Anil" , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-nvme@lists.infradead.org Subject: Re: [PATCH 13/16] block: switch polling to be bio based Message-ID: <20210621072005.GA6651@lst.de> References: <20210615131034.752623-1-hch@lst.de> <20210615131034.752623-14-hch@lst.de> <20210618140147.GA16258@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Fri, Jun 18, 2021 at 10:27:20PM +0800, Ming Lei wrote: > > How? On a block device the caller needs to hold the block device open > > to read/write from it. On a file systems the file systems needs to > > be mounted, which also holds a bdev reference. > > + rcu_read_lock(); > + bio = READ_ONCE(kiocb->private); > + if (bio && bio->bi_bdev) > > The bio may be ended now from another polling job, then the disk is > closed & deleted, and released. Then request queue & hctxs are released. > > + ret = bio_poll(bio, flags); > > But disk & request queue & hctx can still be referred in above bio_poll(). I don't see how this can happen. A bio stashed into kiocb->private needs to belong to the correct device initially. For it to point to the "wrong" device it needs to have been completed on the correct one, and then be reused for a different device. At the point it is reused that device must obviously have been alive, and for it to be freed a RCU grace period must have been passed. And that grace period can't have started earlier than when iocb_bio_iopoll was called. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ACE8C48BE5 for ; Mon, 21 Jun 2021 07:20:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D8F8861026 for ; Mon, 21 Jun 2021 07:20:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8F8861026 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0qclxLZc0H4jMO6avI5m70ymc0GmPL88RDsPD+jfTuk=; b=LigetNealgZW8C wmv0tVcOk1Wc3hw3HElcGg7Q8I/eXA/NeO1f/nrRgXrptzUE4XogFIdLgH8z8MX2oOYiWyajhUVAx ml5voRKUH/JuPaAvfZdCdwov3Xe3o6jFtqKTuGh2Xrtq6Lk1J9D1FBoN87D4iYBqNLml2l4Xe1JTF 9KKq0tnThVXibduYkypCc+J/GL6Bnt0Dym9ZtBM+yP56qzhoqTTSCN1BeMu5KAimbiDhrvGXh27rH YtZgXhDW/eCegfgqeduwtOEWxOicTQi7sOyKoszrAklsVsSQlGAP3iz2Jzf+MyFdsarp72//Tn2Ha 1Gwldb6VyXAPm7GCJQPQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvEEE-002SRx-25; Mon, 21 Jun 2021 07:20:18 +0000 Received: from verein.lst.de ([213.95.11.211]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvEE7-002SPZ-4X for linux-nvme@lists.infradead.org; Mon, 21 Jun 2021 07:20:15 +0000 Received: by verein.lst.de (Postfix, from userid 2407) id 11AB568BFE; Mon, 21 Jun 2021 09:20:06 +0200 (CEST) Date: Mon, 21 Jun 2021 09:20:05 +0200 From: Christoph Hellwig To: Ming Lei Cc: Christoph Hellwig , Jens Axboe , Jeffle Xu , Damien Le Moal , Keith Busch , Sagi Grimberg , "Wunderlich, Mark" , "Vasudevan, Anil" , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-nvme@lists.infradead.org Subject: Re: [PATCH 13/16] block: switch polling to be bio based Message-ID: <20210621072005.GA6651@lst.de> References: <20210615131034.752623-1-hch@lst.de> <20210615131034.752623-14-hch@lst.de> <20210618140147.GA16258@lst.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210621_002011_370601_89AF58F8 X-CRM114-Status: GOOD ( 19.46 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On Fri, Jun 18, 2021 at 10:27:20PM +0800, Ming Lei wrote: > > How? On a block device the caller needs to hold the block device open > > to read/write from it. On a file systems the file systems needs to > > be mounted, which also holds a bdev reference. > > + rcu_read_lock(); > + bio = READ_ONCE(kiocb->private); > + if (bio && bio->bi_bdev) > > The bio may be ended now from another polling job, then the disk is > closed & deleted, and released. Then request queue & hctxs are released. > > + ret = bio_poll(bio, flags); > > But disk & request queue & hctx can still be referred in above bio_poll(). I don't see how this can happen. A bio stashed into kiocb->private needs to belong to the correct device initially. For it to point to the "wrong" device it needs to have been completed on the correct one, and then be reused for a different device. At the point it is reused that device must obviously have been alive, and for it to be freed a RCU grace period must have been passed. And that grace period can't have started earlier than when iocb_bio_iopoll was called. _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme