From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B39D9C2BB1D for ; Tue, 14 Apr 2020 20:58:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B1C52074D for ; Tue, 14 Apr 2020 20:58:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B1C52074D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F35238E0003; Tue, 14 Apr 2020 16:58:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBE068E0001; Tue, 14 Apr 2020 16:58:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5EC08E0003; Tue, 14 Apr 2020 16:58:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id B80A78E0001 for ; Tue, 14 Apr 2020 16:58:56 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 75D458245571 for ; Tue, 14 Apr 2020 20:58:56 +0000 (UTC) X-FDA: 76707675072.20.drink95_883b4993eed14 X-HE-Tag: drink95_883b4993eed14 X-Filterd-Recvd-Size: 8055 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Tue, 14 Apr 2020 20:58:56 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id n16so459226pgb.7 for ; Tue, 14 Apr 2020 13:58:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=2BFbh/NVyd/nEEqnIV7FAkYPsjDu10E1KJ7JMizEGUs=; b=pMPZsbhavVOywsnpMqsyxDjCfWwSZimRnKttrNKuZsOiEHtOC4MzmEV9SmMZX5sZeX 3FIhQEuR5jXFZWyIZUqTHorypDhbp1lEaNNXddYxi1Ux8LJ9qXGki96C9qQ0maViTThP dffSYkGDeoWlzhBS7mo+hJrHVZHMud0jLF0PFpe6uJJuvx0RFGEZ2Aqshvev/m+hW0TQ AhiJtY78ZGP7WjCZvO/ZvdCUj0Lfq08ql0E2gpHxoZCUojQ1uGS2Cefz0tyesa5HoDKv vCwzq8RnJMCNaXsrIXje/Wanz/UaVb3v8/AKvmaF7wzWuXMLFSQs5+LYVlh0jqBsS6es Jjpg== X-Gm-Message-State: AGi0PubH4zZ0eYuLUNGNo9Y9l2CUMcRRnjNHJI56ZrnaVmM8TkJfUZkU XYvSXQs3/BL5xhz2QsmbxUk= X-Google-Smtp-Source: APiQypKmhyUPwU3E+eaaqWqZ9C3MBCiwt3U/HU4Vjk70huVm8L0B8YcyK+w2BA/5pI1GVFdjIpQH2g== X-Received: by 2002:a63:d454:: with SMTP id i20mr22395308pgj.209.1586897935063; Tue, 14 Apr 2020 13:58:55 -0700 (PDT) Received: from 42.do-not-panic.com (42.do-not-panic.com. [157.230.128.187]) by smtp.gmail.com with ESMTPSA id a1sm11817266pfl.188.2020.04.14.13.58.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Apr 2020 13:58:53 -0700 (PDT) Received: by 42.do-not-panic.com (Postfix, from userid 1000) id DA82440277; Tue, 14 Apr 2020 20:58:52 +0000 (UTC) Date: Tue, 14 Apr 2020 20:58:52 +0000 From: Luis Chamberlain To: Christoph Hellwig Cc: axboe@kernel.dk, viro@zeniv.linux.org.uk, bvanassche@acm.org, gregkh@linuxfoundation.org, rostedt@goodmis.org, mingo@redhat.com, jack@suse.cz, ming.lei@redhat.com, nstange@suse.de, akpm@linux-foundation.org, mhocko@suse.com, yukuai3@huawei.com, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Omar Sandoval , Hannes Reinecke , Michal Hocko Subject: Re: [PATCH 5/5] block: revert back to synchronous request_queue removal Message-ID: <20200414205852.GP11244@42.do-not-panic.com> References: <20200414041902.16769-1-mcgrof@kernel.org> <20200414041902.16769-6-mcgrof@kernel.org> <20200414154725.GD25765@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200414154725.GD25765@infradead.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 14, 2020 at 08:47:25AM -0700, Christoph Hellwig wrote: > On Tue, Apr 14, 2020 at 04:19:02AM +0000, Luis Chamberlain wrote: > > Commit dc9edc44de6c ("block: Fix a blk_exit_rl() regression") merged on > > v4.12 moved the work behind blk_release_queue() into a workqueue after a > > splat floated around which indicated some work on blk_release_queue() > > could sleep in blk_exit_rl(). This splat would be possible when a driver > > called blk_put_queue() or blk_cleanup_queue() (which calls blk_put_queue() > > as its final call) from an atomic context. > > > > blk_put_queue() decrements the refcount for the request_queue > > kobject, and upon reaching 0 blk_release_queue() is called. Although > > blk_exit_rl() is now removed through commit db6d9952356 ("block: remove > > request_list code"), we reserve the right to be able to sleep within > > blk_release_queue() context. If you see no other way and *have* be > > in atomic context when you driver calls the last blk_put_queue() > > you can always just increase your block device's reference count with > > bdgrab() as this can be done in atomic context and the request_queue > > removal would be left to upper layers later. We document this bit of > > tribal knowledge as well now, and adjust kdoc format a bit. > > > > We revert back to synchronous request_queue removal because asynchronous > > removal creates a regression with expected userspace interaction with > > several drivers. An example is when removing the loopback driver and > > issues ioctl from userspace to do so, upon return and if successful one > > expects the device to be removed. Moving to asynchronous request_queue > > removal could have broken many scripts which relied on the removal to > > have been completed if there was no error. > > > > Using asynchronous request_queue removal however has helped us find > > other bugs, in the future we can test what could break with this > > arrangement by enabling CONFIG_DEBUG_KOBJECT_RELEASE. > > > > Cc: Bart Van Assche > > Cc: Omar Sandoval > > Cc: Hannes Reinecke > > Cc: Nicolai Stange > > Cc: Greg Kroah-Hartman > > Cc: Michal Hocko > > Cc: yu kuai > > Suggested-by: Nicolai Stange > > Fixes: dc9edc44de6c ("block: Fix a blk_exit_rl() regression") > > Signed-off-by: Luis Chamberlain > > --- > > block/blk-core.c | 19 ++++++++++++++++++- > > block/blk-sysfs.c | 38 +++++++++++++++++--------------------- > > include/linux/blkdev.h | 2 -- > > 3 files changed, 35 insertions(+), 24 deletions(-) > > > > diff --git a/block/blk-core.c b/block/blk-core.c > > index 5aaae7a1b338..8346c7c59ee6 100644 > > --- a/block/blk-core.c > > +++ b/block/blk-core.c > > @@ -301,6 +301,17 @@ void blk_clear_pm_only(struct request_queue *q) > > } > > EXPORT_SYMBOL_GPL(blk_clear_pm_only); > > > > +/** > > + * blk_put_queue - decrement the request_queue refcount > > + * > > + * Decrements the refcount to the request_queue kobject, when this reaches > > + * 0 we'll have blk_release_queue() called. You should avoid calling > > + * this function in atomic context but if you really have to ensure you > > + * first refcount the block device with bdgrab() / bdput() so that the > > + * last decrement happens in blk_cleanup_queue(). > > + * > > + * @q: the request_queue structure to decrement the refcount for > > + */ > > void blk_put_queue(struct request_queue *q) > > { > > kobject_put(&q->kobj); > > @@ -328,10 +339,16 @@ EXPORT_SYMBOL_GPL(blk_set_queue_dying); > > > > /** > > * blk_cleanup_queue - shutdown a request queue > > - * @q: request queue to shutdown > > * > > * Mark @q DYING, drain all pending requests, mark @q DEAD, destroy and > > * put it. All future requests will be failed immediately with -ENODEV. > > + * > > + * You should not call this function in atomic context. If you need to > > + * refcount a request_queue in atomic context, instead refcount the > > + * block device with bdgrab() / bdput(). > > I think this needs a WARN_ON thrown in to enforece the calling context. I considered adding a might_sleep() but upon review with Bart, he noted that this function already has a mutex_lock(), and if you look under the hood of mutex_lock(), it has a might_sleep() at the very top. The warning then is implicit. > > + * > > + * @q: request queue to shutdown > > Moving the argument documentation seems against the usual kerneldoc > style. Would you look at that, Documentation/doc-guide/kernel-doc.rst does say to keep the argument at the top as it was in place before, OK will revert that. Sorry, I used include/net/mac80211.h as my base for style. > Otherwise this look good, I hope it sticks :) I hope that the kdocs / might_sleep() sprinkled should make it stick now. But hey, this uncovered wonderful obscure bugs, it was fun. I'll add a selftest also later to ensure we don't regress on some of this later once again. Luis