From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Kent Date: Thu, 02 Apr 2020 02:52:27 +0000 Subject: Re: Upcoming: Notifications, FS notifications and fsinfo() Message-Id: <36e45eae8ad78f7b8889d9d03b8846e78d735d28.camel@themaw.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit List-Id: References: <20200330211700.g7evnuvvjenq3fzm@wittgenstein> <1445647.1585576702@warthog.procyon.org.uk> <2418286.1585691572@warthog.procyon.org.uk> <20200401144109.GA29945@gardel-login> <2590640.1585757211@warthog.procyon.org.uk> In-Reply-To: To: Miklos Szeredi , David Howells Cc: Lennart Poettering , Christian Brauner , Linus Torvalds , Al Viro , dray@redhat.com, Karel Zak , Miklos Szeredi , Steven Whitehouse , Jeff Layton , andres@anarazel.de, keyrings@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Aleksa Sarai On Wed, 2020-04-01 at 18:40 +0200, Miklos Szeredi wrote: > On Wed, Apr 1, 2020 at 6:07 PM David Howells > wrote: > > Miklos Szeredi wrote: > > > > > I've still not heard a convincing argument in favor of a syscall. > > > > From your own results, scanning 10000 mounts through mountfs and > > reading just > > two values from each is an order of magnitude slower without the > > effect of the > > dentry/inode caches. It gets faster on the second run because the > > mountfs > > dentries and inodes are cached - but at a cost of >205MiB of > > RAM. And it's > > *still* slower than fsinfo(). > > Already told you that we can just delete the dentry on dput_final, so > the memory argument is immaterial. > > And the speed argument also, because there's no use case where that > would make a difference. You keep bringing up the notification queue > overrun when watching a subtree, but that's going to be painful with > fsinfo(2) as well. If that's a relevant use case (not saying it's > true), might as well add a /mnt/MNT_ID/subtree_info (trivial again) > that contains all information for the subtree. Have fun implementing > that with fsinfo(2). Forgive me for not trawling through your patch to work this out but how does a poll on a path get what's needed to get mount info. Or, more specifically, how does one get what's needed to go directly to the place to get mount info. when something in the tree under the polled path changes (mount/umount). IIUC poll alone won't do subtree change monitoring? Don't get me wrong, neither the proc nor the fsinfo implementations deal with the notification storms that cause much of the problem we see now. IMHO that's a separate and very difficult problem in itself that can't even be considered until getting the information efficiently is resolved. Ian From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F0D3C43331 for ; Thu, 2 Apr 2020 02:52:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3C8112072E for ; Thu, 2 Apr 2020 02:52:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="HncnaXzx"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="UUnDipPD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2387406AbgDBCwm (ORCPT ); Wed, 1 Apr 2020 22:52:42 -0400 Received: from wnew2-smtp.messagingengine.com ([64.147.123.27]:36329 "EHLO wnew2-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727135AbgDBCwl (ORCPT ); Wed, 1 Apr 2020 22:52:41 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.west.internal (Postfix) with ESMTP id A44AD679; Wed, 1 Apr 2020 22:52:39 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 01 Apr 2020 22:52:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= message-id:subject:from:to:cc:date:in-reply-to:references :content-type:mime-version:content-transfer-encoding; s=fm2; bh= h7ECITCKfNZMx8k2mxEusSlerH4FqS6pM+JOAcsVU9w=; b=HncnaXzxp07w5tFl gnU6d6fR74L+X606QoPnWwoQm93HnfTUw+g9r7uWRImUy98GPXTCe85aiBPJjZTt Jo9/KHn4chsLHC9ss30kDAhPMKypg2XzX68wgh3jsQfjnLTXnIRNN1/vCqLn2IwW BrguiyXF0bRIIm/4yw1UB48Br98QFdV2QWgWnxCGu7n8+Bt9FDmlMnTDVPFtKP9/ ynx14PW63H/QBL8fM9KCWE8fxdJg9QeYPNG+mekemc/rjvNZ/scu8c23T1jlQQi/ oWbJ9Cf6sNrMd9ltpSoNjqg719NYSDPyLTtxUa/LDx1q39B5UCOJ9DyUvuWNpV3h w0qlPA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=h7ECITCKfNZMx8k2mxEusSlerH4FqS6pM+JOAcsVU 9w=; b=UUnDipPDjurA/G0QzCXe5RkWhjy/3uCnNKWYhiBiQPgtfgAAhHl/X+r3y ub/LgNfdLHMvPMLmlSBJZHlljt3XOwdzgs8BO6SlEXb5b4LZKaJeU+RR/Hif6xLO yPF1kMtYhCV1IoKsgH3aQsYIPOuyCPOWt6D0FzJK1BMhvlyhlZq9PdweEqX1Zg+d kYizbF9G+sA63dinVYakCr5RVUV5Wr7Y85/zO2dpL0Qt3XxQ71UY3rqEI16H9dPH QpykNrJeFn4UN4lEP3urw2o3sDQAIVr3PEvf80tPPn5Ov/ST9drkcKvOJh6O924F UPEhe8y+3qpIBnqXKb5sFsdWf2EmQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrtdefgdeihecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkuffhvfffjghftggfggfgsehtjeertddtreejnecuhfhrohhmpefkrghnucfm vghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecukfhppeduudekrddvtdelrd duieeirddvfedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhf rhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (unknown [118.209.166.232]) by mail.messagingengine.com (Postfix) with ESMTPA id F37B9306CD58; Wed, 1 Apr 2020 22:52:31 -0400 (EDT) Message-ID: <36e45eae8ad78f7b8889d9d03b8846e78d735d28.camel@themaw.net> Subject: Re: Upcoming: Notifications, FS notifications and fsinfo() From: Ian Kent To: Miklos Szeredi , David Howells Cc: Lennart Poettering , Christian Brauner , Linus Torvalds , Al Viro , dray@redhat.com, Karel Zak , Miklos Szeredi , Steven Whitehouse , Jeff Layton , andres@anarazel.de, keyrings@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Aleksa Sarai Date: Thu, 02 Apr 2020 10:52:27 +0800 In-Reply-To: References: <20200330211700.g7evnuvvjenq3fzm@wittgenstein> <1445647.1585576702@warthog.procyon.org.uk> <2418286.1585691572@warthog.procyon.org.uk> <20200401144109.GA29945@gardel-login> <2590640.1585757211@warthog.procyon.org.uk> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.32.5 (3.32.5-1.fc30) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2020-04-01 at 18:40 +0200, Miklos Szeredi wrote: > On Wed, Apr 1, 2020 at 6:07 PM David Howells > wrote: > > Miklos Szeredi wrote: > > > > > I've still not heard a convincing argument in favor of a syscall. > > > > From your own results, scanning 10000 mounts through mountfs and > > reading just > > two values from each is an order of magnitude slower without the > > effect of the > > dentry/inode caches. It gets faster on the second run because the > > mountfs > > dentries and inodes are cached - but at a cost of >205MiB of > > RAM. And it's > > *still* slower than fsinfo(). > > Already told you that we can just delete the dentry on dput_final, so > the memory argument is immaterial. > > And the speed argument also, because there's no use case where that > would make a difference. You keep bringing up the notification queue > overrun when watching a subtree, but that's going to be painful with > fsinfo(2) as well. If that's a relevant use case (not saying it's > true), might as well add a /mnt/MNT_ID/subtree_info (trivial again) > that contains all information for the subtree. Have fun implementing > that with fsinfo(2). Forgive me for not trawling through your patch to work this out but how does a poll on a path get what's needed to get mount info. Or, more specifically, how does one get what's needed to go directly to the place to get mount info. when something in the tree under the polled path changes (mount/umount). IIUC poll alone won't do subtree change monitoring? Don't get me wrong, neither the proc nor the fsinfo implementations deal with the notification storms that cause much of the problem we see now. IMHO that's a separate and very difficult problem in itself that can't even be considered until getting the information efficiently is resolved. Ian