From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69EAAC2D0F4 for ; Thu, 2 Apr 2020 01:38:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3E99E20721 for ; Thu, 2 Apr 2020 01:38:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=themaw.net header.i=@themaw.net header.b="EhjE5WUl"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="ZVR+2Un1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1733257AbgDBBie (ORCPT ); Wed, 1 Apr 2020 21:38:34 -0400 Received: from new1-smtp.messagingengine.com ([66.111.4.221]:46223 "EHLO new1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726319AbgDBBid (ORCPT ); Wed, 1 Apr 2020 21:38:33 -0400 Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id 3ABD45801C4; Wed, 1 Apr 2020 21:38:32 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 01 Apr 2020 21:38:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=themaw.net; h= message-id:subject:from:to:cc:date:in-reply-to:references :content-type:mime-version:content-transfer-encoding; s=fm2; bh= 65E0cCQjZaasR2Q4SReMT3up4B2vZTD5YrrV+eEQPzU=; b=EhjE5WUllq7utAsj QGotnYbb9Kw8jZCsD7odmyKl1n5mYNo16UGKeB+MA/N8pXr1cwCWa1jb2Qz6NnFR OmkHYTPm3ClH8nYV38aHOd8+YkGAGNacEUH924J1y6HJ80OvGw1vOk420eaVRmr1 Wqv16Qg6Prbwz52FOwUoa9pw7JtngLbVwCiM9SbAOY4/IxfKTbu2Bj+zofmYQPAZ 3DbFdozqMh3qo9qJ646FaGfljfvPwyXGuJkQfmzNQxDFzQLOA6lcR1LS1OAtdFYu R1qMp7IVbQ+56/KqtUyLLkVzOUUUIIJ+umm4GYxaZV/Wirk+sjBmfMOp/5UCTFtr +kr2LQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=65E0cCQjZaasR2Q4SReMT3up4B2vZTD5YrrV+eEQP zU=; b=ZVR+2Un1ogFAbTykJdID1KGQHIQCJlxQdHSQLt4iyc695rALIxGzJ/tZx 8ttNUhd77g0OaBXN5/v5Q61v4ileqM3MNUllXQT/wmZHyyTxCGxukO3drlUximUQ Yn0lLHQQ4deECa6MIpgb5BJCB4VMr8RT3NfjFcB+xwsNz55sEXdbwq6kd1MQnzlK Mta2qB0IOK7uVSe1r+MH6dKM/Ym1LVOlW+qAmnUOMTA0qqlWFDEPpyRtftHWjREV czKId4Nepql7rXMwphjcl51Mlljv3xaYYmB6qAyPcxod1+UpKfKizo5hobnmT3m2 byqO5tanFiUouyIa6j4md5wueG0Pg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrtdefgdehtdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefkuffhvfffjghftggfggfgsehtjeertddtreejnecuhfhrohhmpefkrghnucfm vghnthcuoehrrghvvghnsehthhgvmhgrfidrnhgvtheqnecukfhppeduudekrddvtdelrd duieeirddvfedvnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhf rhhomheprhgrvhgvnhesthhhvghmrgifrdhnvght X-ME-Proxy: Received: from mickey.themaw.net (unknown [118.209.166.232]) by mail.messagingengine.com (Postfix) with ESMTPA id 69846306CD83; Wed, 1 Apr 2020 21:38:24 -0400 (EDT) Message-ID: <459876eceda4bc68212faf4ed3d4bcb8570aa105.camel@themaw.net> Subject: Re: [PATCH 00/13] VFS: Filesystem information [ver #19] From: Ian Kent To: Miklos Szeredi , David Howells Cc: Linus Torvalds , Al Viro , Linux NFS list , Andreas Dilger , Anna Schumaker , Theodore Ts'o , Linux API , linux-ext4@vger.kernel.org, Trond Myklebust , Miklos Szeredi , Christian Brauner , Jann Horn , "Darrick J. Wong" , Karel Zak , Jeff Layton , linux-fsdevel@vger.kernel.org, LSM , linux-kernel@vger.kernel.org Date: Thu, 02 Apr 2020 09:38:20 +0800 In-Reply-To: References: <158454408854.2864823.5910520544515668590.stgit@warthog.procyon.org.uk> <50caf93782ba1d66bd6acf098fb8dcb0ecc98610.camel@themaw.net> <2465266.1585729649@warthog.procyon.org.uk> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.32.5 (3.32.5-1.fc30) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2020-04-01 at 10:37 +0200, Miklos Szeredi wrote: > On Wed, Apr 1, 2020 at 10:27 AM David Howells > wrote: > > Miklos Szeredi wrote: > > > > > According to dhowell's measurements processing 100k mounts would > > > take > > > about a few seconds of system time (that's the time spent by the > > > kernel to retrieve the data, > > > > But the inefficiency of mountfs - at least as currently implemented > > - scales > > up with the number of individual values you want to retrieve, both > > in terms of > > memory usage and time taken. > > I've taken that into account when guesstimating a "few seconds per > 100k entries". My guess is that there's probably an order of > magnitude difference between the performance of a fs based interface > and a binary syscall based interface. That could be reduced somewhat > with a readfile(2) type API. > > But the point is: this does not matter. Whether it's .5s or 5s is > completely irrelevant, as neither is going to take down the system, > and userspace processing is probably going to take as much, if not > more time. And remember, we are talking about stopping and starting > the automount daemon, which is something that happens, but it should > not happen often by any measure. Yes, but don't forget, I'm reporting what I saw when testing during development. >From previous discussion we know systemd (and probably the other apps like udisks2, et. al.) gets notified on mount and umount activity so its not going to be just starting and stopping autofs that's a problem with very large mount tables. To get a feel for the real difference we'd need to make the libmount changes for both and then check between the two and check behaviour. The mount and umount lookup case that Karel (and I) talked about should be sufficient. The biggest problem I had with fsinfo() when I was working with earlier series was getting fs specific options, in particular the need to use sb op ->fsinfo(). With this latest series David has made that part of the generic code and your patch also cover it. So the thing that was holding me up is done so we should be getting on with libmount improvements, we need to settle this. I prefer the system call interface and I'm not offering justification for that other than a general dislike (and on occasion outright frustration) of pretty much every proc implementation I have had to look at. > > > With fsinfo(), I've tried to batch values together where it makes > > sense - and > > there's no lingering memory overhead - no extra inodes, dentries > > and files > > required. > > The dentries, inodes and files in your test are single use (except > the > root dentry) and can be made ephemeral if that turns out to be > better. > My guess is that dentries belonging to individual attributes should > be > deleted on final put, while the dentries belonging to the mount > directory can be reclaimed normally. > > Thanks, > Miklos