Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Karel Zak <kzak@redhat.com>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: "Ian Kent" <raven@themaw.net>,
	"Miklos Szeredi" <mszeredi@redhat.com>,
	"James Bottomley" <James.Bottomley@hansenpartnership.com>,
	"Steven Whitehouse" <swhiteho@redhat.com>,
	"David Howells" <dhowells@redhat.com>,
	viro <viro@zeniv.linux.org.uk>,
	"Christian Brauner" <christian@brauner.io>,
	"Jann Horn" <jannh@google.com>,
	"Darrick J. Wong" <darrick.wong@oracle.com>,
	"Linux API" <linux-api@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	lkml <linux-kernel@vger.kernel.org>,
	"Lennart Poettering" <lennart@poettering.net>,
	"Zbigniew Jędrzejewski-Szmek" <zbyszek@in.waw.pl>,
	util-linux@vger.kernel.org
Subject: Re: [PATCH 00/17] VFS: Filesystem information and notifications [ver #17]
Date: Thu, 27 Feb 2020 16:14:21 +0100
Message-ID: <20200227151421.3u74ijhqt6ekbiss@ws.net.home> (raw)
In-Reply-To: <CAJfpegu5xLcR=QbAOnUrL49QTem6X6ok7nPU+kLFnNHdPXSh1A@mail.gmail.com>

On Thu, Feb 27, 2020 at 02:45:27PM +0100, Miklos Szeredi wrote:
> > So the problem I want to see fixed is the effect of very large
> > mount tables on other user space applications, particularly the
> > effect when a large number of mounts or umounts are performed.

Yes, now you have to generate (in kernel) and parse (in
userspace) all mount table to get information about just 
one mount table entry. This is typical for umount or systemd.

> > >  - add a notification mechanism   - lookup a mount based on path
> > >  - and a way to selectively query mount/superblock information
> > based on path ...

For umount-like use-cases we need mountpoint/ to mount entry
conversion; I guess something like open(mountpoint/) + fsinfo() 
should be good enough.

For systemd we need the same, but triggered by notification. The ideal
solution is to get mount entry ID or FD from notification and later use this
ID or FD to ask for details about the mount entry (probably again fsinfo()).
The notification has to be usable with in epoll() set.

This solves 99% of our performance issues I guess.

> > So that means mount table info. needs to be maintained, whether that
> > can be achieved using sysfs I don't know. Creating and maintaining
> > the sysfs tree would be a big challenge I think.

It will be still necessary to get complete mount table sometimes, but 
not in performance sensitive scenarios.

I'm not sure about sysfs/, you need somehow resolve namespaces, order
of the mount entries (which one is the last one), etc. IMHO translate
mountpoint path to sysfs/ path will be complicated.

> > But before trying to work out how to use a notification mechanism
> > just having a way to get the info provided by the proc tables using
> > a path alone should give initial immediate improvement in libmount.
> 
> Adding Karel, Lennart, Zbigniew and util-linux@vger...
> 
> At a quick glance at libmount and systemd code, it appears that just
> switching out the implementation in libmount will not be enough:
> systemd is calling functions like mnt_table_parse_*() when it receives
> a notification that the mount table changed.

We're ready to change this stuff in systemd if there will be something
better (something per-mount-entry).

My plan is add new API to libmount to query information about one
mount entry (but I had no time to play with fsinfo yet).

> What is the end purpose of parsing the mount tables?  Can systemd guys
> comment on that?

If mount/umount is triggered by systemd than it need verification
about success and final version of the mount options. It also reads
information from libmount to get userspace mount options (.e.g.
_netdev -- libmount uses mount source, target and fsroot to join
kernel and userpace stuff).

And don't forget that mount units are part of systemd dependencies, so
umount/mount is important event for systemd and it need details about
the changes (what, where, ... etc.)

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com


  reply index

Thread overview: 117+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-21 18:01 David Howells
2020-02-21 18:01 ` [PATCH 01/17] watch_queue: Add security hooks to rule on setting mount and sb watches " David Howells
2020-02-21 18:02 ` [PATCH 02/17] watch_queue: Implement mount topology and attribute change notifications " David Howells
2020-02-21 18:02 ` [PATCH 03/17] watch_queue: sample: Display mount tree " David Howells
2020-02-21 18:02 ` [PATCH 04/17] watch_queue: Introduce a non-repeating system-unique superblock ID " David Howells
2020-02-21 18:02 ` [PATCH 05/17] watch_queue: Add superblock notifications " David Howells
2020-02-21 18:02 ` [PATCH 06/17] watch_queue: sample: Display " David Howells
2020-02-21 18:02 ` [PATCH 07/17] fsinfo: Add fsinfo() syscall to query filesystem information " David Howells
2020-02-26  2:29   ` Aleksa Sarai
2020-02-28 14:44   ` David Howells
2020-02-21 18:02 ` [PATCH 08/17] fsinfo: Provide a bitmap of supported features " David Howells
2020-02-21 18:03 ` [PATCH 09/17] fsinfo: Allow fsinfo() to look up a mount object by ID " David Howells
2020-02-21 18:03 ` [PATCH 10/17] fsinfo: Allow mount information to be queried " David Howells
2020-03-04 14:58   ` Miklos Szeredi
2020-03-04 16:10   ` Miklos Szeredi
2020-02-21 18:03 ` [PATCH 11/17] fsinfo: sample: Mount listing program " David Howells
2020-02-21 18:03 ` [PATCH 12/17] fsinfo: Allow the mount topology propogation flags to be retrieved " David Howells
2020-02-21 18:03 ` [PATCH 13/17] fsinfo: Query superblock unique ID and notification counter " David Howells
2020-02-21 18:03 ` [PATCH 14/17] fsinfo: Add API documentation " David Howells
2020-02-21 18:03 ` [PATCH 15/17] fsinfo: Add support for AFS " David Howells
2020-02-21 18:03 ` [PATCH 16/17] fsinfo: Add example support for Ext4 " David Howells
2020-02-21 18:04 ` [PATCH 17/17] fsinfo: Add example support for NFS " David Howells
2020-02-21 20:21 ` [PATCH 00/17] VFS: Filesystem information and notifications " James Bottomley
2020-02-24 10:24   ` Miklos Szeredi
2020-02-24 14:55     ` James Bottomley
2020-02-24 15:28       ` Miklos Szeredi
2020-02-25 12:13         ` Steven Whitehouse
2020-02-25 15:28           ` James Bottomley
2020-02-25 15:47             ` Steven Whitehouse
2020-02-26  9:11             ` Miklos Szeredi
2020-02-26 10:51               ` Steven Whitehouse
2020-02-27  5:06               ` Ian Kent
2020-02-27  9:36                 ` Miklos Szeredi
2020-02-27 11:34                   ` Ian Kent
2020-02-27 13:45                     ` Miklos Szeredi
2020-02-27 15:14                       ` Karel Zak [this message]
2020-02-28  0:43                         ` Ian Kent
2020-02-28  8:35                           ` Miklos Szeredi
2020-02-28 12:27                             ` Greg Kroah-Hartman
2020-02-28 16:24                               ` Miklos Szeredi
2020-02-28 17:15                                 ` Al Viro
2020-03-02  8:43                                   ` Miklos Szeredi
2020-03-02 10:34                                 ` Karel Zak
2020-02-28 16:42                               ` David Howells
2020-02-28 15:08                             ` James Bottomley
2020-02-28 15:40                               ` Miklos Szeredi
2020-02-28  0:12                       ` Ian Kent
2020-02-28 15:52             ` Christian Brauner
2020-02-28 16:36             ` David Howells
2020-03-02  9:09               ` Miklos Szeredi
2020-03-02  9:38                 ` Greg Kroah-Hartman
2020-03-03  5:27                 ` Ian Kent
2020-03-03  7:46                   ` Miklos Szeredi
2020-03-06 16:25                     ` Miklos Szeredi
2020-03-06 19:43                       ` Al Viro
2020-03-06 19:54                         ` Miklos Szeredi
2020-03-06 19:58                         ` Al Viro
2020-03-06 20:05                           ` Al Viro
2020-03-06 20:11                             ` Miklos Szeredi
2020-03-06 20:37                             ` Al Viro
2020-03-06 20:38                               ` Al Viro
2020-03-06 20:45                                 ` Al Viro
2020-03-06 20:49                                   ` Al Viro
2020-03-06 20:51                                     ` Miklos Szeredi
2020-03-06 21:28                                       ` Al Viro
2020-03-06 20:56                                     ` Al Viro
2020-03-06 20:51                                   ` Miklos Szeredi
2020-03-07  9:48                       ` Greg Kroah-Hartman
2020-03-07 20:48                         ` Miklos Szeredi
2020-03-03  9:12                   ` David Howells
2020-03-03  9:26                     ` Miklos Szeredi
2020-03-03  9:48                       ` Miklos Szeredi
2020-03-03 10:21                         ` Steven Whitehouse
2020-03-03 10:32                           ` Miklos Szeredi
2020-03-03 11:09                             ` Ian Kent
2020-03-03 10:00                       ` Christian Brauner
2020-03-03 10:13                         ` Miklos Szeredi
2020-03-03 10:25                           ` Christian Brauner
2020-03-03 11:33                             ` Miklos Szeredi
2020-03-03 11:56                               ` Christian Brauner
2020-03-03 11:38                       ` Karel Zak
2020-03-03 13:03                         ` Greg Kroah-Hartman
2020-03-03 13:14                           ` Greg Kroah-Hartman
2020-03-03 13:34                             ` Miklos Szeredi
2020-03-03 13:43                               ` Greg Kroah-Hartman
2020-03-03 14:10                                 ` Greg Kroah-Hartman
2020-03-03 14:13                                   ` Jann Horn
2020-03-03 14:24                                     ` Greg Kroah-Hartman
2020-03-03 15:44                                       ` Jens Axboe
2020-03-03 16:37                                         ` Greg Kroah-Hartman
2020-03-03 16:51                                         ` Jeff Layton
2020-03-03 16:55                                           ` Jens Axboe
2020-03-03 19:02                                             ` Jeff Layton
2020-03-03 19:07                                               ` Jens Axboe
2020-03-03 19:23                                               ` Jens Axboe
2020-03-03 19:43                                                 ` Jeff Layton
2020-03-03 20:33                                                   ` Jens Axboe
2020-03-03 21:03                                                     ` Jeff Layton
2020-03-03 21:20                                                       ` Jens Axboe
2020-03-03 14:10                                 ` Miklos Szeredi
2020-03-03 14:29                                   ` Greg Kroah-Hartman
2020-03-03 14:40                                     ` Jann Horn
2020-03-03 16:51                                       ` Greg Kroah-Hartman
2020-03-03 16:57                                         ` Jann Horn
2020-03-03 20:15                                         ` Greg Kroah-Hartman
2020-03-03 14:40                                   ` David Howells
2020-03-04  4:20                                   ` Ian Kent
2020-03-03 14:19                                 ` David Howells
2020-03-03 16:59                                   ` Greg Kroah-Hartman
2020-03-03 14:23                               ` Christian Brauner
2020-03-03 15:23                                 ` Greg Kroah-Hartman
2020-03-03 15:53                                 ` David Howells
2020-03-04  2:01                           ` Ian Kent
2020-03-04 15:22                             ` Karel Zak
2020-03-04 16:49                               ` Greg Kroah-Hartman
2020-03-04 17:55                                 ` Karel Zak
2020-03-03 14:09                         ` David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200227151421.3u74ijhqt6ekbiss@ws.net.home \
    --to=kzak@redhat.com \
    --cc=James.Bottomley@hansenpartnership.com \
    --cc=christian@brauner.io \
    --cc=darrick.wong@oracle.com \
    --cc=dhowells@redhat.com \
    --cc=jannh@google.com \
    --cc=lennart@poettering.net \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=mszeredi@redhat.com \
    --cc=raven@themaw.net \
    --cc=swhiteho@redhat.com \
    --cc=util-linux@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=zbyszek@in.waw.pl \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git