linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Steven Whitehouse <swhiteho@redhat.com>
Cc: David Howells <dhowells@redhat.com>,
	Miklos Szeredi <miklos@szeredi.hu>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Al Viro <viro@zeniv.linux.org.uk>, Karel Zak <kzak@redhat.com>,
	Jeff Layton <jlayton@redhat.com>,
	Miklos Szeredi <mszeredi@redhat.com>,
	Nicolas Dichtel <nicolas.dichtel@6wind.com>,
	Christian Brauner <christian@brauner.io>,
	Lennart Poettering <lennart@poettering.net>,
	Linux API <linux-api@vger.kernel.org>,
	Ian Kent <raven@themaw.net>,
	LSM <linux-security-module@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: file metadata via fs API
Date: Mon, 17 Aug 2020 10:15:08 -0700	[thread overview]
Message-ID: <CAHk-=wig0ZqWxgWtD9F1xZzE7jEmgLmXRWABhss0+er3ZRtb9g@mail.gmail.com> (raw)
In-Reply-To: <94f907f0-996e-0456-db8a-7823e2ef3d3f@redhat.com>

On Mon, Aug 17, 2020 at 4:33 AM Steven Whitehouse <swhiteho@redhat.com> wrote:
>
> That said, the overall aim here is to solve the problem and if there are
> better solutions available then I'm sure that everyone is very open to
> those. I agree very much that monitoring at kHz frequencies is not
> useful, but at the same time, there are cases which can generate large
> amounts of mount changes in a very short time period.

So the thing is, I absolutely believe in the kernel _notifying_ about
changes so that people don't need to poll. It's why I did merge the
notification queues, although I wanted to make sure that those worked.

> You recently requested some details of real users for the notifications,
> and (I assumed) by extension fsinfo too.

No, fsinfo wasn't on the table there. To me, notifications are a
completely separate issue, because you *can* get the information from
existing sources (ie things like /proc/mounts etc), and notification
seemed to be the much more fundamental issue.

If you poll for changes, parsing something like /proc/mounts is
obviously very heavy indeed. I don't find that particularly
controversial. Plus the notification queues had other uses, even if it
wasn't clear how many or who would use them.

But honestly, the actual fsinfo thing seems (a) overdesigned and (b)
broken. I've now had two different people say how they want to use it
to figure out whether a filesystem supports certain things that aren't
even per-filesystem things in the first place.

And this feature is clearly controversial, with actual discussion about it.

And I find the whole thing confusing and over-engineered. If this was
a better statfs(), that would be one thing. But it is designed to be
this monstoer thing that does many different things, and I find it
distasteful.  Yes, you can query "extended statfs" kind of data with
it and get the per-file attributes. I find it really annoying how the
vfs layer calls to the filesystems, that then call back to the vfs
layer to fill things in, but I guess we have that nasty pattern from
stat() already. I'd rather have the VFS layer just fill in all the
default values and the stuff it already knows about, and then maybe
have the filesystem callback fill in the ones the vfs *doesn't* know
about, but whatever.

But then you can *also* query odd things like mounts that aren't even
visible, and the topology, and completely random error state.

So it has this very complex "random structures of random things"
implementation. It's a huge sign of over-design and "I don't know what
the hell I want to expose, so I'll make this generic thing that can
expose anything, and then I start adding random fields".

Some things are per-file, some things are per-mount, and some things
are per-namespace and cross mount boundaries.

And honestly, the "random binary interfaces" just turns me off a lot.

A simple and straightforward struct? Sure. But this random "whatever
goes" thing? No.

                 Linus

  reply	other threads:[~2020-08-17 17:28 UTC|newest]

Thread overview: 89+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-03 15:27 [GIT PULL] Mount notifications David Howells
2020-08-03 15:49 ` [GIT PULL] Filesystem Information David Howells
2020-08-03 16:42   ` Miklos Szeredi
2020-08-04  2:15     ` Ian Kent
2020-08-04 14:36       ` Miklos Szeredi
2020-08-05  1:33         ` Ian Kent
2020-08-05  8:00           ` Miklos Szeredi
2020-08-05 11:13             ` Ian Kent
2020-08-05  8:24         ` file metadata via fs API (was: [GIT PULL] Filesystem Information) Miklos Szeredi
2020-08-11 13:54           ` Miklos Szeredi
2020-08-11 14:08             ` Al Viro
2020-08-11 14:22               ` Miklos Szeredi
2020-08-11 14:31                 ` Al Viro
     [not found]                   ` <CAAgocE07=vVKpQhG+rjEGO=NEBKZ02gjg4TRPxECAc+RKrzn=Q@mail.gmail.com>
2020-08-11 14:36                     ` Al Viro
2020-08-11 14:36                   ` Miklos Szeredi
2020-08-11 14:42                     ` Al Viro
2020-08-11 14:47                       ` Miklos Szeredi
2020-08-11 15:20             ` Linus Torvalds
2020-08-11 15:30               ` Miklos Szeredi
2020-08-11 16:05                 ` Linus Torvalds
2020-08-11 18:49                   ` Miklos Szeredi
2020-08-11 19:31                     ` Lennart Poettering
2020-08-11 19:50                       ` Christian Brauner
2020-08-11 19:39                   ` Christian Brauner
2020-08-12  0:53                     ` Ian Kent
2020-08-11 15:39               ` Andy Lutomirski
2020-08-11 16:17                 ` Casey Schaufler
2020-08-11 16:30                   ` Linus Torvalds
2020-08-11 20:28                   ` Miklos Szeredi
2020-08-11 20:36                     ` Jann Horn
2020-08-11 20:56                       ` Miklos Szeredi
2020-08-11 21:17                         ` Andy Lutomirski
2020-08-11 21:18                         ` Linus Torvalds
2020-08-12  7:23                           ` Miklos Szeredi
2020-08-12 14:39                             ` Al Viro
2020-08-12 14:46                               ` Miklos Szeredi
2020-08-12 15:08                                 ` Al Viro
2020-08-12 15:13                                   ` Miklos Szeredi
2020-08-12 16:33                                     ` Al Viro
2020-08-12 17:16                                       ` Miklos Szeredi
2020-08-12 17:39                                         ` Al Viro
2020-08-12 18:33                                           ` Al Viro
2020-08-12 21:30                                             ` Al Viro
2020-08-18  9:41                                               ` Miklos Szeredi
2020-08-18  9:30                                             ` Miklos Szeredi
2020-08-12 15:22                                   ` David Howells
2020-08-11 21:20                     ` Al Viro
2020-08-11 21:35                     ` Casey Schaufler
2020-08-11 16:05               ` Al Viro
2020-08-11 16:09                 ` Linus Torvalds
2020-08-11 16:39                   ` Al Viro
2020-08-12 10:14               ` Karel Zak
2020-08-12 13:09                 ` Miklos Szeredi
2020-08-12 13:33                 ` David Howells
2020-08-12 13:54                   ` Miklos Szeredi
2020-08-12  0:05             ` David Howells
2020-08-12  7:55               ` Miklos Szeredi
2020-08-12  8:29               ` David Howells
2020-08-12  8:37                 ` Miklos Szeredi
2020-08-12  9:43                   ` file metadata via fs API Steven Whitehouse
2020-08-12 10:04                     ` Miklos Szeredi
2020-08-12 11:28                       ` Karel Zak
2020-08-12 12:43                         ` Miklos Szeredi
2020-08-13  8:52                           ` Karel Zak
2020-08-12 13:06                         ` David Howells
2020-08-13  1:01                           ` Ian Kent
2020-08-12 18:18               ` file metadata via fs API (was: [GIT PULL] Filesystem Information) Linus Torvalds
2020-08-12 19:34                 ` file metadata via fs API Steven Whitehouse
2020-08-12 19:50                   ` Linus Torvalds
2020-08-13  3:44                     ` Ian Kent
2020-08-13 10:36                     ` Karel Zak
2020-08-14  7:58                     ` Lennart Poettering
2020-08-17 11:32                     ` Steven Whitehouse
2020-08-17 17:15                       ` Linus Torvalds [this message]
2020-08-17 22:44                         ` Linus Torvalds
2020-08-18 12:50                           ` Miklos Szeredi
2020-08-18 18:51                             ` Linus Torvalds
2020-08-18 20:18                               ` Miklos Szeredi
2020-08-18 20:53                                 ` Linus Torvalds
2020-08-21 13:17                                   ` Miklos Szeredi
2020-08-19  2:29                               ` Al Viro
2020-08-13  3:53                 ` file metadata via fs API (was: [GIT PULL] Filesystem Information) Jeffrey E Altman
2020-08-14 17:05                   ` Linus Torvalds
2020-08-18 15:01                     ` Jeffrey E Altman
2020-08-14  8:06                 ` Lennart Poettering
2020-08-12 13:54             ` David Howells
2020-08-12 14:10               ` Miklos Szeredi
2020-08-12 14:23               ` David Howells
2020-08-03 22:48 ` [GIT PULL] Mount notifications Ian Kent

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAHk-=wig0ZqWxgWtD9F1xZzE7jEmgLmXRWABhss0+er3ZRtb9g@mail.gmail.com' \
    --to=torvalds@linux-foundation.org \
    --cc=christian@brauner.io \
    --cc=dhowells@redhat.com \
    --cc=jlayton@redhat.com \
    --cc=kzak@redhat.com \
    --cc=lennart@poettering.net \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-security-module@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=mszeredi@redhat.com \
    --cc=nicolas.dichtel@6wind.com \
    --cc=raven@themaw.net \
    --cc=swhiteho@redhat.com \
    --cc=viro@zeniv.linux.org.uk \
    --subject='Re: file metadata via fs API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).