Re: File monitor problem

From: Amir Goldstein <amir73il@gmail.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Jan Kara <jack@suse.cz>, Mo Re Ra <more7.rev@gmail.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: File monitor problem
Date: Wed, 4 Dec 2019 22:27:31 +0200	[thread overview]
Message-ID: <CAOQ4uxiZWKCUKcpBt-bHOcnHoFAq+nghWmf94rJu=3CTc5VhRA@mail.gmail.com> (raw)
In-Reply-To: <20191204190206.GA8331@bombadil.infradead.org>

On Wed, Dec 4, 2019 at 9:02 PM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Wed, Dec 04, 2019 at 08:37:09PM +0200, Amir Goldstein wrote:
> > On Wed, Dec 4, 2019 at 7:34 PM Jan Kara <jack@suse.cz> wrote:
> > > The problem is there's no better reliable way. For example even if fanotify
> > > event provided a name as in the Amir's commit you reference, this name is
> > > not very useful. Because by the time your application gets to processing
> > > that fanotify event, the file under that name need not exist anymore, or
> >
> > For DELETE event, file is not expected to exist, the filename in event is
> > always "reliable" (i.e. this name was unlinked).
>
> Jan already pointed out that events may be reordered.  So a CREATE event
> and DELETE event may arrive out of order for the same file.  This will
> confuse any agent.
>

Right. Re-ordering of events is an issue that needs to be addressed.
But the truth is that events for the same file are not re-ordered, they
are merged (though a DELETE_SELF and CREATE could be re-ordered
because they are not on the same object).

The way to frame this correctly IMO is that fsnotify events let application
know that "something has changed", without any ordering guaranty
beyond "sometime before the event was read".

So far, that "something" can be a file (by fd), an inode (by fid),
more specifically a directory inode (by fid) where in an entry has
changed.

Adding filename info extends that concept to "something has changed
in the namespace at" (by parent fid+name).
All it means is that application should pay attention to that part of
the namespace and perform a lookup to find out what has changed.

Maybe the way to mitigate wrong assumptions about ordering and
existence of the filename in the namespace is to omit the event type
for "filename events", for example: { FAN_CHANGE, pfid, name }.

I know that defining a good API is challenging, but I have no doubt
that the need is real.
If anyone doubts that, please present an alternative that users can
actually use.

Ignoring users demand and leaving Linux filesystem notification API
functionality decades behind Windows and MacOS is not acceptable IMO.
We have made a big step forward with fanotify dirent events, but I think
we will need to make a few more adjustments to the API before we get to
what most users need.

Thanks,
Amir.