FAN_CREATE_SELF

FAN_CREATE_SELF

[Amir Goldstein]

Hi Jan,

I have a use case with a listener that uses FAN_REPORT_FID mode.
(fid is an index to a db).
Most of the time fid can be resolved to the path and that is sufficient.
If it cannot, the file will be detected by a later dir scan.

I find that with most of the events this use case can handle the events
efficiently without the name info except for FAN_CREATE.

For files, with most applications, FAN_CREATE will be followed by
some other event with the file fid, but it is not guaranteed.
For dirs, a single FAN_CREATE event is more common.

I was thinking that a FAN_CREATE_SELF event could be useful in some
cases, but I don't think it is a must for closing functional gaps.
For example, another group could be used with FAN_REPORT_DFID_NAME
to listen on FAN_CREATE events only, or on FAN_CREATE (without name)
the dir can be scanned, but it is not ideal.

Before composing a patch I wanted to ask your opinion on the
FAN_CREATE_SELF event. Do you have any thoughts on this?

Thanks,
Amir.

Re: FAN_CREATE_SELF

[Jan Kara]

Hi Amir!

On Wed 09-12-20 21:14:53, Amir Goldstein wrote:
> I have a use case with a listener that uses FAN_REPORT_FID mode.
> (fid is an index to a db).
> Most of the time fid can be resolved to the path and that is sufficient.
> If it cannot, the file will be detected by a later dir scan.
> 
> I find that with most of the events this use case can handle the events
> efficiently without the name info except for FAN_CREATE.
> 
> For files, with most applications, FAN_CREATE will be followed by
> some other event with the file fid, but it is not guaranteed.
> For dirs, a single FAN_CREATE event is more common.
> 
> I was thinking that a FAN_CREATE_SELF event could be useful in some
> cases, but I don't think it is a must for closing functional gaps.
> For example, another group could be used with FAN_REPORT_DFID_NAME
> to listen on FAN_CREATE events only, or on FAN_CREATE (without name)
> the dir can be scanned, but it is not ideal.
> 
> Before composing a patch I wanted to ask your opinion on the
> FAN_CREATE_SELF event. Do you have any thoughts on this?

Well, generating FAN_CREATE_SELF event is kind of odd because you have no
mark placed on the inode which is being created. So it would seem more
logical to me that dirent events - create, move, delete - could provide you
with a FID of object that is affected by the operation (i.e., where DFID +
name takes you). That would have to be another event info type.

BTW, what's the problem with just using FAN_REPORT_DFID_NAME? You don't
want to pay the cost of looking up & copying DFID+name instead of FID for
cases you don't care about? Is there such a significant difference?

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: FAN_CREATE_SELF

[Amir Goldstein]

On Thu, Dec 10, 2020 at 1:43 PM Jan Kara <jack@suse.cz> wrote:
>
> Hi Amir!
>
> On Wed 09-12-20 21:14:53, Amir Goldstein wrote:
> > I have a use case with a listener that uses FAN_REPORT_FID mode.
> > (fid is an index to a db).
> > Most of the time fid can be resolved to the path and that is sufficient.
> > If it cannot, the file will be detected by a later dir scan.
> >
> > I find that with most of the events this use case can handle the events
> > efficiently without the name info except for FAN_CREATE.
> >
> > For files, with most applications, FAN_CREATE will be followed by
> > some other event with the file fid, but it is not guaranteed.
> > For dirs, a single FAN_CREATE event is more common.
> >
> > I was thinking that a FAN_CREATE_SELF event could be useful in some
> > cases, but I don't think it is a must for closing functional gaps.
> > For example, another group could be used with FAN_REPORT_DFID_NAME
> > to listen on FAN_CREATE events only, or on FAN_CREATE (without name)
> > the dir can be scanned, but it is not ideal.
> >
> > Before composing a patch I wanted to ask your opinion on the
> > FAN_CREATE_SELF event. Do you have any thoughts on this?
>
> Well, generating FAN_CREATE_SELF event is kind of odd because you have no
> mark placed on the inode which is being created. So it would seem more
> logical to me that dirent events - create, move, delete - could provide you
> with a FID of object that is affected by the operation (i.e., where DFID +
> name takes you). That would have to be another event info type.

FAN_CREATE_SELF makes sense for a filesystem mark. I forgot to
mention that in the description of my use case.
It also makes sense to API IMO because of symmetry with delete and move self
vs. a completely new type of event.
The application is maintaining a map with entries per accessed file
indexed by fid.
When an object appears on the fs, it would have been nice to be able to update
the map right away, but it is not a deal breaker if it is updated later when the
object is observed in another event.

>
> BTW, what's the problem with just using FAN_REPORT_DFID_NAME? You don't
> want to pay the cost of looking up & copying DFID+name instead of FID for
> cases you don't care about? Is there such a significant difference?

That is the reason. Currently, the application uses FAN_REPORT_DFID_NAME,
but I observed that with some configurations, name info is ONLY relevant for
FAN_CREATE events. For those configurations, if we had FAN_CREATE_SELF,
name info copying, variable size event allocation, less efficient merge, all of
those could be avoided. How much does it actually save? I don't know.

So as I said, I see it as a "nice to have" event, certainly not a
must, but it can
help optimize some workloads.
The honest truth is that I think if I try hard enough I will be able
to find a corner
use case where FAN_CREATE_SELF can actually help to close a functional
gap (or the new type of create event that you mentioned), but I am lazy try,
so I try to use these arguments instead:

The API documentation should be pretty straight forward and natural.
The implementation should be trivial.
So the question is - Why not?

Thanks,
Amir.

Re: FAN_CREATE_SELF

[Amir Goldstein]

On Thu, Dec 10, 2020 at 2:23 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Thu, Dec 10, 2020 at 1:43 PM Jan Kara <jack@suse.cz> wrote:
> >
> > Hi Amir!
> >
> > On Wed 09-12-20 21:14:53, Amir Goldstein wrote:
> > > I have a use case with a listener that uses FAN_REPORT_FID mode.
> > > (fid is an index to a db).
> > > Most of the time fid can be resolved to the path and that is sufficient.
> > > If it cannot, the file will be detected by a later dir scan.
> > >
> > > I find that with most of the events this use case can handle the events
> > > efficiently without the name info except for FAN_CREATE.
> > >
> > > For files, with most applications, FAN_CREATE will be followed by
> > > some other event with the file fid, but it is not guaranteed.
> > > For dirs, a single FAN_CREATE event is more common.
> > >
> > > I was thinking that a FAN_CREATE_SELF event could be useful in some
> > > cases, but I don't think it is a must for closing functional gaps.
> > > For example, another group could be used with FAN_REPORT_DFID_NAME
> > > to listen on FAN_CREATE events only, or on FAN_CREATE (without name)
> > > the dir can be scanned, but it is not ideal.
> > >
> > > Before composing a patch I wanted to ask your opinion on the
> > > FAN_CREATE_SELF event. Do you have any thoughts on this?
> >
> > Well, generating FAN_CREATE_SELF event is kind of odd because you have no
> > mark placed on the inode which is being created. So it would seem more
> > logical to me that dirent events - create, move, delete - could provide you
> > with a FID of object that is affected by the operation (i.e., where DFID +
> > name takes you). That would have to be another event info type.
>
> FAN_CREATE_SELF makes sense for a filesystem mark. I forgot to
> mention that in the description of my use case.
> It also makes sense to API IMO because of symmetry with delete and move self
> vs. a completely new type of event.

Scratch that please. I misread your reply and I was thinking about it
the wrong way.
I agree with you that FAN_CREATE_SELF makes little or no sense.

One option is adding another info type as you wrote, with yet another
FAN_REPORT_XXX flag I presume.

Another option is adding another event type (as I thought you had suggested).
Something like FAN_LINK, for when an inode is linked into the namespace
but from the perspective of the inode as opposed to the perspective of the
parent dir where the inode is being linked.

We could match it with FAN_UNLINK, or interpret FAN_LINK as any sort
of change with linkage of the inode to the namespace (create/delete/rename)
maybe name the event FAN_LINKS.

Obviously, FAN_EVENT_INFO_TYPE_FID would be that of the (un)linked
inode and FAN_EVENT_INFO_TYPE_DFID would be that of the parent.

Apart from being able to learn the fid of a new object appearing in the
filesystem or a new child or a watched parent, watcher can be watching
a file and be notified when that file has been unlinked from the namespace,
even if the file still has open references delaying the report of
FAN_DELETE_SELF.

Thoughts?

Amir.