Difference between invalidating and deleting dentry

Difference between invalidating and deleting dentry

[Nikolaus Rath]

Hello,

I just added an example to FUSE that illustrates use of the
fuse_lowlevel_notify_inval_entry() function. However, when writing it I
realized that I don't actually fully understand how this function
differs from fuse_lowlevel_notify_delete(). Could someone shed some
light on this?

Currently, the FUSE documentation says:

fuse_lowlevel_notify_inval_entry:
   Notify to invalidate parent attributes and the dentry matching
   parent/name

fuse_lowlevel_notify_delete:
   Notify to invalidate parent attributes and delete the dentry matching
   parent/name if the dentry's inode number matches child (otherwise it
   will invalidate the matching dentry).


But what exactly is the difference between deleting and invalidating a
dentry? In each case, isn't the resulting behavior the same, in that the
next time someone tries to access this (parent_inode,entry_name)
combination a lookup() request will be send to the FUSE process?


Thanks,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Amir Goldstein]

Hi Nikolaus,

On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
> Hello,
>
> I just added an example to FUSE that illustrates use of the
> fuse_lowlevel_notify_inval_entry() function. However, when writing it I
> realized that I don't actually fully understand how this function
> differs from fuse_lowlevel_notify_delete(). Could someone shed some
> light on this?
>
> Currently, the FUSE documentation says:
>
> fuse_lowlevel_notify_inval_entry:
>    Notify to invalidate parent attributes and the dentry matching
>    parent/name
>
> fuse_lowlevel_notify_delete:
>    Notify to invalidate parent attributes and delete the dentry matching
>    parent/name if the dentry's inode number matches child (otherwise it
>    will invalidate the matching dentry).
>
>
> But what exactly is the difference between deleting and invalidating a
> dentry?

That is the difference:

/*
 * d_drop() unhashes the entry from the parent dentry hashes, so that it won't
 * be found through a VFS lookup any more. Note that this is different from
 * deleting the dentry - d_delete will try to mark the dentry negative if
 * possible, giving a successful _negative_ lookup, while d_drop will
 * just make the cache lookup fail.
 */

But since fuse_lowlevel_notify_delete does among other things:
d_invalidate->...d_drop()
d_delete()

You may still ask yourself what is the purpose of d_delete() after d_drop(),
because there is no cache entry to make negative...

> In each case, isn't the resulting behavior the same, in that the
> next time someone tries to access this (parent_inode,entry_name)
> combination a lookup() request will be send to the FUSE process?

You are right about the next lookup behavior being the same, but there
are other things that d_delete() does which d_invalidate does not, which
are important, like calling fsnotify_nameremove() and update the cached
inode and dentry that are referenced by open files.

Cheers,
Amir.

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Nikolaus Rath]

Hi Amir,

On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
> Hi Nikolaus,
>
> On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
>> Hello,
>>
>> I just added an example to FUSE that illustrates use of the
>> fuse_lowlevel_notify_inval_entry() function. However, when writing it I
>> realized that I don't actually fully understand how this function
>> differs from fuse_lowlevel_notify_delete(). Could someone shed some
>> light on this?
>>
>> Currently, the FUSE documentation says:
>>
>> fuse_lowlevel_notify_inval_entry:
>>    Notify to invalidate parent attributes and the dentry matching
>>    parent/name
>>
>> fuse_lowlevel_notify_delete:
>>    Notify to invalidate parent attributes and delete the dentry matching
>>    parent/name if the dentry's inode number matches child (otherwise it
>>    will invalidate the matching dentry).
>>
>>
>> But what exactly is the difference between deleting and invalidating a
>> dentry?
>
> That is the difference:
>
> /*
>  * d_drop() unhashes the entry from the parent dentry hashes, so that it won't
>  * be found through a VFS lookup any more. Note that this is different from
>  * deleting the dentry - d_delete will try to mark the dentry negative if
>  * possible, giving a successful _negative_ lookup, while d_drop will
>  * just make the cache lookup fail.
>  */

Alright, so at this point I thought I understood the difference and got
ready to update the documentation, but then you got me very confused:

> But since fuse_lowlevel_notify_delete does among other things:
> d_invalidate->...d_drop()
> d_delete()
>
> You may still ask yourself what is the purpose of d_delete() after d_drop(),
> because there is no cache entry to make negative...

So, in other words, FUSE's notify_delete will *not* store a negative
dentry, but will just drop the dentry?

>> In each case, isn't the resulting behavior the same, in that the
>> next time someone tries to access this (parent_inode,entry_name)
>> combination a lookup() request will be send to the FUSE process?
>
> You are right about the next lookup behavior being the same, but there
> are other things that d_delete() does which d_invalidate does not, which
> are important, like calling fsnotify_nameremove() and update the cached
> inode and dentry that are referenced by open files.

Hmm. So when should one use notify_delete() and when
notify_inval_entry()? I understand there is a difference, but I'm
uncertain about the practical consequences...


Thanks!
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Michael Theall]

On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
> Hi Amir,
> 
> On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
> > 
> > Hi Nikolaus,
> > 
> > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org>
> > wrote:
> > > 
> > > Hello,
> > > 
> > > I just added an example to FUSE that illustrates use of the
> > > fuse_lowlevel_notify_inval_entry() function. However, when
> > > writing it I
> > > realized that I don't actually fully understand how this function
> > > differs from fuse_lowlevel_notify_delete(). Could someone shed
> > > some
> > > light on this?
> > > 
> > > Currently, the FUSE documentation says:
> > > 
> > > fuse_lowlevel_notify_inval_entry:
> > >    Notify to invalidate parent attributes and the dentry matching
> > >    parent/name
> > > 
> > > fuse_lowlevel_notify_delete:
> > >    Notify to invalidate parent attributes and delete the dentry
> > > matching
> > >    parent/name if the dentry's inode number matches child
> > > (otherwise it
> > >    will invalidate the matching dentry).
> > > 
> > > 
> > > But what exactly is the difference between deleting and
> > > invalidating a
> > > dentry?
> > That is the difference:
> > 
> > /*
> >  * d_drop() unhashes the entry from the parent dentry hashes, so
> > that it won't
> >  * be found through a VFS lookup any more. Note that this is
> > different from
> >  * deleting the dentry - d_delete will try to mark the dentry
> > negative if
> >  * possible, giving a successful _negative_ lookup, while d_drop
> > will
> >  * just make the cache lookup fail.
> >  */
> Alright, so at this point I thought I understood the difference and
> got
> ready to update the documentation, but then you got me very confused:
> 
> > 
> > But since fuse_lowlevel_notify_delete does among other things:
> > d_invalidate->...d_drop()
> > d_delete()
> > 
> > You may still ask yourself what is the purpose of d_delete() after
> > d_drop(),
> > because there is no cache entry to make negative...
> So, in other words, FUSE's notify_delete will *not* store a negative
> dentry, but will just drop the dentry?
> 
> > 
> > > 
> > > In each case, isn't the resulting behavior the same, in that the
> > > next time someone tries to access this (parent_inode,entry_name)
> > > combination a lookup() request will be send to the FUSE process?
> > You are right about the next lookup behavior being the same, but
> > there
> > are other things that d_delete() does which d_invalidate does not,
> > which
> > are important, like calling fsnotify_nameremove() and update the
> > cached
> > inode and dentry that are referenced by open files.
> Hmm. So when should one use notify_delete() and when
> notify_inval_entry()? I understand there is a difference, but I'm
> uncertain about the practical consequences...
> 
> 
> Thanks!
> -Nikolaus

Hi Nikolaus,

It sounds to me like you want to use notify_delete() for an
unlink/rmdir and you want to use notify_inval_entry for a rename()
(maybe delete the old name and invalidate the new name). You would want
to do this when you know either of these things has happened outside
the knowledge of the kernel, e.g. you know that an external client has
deleted/renamed a file.

Regards,
Michael Theall

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Amir Goldstein]

On Mon, Oct 10, 2016 at 6:55 PM, Michael Theall
<mtheall@linux.vnet.ibm.com> wrote:
> On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
>> Hi Amir,
>>
>> On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
>> >
>> > Hi Nikolaus,
>> >
>> > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org>
>> > wrote:
>> > >
>> > > Hello,
>> > >
>> > > I just added an example to FUSE that illustrates use of the
>> > > fuse_lowlevel_notify_inval_entry() function. However, when
>> > > writing it I
>> > > realized that I don't actually fully understand how this function
>> > > differs from fuse_lowlevel_notify_delete(). Could someone shed
>> > > some
>> > > light on this?
>> > >
>> > > Currently, the FUSE documentation says:
>> > >
>> > > fuse_lowlevel_notify_inval_entry:
>> > >    Notify to invalidate parent attributes and the dentry matching
>> > >    parent/name
>> > >
>> > > fuse_lowlevel_notify_delete:
>> > >    Notify to invalidate parent attributes and delete the dentry
>> > > matching
>> > >    parent/name if the dentry's inode number matches child
>> > > (otherwise it
>> > >    will invalidate the matching dentry).
>> > >
>> > >
>> > > But what exactly is the difference between deleting and
>> > > invalidating a
>> > > dentry?
>> > That is the difference:
>> >
>> > /*
>> >  * d_drop() unhashes the entry from the parent dentry hashes, so
>> > that it won't
>> >  * be found through a VFS lookup any more. Note that this is
>> > different from
>> >  * deleting the dentry - d_delete will try to mark the dentry
>> > negative if
>> >  * possible, giving a successful _negative_ lookup, while d_drop
>> > will
>> >  * just make the cache lookup fail.
>> >  */
>> Alright, so at this point I thought I understood the difference and
>> got
>> ready to update the documentation, but then you got me very confused:
>>
>> >
>> > But since fuse_lowlevel_notify_delete does among other things:
>> > d_invalidate->...d_drop()
>> > d_delete()
>> >
>> > You may still ask yourself what is the purpose of d_delete() after
>> > d_drop(),
>> > because there is no cache entry to make negative...
>> So, in other words, FUSE's notify_delete will *not* store a negative
>> dentry, but will just drop the dentry?
>>
>> >
>> > >
>> > > In each case, isn't the resulting behavior the same, in that the
>> > > next time someone tries to access this (parent_inode,entry_name)
>> > > combination a lookup() request will be send to the FUSE process?
>> > You are right about the next lookup behavior being the same, but
>> > there
>> > are other things that d_delete() does which d_invalidate does not,
>> > which
>> > are important, like calling fsnotify_nameremove() and update the
>> > cached
>> > inode and dentry that are referenced by open files.
>> Hmm. So when should one use notify_delete() and when
>> notify_inval_entry()? I understand there is a difference, but I'm
>> uncertain about the practical consequences...
>>
>>
>> Thanks!
>> -Nikolaus
>
> Hi Nikolaus,
>
> It sounds to me like you want to use notify_delete() for an
> unlink/rmdir and you want to use notify_inval_entry for a rename()
> (maybe delete the old name and invalidate the new name). You would want
> to do this when you know either of these things has happened outside
> the knowledge of the kernel, e.g. you know that an external client has
> deleted/renamed a file.

Agree. Also external client has modified a file may be a cause for invalidate.

>
> Regards,
> Michael Theall
>

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Nikolaus Rath]

On Oct 10 2016, Michael Theall <mtheall@linux.vnet.ibm.com> wrote:
> On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
>> Hi Amir,
>> 
>> On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
>> > 
>> > Hi Nikolaus,
>> > 
>> > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org>
>> > wrote:
>> > > 
>> > > Hello,
>> > > 
>> > > I just added an example to FUSE that illustrates use of the
>> > > fuse_lowlevel_notify_inval_entry() function. However, when
>> > > writing it I
>> > > realized that I don't actually fully understand how this function
>> > > differs from fuse_lowlevel_notify_delete(). Could someone shed
>> > > some
>> > > light on this?
>> > > 
>> > > Currently, the FUSE documentation says:
>> > > 
>> > > fuse_lowlevel_notify_inval_entry:
>> > >    Notify to invalidate parent attributes and the dentry matching
>> > >    parent/name
>> > > 
>> > > fuse_lowlevel_notify_delete:
>> > >    Notify to invalidate parent attributes and delete the dentry
>> > > matching
>> > >    parent/name if the dentry's inode number matches child
>> > > (otherwise it
>> > >    will invalidate the matching dentry).
>> > > 
>> > > 
>> > > But what exactly is the difference between deleting and
>> > > invalidating a
>> > > dentry?
>> > That is the difference:
>> > 
>> > /*
>> >  * d_drop() unhashes the entry from the parent dentry hashes, so
>> > that it won't
>> >  * be found through a VFS lookup any more. Note that this is
>> > different from
>> >  * deleting the dentry - d_delete will try to mark the dentry
>> > negative if
>> >  * possible, giving a successful _negative_ lookup, while d_drop
>> > will
>> >  * just make the cache lookup fail.
>> >  */
>> Alright, so at this point I thought I understood the difference and
>> got
>> ready to update the documentation, but then you got me very confused:
>> 
>> > 
>> > But since fuse_lowlevel_notify_delete does among other things:
>> > d_invalidate->...d_drop()
>> > d_delete()
>> > 
>> > You may still ask yourself what is the purpose of d_delete() after
>> > d_drop(),
>> > because there is no cache entry to make negative...
>> So, in other words, FUSE's notify_delete will *not* store a negative
>> dentry, but will just drop the dentry?
>> > 
>> > > In each case, isn't the resulting behavior the same, in that the
>> > > next time someone tries to access this (parent_inode,entry_name)
>> > > combination a lookup() request will be send to the FUSE process?
>> > You are right about the next lookup behavior being the same, but
>> > there
>> > are other things that d_delete() does which d_invalidate does not,
>> > which
>> > are important, like calling fsnotify_nameremove() and update the
>> > cached
>> > inode and dentry that are referenced by open files.
>> Hmm. So when should one use notify_delete() and when
>> notify_inval_entry()? I understand there is a difference, but I'm
>> uncertain about the practical consequences...
> 
> It sounds to me like you want to use notify_delete() for an
> unlink/rmdir and you want to use notify_inval_entry for a rename()
> (maybe delete the old name and invalidate the new name).

This sounds reasonable, but what are the reasons? Why does the kernel
need to be told about a rename in a different way than about a removal?
Note that we are not transmitting the new name.

How is a rename even technically different from first removing an entry
and entry and then adding a different one for the same inode? For
example, suppose I have a network file system and this happens on the
remote side:

$ echo "contents" > perm_name
$ ln perm_name old_name
$ rm old_name
$ ln perm_name new name

On the local system, does this really need to be signaled to the kernel
differently than

$ echo "contents" > perm_name
$ ln perm_name old_name
$ mv old_name new_name

(Obviously in the file system one is atomic and the other is not, but I
don't see how this matters for the call to the notify_* function).


Also, what is the reason for _delete() falling back to _inval_entry() in
some conditions? I have trouble coming up with the scenario where this
is required / helpful.


Best,
-Nikolaus


-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Michael Theall]

On Mon, 2016-10-10 at 18:57 +0300, Amir Goldstein wrote:
> On Mon, Oct 10, 2016 at 6:55 PM, Michael Theall
> <mtheall@linux.vnet.ibm.com> wrote:
> > 
> > On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
> > > 
> > > Hi Amir,
> > > 
> > > On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
> > > > 
> > > > 
> > > > Hi Nikolaus,
> > > > 
> > > > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.or
> > > > g>
> > > > wrote:
> > > > > 
> > > > > 
> > > > > Hello,
> > > > > 
> > > > > I just added an example to FUSE that illustrates use of the
> > > > > fuse_lowlevel_notify_inval_entry() function. However, when
> > > > > writing it I
> > > > > realized that I don't actually fully understand how this
> > > > > function
> > > > > differs from fuse_lowlevel_notify_delete(). Could someone
> > > > > shed
> > > > > some
> > > > > light on this?
> > > > > 
> > > > > Currently, the FUSE documentation says:
> > > > > 
> > > > > fuse_lowlevel_notify_inval_entry:
> > > > >    Notify to invalidate parent attributes and the dentry
> > > > > matching
> > > > >    parent/name
> > > > > 
> > > > > fuse_lowlevel_notify_delete:
> > > > >    Notify to invalidate parent attributes and delete the
> > > > > dentry
> > > > > matching
> > > > >    parent/name if the dentry's inode number matches child
> > > > > (otherwise it
> > > > >    will invalidate the matching dentry).
> > > > > 
> > > > > 
> > > > > But what exactly is the difference between deleting and
> > > > > invalidating a
> > > > > dentry?
> > > > That is the difference:
> > > > 
> > > > /*
> > > >  * d_drop() unhashes the entry from the parent dentry hashes,
> > > > so
> > > > that it won't
> > > >  * be found through a VFS lookup any more. Note that this is
> > > > different from
> > > >  * deleting the dentry - d_delete will try to mark the dentry
> > > > negative if
> > > >  * possible, giving a successful _negative_ lookup, while
> > > > d_drop
> > > > will
> > > >  * just make the cache lookup fail.
> > > >  */
> > > Alright, so at this point I thought I understood the difference
> > > and
> > > got
> > > ready to update the documentation, but then you got me very
> > > confused:
> > > 
> > > > 
> > > > 
> > > > But since fuse_lowlevel_notify_delete does among other things:
> > > > d_invalidate->...d_drop()
> > > > d_delete()
> > > > 
> > > > You may still ask yourself what is the purpose of d_delete()
> > > > after
> > > > d_drop(),
> > > > because there is no cache entry to make negative...
> > > So, in other words, FUSE's notify_delete will *not* store a
> > > negative
> > > dentry, but will just drop the dentry?
> > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > In each case, isn't the resulting behavior the same, in that
> > > > > the
> > > > > next time someone tries to access this
> > > > > (parent_inode,entry_name)
> > > > > combination a lookup() request will be send to the FUSE
> > > > > process?
> > > > You are right about the next lookup behavior being the same,
> > > > but
> > > > there
> > > > are other things that d_delete() does which d_invalidate does
> > > > not,
> > > > which
> > > > are important, like calling fsnotify_nameremove() and update
> > > > the
> > > > cached
> > > > inode and dentry that are referenced by open files.
> > > Hmm. So when should one use notify_delete() and when
> > > notify_inval_entry()? I understand there is a difference, but I'm
> > > uncertain about the practical consequences...
> > > 
> > > 
> > > Thanks!
> > > -Nikolaus
> > Hi Nikolaus,
> > 
> > It sounds to me like you want to use notify_delete() for an
> > unlink/rmdir and you want to use notify_inval_entry for a rename()
> > (maybe delete the old name and invalidate the new name). You would
> > want
> > to do this when you know either of these things has happened
> > outside
> > the knowledge of the kernel, e.g. you know that an external client
> > has
> > deleted/renamed a file.
> Agree. Also external client has modified a file may be a cause for
> invalidate.

Unless the name has changed, I would use inval_inode() when an external
client has changed the file.

Regards,
Michael Theall

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Nikolaus Rath]

Hello,

*ping*

Really no one able to clarify the difference between the FUSE invalidate
and delete dentry requests?

Best,
-Nikolaus

On Oct 10 2016, Nikolaus Rath <Nikolaus@rath.org> wrote:
> On Oct 10 2016, Michael Theall <mtheall@linux.vnet.ibm.com> wrote:
>> On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
>>> Hi Amir,
>>> 
>>> On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
>>> > 
>>> > Hi Nikolaus,
>>> > 
>>> > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org>
>>> > wrote:
>>> > > 
>>> > > Hello,
>>> > > 
>>> > > I just added an example to FUSE that illustrates use of the
>>> > > fuse_lowlevel_notify_inval_entry() function. However, when
>>> > > writing it I
>>> > > realized that I don't actually fully understand how this function
>>> > > differs from fuse_lowlevel_notify_delete(). Could someone shed
>>> > > some
>>> > > light on this?
>>> > > 
>>> > > Currently, the FUSE documentation says:
>>> > > 
>>> > > fuse_lowlevel_notify_inval_entry:
>>> > >    Notify to invalidate parent attributes and the dentry matching
>>> > >    parent/name
>>> > > 
>>> > > fuse_lowlevel_notify_delete:
>>> > >    Notify to invalidate parent attributes and delete the dentry
>>> > > matching
>>> > >    parent/name if the dentry's inode number matches child
>>> > > (otherwise it
>>> > >    will invalidate the matching dentry).
>>> > > 
>>> > > 
>>> > > But what exactly is the difference between deleting and
>>> > > invalidating a
>>> > > dentry?
>>> > That is the difference:
>>> > 
>>> > /*
>>> >  * d_drop() unhashes the entry from the parent dentry hashes, so
>>> > that it won't
>>> >  * be found through a VFS lookup any more. Note that this is
>>> > different from
>>> >  * deleting the dentry - d_delete will try to mark the dentry
>>> > negative if
>>> >  * possible, giving a successful _negative_ lookup, while d_drop
>>> > will
>>> >  * just make the cache lookup fail.
>>> >  */
>>> Alright, so at this point I thought I understood the difference and
>>> got
>>> ready to update the documentation, but then you got me very confused:
>>> 
>>> > 
>>> > But since fuse_lowlevel_notify_delete does among other things:
>>> > d_invalidate->...d_drop()
>>> > d_delete()
>>> > 
>>> > You may still ask yourself what is the purpose of d_delete() after
>>> > d_drop(),
>>> > because there is no cache entry to make negative...
>>> So, in other words, FUSE's notify_delete will *not* store a negative
>>> dentry, but will just drop the dentry?
>>> > 
>>> > > In each case, isn't the resulting behavior the same, in that the
>>> > > next time someone tries to access this (parent_inode,entry_name)
>>> > > combination a lookup() request will be send to the FUSE process?
>>> > You are right about the next lookup behavior being the same, but
>>> > there
>>> > are other things that d_delete() does which d_invalidate does not,
>>> > which
>>> > are important, like calling fsnotify_nameremove() and update the
>>> > cached
>>> > inode and dentry that are referenced by open files.
>>> Hmm. So when should one use notify_delete() and when
>>> notify_inval_entry()? I understand there is a difference, but I'm
>>> uncertain about the practical consequences...
>> 
>> It sounds to me like you want to use notify_delete() for an
>> unlink/rmdir and you want to use notify_inval_entry for a rename()
>> (maybe delete the old name and invalidate the new name).
>
> This sounds reasonable, but what are the reasons? Why does the kernel
> need to be told about a rename in a different way than about a removal?
> Note that we are not transmitting the new name.
>
> How is a rename even technically different from first removing an entry
> and entry and then adding a different one for the same inode? For
> example, suppose I have a network file system and this happens on the
> remote side:
>
> $ echo "contents" > perm_name
> $ ln perm_name old_name
> $ rm old_name
> $ ln perm_name new name
>
> On the local system, does this really need to be signaled to the kernel
> differently than
>
> $ echo "contents" > perm_name
> $ ln perm_name old_name
> $ mv old_name new_name
>
> (Obviously in the file system one is atomic and the other is not, but I
> don't see how this matters for the call to the notify_* function).
>
>
> Also, what is the reason for _delete() falling back to _inval_entry() in
> some conditions? I have trouble coming up with the scenario where this
> is required / helpful.
>
>
> Best,
> -Nikolaus
>
>
> -- 
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
>              »Time flies like an arrow, fruit flies like a Banana.«


-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Amir Goldstein]

On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
> Hello,
>
> *ping*
>
> Really no one able to clarify the difference between the FUSE invalidate
> and delete dentry requests?

It may sound strange to you, but I think the documentation is clear enough.
The difference is that in delete case, you *know* that the entry has
been deleted
and in invalidate case, you may not know what happened to the entry.
The specific use cases are implementation specific, but maybe you lost
contact with remote server or maybe the remote protocol imposes a timeout
for validity or maybe you do not have enough information
to figure out if the entry has been deleted and re-created.
Invalidate simply means that you no longer know that the entry is valid.

>
> Best,
> -Nikolaus
>
> On Oct 10 2016, Nikolaus Rath <Nikolaus@rath.org> wrote:
>> On Oct 10 2016, Michael Theall <mtheall@linux.vnet.ibm.com> wrote:
>>> On Mon, 2016-10-10 at 08:45 -0700, Nikolaus Rath wrote:
>>>> Hi Amir,
>>>>
>>>> On Oct 10 2016, Amir Goldstein <amir73il@gmail.com> wrote:
>>>> >
>>>> > Hi Nikolaus,
>>>> >
>>>> > On Sun, Oct 9, 2016 at 7:37 AM, Nikolaus Rath <Nikolaus@rath.org>
>>>> > wrote:
>>>> > >
>>>> > > Hello,
>>>> > >
>>>> > > I just added an example to FUSE that illustrates use of the
>>>> > > fuse_lowlevel_notify_inval_entry() function. However, when
>>>> > > writing it I
>>>> > > realized that I don't actually fully understand how this function
>>>> > > differs from fuse_lowlevel_notify_delete(). Could someone shed
>>>> > > some
>>>> > > light on this?
>>>> > >
>>>> > > Currently, the FUSE documentation says:
>>>> > >
>>>> > > fuse_lowlevel_notify_inval_entry:
>>>> > >    Notify to invalidate parent attributes and the dentry matching
>>>> > >    parent/name
>>>> > >
>>>> > > fuse_lowlevel_notify_delete:
>>>> > >    Notify to invalidate parent attributes and delete the dentry
>>>> > > matching
>>>> > >    parent/name if the dentry's inode number matches child
>>>> > > (otherwise it
>>>> > >    will invalidate the matching dentry).
>>>> > >
>>>> > >
>>>> > > But what exactly is the difference between deleting and
>>>> > > invalidating a
>>>> > > dentry?
>>>> > That is the difference:
>>>> >
>>>> > /*
>>>> >  * d_drop() unhashes the entry from the parent dentry hashes, so
>>>> > that it won't
>>>> >  * be found through a VFS lookup any more. Note that this is
>>>> > different from
>>>> >  * deleting the dentry - d_delete will try to mark the dentry
>>>> > negative if
>>>> >  * possible, giving a successful _negative_ lookup, while d_drop
>>>> > will
>>>> >  * just make the cache lookup fail.
>>>> >  */
>>>> Alright, so at this point I thought I understood the difference and
>>>> got
>>>> ready to update the documentation, but then you got me very confused:
>>>>
>>>> >
>>>> > But since fuse_lowlevel_notify_delete does among other things:
>>>> > d_invalidate->...d_drop()
>>>> > d_delete()
>>>> >
>>>> > You may still ask yourself what is the purpose of d_delete() after
>>>> > d_drop(),
>>>> > because there is no cache entry to make negative...
>>>> So, in other words, FUSE's notify_delete will *not* store a negative
>>>> dentry, but will just drop the dentry?
>>>> >
>>>> > > In each case, isn't the resulting behavior the same, in that the
>>>> > > next time someone tries to access this (parent_inode,entry_name)
>>>> > > combination a lookup() request will be send to the FUSE process?
>>>> > You are right about the next lookup behavior being the same, but
>>>> > there
>>>> > are other things that d_delete() does which d_invalidate does not,
>>>> > which
>>>> > are important, like calling fsnotify_nameremove() and update the
>>>> > cached
>>>> > inode and dentry that are referenced by open files.
>>>> Hmm. So when should one use notify_delete() and when
>>>> notify_inval_entry()? I understand there is a difference, but I'm
>>>> uncertain about the practical consequences...
>>>
>>> It sounds to me like you want to use notify_delete() for an
>>> unlink/rmdir and you want to use notify_inval_entry for a rename()
>>> (maybe delete the old name and invalidate the new name).
>>
>> This sounds reasonable, but what are the reasons? Why does the kernel
>> need to be told about a rename in a different way than about a removal?
>> Note that we are not transmitting the new name.
>>
>> How is a rename even technically different from first removing an entry
>> and entry and then adding a different one for the same inode? For
>> example, suppose I have a network file system and this happens on the
>> remote side:
>>
>> $ echo "contents" > perm_name
>> $ ln perm_name old_name
>> $ rm old_name
>> $ ln perm_name new name
>>
>> On the local system, does this really need to be signaled to the kernel
>> differently than
>>
>> $ echo "contents" > perm_name
>> $ ln perm_name old_name
>> $ mv old_name new_name
>>
>> (Obviously in the file system one is atomic and the other is not, but I
>> don't see how this matters for the call to the notify_* function).
>>
>>
>> Also, what is the reason for _delete() falling back to _inval_entry() in
>> some conditions? I have trouble coming up with the scenario where this
>> is required / helpful.
>>
>>
>> Best,
>> -Nikolaus
>>
>>
>> --
>> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
>> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>>
>>              »Time flies like an arrow, fruit flies like a Banana.«
>
>
> --
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
>              »Time flies like an arrow, fruit flies like a Banana.«
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> --
> fuse-devel mailing list
> To unsubscribe or subscribe, visit https://lists.sourceforge.net/lists/listinfo/fuse-devel

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Miklos Szeredi]

On Wed, Oct 19, 2016 at 7:53 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:

>> Really no one able to clarify the difference between the FUSE invalidate
>> and delete dentry requests?
>
> It may sound strange to you, but I think the documentation is clear enough.
> The difference is that in delete case, you *know* that the entry has
> been deleted
> and in invalidate case, you may not know what happened to the entry.
> The specific use cases are implementation specific, but maybe you lost
> contact with remote server or maybe the remote protocol imposes a timeout
> for validity or maybe you do not have enough information
> to figure out if the entry has been deleted and re-created.
> Invalidate simply means that you no longer know that the entry is valid.

That was the intent, yes.  However 'invalidate' + 'lookup negative'
should be equivalent to 'delete'.  And it is.

The reason why 'delete' was introduced by commit 451d0f599934 ("FUSE:
Notifying the kernel of deletion.") is that 'invalidate' wasn't able
to remove in-use subtrees.

Commit bafc9b754f75 ("vfs: More precise tests in d_invalidate")
changed that, so now 'notify_delete' is superfluous and AFAICS the
effect of it is exactly the same as 'notify_invalidate'.

Thanks,
Miklos

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Amir Goldstein]

On Wed, Oct 19, 2016 at 11:01 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Wed, Oct 19, 2016 at 7:53 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
>
>>> Really no one able to clarify the difference between the FUSE invalidate
>>> and delete dentry requests?
>>
>> It may sound strange to you, but I think the documentation is clear enough.
>> The difference is that in delete case, you *know* that the entry has
>> been deleted
>> and in invalidate case, you may not know what happened to the entry.
>> The specific use cases are implementation specific, but maybe you lost
>> contact with remote server or maybe the remote protocol imposes a timeout
>> for validity or maybe you do not have enough information
>> to figure out if the entry has been deleted and re-created.
>> Invalidate simply means that you no longer know that the entry is valid.
>
> That was the intent, yes.  However 'invalidate' + 'lookup negative'
> should be equivalent to 'delete'.  And it is.
>
> The reason why 'delete' was introduced by commit 451d0f599934 ("FUSE:
> Notifying the kernel of deletion.") is that 'invalidate' wasn't able
> to remove in-use subtrees.
>
> Commit bafc9b754f75 ("vfs: More precise tests in d_invalidate")
> changed that, so now 'notify_delete' is superfluous and AFAICS the
> effect of it is exactly the same as 'notify_invalidate'.
>

Minus fsnotify_nameremove() for whatever it is worth.

Amir.

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Miklos Szeredi]

On Wed, Oct 19, 2016 at 10:32 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Oct 19, 2016 at 11:01 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Wed, Oct 19, 2016 at 7:53 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
>>
>>>> Really no one able to clarify the difference between the FUSE invalidate
>>>> and delete dentry requests?
>>>
>>> It may sound strange to you, but I think the documentation is clear enough.
>>> The difference is that in delete case, you *know* that the entry has
>>> been deleted
>>> and in invalidate case, you may not know what happened to the entry.
>>> The specific use cases are implementation specific, but maybe you lost
>>> contact with remote server or maybe the remote protocol imposes a timeout
>>> for validity or maybe you do not have enough information
>>> to figure out if the entry has been deleted and re-created.
>>> Invalidate simply means that you no longer know that the entry is valid.
>>
>> That was the intent, yes.  However 'invalidate' + 'lookup negative'
>> should be equivalent to 'delete'.  And it is.
>>
>> The reason why 'delete' was introduced by commit 451d0f599934 ("FUSE:
>> Notifying the kernel of deletion.") is that 'invalidate' wasn't able
>> to remove in-use subtrees.
>>
>> Commit bafc9b754f75 ("vfs: More precise tests in d_invalidate")
>> changed that, so now 'notify_delete' is superfluous and AFAICS the
>> effect of it is exactly the same as 'notify_invalidate'.
>>
>
> Minus fsnotify_nameremove() for whatever it is worth.

Ah, right.  But fuse lacks proper fsnotify support for remote fs
anyway, so it's not worth much.

Thanks,
Miklos

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Nikolaus Rath]

On Oct 19 2016, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus@rath.org> wrote:
>> Hello,
>>
>> *ping*
>>
>> Really no one able to clarify the difference between the FUSE invalidate
>> and delete dentry requests?
>
> It may sound strange to you, but I think the documentation is clear
> enough.  The difference is that in delete case, you *know* that the
> entry has been deleted and in invalidate case, you may not know what
> happened to the entry.  The specific use cases are implementation
> specific, but maybe you lost contact with remote server or maybe the
> remote protocol imposes a timeout for validity or maybe you do not
> have enough information to figure out if the entry has been deleted
> and re-created.  Invalidate simply means that you no longer know that
> the entry is valid.

This would make sense if delete would cache the negative lookup, but
based on what you said earlier this is not the case.


So what is the practical difference between the two functions? Or,
putting it differently, what are the consequences of sending a delete
when the entry still exists, or an invalidate when it has been deleted?


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Difference between invalidating and deleting dentry

[Nikolaus Rath]

On Oct 19 2016, Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org> wrote:
> On Wed, Oct 19, 2016 at 10:32 AM, Amir Goldstein <amir73il-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On Wed, Oct 19, 2016 at 11:01 AM, Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org> wrote:
>>> On Wed, Oct 19, 2016 at 7:53 AM, Amir Goldstein <amir73il-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> On Wed, Oct 19, 2016 at 4:51 AM, Nikolaus Rath <Nikolaus-BTH8mxji4b0@public.gmane.org> wrote:
>>>
>>>>> Really no one able to clarify the difference between the FUSE invalidate
>>>>> and delete dentry requests?
>>>>
>>>> It may sound strange to you, but I think the documentation is clear enough.
>>>> The difference is that in delete case, you *know* that the entry has
>>>> been deleted
>>>> and in invalidate case, you may not know what happened to the entry.
>>>> The specific use cases are implementation specific, but maybe you lost
>>>> contact with remote server or maybe the remote protocol imposes a timeout
>>>> for validity or maybe you do not have enough information
>>>> to figure out if the entry has been deleted and re-created.
>>>> Invalidate simply means that you no longer know that the entry is valid.
>>>
>>> That was the intent, yes.  However 'invalidate' + 'lookup negative'
>>> should be equivalent to 'delete'.  And it is.
>>>
>>> The reason why 'delete' was introduced by commit 451d0f599934 ("FUSE:
>>> Notifying the kernel of deletion.") is that 'invalidate' wasn't able
>>> to remove in-use subtrees.
>>>
>>> Commit bafc9b754f75 ("vfs: More precise tests in d_invalidate")
>>> changed that, so now 'notify_delete' is superfluous and AFAICS the
>>> effect of it is exactly the same as 'notify_invalidate'.
>>>
>>
>> Minus fsnotify_nameremove() for whatever it is worth.
>
> Ah, right.  But fuse lacks proper fsnotify support for remote fs
> anyway, so it's not worth much.

Thanks Miklos!

So I'll extend the documentation to say that both _invalidate() and
_delete() drop the dentry from the cache (so the next lookup attempt)
will be passed to the fuse file sysem), but that _delete() will also
inform any inotify watchers that the entry has disappeared.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: [fuse-devel] Difference between invalidating and deleting dentry

[John Muir]

> On 2016.10.08, at 21:37 , Nikolaus Rath <Nikolaus@rath.org> wrote:
> 
> fuse_lowlevel_notify_inval_entry:
>   Notify to invalidate parent attributes and the dentry matching
>   parent/name
> 
> fuse_lowlevel_notify_delete:
>   Notify to invalidate parent attributes and delete the dentry matching
>   parent/name if the dentry's inode number matches child (otherwise it
>   will invalidate the matching dentry).
> 
> 
> But what exactly is the difference between deleting and invalidating a
> dentry? In each case, isn't the resulting behavior the same, in that the
> next time someone tries to access this (parent_inode,entry_name)
> combination a lookup() request will be send to the FUSE process?


These are aimed at networked file-systems where changes can be initiated at the other end.

The first clears the cached data for that dentry, so that next time the file lookup occurs the file is still in existence but there is no cached data, forcing the request to go down to the user-space file-system. This would be used during remote rename.

The second actually removes the dentry in the VFS in the kernel. This would be used during remote deletion.

Re: [fuse-devel] Difference between invalidating and deleting dentry

[Nikolaus Rath]

On Oct 31 2016, John Muir <john@jmuir.com> wrote:
>> On 2016.10.08, at 21:37 , Nikolaus Rath <Nikolaus@rath.org> wrote:
>> 
>> fuse_lowlevel_notify_inval_entry:
>>   Notify to invalidate parent attributes and the dentry matching
>>   parent/name
>> 
>> fuse_lowlevel_notify_delete:
>>   Notify to invalidate parent attributes and delete the dentry matching
>>   parent/name if the dentry's inode number matches child (otherwise it
>>   will invalidate the matching dentry).
>> 
>> 
>> But what exactly is the difference between deleting and invalidating a
>> dentry? In each case, isn't the resulting behavior the same, in that the
>> next time someone tries to access this (parent_inode,entry_name)
>> combination a lookup() request will be send to the FUSE process?
>
>
> These are aimed at networked file-systems where changes can be initiated at the other end.
>
> The first clears the cached data for that dentry, so that next time
> the file lookup occurs the file is still in existence but there is no
> cached data, forcing the request to go down to the user-space
> file-system. This would be used during remote rename.
>
> The second actually removes the dentry in the VFS in the kernel. This
> would be used during remote deletion.

Alright, seems there are as many different opinions on this as there are
replies. So I tried to make sense of the code myself.

I think Miklos and Amir are correct: the only difference between the
functions is a call to fsnotify.

At least as of kernel 4.8, both notify_delete and notify_inval_inode
result in a call to fuse_reverse_inval_entry() in the kernel. And that
function calls fuse_invalidate_attr (to invalidate the attributes of the
parent) and fuse_invalidate_entry (which removes the dentry from the
hash via d_invalidate *and* marks it as stale).

The only difference is that when reverse_inval_entry is entered via
notify_delete, it attempts to call d_delete (after the call to
d_invalidate), which seems to have the effect of only calling
fsnotify_nameremove().

John, are we missing something?

Best,
Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«