File alteration events?

File alteration events?

[Ronald Schaten]

Hello everybody...

This is my first mail to this list, and -- as much as I'd like to be --
I'm not a kernel developer. So please forgive me if this isn't the right
place for questions like this. I'm thankful for any pointer into the
right direction.


The question:

I'm looking for a solution to track file changes on a fairly large
filesystem. Many thousands of directories, millions of files.

The volume is shared to other computers, changes can come from different
sources (NFS, Samba, local access...). So I think the kernel or the
filesystem itself are the definitive source of information.

Does BTRFS have any mechanism that could be used to track every event?
Or do you happen to know if the kernel does?


What I tried till now:

- inotify: Doesn't work for a full volume, just for single inodes.

- fanotify: That's almost the solution, but I need to see rename and
  delete events, too.

- btrfs subvolume find-new: As far as I know that only finds changes
  between snapshots. I need something a little more real-timey, and I
  have to know if the file has been changed once or several times.


As I said: I'm thankful for any hint... thanks in advance!


Regards,
Ronald.

-- 
Ronald Schaten :: http://www.schatenseite.de

Re: File alteration events?

[Steven Davies]

On 2019-01-31 12:38, Ronald Schaten wrote:
> Hello everybody...

> I'm looking for a solution to track file changes on a fairly large
> filesystem. Many thousands of directories, millions of files.
> 
> The volume is shared to other computers, changes can come from 
> different
> sources (NFS, Samba, local access...). So I think the kernel or the
> filesystem itself are the definitive source of information.
> 
> Does BTRFS have any mechanism that could be used to track every event?
> Or do you happen to know if the kernel does?
> 
> 
> What I tried till now:
> 
> - inotify: Doesn't work for a full volume, just for single inodes.
> 
> - fanotify: That's almost the solution, but I need to see rename and
>   delete events, too.
> 
> - btrfs subvolume find-new: As far as I know that only finds changes
>   between snapshots. I need something a little more real-timey, and I
>   have to know if the file has been changed once or several times.
> 
> 
> As I said: I'm thankful for any hint... thanks in advance!

As far as I know btrfs doesn't have any hooks you could use for that, 
but perhaps have a look at a utility called inotifywait instead.

-- 
Steve

Re: File alteration events?

[Austin S. Hemmelgarn]

On 2019-01-31 07:38, Ronald Schaten wrote:
> Hello everybody...
> 
> This is my first mail to this list, and -- as much as I'd like to be --
> I'm not a kernel developer. So please forgive me if this isn't the right
> place for questions like this. I'm thankful for any pointer into the
> right direction.
> 
> 
> The question:
> 
> I'm looking for a solution to track file changes on a fairly large
> filesystem. Many thousands of directories, millions of files.
> 
> The volume is shared to other computers, changes can come from different
> sources (NFS, Samba, local access...). So I think the kernel or the
> filesystem itself are the definitive source of information.
> 
> Does BTRFS have any mechanism that could be used to track every event?
> Or do you happen to know if the kernel does?
> 
> 
> What I tried till now:
> 
> - inotify: Doesn't work for a full volume, just for single inodes.
> 
> - fanotify: That's almost the solution, but I need to see rename and
>    delete events, too.
> 
> - btrfs subvolume find-new: As far as I know that only finds changes
>    between snapshots. I need something a little more real-timey, and I
>    have to know if the file has been changed once or several times.
> 
> 
BTRFS does not, anything like this is going to be in the VFS layer.

Given what you've said above, I'd suggest asking the VFS developers 
about the possibility of getting rename and delete events in fanotify, 
as that's likely to be the best option long-term.

Re: File alteration events?

[Nikolay Borisov]

On 31.01.19 г. 14:38 ч., Ronald Schaten wrote:
> Hello everybody...
> 
> This is my first mail to this list, and -- as much as I'd like to be --
> I'm not a kernel developer. So please forgive me if this isn't the right
> place for questions like this. I'm thankful for any pointer into the
> right direction.
> 
> 
> The question:
> 
> I'm looking for a solution to track file changes on a fairly large
> filesystem. Many thousands of directories, millions of files.
> 
> The volume is shared to other computers, changes can come from different
> sources (NFS, Samba, local access...). So I think the kernel or the
> filesystem itself are the definitive source of information.
> 
> Does BTRFS have any mechanism that could be used to track every event?
> Or do you happen to know if the kernel does?
> 
> 
> What I tried till now:
> 
> - inotify: Doesn't work for a full volume, just for single inodes.
> 
> - fanotify: That's almost the solution, but I need to see rename and
>   delete events, too.

As other have said the filesystem is not the place to do this. And as it
turns out rename is already supported by fanotify, based on reading the
code of vfs_rename, right after the out: label

if (!error) {
                fsnotify_move(old_dir, new_dir, old_name.name, is_dir,

                              !(flags & RENAME_EXCHANGE) ? target :
NULL, old_dentry);
                if (flags & RENAME_EXCHANGE) {

                        fsnotify_move(new_dir, old_dir,
old_dentry->d_name.name,
                                      new_is_dir, NULL, new_dentry);

                }

        }

And regarding delete fsnotify is also notified that the link count has
already changed.

> 
> - btrfs subvolume find-new: As far as I know that only finds changes
>   between snapshots. I need something a little more real-timey, and I
>   have to know if the file has been changed once or several times.
> 
> 
> As I said: I'm thankful for any hint... thanks in advance!
> 
> 
> Regards,
> Ronald.
>