[PATCH 0/4] fanotify updates for v5.7

[PATCH 0/4] fanotify updates for v5.7

[Amir Goldstein]

Hi Michael,

Following patches include fixes to earlier FAN_REPORT_FID change from
v5.1 and an update for the new event FAN_DIR_MODIFY that has a new
event format.

Those patches were reviewed by Jan and Matthew earlier on github.

Thanks,
Amir.

Amir Goldstein (4):
  fanotify_mark.2: Clarification about FAN_MARK_MOUNT and FAN_REPORT_FID
  fanotify_mark.2: Clarification about FAN_EVENT_ON_CHILD and new events
  fanotify.7: Fix fanotify_fid.c example
  fanotify.7, fanotify_mark.2: Document FAN_DIR_MODIFY

 man2/fanotify_mark.2 |  89 ++++++++++++++++++++++++++++++++---
 man7/fanotify.7      | 107 +++++++++++++++++++++++++++++++++----------
 2 files changed, 165 insertions(+), 31 deletions(-)

-- 
2.17.1

[PATCH 1/4] fanotify_mark.2: Clarification about FAN_MARK_MOUNT and FAN_REPORT_FID

[Amir Goldstein]

It is not true that FAN_MARK_MOUNT cannot be used with a group that
was initialized with flag FAN_REPORT_FID.

The correct assertion is that events that require a group with flag
FAN_REPORT_FID cannot be requested on a mark mount.

For exaple, a FAN_OPEN event can be requested on a mark mount and
will generate an event with file handle information if the group was
initialized with flag FAN_REPORT_FID.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
---
 man2/fanotify_mark.2 | 41 ++++++++++++++++++++++++++++++++++++-----
 1 file changed, 36 insertions(+), 5 deletions(-)

diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
index 30290d519..687e20f4a 100644
--- a/man2/fanotify_mark.2
+++ b/man2/fanotify_mark.2
@@ -126,12 +126,22 @@ is not itself a mount point, the mount point containing
 will be marked.
 All directories, subdirectories, and the contained files of the mount point
 will be monitored.
-This value cannot be used if the
+The events which require the
 .I fanotify_fd
-file descriptor has been initialized with the flag
-.BR FAN_REPORT_FID
-or if any of the new directory modification events are provided as a
-.IR mask .
+file descriptor to have been intialized with the flag
+.BR FAN_REPORT_FID ,
+such as
+.BR FAN_CREATE ,
+.BR FAN_ATTRIB ,
+.BR FAN_MOVE ,
+and
+.BR FAN_DELETE_SELF ,
+cannot be provided as a
+.IR mask
+when
+.I flags
+contains
+.BR FAN_MARK_MOUNT .
 Attempting to do so will result in the error
 .B EINVAL
 being returned.
@@ -184,34 +194,55 @@ See NOTES for additional details.
 .BR FAN_ATTRIB " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when the metadata for a file or directory has changed.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_CREATE " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a file or directory has been created in a marked
 parent directory.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_DELETE " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a file or directory has been deleted in a marked
 parent directory.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_DELETE_SELF " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a marked file or directory itself is deleted.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_MOVED_FROM " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a file or directory has been moved from a marked
 parent directory.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_MOVED_TO " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a file or directory has been moved to a marked parent
 directory.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .BR FAN_MOVE_SELF " (since Linux 5.1)"
 .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
 Create an event when a marked file or directory itself has been moved.
+An fanotify file descriptor created with
+.B FAN_REPORT_FID
+is required.
 .TP
 .B FAN_Q_OVERFLOW
 Create an event when an overflow of the event queue occurs.
-- 
2.17.1

[PATCH 2/4] fanotify_mark.2: Clarification about FAN_EVENT_ON_CHILD and new events

[Amir Goldstein]

Some of the new event types that were added in v5.1 along with init flag
FAN_REPORT_FID are not eligable for reporting to a directory watching
with FAN_EVENT_ON_CHILD.

Document the events that cannot be generated on children of a watching
parent.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
---
 man2/fanotify_mark.2 | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
index 687e20f4a..7fbcd5e36 100644
--- a/man2/fanotify_mark.2
+++ b/man2/fanotify_mark.2
@@ -314,8 +314,21 @@ Events for the immediate children of marked directories shall be created.
 The flag has no effect when marking mounts and filesystems.
 Note that events are not generated for children of the subdirectories
 of marked directories.
+More specifically, the directory entry modification events
+.BR FAN_CREATE ,
+.BR FAN_DELETE ,
+.BR FAN_MOVED_FROM
+and
+.BR FAN_MOVED_TO
+are not generated for any entry modifications performed inside subdirectories
+of marked directories.
+Note that the events
+.BR FAN_DELETE_SELF
+and
+.BR FAN_MOVE_SELF
+are not generated for children of marked directories.
 To monitor complete directory trees it is necessary to mark the relevant
-mount.
+mount or filesystem.
 .PP
 The following composed values are defined:
 .TP
-- 
2.17.1

[PATCH 3/4] fanotify.7: Fix fanotify_fid.c example

[Amir Goldstein]

- The condition for printing "subdirectory created" was always true.
- The arguments and error check of open_by_handle_at() were incorrect.
- Fix example description inconsistencies.
- Nicer indentation of example output.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 man7/fanotify.7 | 38 ++++++++++++++++++++++----------------
 1 file changed, 22 insertions(+), 16 deletions(-)

diff --git a/man7/fanotify.7 b/man7/fanotify.7
index eaf2acf25..72e7e4fb9 100644
--- a/man7/fanotify.7
+++ b/man7/fanotify.7
@@ -938,21 +938,20 @@ This is followed by the creation of a regular file,
 .IR /home/user/testfile.txt .
 This results in a
 .B FAN_CREATE
-event being created and reported against the file's parent watched
+event being generated and reported against the file's parent watched
 directory object.
 Program execution ends once all events captured within the buffer have
 been processed.
-Program execution ends once all events captured within the buffer are
-processed.
 .PP
 .in +4n
 .EX
 # \fB./fanotify_fid /home/user\fP
 Listening for events.
-FAN_CREATE (file created): Directory /home/user has been modified.
+FAN_CREATE (file created):
+        Directory /home/user has been modified.
 All events processed successfully. Program exiting.
 
-$ \fBtouch /home/user/testing\fP              # In another terminal
+$ \fBtouch /home/user/testfile.txt\fP              # In another terminal
 .EE
 .in
 .PP
@@ -960,11 +959,11 @@ The second session shows a mark being placed on
 .IR /home/user .
 This is followed by the creation of a directory,
 .IR /home/user/testdir .
-This specific action results in the program producing a
+This specific action results in a
 .B FAN_CREATE
-and
+event being generated and is reported with the
 .B FAN_ONDIR
-event.
+flag set.
 .PP
 .in +4n
 .EX
@@ -974,7 +973,7 @@ FAN_CREATE | FAN_ONDIR (subdirectory created):
         Directory /home/user has been modified.
 All events processed successfully. Program exiting.
 
-$ \fBmkdir \-p /home/user/testing\fP          # In another terminal
+$ \fBmkdir \-p /home/user/testdir\fP          # In another terminal
 .EE
 .in
 .SS Program source: fanotify_fid.c
@@ -996,7 +995,7 @@ $ \fBmkdir \-p /home/user/testing\fP          # In another terminal
 int
 main(int argc, char **argv)
 {
-    int fd, ret, event_fd;
+    int fd, ret, event_fd, mount_fd;
     ssize_t len, path_len;
     char path[PATH_MAX];
     char procfd_path[PATH_MAX];
@@ -1010,6 +1009,13 @@ main(int argc, char **argv)
         exit(EXIT_FAILURE);
     }
 
+    mount_fd = open(argv[1], O_DIRECTORY | O_RDONLY);
+    if (mount_fd == \-1) {
+        perror(argv[1]);
+        exit(EXIT_FAILURE);
+    }
+
+
     /* Create an fanotify file descriptor with FAN_REPORT_FID as a flag
        so that program can receive fid events. */
 
@@ -1055,10 +1061,10 @@ main(int argc, char **argv)
         }
 
         if (metadata\->mask == FAN_CREATE)
-            printf("FAN_CREATE (file created):");
+            printf("FAN_CREATE (file created):\en");
 
-        if (metadata\->mask == FAN_CREATE | FAN_ONDIR)
-            printf("FAN_CREATE | FAN_ONDIR (subdirectory created):");
+        if (metadata\->mask == (FAN_CREATE | FAN_ONDIR))
+            printf("FAN_CREATE | FAN_ONDIR (subdirectory created):\en");
 
         /* metadata\->fd is set to FAN_NOFD when FAN_REPORT_FID is enabled.
            To obtain a file descriptor for the file object corresponding to
@@ -1068,8 +1074,8 @@ main(int argc, char **argv)
            to accommodate for the situation where the file handle for the
            object was deleted prior to this system call. */
 
-        event_fd = open_by_handle_at(AT_FDCWD, file_handle, O_RDONLY);
-        if (ret == \-1) {
+        event_fd = open_by_handle_at(mount_fd, file_handle, O_RDONLY);
+        if (event_fd == \-1) {
             if (errno == ESTALE) {
                 printf("File handle is no longer valid. "
                         "File has been deleted\en");
@@ -1077,7 +1083,7 @@ main(int argc, char **argv)
             } else {
                 perror("open_by_handle_at");
                 exit(EXIT_FAILURE);
-	    }
+            }
         }
 
         snprintf(procfd_path, sizeof(procfd_path), "/proc/self/fd/%d",
-- 
2.17.1

[PATCH 4/4] fanotify.7, fanotify_mark.2: Document FAN_DIR_MODIFY

[Amir Goldstein]

Document the details of the new FAN_DIR_MODIFY event, which introduces
entry name information to the fanotify event reporting format.

Enhance the fanotify_fid.c example to also report this event.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
Reviewed-by: Jan Kara <jack@suse.cz>
---
 man2/fanotify_mark.2 | 33 ++++++++++++++++++++
 man7/fanotify.7      | 71 +++++++++++++++++++++++++++++++++++++-------
 2 files changed, 94 insertions(+), 10 deletions(-)

diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
index 7fbcd5e36..e94d8a7e1 100644
--- a/man2/fanotify_mark.2
+++ b/man2/fanotify_mark.2
@@ -244,6 +244,38 @@ An fanotify file descriptor created with
 .B FAN_REPORT_FID
 is required.
 .TP
+.BR FAN_DIR_MODIFY " (since Linux 5.7)"
+.\" commit 44d705b0370b1d581f46ff23e5d33e8b5ff8ec58
+Create an event when one of the entries in a marked directory has been created,
+deleted or moved.
+This event will encapsulate additional information about the name of the
+modified directory entry.
+It is intended for applications that want to monitor the content of a large
+directory without having to reiterate the directory each time it is modified.
+The directory entry name will follow the directory file handle information in
+the reported event.
+See
+.BR fanotify (7)
+for additional details about the reported event format.
+The
+.BR FAN_DIR_MODIFY
+event will be generated in addition to the events
+.BR FAN_CREATE ,
+.BR FAN_DELETE ,
+.BR FAN_MOVED_FROM ,
+.BR FAN_MOVED_TO
+and will never be merged with the aforementioned events.
+An application receiving this event must not assume that the reported entry name
+exists in the marked directory at the time the event is received and should call
+.BR fstatat (2)
+with the entry name to learn about the current content of the directory entry.
+See
+.BR fanotify (7)
+for an example of its usage.
+An fanotify file descriptor created with
+.BR FAN_REPORT_FID
+is required.
+.TP
 .B FAN_Q_OVERFLOW
 Create an event when an overflow of the event queue occurs.
 The size of the event queue is limited to 16384 entries if
@@ -315,6 +347,7 @@ The flag has no effect when marking mounts and filesystems.
 Note that events are not generated for children of the subdirectories
 of marked directories.
 More specifically, the directory entry modification events
+.BR FAN_DIR_MODIFY ,
 .BR FAN_CREATE ,
 .BR FAN_DELETE ,
 .BR FAN_MOVED_FROM
diff --git a/man7/fanotify.7 b/man7/fanotify.7
index 72e7e4fb9..cd258abf4 100644
--- a/man7/fanotify.7
+++ b/man7/fanotify.7
@@ -324,6 +324,9 @@ A watched file or directory was moved.
 .B FAN_MODIFY
 A file was modified.
 .TP
+.B FAN_DIR_MODIFY
+A directory entry was created, deleted or moved.
+.TP
 .B FAN_CLOSE_WRITE
 A file that was opened for writing
 .RB ( O_WRONLY
@@ -395,9 +398,11 @@ For example, when an fanotify file descriptor is created using
 the
 .I info_type
 field of this header is set to
-.BR FAN_EVENT_INFO_TYPE_FID .
-Event listeners can use this field to check that the additional
-information received for an event is of the correct type.
+.BR FAN_EVENT_INFO_TYPE_FID
+or
+.BR FAN_EVENT_INFO_TYPE_DFID_NAME .
+Event listeners can use this field to check what additional information
+is received for an event.
 Additionally, the
 .I fanotify_event_info_header
 also contains a
@@ -443,6 +448,14 @@ and
 will carry the
 .IR file_handle
 information for the child object if the child object is being watched.
+For the event
+.BR FAN_DIR_MODIFY ,
+the
+.I info_type
+field of this header is set to
+.BR FAN_EVENT_INFO_TYPE_DFID_NAME .
+The file handle describes the modified directory and a null terminated
+name of the modified entry follows directly after the file handle buffer.
 .PP
 The following macros are provided to iterate over a buffer containing
 fanotify event metadata returned by a
@@ -627,9 +640,11 @@ events for the monitored directory itself.
 Fanotify monitoring of directories is not recursive:
 to monitor subdirectories under a directory,
 additional marks must be created.
-(But note that the fanotify API provides no way of detecting when a
-subdirectory has been created under a marked directory,
-which makes recursive monitoring difficult.)
+The
+.B FAN_DIR_MODIFY
+event can be used for detecting when a subdirectory has been created under
+a marked directory.
+An additional mark must then be set on the newly created subdirectory.
 Monitoring mounts offers the capability to monitor a whole directory tree.
 Monitoring filesystems offers the capability to monitor changes made from
 any mount of a filesystem instance.
@@ -923,6 +938,8 @@ The program marks the filesystem object that is passed as
 a command-line argument
 and waits until an event of type
 .B FAN_CREATE
+or
+.B FAN_DIR_MODIFY
 has occurred.
 The event mask indicates which type of filesystem object\(emeither
 a file or a directory\(emwas created.
@@ -939,7 +956,9 @@ This is followed by the creation of a regular file,
 This results in a
 .B FAN_CREATE
 event being generated and reported against the file's parent watched
-directory object.
+directory object and a
+.B FAN_DIR_MODIFY
+event being generated and reported with the created file name.
 Program execution ends once all events captured within the buffer have
 been processed.
 .PP
@@ -949,6 +968,9 @@ been processed.
 Listening for events.
 FAN_CREATE (file created):
         Directory /home/user has been modified.
+FAN_DIR_MODIFY (entry changed):
+        Directory /home/user has been modified.
+        Entry 'testfile.txt' is not a subdirectory.
 All events processed successfully. Program exiting.
 
 $ \fBtouch /home/user/testfile.txt\fP              # In another terminal
@@ -963,7 +985,9 @@ This specific action results in a
 .B FAN_CREATE
 event being generated and is reported with the
 .B FAN_ONDIR
-flag set.
+flag set and a
+.B FAN_DIR_MODIFY
+event being generated and reported with the created directory name.
 .PP
 .in +4n
 .EX
@@ -971,6 +995,9 @@ flag set.
 Listening for events.
 FAN_CREATE | FAN_ONDIR (subdirectory created):
         Directory /home/user has been modified.
+FAN_DIR_MODIFY (entry changed):
+        Directory /home/user has been modified.
+        Entry 'testdir' is a subdirectory.
 All events processed successfully. Program exiting.
 
 $ \fBmkdir \-p /home/user/testdir\fP          # In another terminal
@@ -1003,6 +1030,8 @@ main(int argc, char **argv)
     struct file_handle *file_handle;
     struct fanotify_event_metadata *metadata;
     struct fanotify_event_info_fid *fid;
+    const char *file_name;
+    struct stat sb;
 
     if (argc != 2) {
         fprintf(stderr, "Invalid number of command line arguments.\en");
@@ -1028,7 +1057,7 @@ main(int argc, char **argv)
     /* Place a mark on the filesystem object supplied in argv[1]. */
 
     ret = fanotify_mark(fd, FAN_MARK_ADD | FAN_MARK_ONLYDIR,
-                        FAN_CREATE | FAN_ONDIR,
+                        FAN_DIR_MODIFY | FAN_CREATE | FAN_ONDIR,
                         AT_FDCWD, argv[1]);
     if (ret == \-1) {
         perror("fanotify_mark");
@@ -1055,11 +1084,18 @@ main(int argc, char **argv)
 
         /* Ensure that the event info is of the correct type */
 
-        if (fid\->hdr.info_type != FAN_EVENT_INFO_TYPE_FID) {
+        if (fid\->hdr.info_type == FAN_EVENT_INFO_TYPE_FID) {
+            file_name = NULL;
+        } else if (fid\->hdr.info_type == FAN_EVENT_INFO_TYPE_DFID_NAME) {
+            file_name = file_handle->f_handle + file_handle->handle_bytes;
+        } else {
             fprintf(stderr, "Received unexpected event info type.\en");
             exit(EXIT_FAILURE);
         }
 
+        if (metadata\->mask == FAN_DIR_MODIFY)
+            printf("FAN_DIR_MODIFY (entry changed):\en");
+
         if (metadata\->mask == FAN_CREATE)
             printf("FAN_CREATE (file created):\en");
 
@@ -1100,6 +1136,21 @@ main(int argc, char **argv)
         path[path_len] = \(aq\e0\(aq;
         printf("\etDirectory \(aq%s\(aq has been modified.\en", path);
 
+        if (file_name) {
+            ret = fstatat(event_fd, file_name, &sb, 0);
+            if (ret == \-1) {
+                if (errno != ENOENT) {
+                    perror("fstatat");
+                    exit(EXIT_FAILURE);
+                }
+                printf("\etEntry %\(aqs\(aq does not exist.\en", file_name);
+            } else if ((sb.st_mode & S_IFMT) == S_IFDIR) {
+                printf("\etEntry \(aq%s\(aq is a subdirectory.\en", file_name);
+            } else {
+                printf("\etEntry \(aq%s\(aq is not a subdirectory.\en", file_name);
+            }
+        }
+
         /* Close associated file descriptor for this event */
 
         close(event_fd);
-- 
2.17.1

Re: [PATCH 1/4] fanotify_mark.2: Clarification about FAN_MARK_MOUNT and FAN_REPORT_FID

[Michael Kerrisk (man-pages)]

Hello Amir,

On 4/20/20 8:42 PM, Amir Goldstein wrote:
> It is not true that FAN_MARK_MOUNT cannot be used with a group that
> was initialized with flag FAN_REPORT_FID.
> 
> The correct assertion is that events that require a group with flag
> FAN_REPORT_FID cannot be requested on a mark mount.
> 
> For exaple, a FAN_OPEN event can be requested on a mark mount and
> will generate an event with file handle information if the group was
> initialized with flag FAN_REPORT_FID.
> 
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> Reviewed-by: Jan Kara <jack@suse.cz>
> Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>

Thanks. Patch applied.

@Jan, @Matthew: thanks for the reviews!

Cheers,

Michael

> ---
>  man2/fanotify_mark.2 | 41 ++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 36 insertions(+), 5 deletions(-)
> 
> diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
> index 30290d519..687e20f4a 100644
> --- a/man2/fanotify_mark.2
> +++ b/man2/fanotify_mark.2
> @@ -126,12 +126,22 @@ is not itself a mount point, the mount point containing
>  will be marked.
>  All directories, subdirectories, and the contained files of the mount point
>  will be monitored.
> -This value cannot be used if the
> +The events which require the
>  .I fanotify_fd
> -file descriptor has been initialized with the flag
> -.BR FAN_REPORT_FID
> -or if any of the new directory modification events are provided as a
> -.IR mask .
> +file descriptor to have been intialized with the flag
> +.BR FAN_REPORT_FID ,
> +such as
> +.BR FAN_CREATE ,
> +.BR FAN_ATTRIB ,
> +.BR FAN_MOVE ,
> +and
> +.BR FAN_DELETE_SELF ,
> +cannot be provided as a
> +.IR mask
> +when
> +.I flags
> +contains
> +.BR FAN_MARK_MOUNT .
>  Attempting to do so will result in the error
>  .B EINVAL
>  being returned.
> @@ -184,34 +194,55 @@ See NOTES for additional details.
>  .BR FAN_ATTRIB " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when the metadata for a file or directory has changed.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_CREATE " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a file or directory has been created in a marked
>  parent directory.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_DELETE " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a file or directory has been deleted in a marked
>  parent directory.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_DELETE_SELF " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a marked file or directory itself is deleted.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_MOVED_FROM " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a file or directory has been moved from a marked
>  parent directory.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_MOVED_TO " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a file or directory has been moved to a marked parent
>  directory.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .BR FAN_MOVE_SELF " (since Linux 5.1)"
>  .\" commit 235328d1fa4251c6dcb32351219bb553a58838d2
>  Create an event when a marked file or directory itself has been moved.
> +An fanotify file descriptor created with
> +.B FAN_REPORT_FID
> +is required.
>  .TP
>  .B FAN_Q_OVERFLOW
>  Create an event when an overflow of the event queue occurs.
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 2/4] fanotify_mark.2: Clarification about FAN_EVENT_ON_CHILD and new events

[Michael Kerrisk (man-pages)]

Hello Amir,

On 4/20/20 8:42 PM, Amir Goldstein wrote:
> Some of the new event types that were added in v5.1 along with init flag
> FAN_REPORT_FID are not eligable for reporting to a directory watching
> with FAN_EVENT_ON_CHILD.
> 
> Document the events that cannot be generated on children of a watching
> parent.
> 
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> Reviewed-by: Jan Kara <jack@suse.cz>
> Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>

Thanks. Patch applied.

Cheers,

Michael

> ---
>  man2/fanotify_mark.2 | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
> index 687e20f4a..7fbcd5e36 100644
> --- a/man2/fanotify_mark.2
> +++ b/man2/fanotify_mark.2
> @@ -314,8 +314,21 @@ Events for the immediate children of marked directories shall be created.
>  The flag has no effect when marking mounts and filesystems.
>  Note that events are not generated for children of the subdirectories
>  of marked directories.
> +More specifically, the directory entry modification events
> +.BR FAN_CREATE ,
> +.BR FAN_DELETE ,
> +.BR FAN_MOVED_FROM
> +and
> +.BR FAN_MOVED_TO
> +are not generated for any entry modifications performed inside subdirectories
> +of marked directories.
> +Note that the events
> +.BR FAN_DELETE_SELF
> +and
> +.BR FAN_MOVE_SELF
> +are not generated for children of marked directories.
>  To monitor complete directory trees it is necessary to mark the relevant
> -mount.
> +mount or filesystem.
>  .PP
>  The following composed values are defined:
>  .TP
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 3/4] fanotify.7: Fix fanotify_fid.c example

[Michael Kerrisk (man-pages)]

On 4/20/20 8:42 PM, Amir Goldstein wrote:
> - The condition for printing "subdirectory created" was always true.
> - The arguments and error check of open_by_handle_at() were incorrect.
> - Fix example description inconsistencies.
> - Nicer indentation of example output.

Thanks, Amir. Patch applied.

Cheers,

Michael

> 
> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> Reviewed-by: Jan Kara <jack@suse.cz>
> ---
>  man7/fanotify.7 | 38 ++++++++++++++++++++++----------------
>  1 file changed, 22 insertions(+), 16 deletions(-)
> 
> diff --git a/man7/fanotify.7 b/man7/fanotify.7
> index eaf2acf25..72e7e4fb9 100644
> --- a/man7/fanotify.7
> +++ b/man7/fanotify.7
> @@ -938,21 +938,20 @@ This is followed by the creation of a regular file,
>  .IR /home/user/testfile.txt .
>  This results in a
>  .B FAN_CREATE
> -event being created and reported against the file's parent watched
> +event being generated and reported against the file's parent watched
>  directory object.
>  Program execution ends once all events captured within the buffer have
>  been processed.
> -Program execution ends once all events captured within the buffer are
> -processed.
>  .PP
>  .in +4n
>  .EX
>  # \fB./fanotify_fid /home/user\fP
>  Listening for events.
> -FAN_CREATE (file created): Directory /home/user has been modified.
> +FAN_CREATE (file created):
> +        Directory /home/user has been modified.
>  All events processed successfully. Program exiting.
>  
> -$ \fBtouch /home/user/testing\fP              # In another terminal
> +$ \fBtouch /home/user/testfile.txt\fP              # In another terminal
>  .EE
>  .in
>  .PP
> @@ -960,11 +959,11 @@ The second session shows a mark being placed on
>  .IR /home/user .
>  This is followed by the creation of a directory,
>  .IR /home/user/testdir .
> -This specific action results in the program producing a
> +This specific action results in a
>  .B FAN_CREATE
> -and
> +event being generated and is reported with the
>  .B FAN_ONDIR
> -event.
> +flag set.
>  .PP
>  .in +4n
>  .EX
> @@ -974,7 +973,7 @@ FAN_CREATE | FAN_ONDIR (subdirectory created):
>          Directory /home/user has been modified.
>  All events processed successfully. Program exiting.
>  
> -$ \fBmkdir \-p /home/user/testing\fP          # In another terminal
> +$ \fBmkdir \-p /home/user/testdir\fP          # In another terminal
>  .EE
>  .in
>  .SS Program source: fanotify_fid.c
> @@ -996,7 +995,7 @@ $ \fBmkdir \-p /home/user/testing\fP          # In another terminal
>  int
>  main(int argc, char **argv)
>  {
> -    int fd, ret, event_fd;
> +    int fd, ret, event_fd, mount_fd;
>      ssize_t len, path_len;
>      char path[PATH_MAX];
>      char procfd_path[PATH_MAX];
> @@ -1010,6 +1009,13 @@ main(int argc, char **argv)
>          exit(EXIT_FAILURE);
>      }
>  
> +    mount_fd = open(argv[1], O_DIRECTORY | O_RDONLY);
> +    if (mount_fd == \-1) {
> +        perror(argv[1]);
> +        exit(EXIT_FAILURE);
> +    }
> +
> +
>      /* Create an fanotify file descriptor with FAN_REPORT_FID as a flag
>         so that program can receive fid events. */
>  
> @@ -1055,10 +1061,10 @@ main(int argc, char **argv)
>          }
>  
>          if (metadata\->mask == FAN_CREATE)
> -            printf("FAN_CREATE (file created):");
> +            printf("FAN_CREATE (file created):\en");
>  
> -        if (metadata\->mask == FAN_CREATE | FAN_ONDIR)
> -            printf("FAN_CREATE | FAN_ONDIR (subdirectory created):");
> +        if (metadata\->mask == (FAN_CREATE | FAN_ONDIR))
> +            printf("FAN_CREATE | FAN_ONDIR (subdirectory created):\en");
>  
>          /* metadata\->fd is set to FAN_NOFD when FAN_REPORT_FID is enabled.
>             To obtain a file descriptor for the file object corresponding to
> @@ -1068,8 +1074,8 @@ main(int argc, char **argv)
>             to accommodate for the situation where the file handle for the
>             object was deleted prior to this system call. */
>  
> -        event_fd = open_by_handle_at(AT_FDCWD, file_handle, O_RDONLY);
> -        if (ret == \-1) {
> +        event_fd = open_by_handle_at(mount_fd, file_handle, O_RDONLY);
> +        if (event_fd == \-1) {
>              if (errno == ESTALE) {
>                  printf("File handle is no longer valid. "
>                          "File has been deleted\en");
> @@ -1077,7 +1083,7 @@ main(int argc, char **argv)
>              } else {
>                  perror("open_by_handle_at");
>                  exit(EXIT_FAILURE);
> -	    }
> +            }
>          }
>  
>          snprintf(procfd_path, sizeof(procfd_path), "/proc/self/fd/%d",
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 4/4] fanotify.7, fanotify_mark.2: Document FAN_DIR_MODIFY

[Michael Kerrisk (man-pages)]

Hello Amir,

On 4/20/20 8:42 PM, Amir Goldstein wrote:
> Document the details of the new FAN_DIR_MODIFY event, which introduces
> entry name information to the fanotify event reporting format.
> 
> Enhance the fanotify_fid.c example to also report this event.

Thanks. patch applied.

Cheers,

Michael

> Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> Reviewed-by: Matthew Bobrowski <mbobrowski@mbobrowski.org>
> Reviewed-by: Jan Kara <jack@suse.cz>
> ---
>  man2/fanotify_mark.2 | 33 ++++++++++++++++++++
>  man7/fanotify.7      | 71 +++++++++++++++++++++++++++++++++++++-------
>  2 files changed, 94 insertions(+), 10 deletions(-)
> 
> diff --git a/man2/fanotify_mark.2 b/man2/fanotify_mark.2
> index 7fbcd5e36..e94d8a7e1 100644
> --- a/man2/fanotify_mark.2
> +++ b/man2/fanotify_mark.2
> @@ -244,6 +244,38 @@ An fanotify file descriptor created with
>  .B FAN_REPORT_FID
>  is required.
>  .TP
> +.BR FAN_DIR_MODIFY " (since Linux 5.7)"
> +.\" commit 44d705b0370b1d581f46ff23e5d33e8b5ff8ec58
> +Create an event when one of the entries in a marked directory has been created,
> +deleted or moved.
> +This event will encapsulate additional information about the name of the
> +modified directory entry.
> +It is intended for applications that want to monitor the content of a large
> +directory without having to reiterate the directory each time it is modified.
> +The directory entry name will follow the directory file handle information in
> +the reported event.
> +See
> +.BR fanotify (7)
> +for additional details about the reported event format.
> +The
> +.BR FAN_DIR_MODIFY
> +event will be generated in addition to the events
> +.BR FAN_CREATE ,
> +.BR FAN_DELETE ,
> +.BR FAN_MOVED_FROM ,
> +.BR FAN_MOVED_TO
> +and will never be merged with the aforementioned events.
> +An application receiving this event must not assume that the reported entry name
> +exists in the marked directory at the time the event is received and should call
> +.BR fstatat (2)
> +with the entry name to learn about the current content of the directory entry.
> +See
> +.BR fanotify (7)
> +for an example of its usage.
> +An fanotify file descriptor created with
> +.BR FAN_REPORT_FID
> +is required.
> +.TP
>  .B FAN_Q_OVERFLOW
>  Create an event when an overflow of the event queue occurs.
>  The size of the event queue is limited to 16384 entries if
> @@ -315,6 +347,7 @@ The flag has no effect when marking mounts and filesystems.
>  Note that events are not generated for children of the subdirectories
>  of marked directories.
>  More specifically, the directory entry modification events
> +.BR FAN_DIR_MODIFY ,
>  .BR FAN_CREATE ,
>  .BR FAN_DELETE ,
>  .BR FAN_MOVED_FROM
> diff --git a/man7/fanotify.7 b/man7/fanotify.7
> index 72e7e4fb9..cd258abf4 100644
> --- a/man7/fanotify.7
> +++ b/man7/fanotify.7
> @@ -324,6 +324,9 @@ A watched file or directory was moved.
>  .B FAN_MODIFY
>  A file was modified.
>  .TP
> +.B FAN_DIR_MODIFY
> +A directory entry was created, deleted or moved.
> +.TP
>  .B FAN_CLOSE_WRITE
>  A file that was opened for writing
>  .RB ( O_WRONLY
> @@ -395,9 +398,11 @@ For example, when an fanotify file descriptor is created using
>  the
>  .I info_type
>  field of this header is set to
> -.BR FAN_EVENT_INFO_TYPE_FID .
> -Event listeners can use this field to check that the additional
> -information received for an event is of the correct type.
> +.BR FAN_EVENT_INFO_TYPE_FID
> +or
> +.BR FAN_EVENT_INFO_TYPE_DFID_NAME .
> +Event listeners can use this field to check what additional information
> +is received for an event.
>  Additionally, the
>  .I fanotify_event_info_header
>  also contains a
> @@ -443,6 +448,14 @@ and
>  will carry the
>  .IR file_handle
>  information for the child object if the child object is being watched.
> +For the event
> +.BR FAN_DIR_MODIFY ,
> +the
> +.I info_type
> +field of this header is set to
> +.BR FAN_EVENT_INFO_TYPE_DFID_NAME .
> +The file handle describes the modified directory and a null terminated
> +name of the modified entry follows directly after the file handle buffer.
>  .PP
>  The following macros are provided to iterate over a buffer containing
>  fanotify event metadata returned by a
> @@ -627,9 +640,11 @@ events for the monitored directory itself.
>  Fanotify monitoring of directories is not recursive:
>  to monitor subdirectories under a directory,
>  additional marks must be created.
> -(But note that the fanotify API provides no way of detecting when a
> -subdirectory has been created under a marked directory,
> -which makes recursive monitoring difficult.)
> +The
> +.B FAN_DIR_MODIFY
> +event can be used for detecting when a subdirectory has been created under
> +a marked directory.
> +An additional mark must then be set on the newly created subdirectory.
>  Monitoring mounts offers the capability to monitor a whole directory tree.
>  Monitoring filesystems offers the capability to monitor changes made from
>  any mount of a filesystem instance.
> @@ -923,6 +938,8 @@ The program marks the filesystem object that is passed as
>  a command-line argument
>  and waits until an event of type
>  .B FAN_CREATE
> +or
> +.B FAN_DIR_MODIFY
>  has occurred.
>  The event mask indicates which type of filesystem object\(emeither
>  a file or a directory\(emwas created.
> @@ -939,7 +956,9 @@ This is followed by the creation of a regular file,
>  This results in a
>  .B FAN_CREATE
>  event being generated and reported against the file's parent watched
> -directory object.
> +directory object and a
> +.B FAN_DIR_MODIFY
> +event being generated and reported with the created file name.
>  Program execution ends once all events captured within the buffer have
>  been processed.
>  .PP
> @@ -949,6 +968,9 @@ been processed.
>  Listening for events.
>  FAN_CREATE (file created):
>          Directory /home/user has been modified.
> +FAN_DIR_MODIFY (entry changed):
> +        Directory /home/user has been modified.
> +        Entry 'testfile.txt' is not a subdirectory.
>  All events processed successfully. Program exiting.
>  
>  $ \fBtouch /home/user/testfile.txt\fP              # In another terminal
> @@ -963,7 +985,9 @@ This specific action results in a
>  .B FAN_CREATE
>  event being generated and is reported with the
>  .B FAN_ONDIR
> -flag set.
> +flag set and a
> +.B FAN_DIR_MODIFY
> +event being generated and reported with the created directory name.
>  .PP
>  .in +4n
>  .EX
> @@ -971,6 +995,9 @@ flag set.
>  Listening for events.
>  FAN_CREATE | FAN_ONDIR (subdirectory created):
>          Directory /home/user has been modified.
> +FAN_DIR_MODIFY (entry changed):
> +        Directory /home/user has been modified.
> +        Entry 'testdir' is a subdirectory.
>  All events processed successfully. Program exiting.
>  
>  $ \fBmkdir \-p /home/user/testdir\fP          # In another terminal
> @@ -1003,6 +1030,8 @@ main(int argc, char **argv)
>      struct file_handle *file_handle;
>      struct fanotify_event_metadata *metadata;
>      struct fanotify_event_info_fid *fid;
> +    const char *file_name;
> +    struct stat sb;
>  
>      if (argc != 2) {
>          fprintf(stderr, "Invalid number of command line arguments.\en");
> @@ -1028,7 +1057,7 @@ main(int argc, char **argv)
>      /* Place a mark on the filesystem object supplied in argv[1]. */
>  
>      ret = fanotify_mark(fd, FAN_MARK_ADD | FAN_MARK_ONLYDIR,
> -                        FAN_CREATE | FAN_ONDIR,
> +                        FAN_DIR_MODIFY | FAN_CREATE | FAN_ONDIR,
>                          AT_FDCWD, argv[1]);
>      if (ret == \-1) {
>          perror("fanotify_mark");
> @@ -1055,11 +1084,18 @@ main(int argc, char **argv)
>  
>          /* Ensure that the event info is of the correct type */
>  
> -        if (fid\->hdr.info_type != FAN_EVENT_INFO_TYPE_FID) {
> +        if (fid\->hdr.info_type == FAN_EVENT_INFO_TYPE_FID) {
> +            file_name = NULL;
> +        } else if (fid\->hdr.info_type == FAN_EVENT_INFO_TYPE_DFID_NAME) {
> +            file_name = file_handle->f_handle + file_handle->handle_bytes;
> +        } else {
>              fprintf(stderr, "Received unexpected event info type.\en");
>              exit(EXIT_FAILURE);
>          }
>  
> +        if (metadata\->mask == FAN_DIR_MODIFY)
> +            printf("FAN_DIR_MODIFY (entry changed):\en");
> +
>          if (metadata\->mask == FAN_CREATE)
>              printf("FAN_CREATE (file created):\en");
>  
> @@ -1100,6 +1136,21 @@ main(int argc, char **argv)
>          path[path_len] = \(aq\e0\(aq;
>          printf("\etDirectory \(aq%s\(aq has been modified.\en", path);
>  
> +        if (file_name) {
> +            ret = fstatat(event_fd, file_name, &sb, 0);
> +            if (ret == \-1) {
> +                if (errno != ENOENT) {
> +                    perror("fstatat");
> +                    exit(EXIT_FAILURE);
> +                }
> +                printf("\etEntry %\(aqs\(aq does not exist.\en", file_name);
> +            } else if ((sb.st_mode & S_IFMT) == S_IFDIR) {
> +                printf("\etEntry \(aq%s\(aq is a subdirectory.\en", file_name);
> +            } else {
> +                printf("\etEntry \(aq%s\(aq is not a subdirectory.\en", file_name);
> +            }
> +        }
> +
>          /* Close associated file descriptor for this event */
>  
>          close(event_fd);
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 0/4] fanotify updates for v5.7

[Michael Kerrisk (man-pages)]

Hello Amir,

On 4/20/20 8:42 PM, Amir Goldstein wrote:
> Hi Michael,
> 
> Following patches include fixes to earlier FAN_REPORT_FID change from
> v5.1 and an update for the new event FAN_DIR_MODIFY that has a new
> event format.
> 
> Those patches were reviewed by Jan and Matthew earlier on github.
> 
> Thanks,
> Amir.
> 
> Amir Goldstein (4):
>   fanotify_mark.2: Clarification about FAN_MARK_MOUNT and FAN_REPORT_FID
>   fanotify_mark.2: Clarification about FAN_EVENT_ON_CHILD and new events
>   fanotify.7: Fix fanotify_fid.c example
>   fanotify.7, fanotify_mark.2: Document FAN_DIR_MODIFY
> 
>  man2/fanotify_mark.2 |  89 ++++++++++++++++++++++++++++++++---
>  man7/fanotify.7      | 107 +++++++++++++++++++++++++++++++++----------
>  2 files changed, 165 insertions(+), 31 deletions(-)

Thanks for the nicely prepared patches! There was very little for 
me to edit.

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/