[PATCH v3 00/13] fanotify: add support for more event types

[Amir Goldstein <amir73il@gmail.com>]

Jan,

This is the 3rd revision of patch series to add support for filesystem
change monitoring to fanotify.
It incorporates the changes you requested in review of v2 patches.
The complete work is available on fanotify_dirent branch [1] on my tree.

The end game is to use:
  fd = fanotify_init(FAN_CLASS_NOTIF|FAN_REPORT_FID, ...);
  rc = fanotify_mark(fd, FAN_MARK_FILESYSTEM, FAN_CREATE|FAN_DELETE...);
to monitor changes to a large scale namespace.

This functionality was not available with inotify API, which does not
scale well with recursive directory watches and was not available
with fanotify API, which did not support directory modification events.

I have tested this work with some preliminary LTP tests [2] and with
a prototype of global filesystem monitor based on inotify-tools [3].
Please see below a demo output [4] from filesystem monitor prototype.
Note that the "watches" in the prototype are userland entries to
map fid to path. The kernel has but one mark per super block.

Matthew Bobrowski has agreed to help me with writing more tests and
man pages (thanks Matthew!).

Thanks,
Amir.

Changes since v2:
- Discard FSNOTIFY_EVENT_DENTRY data type changes
- Cache fsid in connector instead of calling vfs_statfs() on every event
- Deny setting fid watch on filesystem with no fsid (tmpfs)
- Deny setting fid watch on filesystem with non root fsid (btrfs subvol)
- Report FAN_ONDIR for all event types with FAN_REPORT_FID

[1] https://github.com/amir73il/linux/commits/fanotify_dirent
[2] https://github.com/amir73il/ltp/commits/fanotify_dirent
[3] https://github.com/amir73il/inotify-tools/commits/fanotify_dirent
[4] Demo run of inotifywait monitor on 2 filesystems and 1 subvolume:
=================
...
/dev/vdf on /vdf type xfs (rw,attr2,inode64,noquota)
/dev/vde on /mnt type btrfs (rw,space_cache,subvolid=257,subvol=/subvol)
/dev/vde on /vde type btrfs (rw,space_cache,subvolid=5,subvol=/)

root@kvm-xfstests:~# inotifywait -m -g /vde /vdf
Setting up global filesystem watches.
Watches established.

/vde/ OPEN,ISDIR
/vdf/ OPEN,ISDIR

root@kvm-xfstests:~# mkdir -p /mnt/a/b/c/d/e/ && touch /mnt/a/b/c/d/e/x
...Start watching /vde/subvol (fid=f74e7a26.a635b2c5.100...)
/vde/subvol CREATE,ISDIR
...Start watching /vde/subvol/a (fid=f74e7a26.a635b2c5.101...)
/vde/subvol/a CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR
/vde/subvol CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
...Start watching /vde/subvol/a/b (fid=f74e7a26.a635b2c5.102...)
/vde/subvol/a/b CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a/b CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a/b CREATE,ISDIR
...Start watching /vde/subvol/a/b/c (fid=f74e7a26.a635b2c5.103...)
/vde/subvol/a/b/c CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a/b/c CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a/b/c CREATE,ISDIR
...Start watching /vde/subvol/a/b/c/d (fid=f74e7a26.a635b2c5.104...)
/vde/subvol/a/b/c/d CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR
/vde/subvol/a/b/c/d CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
...Start watching /vde/subvol/a/b/c/d/e (fid=f74e7a26.a635b2c5.105...)
/vde/subvol/a/b/c/d/e CREATE
...Start watching /vde/subvol/a/b/c/d/e/x (fid=f74e7a26.a635b2c5.106...)
/vde/subvol/a/b/c/d/e/x OPEN
/vde/subvol/a/b/c/d/e CLOSE_NOWRITE,OPEN,CLOSE,ISDIR
/vde/subvol/a/b/c/d/e/x CLOSE_NOWRITE,OPEN,CLOSE
/vde/subvol/a/b/c/d/e/x ATTRIB
/vde/subvol/a/b/c/d/e/x CLOSE_WRITE,CLOSE

root@kvm-xfstests:~# touch /vde/a /vdf/a /mnt/a
/vde/ CREATE
...Start watching /vde/a (fid=f74e7a26.a635b2c5.101...)
/vde/a OPEN
/vde/a ATTRIB
/vde/a CLOSE_NOWRITE,OPEN,CLOSE
/vde/a CLOSE_WRITE,CLOSE
/vdf/ CREATE
...Start watching /vdf/a (fid=fd50.0.105...)
/vdf/a OPEN
/vdf/a CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a ATTRIB,CLOSE_WRITE,CLOSE
/vde/subvol/a ATTRIB,ISDIR

root@kvm-xfstests:~# rm -rf /vde/a /vdf/a /mnt/a
/vde/a ATTRIB,DELETE_SELF
/vde/ DELETE
/vdf/a ATTRIB,DELETE_SELF
/vdf/ DELETE
/vde/subvol/a OPEN,ISDIR
/vde/subvol/a ACCESS,ISDIR
/vde/subvol/a CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a OPEN,ISDIR
/vde/subvol/a ACCESS,ISDIR
/vde/subvol/a ACCESS,ISDIR
/vde/subvol/a/b OPEN,ISDIR
/vde/subvol/a/b ACCESS,ISDIR
/vde/subvol/a/b CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b OPEN,ISDIR
/vde/subvol/a/b ACCESS,ISDIR
/vde/subvol/a/b ACCESS,ISDIR
/vde/subvol/a/b/c ACCESS,OPEN,ISDIR
/vde/subvol/a/b/c CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c OPEN,ISDIR
/vde/subvol/a/b/c ACCESS,ISDIR
/vde/subvol/a/b/c/d OPEN,ISDIR
/vde/subvol/a/b/c/d ACCESS,ISDIR
/vde/subvol/a/b/c/d CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c/d OPEN,ISDIR
/vde/subvol/a/b/c/d ACCESS,ISDIR
/vde/subvol/a/b/c/d ACCESS,ISDIR
/vde/subvol/a/b/c/d/e OPEN,ISDIR
/vde/subvol/a/b/c/d/e ACCESS,ISDIR
/vde/subvol/a/b/c/d/e CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c/d/e OPEN,ISDIR
/vde/subvol/a/b/c/d/e ACCESS,ISDIR
/vde/subvol/a/b/c/d/e ACCESS,ISDIR
/vde/subvol/a/b/c/d/e/x ATTRIB,DELETE_SELF
/vde/subvol/a/b/c/d/e DELETE
/vde/subvol/a/b/c/d/e CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c/d/e DELETE_SELF
/vde/subvol/a/b/c/d DELETE,ISDIR
/vde/subvol/a/b/c/d CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c/d DELETE_SELF
/vde/subvol/a/b/c DELETE,ISDIR
/vde/subvol/a/b/c CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b/c DELETE_SELF
/vde/subvol/a/b DELETE,ISDIR
/vde/subvol/a/b CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a/b DELETE_SELF
/vde/subvol/a DELETE,ISDIR
/vde/subvol/a CLOSE_NOWRITE,CLOSE,ISDIR
/vde/subvol/a DELETE_SELF
/vde/subvol DELETE,ISDIR
=====================

Amir Goldstein (13):
  fsnotify: annotate directory entry modification events
  fsnotify: send all event types to super block marks
  fanotify: rename struct fanotify_{,perm_}event_info
  fanotify: define the structures to report a unique file identifier
  fanotify: classify events that hold a file identifier
  fanotify: encode file identifier for FAN_REPORT_FID
  fanotify: copy event fid info to user
  fanotify: enable FAN_REPORT_FID init flag
  fanotify: cache fsid in fsnotify_mark_connector
  fanotify: check FS_ISDIR flag instead of d_is_dir()
  fanotify: support events with data type FSNOTIFY_EVENT_INODE
  fanotify: add support for create/attrib/move/delete events
  fanotify: report FAN_ONDIR to listener with FAN_REPORT_FID

 fs/notify/fanotify/fanotify.c      | 214 ++++++++++++++++++++++++-----
 fs/notify/fanotify/fanotify.h      |  77 +++++++++--
 fs/notify/fanotify/fanotify_user.c | 213 +++++++++++++++++++++++-----
 fs/notify/fsnotify.c               |  15 +-
 fs/notify/mark.c                   |  77 +++++++++--
 fs/statfs.c                        |   4 +-
 include/linux/fanotify.h           |  33 ++++-
 include/linux/fsnotify.h           |  46 +++++--
 include/linux/fsnotify_backend.h   |  60 +++++---
 include/linux/statfs.h             |   3 +
 include/uapi/linux/fanotify.h      |  46 ++++++-
 11 files changed, 642 insertions(+), 146 deletions(-)

--
2.17.1