linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/9] fanotify: add support for more event types
@ 2018-11-15 18:45 Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 1/9] fanotify: rename struct fanotify_{,perm_}event_info Amir Goldstein
                   ` (9 more replies)
  0 siblings, 10 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

Jan,

This is the final part of patch series to add support for filesystem
change monitoring to fanotify.

The end game is to use:
  fd = fanotify_init(FAN_CLASS_NOTIF|FAN_REPORT_FID, ...);
  rc = fanotify_mark(fd, FAN_MARK_FILESYSTEM, FAN_CREATE|FAN_DELETE...);
to monitor changes to a large scale namespace.

This functionality was not available with inotify API, which does not
scale well with recursive directory watches and was not available
with fanotify API, which did not support directory modification events.

This patch set depends on the fsnotify prep patches posted yesterday.
The entire work based on your fsnotify branch is available on my
fanotify_dentry branch [1].

I have tested this work with some preliminary LTP tests [2] and a demo
program [3]. Matthew Bobrowski has agreed to help me with writing more
tests and man pages (thanks Matthew!).

Last patch raises a question about how to deal with FAN_ONDIR flag with
new event types and proposed a minimal viable implementation that could
be used as a base for further discussion.

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/fanotify_dentry
[2] https://github.com/amir73il/ltp/commits/fanotify_dentry
[3] https://github.com/amir73il/fsnotify-utils/blob/master/src/test/fanotify_demo.c

Amir Goldstein (9):
  fanotify: rename struct fanotify_{,perm_}event_info
  fanotify: define the structures to report a unique file identifier
  fanotify: classify events that hold a file identifier
  fanotify: encode file identifier for FAN_REPORT_FID
  fanotify: copy event fid info to user
  fanotify: enable FAN_REPORT_FID init flag
  fanotify: support events with data type FSNOTIFY_EVENT_DENTRY
  fanotify: add support for create/attrib/move/delete events
  fanotify: report FAN_ONDIR to listener for filename events

 fs/notify/fanotify/fanotify.c      | 199 ++++++++++++++++++++++++-----
 fs/notify/fanotify/fanotify.h      |  76 ++++++++---
 fs/notify/fanotify/fanotify_user.c | 131 ++++++++++++++++---
 fs/statfs.c                        |   3 +-
 include/linux/fanotify.h           |  31 ++++-
 include/linux/statfs.h             |   3 +
 include/uapi/linux/fanotify.h      |  45 ++++++-
 7 files changed, 408 insertions(+), 80 deletions(-)

-- 
2.17.1

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH v2 1/9] fanotify: rename struct fanotify_{,perm_}event_info
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 2/9] fanotify: define the structures to report a unique file identifier Amir Goldstein
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

struct fanotify_event_info "inherits" from struct fsnotify_event and
therefore a more appropriate (and short) name for it is fanotify_event.
Same for struct fanotify_perm_event_info, which now "inherits" from
struct fanotify_event.

We plan to reuse the name struct fanotify_event_info for user visible
event info record format.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c      | 14 +++++++-------
 fs/notify/fanotify/fanotify.h      | 16 ++++++++--------
 fs/notify/fanotify/fanotify_user.c | 20 ++++++++++----------
 3 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 3723f3d18d20..ecd5f4aec624 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -19,7 +19,7 @@
 static bool should_merge(struct fsnotify_event *old_fsn,
 			 struct fsnotify_event *new_fsn)
 {
-	struct fanotify_event_info *old, *new;
+	struct fanotify_event *old, *new;
 
 	pr_debug("%s: old=%p new=%p\n", __func__, old_fsn, new_fsn);
 	old = FANOTIFY_E(old_fsn);
@@ -58,7 +58,7 @@ static int fanotify_merge(struct list_head *list, struct fsnotify_event *event)
 }
 
 static int fanotify_get_response(struct fsnotify_group *group,
-				 struct fanotify_perm_event_info *event,
+				 struct fanotify_perm_event *event,
 				 struct fsnotify_iter_info *iter_info)
 {
 	int ret;
@@ -141,11 +141,11 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		~marks_ignored_mask;
 }
 
-struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
+struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
 						 struct inode *inode, u32 mask,
 						 const struct path *path)
 {
-	struct fanotify_event_info *event = NULL;
+	struct fanotify_event *event = NULL;
 	gfp_t gfp = GFP_KERNEL_ACCOUNT;
 
 	/*
@@ -160,7 +160,7 @@ struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
 	memalloc_use_memcg(group->memcg);
 
 	if (fanotify_is_perm_event(mask)) {
-		struct fanotify_perm_event_info *pevent;
+		struct fanotify_perm_event *pevent;
 
 		pevent = kmem_cache_alloc(fanotify_perm_event_cachep, gfp);
 		if (!pevent)
@@ -197,7 +197,7 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 				 struct fsnotify_iter_info *iter_info)
 {
 	int ret = 0;
-	struct fanotify_event_info *event;
+	struct fanotify_event *event;
 	struct fsnotify_event *fsn_event;
 
 	BUILD_BUG_ON(FAN_ACCESS != FS_ACCESS);
@@ -275,7 +275,7 @@ static void fanotify_free_group_priv(struct fsnotify_group *group)
 
 static void fanotify_free_event(struct fsnotify_event *fsn_event)
 {
-	struct fanotify_event_info *event;
+	struct fanotify_event *event;
 
 	event = FANOTIFY_E(fsn_event);
 	path_put(&event->path);
diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index ea05b8a401e7..fb84dd3289f8 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -12,7 +12,7 @@ extern struct kmem_cache *fanotify_perm_event_cachep;
  * fanotify_handle_event() and freed when the information is retrieved by
  * userspace
  */
-struct fanotify_event_info {
+struct fanotify_event {
 	struct fsnotify_event fse;
 	/*
 	 * We hold ref to this path so it may be dereferenced at any point
@@ -29,16 +29,16 @@ struct fanotify_event_info {
  * group->notification_list to group->fanotify_data.access_list to wait for
  * user response.
  */
-struct fanotify_perm_event_info {
-	struct fanotify_event_info fae;
+struct fanotify_perm_event {
+	struct fanotify_event fae;
 	int response;	/* userspace answer to question */
 	int fd;		/* fd we passed to userspace for this event */
 };
 
-static inline struct fanotify_perm_event_info *
+static inline struct fanotify_perm_event *
 FANOTIFY_PE(struct fsnotify_event *fse)
 {
-	return container_of(fse, struct fanotify_perm_event_info, fae.fse);
+	return container_of(fse, struct fanotify_perm_event, fae.fse);
 }
 
 static inline bool fanotify_is_perm_event(u32 mask)
@@ -47,11 +47,11 @@ static inline bool fanotify_is_perm_event(u32 mask)
 		mask & FANOTIFY_PERM_EVENTS;
 }
 
-static inline struct fanotify_event_info *FANOTIFY_E(struct fsnotify_event *fse)
+static inline struct fanotify_event *FANOTIFY_E(struct fsnotify_event *fse)
 {
-	return container_of(fse, struct fanotify_event_info, fse);
+	return container_of(fse, struct fanotify_event, fse);
 }
 
-struct fanotify_event_info *fanotify_alloc_event(struct fsnotify_group *group,
+struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
 						 struct inode *inode, u32 mask,
 						 const struct path *path);
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index e03be5071362..2dbb2662a92f 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -73,7 +73,7 @@ static struct fsnotify_event *get_one_event(struct fsnotify_group *group,
 }
 
 static int create_fd(struct fsnotify_group *group,
-		     struct fanotify_event_info *event,
+		     struct fanotify_event *event,
 		     struct file **file)
 {
 	int client_fd;
@@ -120,13 +120,13 @@ static int fill_event_metadata(struct fsnotify_group *group,
 			       struct file **file)
 {
 	int ret = 0;
-	struct fanotify_event_info *event;
+	struct fanotify_event *event;
 
 	pr_debug("%s: group=%p metadata=%p event=%p\n", __func__,
 		 group, metadata, fsn_event);
 
 	*file = NULL;
-	event = container_of(fsn_event, struct fanotify_event_info, fse);
+	event = container_of(fsn_event, struct fanotify_event, fse);
 	metadata->event_len = FAN_EVENT_METADATA_LEN;
 	metadata->metadata_len = FAN_EVENT_METADATA_LEN;
 	metadata->vers = FANOTIFY_METADATA_VERSION;
@@ -144,10 +144,10 @@ static int fill_event_metadata(struct fsnotify_group *group,
 	return ret;
 }
 
-static struct fanotify_perm_event_info *dequeue_event(
+static struct fanotify_perm_event *dequeue_event(
 				struct fsnotify_group *group, int fd)
 {
-	struct fanotify_perm_event_info *event, *return_e = NULL;
+	struct fanotify_perm_event *event, *return_e = NULL;
 
 	spin_lock(&group->notification_lock);
 	list_for_each_entry(event, &group->fanotify_data.access_list,
@@ -169,7 +169,7 @@ static struct fanotify_perm_event_info *dequeue_event(
 static int process_access_response(struct fsnotify_group *group,
 				   struct fanotify_response *response_struct)
 {
-	struct fanotify_perm_event_info *event;
+	struct fanotify_perm_event *event;
 	int fd = response_struct->fd;
 	int response = response_struct->response;
 
@@ -364,7 +364,7 @@ static ssize_t fanotify_write(struct file *file, const char __user *buf, size_t
 static int fanotify_release(struct inode *ignored, struct file *file)
 {
 	struct fsnotify_group *group = file->private_data;
-	struct fanotify_perm_event_info *event, *next;
+	struct fanotify_perm_event *event, *next;
 	struct fsnotify_event *fsn_event;
 
 	/*
@@ -682,7 +682,7 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)
 	struct fsnotify_group *group;
 	int f_flags, fd;
 	struct user_struct *user;
-	struct fanotify_event_info *oevent;
+	struct fanotify_event *oevent;
 
 	pr_debug("%s: flags=%x event_f_flags=%x\n",
 		 __func__, flags, event_f_flags);
@@ -949,10 +949,10 @@ static int __init fanotify_user_setup(void)
 
 	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
 					 SLAB_PANIC|SLAB_ACCOUNT);
-	fanotify_event_cachep = KMEM_CACHE(fanotify_event_info, SLAB_PANIC);
+	fanotify_event_cachep = KMEM_CACHE(fanotify_event, SLAB_PANIC);
 	if (IS_ENABLED(CONFIG_FANOTIFY_ACCESS_PERMISSIONS)) {
 		fanotify_perm_event_cachep =
-			KMEM_CACHE(fanotify_perm_event_info, SLAB_PANIC);
+			KMEM_CACHE(fanotify_perm_event, SLAB_PANIC);
 	}
 
 	return 0;
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 2/9] fanotify: define the structures to report a unique file identifier
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 1/9] fanotify: rename struct fanotify_{,perm_}event_info Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 3/9] fanotify: classify events that hold a " Amir Goldstein
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

When user requests the flag FAN_REPORT_FID in fanotify_init(),
a unique file indetifier of the event target object will be reported
with the event.

This commit only defines the internal and user visible structures used
to store and report the unique file identifier.

The file identifier includes the filesystem's fsid (i.e. from statfs(2))
and an NFS file handle of the file (i.e. from name_to_handle_at(2)).

The file identifier makes holding the path reference and passing a file
descriptor to user redundant, so those are disabled in a group with
FAN_REPORT_FID.

Cc: <linux-api@vger.kernel.org>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c      |  2 +-
 fs/notify/fanotify/fanotify.h      | 26 ++++++++++++++++----
 fs/notify/fanotify/fanotify_user.c |  5 ++--
 include/uapi/linux/fanotify.h      | 38 +++++++++++++++++++++++++-----
 4 files changed, 57 insertions(+), 14 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index ecd5f4aec624..59d093923c97 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -178,7 +178,7 @@ init: __maybe_unused
 		event->pid = get_pid(task_pid(current));
 	else
 		event->pid = get_pid(task_tgid(current));
-	if (path) {
+	if (path && !FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
 		event->path = *path;
 		path_get(&event->path);
 	} else {
diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index fb84dd3289f8..2e4fca30afda 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -7,6 +7,14 @@ extern struct kmem_cache *fanotify_mark_cache;
 extern struct kmem_cache *fanotify_event_cachep;
 extern struct kmem_cache *fanotify_perm_event_cachep;
 
+/* The size of the variable length buffer storing fsid and file handle */
+#define FANOTIFY_FID_LEN(handle_bytes)	\
+	(sizeof(struct fanotify_event_fid) + (handle_bytes))
+
+struct fanotify_info {
+	struct fanotify_event_fid *fid;
+};
+
 /*
  * Structure for normal fanotify events. It gets allocated in
  * fanotify_handle_event() and freed when the information is retrieved by
@@ -14,11 +22,19 @@ extern struct kmem_cache *fanotify_perm_event_cachep;
  */
 struct fanotify_event {
 	struct fsnotify_event fse;
-	/*
-	 * We hold ref to this path so it may be dereferenced at any point
-	 * during this object's lifetime
-	 */
-	struct path path;
+	union {
+		/*
+		 * We hold ref to this path so it may be dereferenced at any
+		 * point during this object's lifetime
+		 */
+		struct path path;
+		/*
+		 * With FAN_REPORT_FID, we do not hold any reference on the
+		 * victim object. Instead we store its NFS file handle and its
+		 * filesystem's fsid as a unique identifier.
+		 */
+		struct fanotify_info info;
+	};
 	struct pid *pid;
 };
 
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 2dbb2662a92f..93e1aa2a389f 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -133,9 +133,10 @@ static int fill_event_metadata(struct fsnotify_group *group,
 	metadata->reserved = 0;
 	metadata->mask = fsn_event->mask & FANOTIFY_OUTGOING_EVENTS;
 	metadata->pid = pid_vnr(event->pid);
-	if (unlikely(fsn_event->mask & FAN_Q_OVERFLOW))
+	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID) ||
+	    unlikely(fsn_event->mask & FAN_Q_OVERFLOW)) {
 		metadata->fd = FAN_NOFD;
-	else {
+	} else {
 		metadata->fd = create_fd(group, event, file);
 		if (metadata->fd < 0)
 			ret = metadata->fd;
diff --git a/include/uapi/linux/fanotify.h b/include/uapi/linux/fanotify.h
index 909c98fcace2..aa510aec7968 100644
--- a/include/uapi/linux/fanotify.h
+++ b/include/uapi/linux/fanotify.h
@@ -44,6 +44,7 @@
 
 /* Flags to determine fanotify event format */
 #define FAN_REPORT_TID		0x00000100	/* event->pid is thread id */
+#define FAN_REPORT_FID		0x00000200	/* Report unique file id */
 
 /* Deprecated - do not use this in programs and do not add new flags here! */
 #define FAN_ALL_INIT_FLAGS	(FAN_CLOEXEC | FAN_NONBLOCK | \
@@ -106,6 +107,24 @@ struct fanotify_event_metadata {
 	__s32 pid;
 };
 
+#define FAN_EVENT_INFO_TYPE_FID		1
+
+/* Variable length info record header following event metadata */
+struct fanotify_event_info {
+	__u8 info_type;
+	__u8 reserved;
+	__u16 info_len;
+	unsigned char info[0];
+};
+
+/* Unique file identifier info record */
+struct fanotify_event_fid {
+	__kernel_fsid_t fsid;
+	__u32 handle_bytes;
+	__s32 handle_type;
+	unsigned char f_handle[0];
+};
+
 struct fanotify_response {
 	__s32 fd;
 	__u32 response;
@@ -122,12 +141,19 @@ struct fanotify_response {
 /* Helper functions to deal with fanotify_event_metadata buffers */
 #define FAN_EVENT_METADATA_LEN (sizeof(struct fanotify_event_metadata))
 
-#define FAN_EVENT_NEXT(meta, len) ((len) -= (meta)->event_len, \
-				   (struct fanotify_event_metadata*)(((char *)(meta)) + \
-				   (meta)->event_len))
+#define FAN_EVENT_NEXT(meta, len) \
+	((len) -= (meta)->event_len, \
+	 (struct fanotify_event_metadata *)(((char *)(meta)) + \
+					   (meta)->event_len))
+
+#define FAN_EVENT_OK(meta, len)	\
+	((long)(len) >= (long)FAN_EVENT_METADATA_LEN && \
+	 (long)(meta)->event_len >= (long)FAN_EVENT_METADATA_LEN && \
+	 (long)(meta)->event_len <= (long)(len))
 
-#define FAN_EVENT_OK(meta, len)	((long)(len) >= (long)FAN_EVENT_METADATA_LEN && \
-				(long)(meta)->event_len >= (long)FAN_EVENT_METADATA_LEN && \
-				(long)(meta)->event_len <= (long)(len))
+/* Get the first event info record if one exists */
+#define FAN_EVENT_INFO(meta)	\
+	((long)(meta)->event_len > (long)FAN_EVENT_METADATA_LEN ? \
+	 (struct fanotify_event_info *)((meta) + 1) : NULL)
 
 #endif /* _UAPI_LINUX_FANOTIFY_H */
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 3/9] fanotify: classify events that hold a file identifier
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 1/9] fanotify: rename struct fanotify_{,perm_}event_info Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 2/9] fanotify: define the structures to report a unique file identifier Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 4/9] fanotify: encode file identifier for FAN_REPORT_FID Amir Goldstein
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

With group flag FAN_REPORT_FID, we do not store event->path.
Instead, we will store the file handle and fsid in event->info.fid.

Because event->info and event->path share the same union space, set
bit 0 of event->info.flags, so we know how to free and compare the
event objects.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c      | 35 ++++++++++++++++++++++++------
 fs/notify/fanotify/fanotify.h      | 27 +++++++++++++++++++++++
 fs/notify/fanotify/fanotify_user.c |  3 +++
 3 files changed, 58 insertions(+), 7 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 59d093923c97..8192d4b1db21 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -25,11 +25,16 @@ static bool should_merge(struct fsnotify_event *old_fsn,
 	old = FANOTIFY_E(old_fsn);
 	new = FANOTIFY_E(new_fsn);
 
-	if (old_fsn->inode == new_fsn->inode && old->pid == new->pid &&
-	    old->path.mnt == new->path.mnt &&
-	    old->path.dentry == new->path.dentry)
-		return true;
-	return false;
+	if (old_fsn->inode != new_fsn->inode || old->pid != new->pid)
+		return false;
+
+	if (FANOTIFY_HAS_FID(new)) {
+		return FANOTIFY_HAS_FID(old) &&
+			fanotify_fid_equal(old->info.fid, new->info.fid);
+	} else {
+		return old->path.mnt == new->path.mnt &&
+			old->path.dentry == new->path.dentry;
+	}
 }
 
 /* and the list better be locked by something too! */
@@ -178,7 +183,10 @@ init: __maybe_unused
 		event->pid = get_pid(task_pid(current));
 	else
 		event->pid = get_pid(task_tgid(current));
-	if (path && !FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+		/* TODO: allocate buffer and encode file handle */
+		fanotify_set_fid(event, NULL);
+	} else if (path) {
 		event->path = *path;
 		path_get(&event->path);
 	} else {
@@ -277,8 +285,21 @@ static void fanotify_free_event(struct fsnotify_event *fsn_event)
 {
 	struct fanotify_event *event;
 
+	/*
+	 * event->path and event->info are a union. Make sure that
+	 * event->info.flags overlaps with path.dentry pointer, so it is safe
+	 * to use bit 0 to classify the event info type (FANOTIFY_HAS_FID).
+	 */
+	BUILD_BUG_ON(offsetof(struct fanotify_event, path.dentry) !=
+		     offsetof(struct fanotify_event, info.flags));
+	BUILD_BUG_ON(!__builtin_types_compatible_p(typeof(event->info.flags),
+						   unsigned long));
+
 	event = FANOTIFY_E(fsn_event);
-	path_put(&event->path);
+	if (FANOTIFY_HAS_FID(event))
+		kfree(event->info.fid);
+	else
+		path_put(&event->path);
 	put_pid(event->pid);
 	if (fanotify_is_perm_event(fsn_event->mask)) {
 		kmem_cache_free(fanotify_perm_event_cachep,
diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index 2e4fca30afda..d6ed31134321 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -13,8 +13,20 @@ extern struct kmem_cache *fanotify_perm_event_cachep;
 
 struct fanotify_info {
 	struct fanotify_event_fid *fid;
+	unsigned long flags;
 };
 
+static inline bool fanotify_fid_equal(const struct fanotify_event_fid *fid1,
+				      const struct fanotify_event_fid *fid2)
+{
+	if (fid1 == fid2)
+		return true;
+
+	return fid1 && fid2 && fid1->handle_bytes == fid2->handle_bytes &&
+		!memcmp((void *)fid1, (void *)fid2,
+			FANOTIFY_FID_LEN(fid1->handle_bytes));
+}
+
 /*
  * Structure for normal fanotify events. It gets allocated in
  * fanotify_handle_event() and freed when the information is retrieved by
@@ -38,6 +50,21 @@ struct fanotify_event {
 	struct pid *pid;
 };
 
+/*
+ * Bit 0 of info.flags is set when event has fid information.
+ * event->info shares the same union address with event->path, so this helps
+ * us tell if event has fid or path.
+ */
+#define __FANOTIFY_FID		1UL
+#define FANOTIFY_HAS_FID(event)	((event)->info.flags & __FANOTIFY_FID)
+
+static inline void fanotify_set_fid(struct fanotify_event *event,
+				    struct fanotify_event_fid *fid)
+{
+	event->info.fid = fid;
+	event->info.flags = fid ? __FANOTIFY_FID : 0;
+}
+
 /*
  * Structure for permission fanotify events. It gets allocated and freed in
  * fanotify_handle_event() since we wait there for user response. When the
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 93e1aa2a389f..29f61060af66 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -81,6 +81,9 @@ static int create_fd(struct fsnotify_group *group,
 
 	pr_debug("%s: group=%p event=%p\n", __func__, group, event);
 
+	if (WARN_ON_ONCE(FANOTIFY_HAS_FID(event)))
+		return -EBADF;
+
 	client_fd = get_unused_fd_flags(group->fanotify_data.f_flags);
 	if (client_fd < 0)
 		return client_fd;
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 4/9] fanotify: encode file identifier for FAN_REPORT_FID
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (2 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 3/9] fanotify: classify events that hold a " Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 5/9] fanotify: copy event fid info to user Amir Goldstein
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

Allocate and encode event->info.fid for a group with FAN_REPORT_FID.
Treat failure to allocate fid buffer as failure to allocate event.
Treat failure to encode fid by printing a warning but queueing
the event without the fid information.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c | 59 +++++++++++++++++++++++++++++++++--
 1 file changed, 57 insertions(+), 2 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 8192d4b1db21..844b748f0b74 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -13,6 +13,8 @@
 #include <linux/wait.h>
 #include <linux/audit.h>
 #include <linux/sched/mm.h>
+#include <linux/statfs.h>
+#include <linux/exportfs.h>
 
 #include "fanotify.h"
 
@@ -146,11 +148,55 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		~marks_ignored_mask;
 }
 
+static struct fanotify_event_fid *fanotify_alloc_fid(const struct path *path,
+						     gfp_t gfp)
+{
+	struct fanotify_event_fid *fid = NULL;
+	int dwords, bytes = 0;
+	struct kstatfs stat;
+	int err, type;
+
+	dwords = 0;
+	err = -ENOENT;
+	type = exportfs_encode_fh(path->dentry, NULL, &dwords,  0);
+	if (!dwords)
+		goto out_err;
+
+	err = vfs_statfs(path, &stat);
+	if (err)
+		goto out_err;
+
+	/* Treat failure to allocate fid as failure to allocate event */
+	bytes = dwords << 2;
+	fid = kmalloc(FANOTIFY_FID_LEN(bytes), gfp);
+	if (!fid)
+		return NULL;
+
+	type = exportfs_encode_fh(path->dentry, (struct fid *)fid->f_handle,
+				  &dwords,  0);
+	err = -EINVAL;
+	if (type == FILEID_INVALID || bytes != dwords << 2)
+		goto out_err;
+
+	fid->handle_bytes = bytes;
+	fid->handle_type = type;
+	fid->fsid = stat.f_fsid;
+
+	return fid;
+
+out_err:
+	pr_warn_ratelimited("fanotify: failed to encode fid of %pd2 (bytes=%d, err=%i)\n",
+			    path->dentry, bytes, err);
+	kfree(fid);
+	return ERR_PTR(err);
+}
+
 struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
 						 struct inode *inode, u32 mask,
 						 const struct path *path)
 {
 	struct fanotify_event *event = NULL;
+	struct fanotify_event_fid *fid = NULL;
 	gfp_t gfp = GFP_KERNEL_ACCOUNT;
 
 	/*
@@ -164,6 +210,16 @@ struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
 	/* Whoever is interested in the event, pays for the allocation. */
 	memalloc_use_memcg(group->memcg);
 
+	if (path && FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+		fid = fanotify_alloc_fid(path, gfp);
+		/* Treat failure to allocate fid as failure to allocate event */
+		if (!fid)
+			goto out;
+		/* Report the event without a file identifier on encode error */
+		if (IS_ERR(fid))
+			fid = NULL;
+	}
+
 	if (fanotify_is_perm_event(mask)) {
 		struct fanotify_perm_event *pevent;
 
@@ -184,8 +240,7 @@ init: __maybe_unused
 	else
 		event->pid = get_pid(task_tgid(current));
 	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
-		/* TODO: allocate buffer and encode file handle */
-		fanotify_set_fid(event, NULL);
+		fanotify_set_fid(event, fid);
 	} else if (path) {
 		event->path = *path;
 		path_get(&event->path);
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 5/9] fanotify: copy event fid info to user
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (3 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 4/9] fanotify: encode file identifier for FAN_REPORT_FID Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 6/9] fanotify: enable FAN_REPORT_FID init flag Amir Goldstein
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

If group requested FAN_REPORT_FID and event has file identifier
copy that information to user reading the event after reading
event metadata.
metadata->event_len includes the length of the fid information.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.h      |  3 ++
 fs/notify/fanotify/fanotify_user.c | 72 +++++++++++++++++++++++++++---
 2 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index d6ed31134321..26752d2f2bd6 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -10,6 +10,9 @@ extern struct kmem_cache *fanotify_perm_event_cachep;
 /* The size of the variable length buffer storing fsid and file handle */
 #define FANOTIFY_FID_LEN(handle_bytes)	\
 	(sizeof(struct fanotify_event_fid) + (handle_bytes))
+#define FANOTIFY_FID_INFO_LEN(event)	\
+	(sizeof(struct fanotify_event_info) + \
+	FANOTIFY_FID_LEN((event)->info.fid->handle_bytes))
 
 struct fanotify_info {
 	struct fanotify_event_fid *fid;
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 29f61060af66..7e38d7a1854b 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -47,6 +47,16 @@ struct kmem_cache *fanotify_mark_cache __read_mostly;
 struct kmem_cache *fanotify_event_cachep __read_mostly;
 struct kmem_cache *fanotify_perm_event_cachep __read_mostly;
 
+static int round_event_fid_len(struct fsnotify_event *fsn_event)
+{
+	struct fanotify_event *event = FANOTIFY_E(fsn_event);
+
+	if (!FANOTIFY_HAS_FID(event))
+		return 0;
+
+	return roundup(FANOTIFY_FID_INFO_LEN(event), FAN_EVENT_METADATA_LEN);
+}
+
 /*
  * Get an fsnotify notification event if one exists and is small
  * enough to fit in "count". Return an error pointer if the count
@@ -57,6 +67,9 @@ struct kmem_cache *fanotify_perm_event_cachep __read_mostly;
 static struct fsnotify_event *get_one_event(struct fsnotify_group *group,
 					    size_t count)
 {
+	size_t event_size = FAN_EVENT_METADATA_LEN;
+	struct fsnotify_event *event;
+
 	assert_spin_locked(&group->notification_lock);
 
 	pr_debug("%s: group=%p count=%zd\n", __func__, group, count);
@@ -64,11 +77,18 @@ static struct fsnotify_event *get_one_event(struct fsnotify_group *group,
 	if (fsnotify_notify_queue_is_empty(group))
 		return NULL;
 
-	if (FAN_EVENT_METADATA_LEN > count)
+	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+		event = fsnotify_peek_first_event(group);
+		event_size += round_event_fid_len(event);
+	}
+
+	if (event_size > count)
 		return ERR_PTR(-EINVAL);
 
-	/* held the notification_lock the whole time, so this is the
-	 * same event we peeked above */
+	/*
+	 * Held the notification_lock the whole time, so this is the
+	 * same event we peeked above
+	 */
 	return fsnotify_remove_first_event(group);
 }
 
@@ -129,7 +149,7 @@ static int fill_event_metadata(struct fsnotify_group *group,
 		 group, metadata, fsn_event);
 
 	*file = NULL;
-	event = container_of(fsn_event, struct fanotify_event, fse);
+	event = FANOTIFY_E(fsn_event);
 	metadata->event_len = FAN_EVENT_METADATA_LEN;
 	metadata->metadata_len = FAN_EVENT_METADATA_LEN;
 	metadata->vers = FANOTIFY_METADATA_VERSION;
@@ -139,6 +159,7 @@ static int fill_event_metadata(struct fsnotify_group *group,
 	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID) ||
 	    unlikely(fsn_event->mask & FAN_Q_OVERFLOW)) {
 		metadata->fd = FAN_NOFD;
+		metadata->event_len += round_event_fid_len(fsn_event);
 	} else {
 		metadata->fd = create_fd(group, event, file);
 		if (metadata->fd < 0)
@@ -208,6 +229,38 @@ static int process_access_response(struct fsnotify_group *group,
 	return 0;
 }
 
+static int copy_fid_to_user(struct fsnotify_event *fsn_event, char __user *buf)
+{
+	struct fanotify_event *event = FANOTIFY_E(fsn_event);
+	struct fanotify_event_info ei;
+	size_t fid_len;
+	size_t pad_len = round_event_fid_len(fsn_event);
+
+	if (!pad_len)
+		return 0;
+
+	/* Copy event info header followed by fid buffer */
+	fid_len = FANOTIFY_FID_INFO_LEN(event);
+	pad_len -= fid_len;
+	ei.info_type = FAN_EVENT_INFO_TYPE_FID;
+	ei.reserved = 0;
+	ei.info_len = fid_len;
+	if (copy_to_user(buf, &ei, sizeof(ei)))
+		return -EFAULT;
+
+	buf += sizeof(ei);
+	fid_len -= sizeof(ei);
+	if (copy_to_user(buf, event->info.fid, fid_len))
+		return -EFAULT;
+
+	/* Pad with 0's */
+	buf += fid_len;
+	if (pad_len && clear_user(buf, pad_len))
+		return -EFAULT;
+
+	return 0;
+}
+
 static ssize_t copy_event_to_user(struct fsnotify_group *group,
 				  struct fsnotify_event *event,
 				  char __user *buf)
@@ -224,15 +277,20 @@ static ssize_t copy_event_to_user(struct fsnotify_group *group,
 
 	fd = fanotify_event_metadata.fd;
 	ret = -EFAULT;
-	if (copy_to_user(buf, &fanotify_event_metadata,
-			 fanotify_event_metadata.event_len))
+	if (copy_to_user(buf, &fanotify_event_metadata, FAN_EVENT_METADATA_LEN))
 		goto out_close_fd;
 
 	if (fanotify_is_perm_event(event->mask))
 		FANOTIFY_PE(event)->fd = fd;
 
-	if (fd != FAN_NOFD)
+	if (fd != FAN_NOFD) {
 		fd_install(fd, f);
+	} else if (FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+		ret = copy_fid_to_user(event, buf + FAN_EVENT_METADATA_LEN);
+		if (ret < 0)
+			return ret;
+	}
+
 	return fanotify_event_metadata.event_len;
 
 out_close_fd:
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 6/9] fanotify: enable FAN_REPORT_FID init flag
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (4 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 5/9] fanotify: copy event fid info to user Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 7/9] fanotify: support events with data type FSNOTIFY_EVENT_DENTRY Amir Goldstein
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

When setting up an fanotify listener, user may request to get fid
information in event instead of an open file descriptor.

The fid obtained with event on a watched object contains the file
handle returned by name_to_handle_at(2) and fsid returned by statfs(2).
Both name_to_handle_at(2) and statfs(2) are allowed for unprivileged
users that have access to a given path, so we also allow an unprivileged
listener to get fid information, because with unprivileged listener,
users can only set marks on inode objects they have read access to.

When setting a mark, we need to make sure that the filesystem
supports file handles, so user can use name_to_handle_at(2) to compare
fid returned with event to the file handle of watched objects.

Cc: <linux-api@vger.kernel.org>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify_user.c | 17 ++++++++++++++++-
 include/linux/fanotify.h           |  2 +-
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 7e38d7a1854b..8ded9b5c0e76 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -17,6 +17,7 @@
 #include <linux/compat.h>
 #include <linux/sched/signal.h>
 #include <linux/memcontrol.h>
+#include <linux/exportfs.h>
 
 #include <asm/ioctls.h>
 
@@ -942,6 +943,19 @@ static int do_fanotify_mark(int fanotify_fd, unsigned int flags, __u64 mask,
 	if (ret)
 		goto fput_and_out;
 
+	/*
+	 * We need to make sure whether the file system supports at least
+	 * encoding a file handle so user can use name_to_handle_at() to
+	 * compare fid returned with event to the file handle of watched
+	 * objects. However, name_to_handle_at() requires that filesystem
+	 * supports decoding file handles as well.
+	 */
+	ret = -EOPNOTSUPP;
+	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID) &&
+	    (!path.dentry->d_sb->s_export_op ||
+	     !path.dentry->d_sb->s_export_op->fh_to_dentry))
+		goto path_put_and_out;
+
 	/* inode held in place by reference to path; group by fget on fd */
 	if (mark_type == FAN_MARK_INODE)
 		inode = path.dentry->d_inode;
@@ -970,6 +984,7 @@ static int do_fanotify_mark(int fanotify_fd, unsigned int flags, __u64 mask,
 		ret = -EINVAL;
 	}
 
+path_put_and_out:
 	path_put(&path);
 fput_and_out:
 	fdput(f);
@@ -1006,7 +1021,7 @@ COMPAT_SYSCALL_DEFINE6(fanotify_mark,
  */
 static int __init fanotify_user_setup(void)
 {
-	BUILD_BUG_ON(HWEIGHT32(FANOTIFY_INIT_FLAGS) != 7);
+	BUILD_BUG_ON(HWEIGHT32(FANOTIFY_INIT_FLAGS) != 8);
 	BUILD_BUG_ON(HWEIGHT32(FANOTIFY_MARK_FLAGS) != 9);
 
 	fanotify_mark_cache = KMEM_CACHE(fsnotify_mark,
diff --git a/include/linux/fanotify.h b/include/linux/fanotify.h
index 9e2142795335..f59be967f72b 100644
--- a/include/linux/fanotify.h
+++ b/include/linux/fanotify.h
@@ -19,7 +19,7 @@
 				 FAN_CLASS_PRE_CONTENT)
 
 #define FANOTIFY_INIT_FLAGS	(FANOTIFY_CLASS_BITS | \
-				 FAN_REPORT_TID | \
+				 FAN_REPORT_TID | FAN_REPORT_FID | \
 				 FAN_CLOEXEC | FAN_NONBLOCK | \
 				 FAN_UNLIMITED_QUEUE | FAN_UNLIMITED_MARKS)
 
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 7/9] fanotify: support events with data type FSNOTIFY_EVENT_DENTRY
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (5 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 6/9] fanotify: enable FAN_REPORT_FID init flag Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 8/9] fanotify: add support for create/attrib/move/delete events Amir Goldstein
                   ` (2 subsequent siblings)
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

When event data type is FSNOTIFY_EVENT_DENTRY, we don't have a refernece
to the mount, so we will not be able to open a file descriptor when user
reads the event. However, if the listener has enabled reporting file
identifier with the FAN_REPORT_FID init flag, we allow repoting those
events and we use the dentry to encode fid.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c      | 63 ++++++++++++++++++++----------
 fs/notify/fanotify/fanotify.h      |  4 +-
 fs/notify/fanotify/fanotify_user.c |  6 ++-
 fs/statfs.c                        |  3 +-
 include/linux/statfs.h             |  3 ++
 5 files changed, 53 insertions(+), 26 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 844b748f0b74..9ee15af2f83d 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -92,10 +92,26 @@ static int fanotify_get_response(struct fsnotify_group *group,
 
 	pr_debug("%s: group=%p event=%p about to return ret=%d\n", __func__,
 		 group, event, ret);
-	
+
 	return ret;
 }
 
+static struct dentry *fanotify_event_dentry(const void *data, int data_type)
+{
+	struct dentry *dentry = NULL;
+
+	if (data_type == FSNOTIFY_EVENT_PATH) {
+		dentry = ((struct path *)data)->dentry;
+		/* Path type events are only relevant for files and dirs */
+		if (!d_is_reg(dentry) && !d_can_lookup(dentry))
+			return NULL;
+	} else if (data_type == FSNOTIFY_EVENT_DENTRY) {
+		dentry = (struct dentry *)data;
+	}
+
+	return dentry;
+}
+
 /*
  * This function returns a mask for an event that only contains the flags
  * that have been specifically requested by the user. Flags that may have
@@ -107,7 +123,7 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 				       int data_type)
 {
 	__u32 marks_mask = 0, marks_ignored_mask = 0;
-	const struct path *path = data;
+	struct dentry *dentry = fanotify_event_dentry(data, data_type);
 	struct fsnotify_mark *mark;
 	int type;
 
@@ -115,12 +131,7 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		 __func__, iter_info->report_mask, event_mask, data, data_type);
 
 	/* If we don't have enough info to send an event to userspace say no */
-	if (data_type != FSNOTIFY_EVENT_PATH)
-		return 0;
-
-	/* Sorry, fanotify only gives a damn about files and dirs */
-	if (!d_is_reg(path->dentry) &&
-	    !d_can_lookup(path->dentry))
+	if (!dentry)
 		return 0;
 
 	fsnotify_foreach_obj_type(type) {
@@ -140,7 +151,7 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		marks_ignored_mask |= mark->ignored_mask;
 	}
 
-	if (d_is_dir(path->dentry) &&
+	if (d_is_dir(dentry) &&
 	    !(marks_mask & FS_ISDIR & ~marks_ignored_mask))
 		return 0;
 
@@ -148,7 +159,7 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		~marks_ignored_mask;
 }
 
-static struct fanotify_event_fid *fanotify_alloc_fid(const struct path *path,
+static struct fanotify_event_fid *fanotify_alloc_fid(struct dentry *dentry,
 						     gfp_t gfp)
 {
 	struct fanotify_event_fid *fid = NULL;
@@ -158,11 +169,11 @@ static struct fanotify_event_fid *fanotify_alloc_fid(const struct path *path,
 
 	dwords = 0;
 	err = -ENOENT;
-	type = exportfs_encode_fh(path->dentry, NULL, &dwords,  0);
+	type = exportfs_encode_fh(dentry, NULL, &dwords,  0);
 	if (!dwords)
 		goto out_err;
 
-	err = vfs_statfs(path, &stat);
+	err = statfs_by_dentry(dentry, &stat);
 	if (err)
 		goto out_err;
 
@@ -172,7 +183,7 @@ static struct fanotify_event_fid *fanotify_alloc_fid(const struct path *path,
 	if (!fid)
 		return NULL;
 
-	type = exportfs_encode_fh(path->dentry, (struct fid *)fid->f_handle,
+	type = exportfs_encode_fh(dentry, (struct fid *)fid->f_handle,
 				  &dwords,  0);
 	err = -EINVAL;
 	if (type == FILEID_INVALID || bytes != dwords << 2)
@@ -186,15 +197,16 @@ static struct fanotify_event_fid *fanotify_alloc_fid(const struct path *path,
 
 out_err:
 	pr_warn_ratelimited("fanotify: failed to encode fid of %pd2 (bytes=%d, err=%i)\n",
-			    path->dentry, bytes, err);
+			    dentry, bytes, err);
 	kfree(fid);
 	return ERR_PTR(err);
 }
 
 struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
-						 struct inode *inode, u32 mask,
-						 const struct path *path)
+					    struct inode *inode, u32 mask,
+					    const void *data, int data_type)
 {
+	struct dentry *dentry = fanotify_event_dentry(data, data_type);
 	struct fanotify_event *event = NULL;
 	struct fanotify_event_fid *fid = NULL;
 	gfp_t gfp = GFP_KERNEL_ACCOUNT;
@@ -210,8 +222,8 @@ struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
 	/* Whoever is interested in the event, pays for the allocation. */
 	memalloc_use_memcg(group->memcg);
 
-	if (path && FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
-		fid = fanotify_alloc_fid(path, gfp);
+	if (dentry && FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
+		fid = fanotify_alloc_fid(dentry, gfp);
 		/* Treat failure to allocate fid as failure to allocate event */
 		if (!fid)
 			goto out;
@@ -241,9 +253,13 @@ init: __maybe_unused
 		event->pid = get_pid(task_tgid(current));
 	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID)) {
 		fanotify_set_fid(event, fid);
-	} else if (path) {
-		event->path = *path;
+	} else if (data && data_type == FSNOTIFY_EVENT_PATH) {
+		event->path = *((const struct path *)data);
 		path_get(&event->path);
+		pr_debug("%s: mnt=%p, dentry=%p parent=%p d_flags=%x\n",
+				__func__, event->path.mnt, event->path.dentry,
+				event->path.dentry->d_parent,
+				event->path.dentry->d_flags);
 	} else {
 		event->path.mnt = NULL;
 		event->path.dentry = NULL;
@@ -278,6 +294,11 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 
 	BUILD_BUG_ON(HWEIGHT32(ALL_FANOTIFY_EVENT_BITS) != 12);
 
+	/* Dentry type events cannot report fd, so require FAN_REPORT_FID */
+	if (data_type == FSNOTIFY_EVENT_DENTRY &&
+	    !FAN_GROUP_FLAG(group, FAN_REPORT_FID))
+		return 0;
+
 	mask = fanotify_group_event_mask(iter_info, mask, data, data_type);
 	if (!mask)
 		return 0;
@@ -294,7 +315,7 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 			return 0;
 	}
 
-	event = fanotify_alloc_event(group, inode, mask, data);
+	event = fanotify_alloc_event(group, inode, mask, data, data_type);
 	ret = -ENOMEM;
 	if (unlikely(!event)) {
 		/*
diff --git a/fs/notify/fanotify/fanotify.h b/fs/notify/fanotify/fanotify.h
index 26752d2f2bd6..0ecbbe04798e 100644
--- a/fs/notify/fanotify/fanotify.h
+++ b/fs/notify/fanotify/fanotify.h
@@ -99,5 +99,5 @@ static inline struct fanotify_event *FANOTIFY_E(struct fsnotify_event *fse)
 }
 
 struct fanotify_event *fanotify_alloc_event(struct fsnotify_group *group,
-						 struct inode *inode, u32 mask,
-						 const struct path *path);
+					    struct inode *inode, u32 mask,
+					    const void *data, int data_type);
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index 8ded9b5c0e76..aa68f617964b 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -158,7 +158,8 @@ static int fill_event_metadata(struct fsnotify_group *group,
 	metadata->mask = fsn_event->mask & FANOTIFY_OUTGOING_EVENTS;
 	metadata->pid = pid_vnr(event->pid);
 	if (FAN_GROUP_FLAG(group, FAN_REPORT_FID) ||
-	    unlikely(fsn_event->mask & FAN_Q_OVERFLOW)) {
+	    unlikely(fsn_event->mask & FAN_Q_OVERFLOW) ||
+	    !event->path.mnt || !event->path.dentry) {
 		metadata->fd = FAN_NOFD;
 		metadata->event_len += round_event_fid_len(fsn_event);
 	} else {
@@ -796,7 +797,8 @@ SYSCALL_DEFINE2(fanotify_init, unsigned int, flags, unsigned int, event_f_flags)
 	atomic_inc(&user->fanotify_listeners);
 	group->memcg = get_mem_cgroup_from_mm(current->mm);
 
-	oevent = fanotify_alloc_event(group, NULL, FS_Q_OVERFLOW, NULL);
+	oevent = fanotify_alloc_event(group, NULL, FS_Q_OVERFLOW, NULL,
+				      FSNOTIFY_EVENT_NONE);
 	if (unlikely(!oevent)) {
 		fd = -ENOMEM;
 		goto out_destroy_group;
diff --git a/fs/statfs.c b/fs/statfs.c
index f0216629621d..fd718ea14b14 100644
--- a/fs/statfs.c
+++ b/fs/statfs.c
@@ -50,7 +50,8 @@ static int calculate_f_flags(struct vfsmount *mnt)
 		flags_by_sb(mnt->mnt_sb->s_flags);
 }
 
-static int statfs_by_dentry(struct dentry *dentry, struct kstatfs *buf)
+/* Does not set buf->f_flags */
+int statfs_by_dentry(struct dentry *dentry, struct kstatfs *buf)
 {
 	int retval;
 
diff --git a/include/linux/statfs.h b/include/linux/statfs.h
index 3142e98546ac..c733f5a9503c 100644
--- a/include/linux/statfs.h
+++ b/include/linux/statfs.h
@@ -41,4 +41,7 @@ struct kstatfs {
 #define ST_NODIRATIME	0x0800	/* do not update directory access times */
 #define ST_RELATIME	0x1000	/* update atime relative to mtime/ctime */
 
+struct dentry;
+extern int statfs_by_dentry(struct dentry *dentry, struct kstatfs *buf);
+
 #endif
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 8/9] fanotify: add support for create/attrib/move/delete events
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (6 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 7/9] fanotify: support events with data type FSNOTIFY_EVENT_DENTRY Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-15 18:45 ` [PATCH v2 9/9] fanotify: report FAN_ONDIR to listener for filename events Amir Goldstein
  2018-11-18 12:09 ` [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

Add support for create/attrib/move/delete events with data type
FSNOTIFY_EVENT_DENTRY for inode and filesystem mark types.

The "dentry events" do not carry enough inormation (i.e. path) to
report event->fd, so we do not allow setting a mask for dentry events
unless group supports reporting fid.

The "dentry events" are not supported on a mount mark, because they do
not carry enough inormation (i.e. path) to be filtered by mount point.

The "filename" events (create/move/delete) report the fid of the parent
directry where events took place without specifying the filename of the
child. In the future, fanotify may get support fot reporting filename
information for those events.

Two legacy inotify events that are still not supported are DELETE_SELF
and the ATTRIB event of inode nlink change. The events have event data
type FSNOTIFY_EVENT_INODE.

Cc: <linux-api@vger.kernel.org>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c      |  8 +++++++-
 fs/notify/fanotify/fanotify_user.c | 12 ++++++++++++
 include/linux/fanotify.h           | 20 ++++++++++++++++++--
 include/uapi/linux/fanotify.h      |  7 +++++++
 4 files changed, 44 insertions(+), 3 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index 9ee15af2f83d..a2d63e86ba47 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -281,9 +281,15 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 
 	BUILD_BUG_ON(FAN_ACCESS != FS_ACCESS);
 	BUILD_BUG_ON(FAN_MODIFY != FS_MODIFY);
+	BUILD_BUG_ON(FAN_ATTRIB != FS_ATTRIB);
 	BUILD_BUG_ON(FAN_CLOSE_NOWRITE != FS_CLOSE_NOWRITE);
 	BUILD_BUG_ON(FAN_CLOSE_WRITE != FS_CLOSE_WRITE);
 	BUILD_BUG_ON(FAN_OPEN != FS_OPEN);
+	BUILD_BUG_ON(FAN_MOVED_TO != FS_MOVED_TO);
+	BUILD_BUG_ON(FAN_MOVED_FROM != FS_MOVED_FROM);
+	BUILD_BUG_ON(FAN_CREATE != FS_CREATE);
+	BUILD_BUG_ON(FAN_DELETE != FS_DELETE);
+	BUILD_BUG_ON(FAN_MOVE_SELF != FS_MOVE_SELF);
 	BUILD_BUG_ON(FAN_EVENT_ON_CHILD != FS_EVENT_ON_CHILD);
 	BUILD_BUG_ON(FAN_Q_OVERFLOW != FS_Q_OVERFLOW);
 	BUILD_BUG_ON(FAN_OPEN_PERM != FS_OPEN_PERM);
@@ -292,7 +298,7 @@ static int fanotify_handle_event(struct fsnotify_group *group,
 	BUILD_BUG_ON(FAN_OPEN_EXEC != FS_OPEN_EXEC);
 	BUILD_BUG_ON(FAN_OPEN_EXEC_PERM != FS_OPEN_EXEC_PERM);
 
-	BUILD_BUG_ON(HWEIGHT32(ALL_FANOTIFY_EVENT_BITS) != 12);
+	BUILD_BUG_ON(HWEIGHT32(ALL_FANOTIFY_EVENT_BITS) != 18);
 
 	/* Dentry type events cannot report fd, so require FAN_REPORT_FID */
 	if (data_type == FSNOTIFY_EVENT_DENTRY &&
diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
index aa68f617964b..3364ed6ecb3e 100644
--- a/fs/notify/fanotify/fanotify_user.c
+++ b/fs/notify/fanotify/fanotify_user.c
@@ -930,6 +930,18 @@ static int do_fanotify_mark(int fanotify_fd, unsigned int flags, __u64 mask,
 	    group->priority == FS_PRIO_0)
 		goto fput_and_out;
 
+	/*
+	 * dentry events do not carry enough inormation (i.e. path) to report
+	 * event->fd, so we do not allow setting a mask for dentry events unless
+	 * group supports reporting fid.
+	 * dentry events are not supported on a mount mark, because they do not
+	 * carry enough inormation (i.e. path) to be filtered by mount point.
+	 */
+	if (mask & FANOTIFY_DENTRY_EVENTS &&
+	    (!FAN_GROUP_FLAG(group, FAN_REPORT_FID) ||
+	     mark_type == FAN_MARK_MOUNT))
+		goto fput_and_out;
+
 	if (flags & FAN_MARK_FLUSH) {
 		ret = 0;
 		if (mark_type == FAN_MARK_MOUNT)
diff --git a/include/linux/fanotify.h b/include/linux/fanotify.h
index f59be967f72b..fbda82b25ae3 100644
--- a/include/linux/fanotify.h
+++ b/include/linux/fanotify.h
@@ -35,10 +35,26 @@
 				 FAN_MARK_IGNORED_SURV_MODIFY | \
 				 FAN_MARK_FLUSH)
 
-/* Events that user can request to be notified on */
-#define FANOTIFY_EVENTS		(FAN_ACCESS | FAN_MODIFY | \
+/* Events reported with data type FSNOTIFY_EVENT_PATH */
+#define FANOTIFY_PATH_EVENTS	(FAN_ACCESS | FAN_MODIFY | \
 				 FAN_CLOSE | FAN_OPEN | FAN_OPEN_EXEC)
 
+/*
+ * Events whose reported fid is the parent directory.
+ * fanotify may get support for reporting the filename in the futute.
+ * For now, listener only gets notified that a create/delete/rename took
+ * place in that directory.
+ */
+#define FANOTIFY_FILENAME_EVENTS	(FAN_MOVE | FAN_CREATE | FAN_DELETE)
+
+/* Events reported with data type FSNOTIFY_EVENT_DENTRY */
+#define FANOTIFY_DENTRY_EVENTS	(FANOTIFY_FILENAME_EVENTS | \
+				 FAN_ATTRIB | FAN_MOVE_SELF)
+
+/* Events that user can request to be notified on */
+#define FANOTIFY_EVENTS		(FANOTIFY_PATH_EVENTS | \
+				 FANOTIFY_DENTRY_EVENTS)
+
 /* Events that require a permission response from user */
 #define FANOTIFY_PERM_EVENTS	(FAN_OPEN_PERM | FAN_ACCESS_PERM | \
 				 FAN_OPEN_EXEC_PERM)
diff --git a/include/uapi/linux/fanotify.h b/include/uapi/linux/fanotify.h
index aa510aec7968..3c9b7c86b67b 100644
--- a/include/uapi/linux/fanotify.h
+++ b/include/uapi/linux/fanotify.h
@@ -7,9 +7,15 @@
 /* the following events that user-space can register for */
 #define FAN_ACCESS		0x00000001	/* File was accessed */
 #define FAN_MODIFY		0x00000002	/* File was modified */
+#define FAN_ATTRIB		0x00000004	/* Metadata changed */
 #define FAN_CLOSE_WRITE		0x00000008	/* Writtable file closed */
 #define FAN_CLOSE_NOWRITE	0x00000010	/* Unwrittable file closed */
 #define FAN_OPEN		0x00000020	/* File was opened */
+#define FAN_MOVED_FROM		0x00000040	/* File was moved from X */
+#define FAN_MOVED_TO		0x00000080	/* File was moved to Y */
+#define FAN_CREATE		0x00000100	/* Subfile was created */
+#define FAN_DELETE		0x00000200	/* Subfile was deleted */
+#define FAN_MOVE_SELF		0x00000800	/* Self was moved */
 #define FAN_OPEN_EXEC		0x00001000	/* File was opened for exec */
 
 #define FAN_Q_OVERFLOW		0x00004000	/* Event queued overflowed */
@@ -24,6 +30,7 @@
 
 /* helper events */
 #define FAN_CLOSE		(FAN_CLOSE_WRITE | FAN_CLOSE_NOWRITE) /* close */
+#define FAN_MOVE		(FAN_MOVED_FROM | FAN_MOVED_TO) /* moves */
 
 /* flags used for fanotify_init() */
 #define FAN_CLOEXEC		0x00000001
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH v2 9/9] fanotify: report FAN_ONDIR to listener for filename events
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (7 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 8/9] fanotify: add support for create/attrib/move/delete events Amir Goldstein
@ 2018-11-15 18:45 ` Amir Goldstein
  2018-11-18 12:09 ` [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
  9 siblings, 0 replies; 12+ messages in thread
From: Amir Goldstein @ 2018-11-15 18:45 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api

For events reporting fid of parent (create/delete/move), we report
FAN_ONDIR, so user can differentiate mkdir/rmdir from creat/unlink events.
Unlike legacy fanotify events (open/access/close), mkdir/rmdir will be
reported regardless if user requested FAN_ONDIR, but the FAN_ONDIR flag
itself will only be reported if the user asked for it.

For backward compatibility, we do not report FAN_ONDIR to user for legacy
fanotify events.  While it is not documented that this event flags is NOT
reported, it is better to be on the safe side and preserve the existing
behavior.

At the moment the new event FAN_ATTRIB (setattr/setxattr) behaves like
the legacy events, because like them, it reports the fid of the victim,
so user has a way to identify the victim as a directory. Not sure if
FAN_ATTRIB should behave like filename events w.r.t FAN_ONDIR and not
sure whether legacy events behavior should change with FAN_REPORT_FID.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/notify/fanotify/fanotify.c | 42 ++++++++++++++++++++++++++++++-----
 include/linux/fanotify.h      |  9 +++++---
 2 files changed, 43 insertions(+), 8 deletions(-)

diff --git a/fs/notify/fanotify/fanotify.c b/fs/notify/fanotify/fanotify.c
index a2d63e86ba47..f1a431b1a4d9 100644
--- a/fs/notify/fanotify/fanotify.c
+++ b/fs/notify/fanotify/fanotify.c
@@ -119,10 +119,12 @@ static struct dentry *fanotify_event_dentry(const void *data, int data_type)
  * requested by the user, will not be present in the returned mask.
  */
 static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
-				       u32 event_mask, const void *data,
-				       int data_type)
+				     u32 event_mask, const void *data,
+				     int data_type)
 {
 	__u32 marks_mask = 0, marks_ignored_mask = 0;
+	__u32 test_mask, user_mask = FANOTIFY_EVENT_TYPES;
+	bool is_filename = (event_mask & FANOTIFY_FILENAME_EVENTS);
 	struct dentry *dentry = fanotify_event_dentry(data, data_type);
 	struct fsnotify_mark *mark;
 	int type;
@@ -151,12 +153,42 @@ static u32 fanotify_group_event_mask(struct fsnotify_iter_info *iter_info,
 		marks_ignored_mask |= mark->ignored_mask;
 	}
 
-	if (d_is_dir(dentry) &&
+	if (is_filename) {
+		/*
+		 * For filename events (create/delete/move), we report
+		 * FAN_ONDIR for mkdir/rmdir so user can differentiate them
+		 * from creat/unlink events. Unlike legacy fanotify events
+		 * (open/access/close), mkdir/rmdir will be reported regardless
+		 * if user requested FAN_ONDIR, but the FAN_ONDIR flag itself
+		 * will only be reported if the user asked for it.
+		 */
+		user_mask |= FAN_ONDIR;
+	} else {
+		/*
+		 * For backward compatibility, do not report FAN_ONDIR to user
+		 * in legacy fanotify mode (reporting fd). While it is not
+		 * documented that this event flag is NOT reported, it is
+		 * better to be on the safe side and preserve the existing
+		 * behavior.
+		 */
+		BUILD_BUG_ON(FANOTIFY_EVENT_TYPES & FANOTIFY_EVENT_FLAGS);
+	}
+
+	/* Do not report FAN_ONDIR without an event type */
+	test_mask = event_mask & marks_mask & ~marks_ignored_mask;
+	if (!(test_mask & FANOTIFY_EVENT_TYPES))
+		return 0;
+
+	/*
+	 * Unlike legacy fanotify events (open/access/close), mkdir/rmdir
+	 * events will be reported even if FAN_ONDIR is in ignore mask, but
+	 * the FAN_ONDIR flag itself will not be reported in this case.
+	 */
+	if (d_is_dir(dentry) && !is_filename &&
 	    !(marks_mask & FS_ISDIR & ~marks_ignored_mask))
 		return 0;
 
-	return event_mask & FANOTIFY_OUTGOING_EVENTS & marks_mask &
-		~marks_ignored_mask;
+	return test_mask & user_mask;
 }
 
 static struct fanotify_event_fid *fanotify_alloc_fid(struct dentry *dentry,
diff --git a/include/linux/fanotify.h b/include/linux/fanotify.h
index fbda82b25ae3..7ccbcebc35e7 100644
--- a/include/linux/fanotify.h
+++ b/include/linux/fanotify.h
@@ -59,13 +59,16 @@
 #define FANOTIFY_PERM_EVENTS	(FAN_OPEN_PERM | FAN_ACCESS_PERM | \
 				 FAN_OPEN_EXEC_PERM)
 
+/* Events types that may be reported from vfs */
+#define FANOTIFY_EVENT_TYPES	(FANOTIFY_EVENTS | \
+				 FANOTIFY_PERM_EVENTS)
+
 /* Extra flags that may be reported with event or control handling of events */
 #define FANOTIFY_EVENT_FLAGS	(FAN_EVENT_ON_CHILD | FAN_ONDIR)
 
 /* Events that may be reported to user */
-#define FANOTIFY_OUTGOING_EVENTS	(FANOTIFY_EVENTS | \
-					 FANOTIFY_PERM_EVENTS | \
-					 FAN_Q_OVERFLOW)
+#define FANOTIFY_OUTGOING_EVENTS	(FANOTIFY_EVENT_TYPES | \
+					 FAN_Q_OVERFLOW | FAN_ONDIR)
 
 #define ALL_FANOTIFY_EVENT_BITS		(FANOTIFY_OUTGOING_EVENTS | \
 					 FANOTIFY_EVENT_FLAGS)
-- 
2.17.1

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 0/9] fanotify: add support for more event types
  2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
                   ` (8 preceding siblings ...)
  2018-11-15 18:45 ` [PATCH v2 9/9] fanotify: report FAN_ONDIR to listener for filename events Amir Goldstein
@ 2018-11-18 12:09 ` Amir Goldstein
  2018-11-20 10:34   ` Jan Kara
  9 siblings, 1 reply; 12+ messages in thread
From: Amir Goldstein @ 2018-11-18 12:09 UTC (permalink / raw)
  To: Jan Kara; +Cc: Matthew Bobrowski, linux-fsdevel, linux-api, Radu Voicilas

On Thu, Nov 15, 2018 at 8:45 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> Jan,
>
> This is the final part of patch series to add support for filesystem
> change monitoring to fanotify.
>
> The end game is to use:
>   fd = fanotify_init(FAN_CLASS_NOTIF|FAN_REPORT_FID, ...);
>   rc = fanotify_mark(fd, FAN_MARK_FILESYSTEM, FAN_CREATE|FAN_DELETE...);
> to monitor changes to a large scale namespace.
>
> This functionality was not available with inotify API, which does not
> scale well with recursive directory watches and was not available
> with fanotify API, which did not support directory modification events.
>

And to demonstrate the power unleashed by FAN_MARK_FILESYSTEM
with FAN_REPORT_FID, I have made a prototype of "global filesystem
monitor" based on inotify-tools:
https://github.com/amir73il/inotify-tools/commits/fanotify_fid

=== This is how legacy "recursive inotify" monitoring looks like: ===

~# /vtmp/inotifywait -m -r /vdf &
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
root@kvm-xfstests:~# mkdir -p /vdf/a/b/c/d/e/ && touch /vdf/a/b/c/d/e/x
/vdf/ CREATE,ISDIR a
/vdf/ OPEN,ISDIR a
/vdf/ CLOSE_NOWRITE,CLOSE,ISDIR a
/vdf/ OPEN,ISDIR a
/vdf/ ACCESS,ISDIR a
/vdf/ CLOSE_NOWRITE,CLOSE,ISDIR a
/vdf/a/b/c/d/e/ CREATE x
/vdf/a/b/c/d/e/ OPEN x
/vdf/a/b/c/d/e/ ATTRIB x
/vdf/a/b/c/d/e/ CLOSE_WRITE,CLOSE x

1. The inherent recursive watch race missed most dir create events.
2. Watches setup time depends on the size of directory tree.
3. Recursive watches pins to inode cache all directory inodes in the tree.
4. CREATE events carry the created filename information
5. Events are generated only under the watched tree

=== And this is how "global fanotify" monitoring looks like: ===

root@kvm-xfstests:~# /vtmp/inotifywait -m -g /vdf &
Setting up global filesystem watches.
Watches established.

root@kvm-xfstests:~# mkdir -p /vdf/a/b/c/d/e/ && touch /vdf/a/b/c/d/e/x

/vdf/ CREATE,ISDIR
Start watching /vdf/a (fid=10001f1...).
/vdf/a CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a CREATE,ISDIR
/vdf/a CLOSE_NOWRITE,OPEN,CLOSE
Start watching /vdf/a/b (fid=200e4a6...).
/vdf/a/b CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR
/vdf/a/b CLOSE_NOWRITE,OPEN,CLOSE
Start watching /vdf/a/b/c (fid=3000159...).
/vdf/a/b/c CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a/b/c CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a/b/c CREATE,ISDIR
Start watching /vdf/a/b/c/d (fid=105...).
/vdf/a/b/c/d OPEN
/vdf/a/b/c/d CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a/b/c/d CLOSE_NOWRITE,CLOSE
/vdf/a/b/c/d CREATE,ISDIR
Start watching /vdf/a/b/c/d/e (fid=10001f2...).
/vdf/a/b/c/d/e CREATE
Start watching /vdf/a/b/c/d/e/x (fid=10001f3...).
/vdf/a/b/c/d/e/x ATTRIB,CLOSE_WRITE,OPEN,CLOSE
/vdf/a/b/c/d/e CLOSE_NOWRITE,OPEN,CLOSE
/vdf/a/b/c/d/e/x CLOSE_NOWRITE,OPEN,CLOSE

1. No directory create/access events are missed
2. Setup time in O(1)
3. No directory inodes are pinned to inode cache
4. CREATE events carry only the directory where file
    was created and the ISDIR flag if this was a mkdir
5. Events are generated for any object in the watched
    filesystem (can be filtered in userspace by regex on path)

NOTE: "Start watching..." in global watch means that
userspace adds an entry to fid => path map, but there is
no object associated with that "watch" in the kernel and no
kernel resource is consumed per "watch".

NOTE #2: This is just a prototype. Some use cases like
monitoring several filesystems and renaming userspace
watch entries are not supported.

Thanks,
Amir.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH v2 0/9] fanotify: add support for more event types
  2018-11-18 12:09 ` [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
@ 2018-11-20 10:34   ` Jan Kara
  0 siblings, 0 replies; 12+ messages in thread
From: Jan Kara @ 2018-11-20 10:34 UTC (permalink / raw)
  To: Amir Goldstein
  Cc: Jan Kara, Matthew Bobrowski, linux-fsdevel, linux-api, Radu Voicilas

On Sun 18-11-18 14:09:34, Amir Goldstein wrote:
> On Thu, Nov 15, 2018 at 8:45 PM Amir Goldstein <amir73il@gmail.com> wrote:
> >
> > Jan,
> >
> > This is the final part of patch series to add support for filesystem
> > change monitoring to fanotify.
> >
> > The end game is to use:
> >   fd = fanotify_init(FAN_CLASS_NOTIF|FAN_REPORT_FID, ...);
> >   rc = fanotify_mark(fd, FAN_MARK_FILESYSTEM, FAN_CREATE|FAN_DELETE...);
> > to monitor changes to a large scale namespace.
> >
> > This functionality was not available with inotify API, which does not
> > scale well with recursive directory watches and was not available
> > with fanotify API, which did not support directory modification events.
> >
> 
> And to demonstrate the power unleashed by FAN_MARK_FILESYSTEM
> with FAN_REPORT_FID, I have made a prototype of "global filesystem
> monitor" based on inotify-tools:
> https://github.com/amir73il/inotify-tools/commits/fanotify_fid

This is indeed impressive :).

								Honza

> === This is how legacy "recursive inotify" monitoring looks like: ===
> 
> ~# /vtmp/inotifywait -m -r /vdf &
> Setting up watches.  Beware: since -r was given, this may take a while!
> Watches established.
> root@kvm-xfstests:~# mkdir -p /vdf/a/b/c/d/e/ && touch /vdf/a/b/c/d/e/x
> /vdf/ CREATE,ISDIR a
> /vdf/ OPEN,ISDIR a
> /vdf/ CLOSE_NOWRITE,CLOSE,ISDIR a
> /vdf/ OPEN,ISDIR a
> /vdf/ ACCESS,ISDIR a
> /vdf/ CLOSE_NOWRITE,CLOSE,ISDIR a
> /vdf/a/b/c/d/e/ CREATE x
> /vdf/a/b/c/d/e/ OPEN x
> /vdf/a/b/c/d/e/ ATTRIB x
> /vdf/a/b/c/d/e/ CLOSE_WRITE,CLOSE x
> 
> 1. The inherent recursive watch race missed most dir create events.
> 2. Watches setup time depends on the size of directory tree.
> 3. Recursive watches pins to inode cache all directory inodes in the tree.
> 4. CREATE events carry the created filename information
> 5. Events are generated only under the watched tree
> 
> === And this is how "global fanotify" monitoring looks like: ===
> 
> root@kvm-xfstests:~# /vtmp/inotifywait -m -g /vdf &
> Setting up global filesystem watches.
> Watches established.
> 
> root@kvm-xfstests:~# mkdir -p /vdf/a/b/c/d/e/ && touch /vdf/a/b/c/d/e/x
> 
> /vdf/ CREATE,ISDIR
> Start watching /vdf/a (fid=10001f1...).
> /vdf/a CLOSE_NOWRITE,OPEN,CLOSE
> /vdf/a CREATE,ISDIR
> /vdf/a CLOSE_NOWRITE,OPEN,CLOSE
> Start watching /vdf/a/b (fid=200e4a6...).
> /vdf/a/b CLOSE_NOWRITE,OPEN,CREATE,CLOSE,ISDIR
> /vdf/a/b CLOSE_NOWRITE,OPEN,CLOSE
> Start watching /vdf/a/b/c (fid=3000159...).
> /vdf/a/b/c CLOSE_NOWRITE,OPEN,CLOSE
> /vdf/a/b/c CLOSE_NOWRITE,OPEN,CLOSE
> /vdf/a/b/c CREATE,ISDIR
> Start watching /vdf/a/b/c/d (fid=105...).
> /vdf/a/b/c/d OPEN
> /vdf/a/b/c/d CLOSE_NOWRITE,OPEN,CLOSE
> /vdf/a/b/c/d CLOSE_NOWRITE,CLOSE
> /vdf/a/b/c/d CREATE,ISDIR
> Start watching /vdf/a/b/c/d/e (fid=10001f2...).
> /vdf/a/b/c/d/e CREATE
> Start watching /vdf/a/b/c/d/e/x (fid=10001f3...).
> /vdf/a/b/c/d/e/x ATTRIB,CLOSE_WRITE,OPEN,CLOSE
> /vdf/a/b/c/d/e CLOSE_NOWRITE,OPEN,CLOSE
> /vdf/a/b/c/d/e/x CLOSE_NOWRITE,OPEN,CLOSE
> 
> 1. No directory create/access events are missed
> 2. Setup time in O(1)
> 3. No directory inodes are pinned to inode cache
> 4. CREATE events carry only the directory where file
>     was created and the ISDIR flag if this was a mkdir
> 5. Events are generated for any object in the watched
>     filesystem (can be filtered in userspace by regex on path)
> 
> NOTE: "Start watching..." in global watch means that
> userspace adds an entry to fid => path map, but there is
> no object associated with that "watch" in the kernel and no
> kernel resource is consumed per "watch".
> 
> NOTE #2: This is just a prototype. Some use cases like
> monitoring several filesystems and renaming userspace
> watch entries are not supported.
> 
> Thanks,
> Amir.
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2018-11-20 21:02 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-15 18:45 [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 1/9] fanotify: rename struct fanotify_{,perm_}event_info Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 2/9] fanotify: define the structures to report a unique file identifier Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 3/9] fanotify: classify events that hold a " Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 4/9] fanotify: encode file identifier for FAN_REPORT_FID Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 5/9] fanotify: copy event fid info to user Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 6/9] fanotify: enable FAN_REPORT_FID init flag Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 7/9] fanotify: support events with data type FSNOTIFY_EVENT_DENTRY Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 8/9] fanotify: add support for create/attrib/move/delete events Amir Goldstein
2018-11-15 18:45 ` [PATCH v2 9/9] fanotify: report FAN_ONDIR to listener for filename events Amir Goldstein
2018-11-18 12:09 ` [PATCH v2 0/9] fanotify: add support for more event types Amir Goldstein
2018-11-20 10:34   ` Jan Kara

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).