linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v3 00/15] File system wide monitoring
@ 2021-06-29 19:10 Gabriel Krisman Bertazi
  2021-06-29 19:10 ` [PATCH v3 01/15] fsnotify: Don't insert unmergeable events in hashtable Gabriel Krisman Bertazi
                   ` (15 more replies)
  0 siblings, 16 replies; 61+ messages in thread
From: Gabriel Krisman Bertazi @ 2021-06-29 19:10 UTC (permalink / raw)
  To: amir73il
  Cc: djwong, tytso, david, jack, dhowells, khazhy, linux-fsdevel,
	linux-ext4, Gabriel Krisman Bertazi, kernel

Hi,

This is the third version of the FAN_FS_ERROR patches.  The main change
in this version is the inode information being reported through an FID
record, which means it requires the group to be created with
FAN_REPORT_FID.  It indeed simplifies a lot the FAN_FS_ERROR patch
itself.

This change raises the question of how we report non-inode errors.  On
one hand, we could omit the FID report, but then fsid would also be
ommited.  I chose to report these kind of errors against the root
inode.

The other changes in this iteration were made to attend to Amir
feedback.  Thank you again for your very detailed input.  It is really
appreciated.

This was tested with LTP for regressions, and also using the sample on
the last patch, with a corrupted image.  I can publish the bad image
upon request.

I also pushed the full series to:

  https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot

Thank you

Original cover letter
---------------------
Hi,

This series follow up on my previous proposal [1] to support file system
wide monitoring.  As suggested by Amir, this proposal drops the ring
buffer in favor of a single slot associated with each mark.  This
simplifies a bit the implementation, as you can see in the code.

As a reminder, This proposal is limited to an interface for
administrators to monitor the health of a file system, instead of a
generic inteface for file errors.  Therefore, this doesn't solve the
problem of writeback errors or the need to watch a specific subtree.

In comparison to the previous RFC, this implementation also drops the
per-fs data and location, and leave those as future extensions.

* Implementation

The feature is implemented on top of fanotify, as a new type of fanotify
mark, FAN_ERROR, which a file system monitoring tool can register to
receive error notifications.  When an error occurs a new notification is
generated, in addition followed by this info field:

 - FS generic data: A file system agnostic structure that has a generic
 error code and identifies the filesystem.  Basically, it let's
 userspace know something happened on a monitored filesystem.  Since
 only the first error is recorded since the last read, this also
 includes a counter of errors that happened since the last read.

* Testing

This was tested by watching notifications flowing from an intentionally
corrupted filesystem in different places.  In addition, other events
were watched in an attempt to detect regressions.

Is there a specific testsuite for fanotify I should be running?

* Patches

This patchset is divided as follows: Patch 1 through 5 are refactoring
to fsnotify/fanotify in preparation for FS_ERROR/FAN_ERROR; patch 6 and
7 implement the FS_ERROR API for filesystems to report error; patch 8
add support for FAN_ERROR in fanotify; Patch 9 is an example
implementation for ext4; patch 10 and 11 provide a sample userspace code
and documentation.

I also pushed the full series to:

  https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot

[1] https://lwn.net/Articles/854545/
[2] https://lwn.net/Articles/856916/

Cc: Darrick J. Wong <djwong@kernel.org>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Dave Chinner <david@fromorbit.com>
Cc: jack@suse.com
To: amir73il@gmail.com
Cc: dhowells@redhat.com
Cc: khazhy@google.com
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-ext4@vger.kernel.org

Amir Goldstein (1):
  fsnotify: pass arguments of fsnotify() in struct fsnotify_event_info

Gabriel Krisman Bertazi (14):
  fsnotify: Don't insert unmergeable events in hashtable
  fanotify: Fold event size calculation to its own function
  fanotify: Split fsid check from other fid mode checks
  fanotify: Split superblock marks out to a new cache
  inotify: Don't force FS_IN_IGNORED
  fsnotify: Add helper to detect overflow_event
  fsnotify: Support passing argument to insert callback on add_event
  fsnotify: Always run the merge hook
  fsnotify: Support FS_ERROR event type
  fsnotify: Introduce helpers to send error_events
  fanotify: Introduce FAN_FS_ERROR event
  ext4: Send notifications on error
  samples: Add fs error monitoring example
  docs: Document the FAN_FS_ERROR event

 .../admin-guide/filesystem-monitoring.rst     |  70 +++++++
 Documentation/admin-guide/index.rst           |   1 +
 fs/ext4/super.c                               |   8 +
 fs/notify/fanotify/fanotify.c                 | 189 ++++++++++++-----
 fs/notify/fanotify/fanotify.h                 |  69 ++++++-
 fs/notify/fanotify/fanotify_user.c            | 195 +++++++++++++++---
 fs/notify/fsnotify.c                          |  85 ++++----
 fs/notify/inotify/inotify_fsnotify.c          |   5 +-
 fs/notify/inotify/inotify_user.c              |   6 +-
 fs/notify/notification.c                      |   8 +-
 include/linux/fanotify.h                      |  11 +-
 include/linux/fsnotify.h                      |  28 ++-
 include/linux/fsnotify_backend.h              | 104 ++++++++--
 include/uapi/linux/fanotify.h                 |   8 +
 samples/Kconfig                               |   9 +
 samples/Makefile                              |   1 +
 samples/fanotify/Makefile                     |   3 +
 samples/fanotify/fs-monitor.c                 | 134 ++++++++++++
 18 files changed, 777 insertions(+), 157 deletions(-)
 create mode 100644 Documentation/admin-guide/filesystem-monitoring.rst
 create mode 100644 samples/fanotify/Makefile
 create mode 100644 samples/fanotify/fs-monitor.c

-- 
2.32.0


^ permalink raw reply	[flat|nested] 61+ messages in thread

end of thread, other threads:[~2021-07-23  1:35 UTC | newest]

Thread overview: 61+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-29 19:10 [PATCH v3 00/15] File system wide monitoring Gabriel Krisman Bertazi
2021-06-29 19:10 ` [PATCH v3 01/15] fsnotify: Don't insert unmergeable events in hashtable Gabriel Krisman Bertazi
2021-06-30  3:12   ` Amir Goldstein
2021-07-07 19:21   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 02/15] fanotify: Fold event size calculation to its own function Gabriel Krisman Bertazi
2021-07-07 19:22   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 03/15] fanotify: Split fsid check from other fid mode checks Gabriel Krisman Bertazi
2021-06-30  3:14   ` Amir Goldstein
2021-07-07 19:24   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 04/15] fanotify: Split superblock marks out to a new cache Gabriel Krisman Bertazi
2021-06-30  3:17   ` Amir Goldstein
2021-07-07 20:13   ` Jan Kara
2021-07-08  6:16     ` Amir Goldstein
2021-06-29 19:10 ` [PATCH v3 05/15] inotify: Don't force FS_IN_IGNORED Gabriel Krisman Bertazi
2021-07-07 19:37   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 06/15] fsnotify: Add helper to detect overflow_event Gabriel Krisman Bertazi
2021-07-07 20:14   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 07/15] fsnotify: pass arguments of fsnotify() in struct fsnotify_event_info Gabriel Krisman Bertazi
2021-06-29 20:39   ` kernel test robot
2021-06-30  0:10   ` kernel test robot
2021-06-30  3:29   ` Amir Goldstein
2021-06-30  8:11   ` Dan Carpenter
2021-06-30  8:35     ` Amir Goldstein
2021-06-30  8:45       ` Dan Carpenter
2021-06-30  9:32         ` Amir Goldstein
2021-06-30  9:34           ` Amir Goldstein
2021-06-30 10:49           ` Dan Carpenter
2021-06-30 12:51             ` Amir Goldstein
2021-07-08 10:43   ` Jan Kara
2021-07-08 11:09     ` Amir Goldstein
2021-07-08 11:37       ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 08/15] fsnotify: Support passing argument to insert callback on add_event Gabriel Krisman Bertazi
2021-06-30  3:40   ` Amir Goldstein
2021-07-08 10:48   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 09/15] fsnotify: Always run the merge hook Gabriel Krisman Bertazi
2021-06-30  3:42   ` Amir Goldstein
2021-06-29 19:10 ` [PATCH v3 10/15] fsnotify: Support FS_ERROR event type Gabriel Krisman Bertazi
2021-07-08 10:53   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 11/15] fsnotify: Introduce helpers to send error_events Gabriel Krisman Bertazi
2021-06-30  3:44   ` Amir Goldstein
2021-07-08 11:02   ` Jan Kara
2021-06-29 19:10 ` [PATCH v3 12/15] fanotify: Introduce FAN_FS_ERROR event Gabriel Krisman Bertazi
2021-06-30 10:26   ` Amir Goldstein
2021-06-30 17:43     ` Gabriel Krisman Bertazi
2021-07-01  6:37       ` Amir Goldstein
2021-06-30 14:03   ` Amir Goldstein
2021-06-29 19:10 ` [PATCH v3 13/15] ext4: Send notifications on error Gabriel Krisman Bertazi
2021-06-29 19:10 ` [PATCH v3 14/15] samples: Add fs error monitoring example Gabriel Krisman Bertazi
2021-06-30  2:42   ` kernel test robot
2021-07-19 14:36     ` Gabriel Krisman Bertazi
2021-07-20 19:49       ` Dan Carpenter
2021-07-22 12:54         ` Chen, Rong A
2021-07-22 16:15           ` Gabriel Krisman Bertazi
2021-07-23  1:35             ` Chen, Rong A
2021-06-30  3:46   ` kernel test robot
2021-06-30  3:58   ` Amir Goldstein
2021-06-29 19:10 ` [PATCH v3 15/15] docs: Document the FAN_FS_ERROR event Gabriel Krisman Bertazi
2021-06-30  4:18   ` Amir Goldstein
2021-06-30  5:10 ` [PATCH v3 00/15] File system wide monitoring Amir Goldstein
2021-07-08 11:32   ` Jan Kara
2021-07-08 12:25     ` Amir Goldstein

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).