From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,UNPARSEABLE_RELAY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC2CDC11F69 for ; Tue, 29 Jun 2021 19:10:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C55A661DD6 for ; Tue, 29 Jun 2021 19:10:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235452AbhF2TNQ (ORCPT ); Tue, 29 Jun 2021 15:13:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233916AbhF2TNQ (ORCPT ); Tue, 29 Jun 2021 15:13:16 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D8AA7C061760; Tue, 29 Jun 2021 12:10:48 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: krisman) with ESMTPSA id 5E9E21F431AF From: Gabriel Krisman Bertazi To: amir73il@gmail.com Cc: djwong@kernel.org, tytso@mit.edu, david@fromorbit.com, jack@suse.com, dhowells@redhat.com, khazhy@google.com, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, Gabriel Krisman Bertazi , kernel@collabora.com Subject: [PATCH v3 00/15] File system wide monitoring Date: Tue, 29 Jun 2021 15:10:20 -0400 Message-Id: <20210629191035.681913-1-krisman@collabora.com> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Hi, This is the third version of the FAN_FS_ERROR patches. The main change in this version is the inode information being reported through an FID record, which means it requires the group to be created with FAN_REPORT_FID. It indeed simplifies a lot the FAN_FS_ERROR patch itself. This change raises the question of how we report non-inode errors. On one hand, we could omit the FID report, but then fsid would also be ommited. I chose to report these kind of errors against the root inode. The other changes in this iteration were made to attend to Amir feedback. Thank you again for your very detailed input. It is really appreciated. This was tested with LTP for regressions, and also using the sample on the last patch, with a corrupted image. I can publish the bad image upon request. I also pushed the full series to: https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot Thank you Original cover letter --------------------- Hi, This series follow up on my previous proposal [1] to support file system wide monitoring. As suggested by Amir, this proposal drops the ring buffer in favor of a single slot associated with each mark. This simplifies a bit the implementation, as you can see in the code. As a reminder, This proposal is limited to an interface for administrators to monitor the health of a file system, instead of a generic inteface for file errors. Therefore, this doesn't solve the problem of writeback errors or the need to watch a specific subtree. In comparison to the previous RFC, this implementation also drops the per-fs data and location, and leave those as future extensions. * Implementation The feature is implemented on top of fanotify, as a new type of fanotify mark, FAN_ERROR, which a file system monitoring tool can register to receive error notifications. When an error occurs a new notification is generated, in addition followed by this info field: - FS generic data: A file system agnostic structure that has a generic error code and identifies the filesystem. Basically, it let's userspace know something happened on a monitored filesystem. Since only the first error is recorded since the last read, this also includes a counter of errors that happened since the last read. * Testing This was tested by watching notifications flowing from an intentionally corrupted filesystem in different places. In addition, other events were watched in an attempt to detect regressions. Is there a specific testsuite for fanotify I should be running? * Patches This patchset is divided as follows: Patch 1 through 5 are refactoring to fsnotify/fanotify in preparation for FS_ERROR/FAN_ERROR; patch 6 and 7 implement the FS_ERROR API for filesystems to report error; patch 8 add support for FAN_ERROR in fanotify; Patch 9 is an example implementation for ext4; patch 10 and 11 provide a sample userspace code and documentation. I also pushed the full series to: https://gitlab.collabora.com/krisman/linux -b fanotify-notifications-single-slot [1] https://lwn.net/Articles/854545/ [2] https://lwn.net/Articles/856916/ Cc: Darrick J. Wong Cc: Theodore Ts'o Cc: Dave Chinner Cc: jack@suse.com To: amir73il@gmail.com Cc: dhowells@redhat.com Cc: khazhy@google.com Cc: linux-fsdevel@vger.kernel.org Cc: linux-ext4@vger.kernel.org Amir Goldstein (1): fsnotify: pass arguments of fsnotify() in struct fsnotify_event_info Gabriel Krisman Bertazi (14): fsnotify: Don't insert unmergeable events in hashtable fanotify: Fold event size calculation to its own function fanotify: Split fsid check from other fid mode checks fanotify: Split superblock marks out to a new cache inotify: Don't force FS_IN_IGNORED fsnotify: Add helper to detect overflow_event fsnotify: Support passing argument to insert callback on add_event fsnotify: Always run the merge hook fsnotify: Support FS_ERROR event type fsnotify: Introduce helpers to send error_events fanotify: Introduce FAN_FS_ERROR event ext4: Send notifications on error samples: Add fs error monitoring example docs: Document the FAN_FS_ERROR event .../admin-guide/filesystem-monitoring.rst | 70 +++++++ Documentation/admin-guide/index.rst | 1 + fs/ext4/super.c | 8 + fs/notify/fanotify/fanotify.c | 189 ++++++++++++----- fs/notify/fanotify/fanotify.h | 69 ++++++- fs/notify/fanotify/fanotify_user.c | 195 +++++++++++++++--- fs/notify/fsnotify.c | 85 ++++---- fs/notify/inotify/inotify_fsnotify.c | 5 +- fs/notify/inotify/inotify_user.c | 6 +- fs/notify/notification.c | 8 +- include/linux/fanotify.h | 11 +- include/linux/fsnotify.h | 28 ++- include/linux/fsnotify_backend.h | 104 ++++++++-- include/uapi/linux/fanotify.h | 8 + samples/Kconfig | 9 + samples/Makefile | 1 + samples/fanotify/Makefile | 3 + samples/fanotify/fs-monitor.c | 134 ++++++++++++ 18 files changed, 777 insertions(+), 157 deletions(-) create mode 100644 Documentation/admin-guide/filesystem-monitoring.rst create mode 100644 samples/fanotify/Makefile create mode 100644 samples/fanotify/fs-monitor.c -- 2.32.0