All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: torvalds@linux-foundation.org
Cc: dhowells@redhat.com, viro@zeniv.linux.org.uk, dray@redhat.com,
	kzak@redhat.com, mszeredi@redhat.com, swhiteho@redhat.com,
	jlayton@redhat.com, raven@themaw.net, andres@anarazel.de,
	christian.brauner@ubuntu.com, keyrings@vger.kernel.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [GIT PULL] Mount and superblock notifications
Date: Mon, 30 Mar 2020 14:36:54 +0000	[thread overview]
Message-ID: <1449543.1585579014@warthog.procyon.org.uk> (raw)
In-Reply-To: 1445647.1585576702@warthog.procyon.org.uk

Hi Linus,

If you could consider pulling this - or would you prefer it to go through
Al?  It adds a couple of VFS-related event sources for the general
notification mechanism:

 (1) Mount topology events, such as mounting, unmounting, mount expiry,
     mount reconfiguration.

 (2) Superblock events, such as R/W<->R/O changes, quota overrun and I/O
     errors (not complete yet).


WHY
===

 (1) Mount notifications.

     This one is wanted to avoid repeated trawling of /proc/mounts or
     similar to work out changes to the mount object attributes and mount
     topology.  I'm told that the proc file holding the namespace_sem is a
     point of contention, especially as the process of generating the text
     descriptions of the mounts/superblocks can be quite involved.

     Whilst you can use poll() on /proc/mounts, it doesn't give you any
     clues as to what changed.  The notification generated here directly
     indicates the mounts involved in any particular event and gives an
     idea of what the change was.

     This is combined with a new fsinfo() system call that allows, amongst
     other things, the ability to retrieve in one go an { id,
     change_counter } tuple from all the children of a specified mount,
     allowing buffer overruns to be dealt with quickly.

     This can be used by systemd to improve efficiency:

	https://lore.kernel.org/linux-fsdevel/20200227151421.3u74ijhqt6ekbiss@ws.net.home/

     And it's not just Red Hat that's potentially interested in this:

	https://lore.kernel.org/linux-fsdevel/293c9bd3-f530-d75e-c353-ddeabac27cf6@6wind.com/

     Also, this can be used to improve management of containers by allowing
     watches to be set in foreign mount namespaces, such as are in a
     container.

 (2) Superblock notifications.

     This one is provided to allow systemd or the desktop to more easily
     detect events such as I/O errors and EDQUOT/ENOSPC.  This would be of
     interest to Postgres:

	https://lore.kernel.org/linux-fsdevel/20200211005626.7yqjf5rbs3vbwagd@alap3.anarazel.de/

     But could also be used to indicate to systemd when a superblock has
     had its configuration changed.

Thanks,
David
---
The following changes since commit 694435dbde3d1da79aafaf4cd680802f9eb229b7:

  smack: Implement the watch_key and post_notification hooks (2020-03-19 17:31:09 +0000)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/notifications-fs-20200330

for you to fetch changes up to 8dbf1aa122da5bbb4ede0f363a8a18dfc723be33:

  watch_queue: sample: Display superblock notifications (2020-03-19 17:31:09 +0000)

----------------------------------------------------------------
Filesystem notifications

----------------------------------------------------------------
David Howells (6):
      watch_queue: Add security hooks to rule on setting mount and sb watches
      watch_queue: Implement mount topology and attribute change notifications
      watch_queue: sample: Display mount tree change notifications
      watch_queue: Introduce a non-repeating system-unique superblock ID
      watch_queue: Add superblock notifications
      watch_queue: sample: Display superblock notifications

 Documentation/watch_queue.rst               |  24 ++-
 arch/alpha/kernel/syscalls/syscall.tbl      |   2 +
 arch/arm/tools/syscall.tbl                  |   2 +
 arch/arm64/include/asm/unistd.h             |   2 +-
 arch/arm64/include/asm/unistd32.h           |   4 +
 arch/ia64/kernel/syscalls/syscall.tbl       |   2 +
 arch/m68k/kernel/syscalls/syscall.tbl       |   2 +
 arch/microblaze/kernel/syscalls/syscall.tbl |   2 +
 arch/mips/kernel/syscalls/syscall_n32.tbl   |   2 +
 arch/mips/kernel/syscalls/syscall_n64.tbl   |   2 +
 arch/mips/kernel/syscalls/syscall_o32.tbl   |   2 +
 arch/parisc/kernel/syscalls/syscall.tbl     |   2 +
 arch/powerpc/kernel/syscalls/syscall.tbl    |   2 +
 arch/s390/kernel/syscalls/syscall.tbl       |   2 +
 arch/sh/kernel/syscalls/syscall.tbl         |   2 +
 arch/sparc/kernel/syscalls/syscall.tbl      |   2 +
 arch/x86/entry/syscalls/syscall_32.tbl      |   2 +
 arch/x86/entry/syscalls/syscall_64.tbl      |   2 +
 arch/xtensa/kernel/syscalls/syscall.tbl     |   2 +
 fs/Kconfig                                  |  21 +++
 fs/Makefile                                 |   1 +
 fs/internal.h                               |   1 +
 fs/mount.h                                  |  21 +++
 fs/mount_notify.c                           | 228 ++++++++++++++++++++++++++++
 fs/namespace.c                              |  22 +++
 fs/super.c                                  | 205 +++++++++++++++++++++++++
 include/linux/dcache.h                      |   1 +
 include/linux/fs.h                          |  62 ++++++++
 include/linux/lsm_hooks.h                   |  24 +++
 include/linux/security.h                    |  16 ++
 include/linux/syscalls.h                    |   4 +
 include/uapi/asm-generic/unistd.h           |   6 +-
 include/uapi/linux/watch_queue.h            |  65 +++++++-
 kernel/sys_ni.c                             |   6 +
 samples/watch_queue/watch_test.c            |  81 +++++++++-
 security/security.c                         |  14 ++
 36 files changed, 835 insertions(+), 5 deletions(-)
 create mode 100644 fs/mount_notify.c

WARNING: multiple messages have this Message-ID (diff)
From: David Howells <dhowells@redhat.com>
To: torvalds@linux-foundation.org
Cc: dhowells@redhat.com, viro@zeniv.linux.org.uk, dray@redhat.com,
	kzak@redhat.com, mszeredi@redhat.com, swhiteho@redhat.com,
	jlayton@redhat.com, raven@themaw.net, andres@anarazel.de,
	christian.brauner@ubuntu.com, keyrings@vger.kernel.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [GIT PULL] Mount and superblock notifications
Date: Mon, 30 Mar 2020 15:36:54 +0100	[thread overview]
Message-ID: <1449543.1585579014@warthog.procyon.org.uk> (raw)
In-Reply-To: 1445647.1585576702@warthog.procyon.org.uk

Hi Linus,

If you could consider pulling this - or would you prefer it to go through
Al?  It adds a couple of VFS-related event sources for the general
notification mechanism:

 (1) Mount topology events, such as mounting, unmounting, mount expiry,
     mount reconfiguration.

 (2) Superblock events, such as R/W<->R/O changes, quota overrun and I/O
     errors (not complete yet).


WHY
===

 (1) Mount notifications.

     This one is wanted to avoid repeated trawling of /proc/mounts or
     similar to work out changes to the mount object attributes and mount
     topology.  I'm told that the proc file holding the namespace_sem is a
     point of contention, especially as the process of generating the text
     descriptions of the mounts/superblocks can be quite involved.

     Whilst you can use poll() on /proc/mounts, it doesn't give you any
     clues as to what changed.  The notification generated here directly
     indicates the mounts involved in any particular event and gives an
     idea of what the change was.

     This is combined with a new fsinfo() system call that allows, amongst
     other things, the ability to retrieve in one go an { id,
     change_counter } tuple from all the children of a specified mount,
     allowing buffer overruns to be dealt with quickly.

     This can be used by systemd to improve efficiency:

	https://lore.kernel.org/linux-fsdevel/20200227151421.3u74ijhqt6ekbiss@ws.net.home/

     And it's not just Red Hat that's potentially interested in this:

	https://lore.kernel.org/linux-fsdevel/293c9bd3-f530-d75e-c353-ddeabac27cf6@6wind.com/

     Also, this can be used to improve management of containers by allowing
     watches to be set in foreign mount namespaces, such as are in a
     container.

 (2) Superblock notifications.

     This one is provided to allow systemd or the desktop to more easily
     detect events such as I/O errors and EDQUOT/ENOSPC.  This would be of
     interest to Postgres:

	https://lore.kernel.org/linux-fsdevel/20200211005626.7yqjf5rbs3vbwagd@alap3.anarazel.de/

     But could also be used to indicate to systemd when a superblock has
     had its configuration changed.

Thanks,
David
---
The following changes since commit 694435dbde3d1da79aafaf4cd680802f9eb229b7:

  smack: Implement the watch_key and post_notification hooks (2020-03-19 17:31:09 +0000)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/notifications-fs-20200330

for you to fetch changes up to 8dbf1aa122da5bbb4ede0f363a8a18dfc723be33:

  watch_queue: sample: Display superblock notifications (2020-03-19 17:31:09 +0000)

----------------------------------------------------------------
Filesystem notifications

----------------------------------------------------------------
David Howells (6):
      watch_queue: Add security hooks to rule on setting mount and sb watches
      watch_queue: Implement mount topology and attribute change notifications
      watch_queue: sample: Display mount tree change notifications
      watch_queue: Introduce a non-repeating system-unique superblock ID
      watch_queue: Add superblock notifications
      watch_queue: sample: Display superblock notifications

 Documentation/watch_queue.rst               |  24 ++-
 arch/alpha/kernel/syscalls/syscall.tbl      |   2 +
 arch/arm/tools/syscall.tbl                  |   2 +
 arch/arm64/include/asm/unistd.h             |   2 +-
 arch/arm64/include/asm/unistd32.h           |   4 +
 arch/ia64/kernel/syscalls/syscall.tbl       |   2 +
 arch/m68k/kernel/syscalls/syscall.tbl       |   2 +
 arch/microblaze/kernel/syscalls/syscall.tbl |   2 +
 arch/mips/kernel/syscalls/syscall_n32.tbl   |   2 +
 arch/mips/kernel/syscalls/syscall_n64.tbl   |   2 +
 arch/mips/kernel/syscalls/syscall_o32.tbl   |   2 +
 arch/parisc/kernel/syscalls/syscall.tbl     |   2 +
 arch/powerpc/kernel/syscalls/syscall.tbl    |   2 +
 arch/s390/kernel/syscalls/syscall.tbl       |   2 +
 arch/sh/kernel/syscalls/syscall.tbl         |   2 +
 arch/sparc/kernel/syscalls/syscall.tbl      |   2 +
 arch/x86/entry/syscalls/syscall_32.tbl      |   2 +
 arch/x86/entry/syscalls/syscall_64.tbl      |   2 +
 arch/xtensa/kernel/syscalls/syscall.tbl     |   2 +
 fs/Kconfig                                  |  21 +++
 fs/Makefile                                 |   1 +
 fs/internal.h                               |   1 +
 fs/mount.h                                  |  21 +++
 fs/mount_notify.c                           | 228 ++++++++++++++++++++++++++++
 fs/namespace.c                              |  22 +++
 fs/super.c                                  | 205 +++++++++++++++++++++++++
 include/linux/dcache.h                      |   1 +
 include/linux/fs.h                          |  62 ++++++++
 include/linux/lsm_hooks.h                   |  24 +++
 include/linux/security.h                    |  16 ++
 include/linux/syscalls.h                    |   4 +
 include/uapi/asm-generic/unistd.h           |   6 +-
 include/uapi/linux/watch_queue.h            |  65 +++++++-
 kernel/sys_ni.c                             |   6 +
 samples/watch_queue/watch_test.c            |  81 +++++++++-
 security/security.c                         |  14 ++
 36 files changed, 835 insertions(+), 5 deletions(-)
 create mode 100644 fs/mount_notify.c


  parent reply	other threads:[~2020-03-30 14:36 UTC|newest]

Thread overview: 199+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-30 13:58 Upcoming: Notifications, FS notifications and fsinfo() David Howells
2020-03-30 13:58 ` David Howells
2020-03-30 14:31 ` [GIT PULL] General notification queue and key notifications David Howells
2020-03-30 14:31   ` David Howells
2020-03-31  6:51   ` Stephen Rothwell
2020-03-31  6:51     ` Stephen Rothwell
2020-06-02 15:55   ` David Howells
2020-06-02 15:55     ` David Howells
2020-06-03  2:15     ` Ian Kent
2020-06-03  2:15       ` Ian Kent
2020-06-08  0:49       ` Ian Kent
2020-06-08  0:49         ` Ian Kent
2020-06-10  9:56     ` Christian Brauner
2020-06-10  9:56       ` Christian Brauner
2020-06-10 11:12     ` Karel Zak
2020-06-10 11:12       ` Karel Zak
2020-06-12 21:32       ` Linus Torvalds
2020-06-12 21:32         ` Linus Torvalds
2020-06-12 22:01       ` Linus Torvalds
2020-06-12 22:01         ` Linus Torvalds
2020-06-13 13:04       ` David Howells
2020-06-13 13:04         ` David Howells
2020-06-13 16:47         ` Linus Torvalds
2020-06-13 16:47           ` Linus Torvalds
2020-06-13 17:03           ` Linus Torvalds
2020-06-13 17:03             ` Linus Torvalds
2020-06-13 19:22         ` Miklos Szeredi
2020-06-13 19:22           ` Miklos Szeredi
2020-06-13 13:24       ` David Howells
2020-06-13 13:24         ` David Howells
2020-06-13 18:00     ` pr-tracker-bot
2020-06-13 18:00       ` pr-tracker-bot
2020-06-17  1:15     ` Williams, Dan J
2020-06-17  1:15       ` Williams, Dan J
2020-06-17  1:15       ` Williams, Dan J
2020-06-23 23:38       ` Dan Williams
2020-06-23 23:38         ` Dan Williams
2020-06-23 23:38         ` Dan Williams
2020-06-24  0:55       ` David Howells
2020-06-24  0:55         ` David Howells
2020-06-24  0:55         ` David Howells
2020-06-24  1:03         ` Dan Williams
2020-06-24  1:03           ` Dan Williams
2020-06-24  1:03           ` Dan Williams
2020-06-24  1:17         ` David Howells
2020-06-24  1:17           ` David Howells
2020-06-24  1:17           ` David Howells
2020-03-30 14:36 ` David Howells [this message]
2020-03-30 14:36   ` [GIT PULL] Mount and superblock notifications David Howells
2020-04-04 21:13   ` Linus Torvalds
2020-04-04 21:13     ` Linus Torvalds
2020-04-05 22:52     ` Andres Freund
2020-04-05 22:52       ` Andres Freund
2020-03-30 14:43 ` [GIT PULL] fsinfo: Filesystem information query David Howells
2020-03-30 14:43   ` David Howells
2020-03-30 20:28 ` Upcoming: Notifications, FS notifications and fsinfo() Miklos Szeredi
2020-03-30 20:28   ` Miklos Szeredi
2020-03-31  9:21   ` Karel Zak
2020-03-31  9:21     ` Karel Zak
2020-03-30 21:17 ` Christian Brauner
2020-03-30 21:17   ` Christian Brauner
2020-03-31  5:11   ` Miklos Szeredi
2020-03-31  5:11     ` Miklos Szeredi
2020-03-31  8:15     ` Christian Brauner
2020-03-31  8:15       ` Christian Brauner
2020-03-31  8:34       ` Miklos Szeredi
2020-03-31  8:34         ` Miklos Szeredi
2020-03-31  8:34     ` Karel Zak
2020-03-31  8:34       ` Karel Zak
2020-03-31  8:56       ` Miklos Szeredi
2020-03-31  8:56         ` Miklos Szeredi
2020-03-31  9:49         ` Karel Zak
2020-03-31  9:49           ` Karel Zak
2020-03-31 12:25         ` Lennart Poettering
2020-03-31 12:25           ` Lennart Poettering
2020-03-31 15:10           ` Miklos Szeredi
2020-03-31 15:10             ` Miklos Szeredi
2020-03-31 15:24             ` Lennart Poettering
2020-03-31 15:24               ` Lennart Poettering
2020-03-31 21:56         ` David Howells
2020-03-31 21:56           ` David Howells
2020-03-31 21:54     ` David Howells
2020-03-31 21:54       ` David Howells
2020-04-01  8:43       ` Karel Zak
2020-04-01  8:43         ` Karel Zak
2020-03-31  7:22   ` Lennart Poettering
2020-03-31  7:22     ` Lennart Poettering
2020-03-31 17:31 ` David Howells
2020-03-31 17:31   ` David Howells
2020-03-31 19:42   ` Miklos Szeredi
2020-03-31 19:42     ` Miklos Szeredi
2020-03-31 19:47   ` David Howells
2020-03-31 19:47     ` David Howells
2020-03-31 21:14   ` David Howells
2020-03-31 21:14     ` David Howells
2020-03-31 21:23   ` David Howells
2020-03-31 21:23     ` David Howells
2020-03-31 21:52 ` David Howells
2020-03-31 21:52   ` David Howells
2020-04-01  9:04   ` Karel Zak
2020-04-01  9:04     ` Karel Zak
2020-04-01 13:34     ` Miklos Szeredi
2020-04-01 13:34       ` Miklos Szeredi
2020-04-01 13:55     ` David Howells
2020-04-01 13:55       ` David Howells
2020-04-01 13:58     ` David Howells
2020-04-01 13:58       ` David Howells
2020-04-01 15:25       ` Miklos Szeredi
2020-04-01 15:25         ` Miklos Szeredi
2020-04-03  9:11         ` Karel Zak
2020-04-03  9:11           ` Karel Zak
2020-04-01 16:01       ` David Howells
2020-04-01 16:01         ` David Howells
2020-04-01 16:30         ` Miklos Szeredi
2020-04-01 16:30           ` Miklos Szeredi
2020-04-02 15:22         ` David Howells
2020-04-02 15:22           ` David Howells
2020-04-02 15:24           ` Miklos Szeredi
2020-04-02 15:24             ` Miklos Szeredi
2020-04-02 15:42           ` David Howells
2020-04-02 15:42             ` David Howells
2020-04-02 15:24         ` David Howells
2020-04-02 15:24           ` David Howells
2020-04-01 14:41   ` Lennart Poettering
2020-04-01 14:41     ` Lennart Poettering
2020-04-01 15:33     ` Miklos Szeredi
2020-04-01 15:33       ` Miklos Szeredi
2020-04-01 16:06     ` David Howells
2020-04-01 16:06       ` David Howells
2020-04-01 16:40       ` Miklos Szeredi
2020-04-01 16:40         ` Miklos Szeredi
2020-04-02  2:52         ` Ian Kent
2020-04-02  2:52           ` Ian Kent
2020-04-02 13:52           ` Miklos Szeredi
2020-04-02 13:52             ` Miklos Szeredi
2020-04-02 14:36             ` Lennart Poettering
2020-04-02 14:36               ` Lennart Poettering
2020-04-02 15:22               ` Miklos Szeredi
2020-04-02 15:22                 ` Miklos Szeredi
2020-04-02 15:28                 ` Lennart Poettering
2020-04-02 15:28                   ` Lennart Poettering
2020-04-02 15:35                   ` Miklos Szeredi
2020-04-02 15:35                     ` Miklos Szeredi
2020-04-02 15:50                     ` Lennart Poettering
2020-04-02 15:50                       ` Lennart Poettering
2020-04-02 17:20                       ` Miklos Szeredi
2020-04-02 17:20                         ` Miklos Szeredi
2020-04-03 11:08                         ` Lennart Poettering
2020-04-03 11:08                           ` Lennart Poettering
2020-04-03 11:48                           ` Miklos Szeredi
2020-04-03 11:48                             ` Miklos Szeredi
2020-04-03 15:01                             ` Lennart Poettering
2020-04-03 15:01                               ` Lennart Poettering
2020-04-06  9:22                               ` Miklos Szeredi
2020-04-06  9:22                                 ` Miklos Szeredi
2020-04-06 17:29                                 ` Lennart Poettering
2020-04-06 17:29                                   ` Lennart Poettering
2020-04-07  2:21                                   ` Ian Kent
2020-04-07  2:21                                     ` Ian Kent
2020-04-07 13:59                                     ` Miklos Szeredi
2020-04-07 13:59                                       ` Miklos Szeredi
2020-04-07 15:53                                       ` Lennart Poettering
2020-04-07 15:53                                         ` Lennart Poettering
2020-04-07 16:06                                         ` Miklos Szeredi
2020-04-07 16:06                                           ` Miklos Szeredi
2020-04-02 15:51                 ` David Howells
2020-04-02 15:51                   ` David Howells
2020-04-02 15:56                 ` David Howells
2020-04-02 15:56                   ` David Howells
2020-04-03  1:44             ` Ian Kent
2020-04-03  1:44               ` Ian Kent
2020-04-03 11:11               ` Lennart Poettering
2020-04-03 11:11                 ` Lennart Poettering
2020-04-03 11:38                 ` Miklos Szeredi
2020-04-03 11:38                   ` Miklos Szeredi
2020-04-03 12:05                   ` Richard Weinberger
2020-04-03 12:05                     ` Richard Weinberger
2020-04-03 15:12                   ` Lennart Poettering
2020-04-03 15:12                     ` Lennart Poettering
2020-04-03 20:30                     ` J. Bruce Fields
2020-04-03 20:30                       ` J. Bruce Fields
2020-04-06  8:35                       ` Miklos Szeredi
2020-04-06  8:35                         ` Miklos Szeredi
2020-04-06 16:07                         ` J. Bruce Fields
2020-04-06 16:07                           ` J. Bruce Fields
2020-04-06  9:17                       ` Karel Zak
2020-04-06  9:17                         ` Karel Zak
2020-04-06 16:34                         ` Linus Torvalds
2020-04-06 16:34                           ` Linus Torvalds
2020-04-06 18:46                           ` J. Bruce Fields
2020-04-06 18:46                             ` J. Bruce Fields
2020-04-06 18:48                           ` Lennart Poettering
2020-04-06 18:48                             ` Lennart Poettering
2020-04-08  3:36                             ` Linus Torvalds
2020-04-08  3:36                               ` Linus Torvalds
2020-04-03 15:36                   ` David Howells
2020-04-03 15:36                     ` David Howells
2020-04-03 15:41                     ` Lennart Poettering
2020-04-03 15:41                       ` Lennart Poettering

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1449543.1585579014@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=andres@anarazel.de \
    --cc=christian.brauner@ubuntu.com \
    --cc=dray@redhat.com \
    --cc=jlayton@redhat.com \
    --cc=keyrings@vger.kernel.org \
    --cc=kzak@redhat.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mszeredi@redhat.com \
    --cc=raven@themaw.net \
    --cc=swhiteho@redhat.com \
    --cc=torvalds@linux-foundation.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.