All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Lehman <dlehman@redhat.com>
To: Ray Strode <rstrode@redhat.com>, Steven Whitehouse <swhiteho@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	David Howells <dhowells@redhat.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Nicolas Dichtel <nicolas.dichtel@6wind.com>,
	raven@themaw.net, keyrings@vger.kernel.org,
	linux-usb@vger.kernel.org,
	linux-block <linux-block@vger.kernel.org>,
	Christian Brauner <christian@brauner.io>,
	LSM List <linux-security-module@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Linux API <linux-api@vger.kernel.org>,
	Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
	Ian Kent <ikent@redhat.com>
Subject: Re: Why add the general notification queue and its sources
Date: Thu, 05 Sep 2019 16:09:58 -0400	[thread overview]
Message-ID: <d19e8783e7fe47e51fbc12bf33c95fea16c93070.camel@redhat.com> (raw)
In-Reply-To: <CAKCoTu7ms4ckwDA_-onuJg+famnMzGZE9gGUcqqMz0kCAAECRg@mail.gmail.com>

On Thu, 2019-09-05 at 14:51 -0400, Ray Strode wrote:
> Hi,
> 
> On Thu, Sep 5, 2019 at 2:37 PM Steven Whitehouse <swhiteho@redhat.com
> > wrote:
> > The original reason for the mount notification mechanism was so
> > that we
> > are able to provide information to GUIs and similar filesystem and
> > storage management tools, matching the state of the filesystem with
> > the
> > state of the underlying devices. This is part of a larger project
> > entitled "Project Springfield" to try and provide better management
> > tools for storage and filesystems. I've copied David Lehman in,
> > since he
> > can provide a wider view on this topic.
> So one problem that I've heard discussed before is what happens in a
> thinp
> setup when the disk space is overallocated and gets used up. IIRC,
> the
> volumes just sort of eat themselves?
> 
> Getting proper notification of looming catastrophic failure to the
> workstation user
> before it's too late would be useful, indeed.
> 
> I don't know if this new mechanism dhowells has development can help
> with that,

My understanding is that there is already a dm devent that gets sent
when the low water mark is crossed for a thin pool, but there is
nothing in userspace that knows how to effectively get the user's
attention at that time.

> and/or if solving that problem is part of the Project Springfield
> initiative or not. Do you
> know off hand?

We have been looking into building a userspace event notification
service (for storage, initially) to aggregate and add context to low-
level events such as these, providing a single source for all kinds of
storage events with an excellent signal:noise ratio. Thin pool
exhaustion is high on the list of problems we would want to address.


David


WARNING: multiple messages have this Message-ID (diff)
From: David Lehman <dlehman@redhat.com>
To: Ray Strode <rstrode@redhat.com>, Steven Whitehouse <swhiteho@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	David Howells <dhowells@redhat.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Nicolas Dichtel <nicolas.dichtel@6wind.com>,
	raven@themaw.net, keyrings@vger.kernel.org,
	linux-usb@vger.kernel.org,
	linux-block <linux-block@vger.kernel.org>,
	Christian Brauner <christian@brauner.io>,
	LSM List <linux-security-module@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Linux API <linux-api@vger.kernel.org>,
	Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
	Ian Kent <ikent@redhat.com>
Subject: Re: Why add the general notification queue and its sources
Date: Thu, 05 Sep 2019 20:09:58 +0000	[thread overview]
Message-ID: <d19e8783e7fe47e51fbc12bf33c95fea16c93070.camel@redhat.com> (raw)
In-Reply-To: <CAKCoTu7ms4ckwDA_-onuJg+famnMzGZE9gGUcqqMz0kCAAECRg@mail.gmail.com>

On Thu, 2019-09-05 at 14:51 -0400, Ray Strode wrote:
> Hi,
> 
> On Thu, Sep 5, 2019 at 2:37 PM Steven Whitehouse <swhiteho@redhat.com
> > wrote:
> > The original reason for the mount notification mechanism was so
> > that we
> > are able to provide information to GUIs and similar filesystem and
> > storage management tools, matching the state of the filesystem with
> > the
> > state of the underlying devices. This is part of a larger project
> > entitled "Project Springfield" to try and provide better management
> > tools for storage and filesystems. I've copied David Lehman in,
> > since he
> > can provide a wider view on this topic.
> So one problem that I've heard discussed before is what happens in a
> thinp
> setup when the disk space is overallocated and gets used up. IIRC,
> the
> volumes just sort of eat themselves?
> 
> Getting proper notification of looming catastrophic failure to the
> workstation user
> before it's too late would be useful, indeed.
> 
> I don't know if this new mechanism dhowells has development can help
> with that,

My understanding is that there is already a dm devent that gets sent
when the low water mark is crossed for a thin pool, but there is
nothing in userspace that knows how to effectively get the user's
attention at that time.

> and/or if solving that problem is part of the Project Springfield
> initiative or not. Do you
> know off hand?

We have been looking into building a userspace event notification
service (for storage, initially) to aggregate and add context to low-
level events such as these, providing a single source for all kinds of
storage events with an excellent signal:noise ratio. Thin pool
exhaustion is high on the list of problems we would want to address.


David

  reply	other threads:[~2019-09-05 20:10 UTC|newest]

Thread overview: 234+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-04 22:15 [PATCH 00/11] Keyrings, Block and USB notifications [ver #8] David Howells
2019-09-04 22:15 ` David Howells
2019-09-04 22:15 ` David Howells
2019-09-04 22:15 ` [PATCH 01/11] uapi: General notification ring definitions " David Howells
2019-09-04 22:15   ` David Howells
2019-09-04 22:15   ` David Howells
2019-09-04 22:16 ` [PATCH 02/11] security: Add hooks to rule on setting a watch " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 03/11] security: Add a hook for the point of notification insertion " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 04/11] General notification queue with user mmap()'able ring buffer " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 05/11] keys: Add a notification facility " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 06/11] Add a general, global device notification watch list " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 07/11] block: Add block layer notifications " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16 ` [PATCH 08/11] usb: Add USB subsystem " David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:16   ` David Howells
2019-09-04 22:17 ` [PATCH 09/11] Add sample notification program " David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:17 ` [PATCH 10/11] selinux: Implement the watch_key security hook " David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:17 ` [PATCH 11/11] smack: Implement the watch_key and post_notification hooks " David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:17   ` David Howells
2019-09-04 22:28 ` [PATCH 00/11] Keyrings, Block and USB notifications " Linus Torvalds
2019-09-04 22:28   ` Linus Torvalds
2019-09-05 17:01 ` Why add the general notification queue and its sources David Howells
2019-09-05 17:01   ` David Howells
2019-09-05 17:19   ` Linus Torvalds
2019-09-05 17:19     ` Linus Torvalds
2019-09-05 18:32     ` Ray Strode
2019-09-05 18:32       ` Ray Strode
2019-09-05 18:32       ` Ray Strode
2019-09-05 20:39       ` Linus Torvalds
2019-09-05 20:39         ` Linus Torvalds
2019-09-06 19:32         ` Ray Strode
2019-09-06 19:32           ` Ray Strode
2019-09-06 19:41           ` Ray Strode
2019-09-06 19:41             ` Ray Strode
2019-09-06 19:53           ` Robbie Harwood
2019-09-06 19:53             ` Robbie Harwood
2019-09-06 19:53             ` Robbie Harwood
2019-09-05 21:32       ` David Howells
2019-09-05 21:32         ` David Howells
2019-09-05 22:08         ` Linus Torvalds
2019-09-05 22:08           ` Linus Torvalds
2019-09-05 23:18         ` David Howells
2019-09-05 23:18           ` David Howells
2019-09-06  0:07           ` Linus Torvalds
2019-09-06  0:07             ` Linus Torvalds
2019-09-06 10:09           ` David Howells
2019-09-06 10:09             ` David Howells
2019-09-06 15:35             ` Linus Torvalds
2019-09-06 15:35               ` Linus Torvalds
2019-09-06 15:53               ` Linus Torvalds
2019-09-06 15:53                 ` Linus Torvalds
2019-09-06 16:12                 ` Steven Whitehouse
2019-09-06 16:12                   ` Steven Whitehouse
2019-09-06 17:07                   ` Linus Torvalds
2019-09-06 17:07                     ` Linus Torvalds
2019-09-06 17:14                     ` Linus Torvalds
2019-09-06 17:14                       ` Linus Torvalds
2019-09-06 21:19                       ` David Howells
2019-09-06 21:19                         ` David Howells
2019-09-06 17:14                   ` Andy Lutomirski
2019-09-06 17:14                     ` Andy Lutomirski
2019-09-05 18:37     ` Steven Whitehouse
2019-09-05 18:37       ` Steven Whitehouse
2019-09-05 18:51       ` Ray Strode
2019-09-05 18:51         ` Ray Strode
2019-09-05 20:09         ` David Lehman [this message]
2019-09-05 20:09           ` David Lehman
2019-09-05 18:33   ` Greg Kroah-Hartman
2019-09-05 18:33     ` Greg Kroah-Hartman
     [not found] <20190903085706.7700-1-hdanton@sina.com>
2019-08-30 13:57 ` [PATCH 00/11] Keyrings, Block and USB notifications [ver #7] David Howells
2019-08-30 13:57   ` David Howells
2019-08-30 13:57   ` David Howells
2019-08-30 13:57   ` [PATCH 01/11] uapi: General notification ring definitions " David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57   ` [PATCH 02/11] security: Add hooks to rule on setting a watch " David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57   ` [PATCH 03/11] security: Add a hook for the point of notification insertion " David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57   ` [PATCH 04/11] General notification queue with user mmap()'able ring buffer " David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57   ` [PATCH 05/11] keys: Add a notification facility " David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:57     ` David Howells
2019-08-30 13:58   ` [PATCH 06/11] Add a general, global device notification watch list " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-09-03  8:34     ` Yoshihiro Shimoda
2019-09-03  8:34       ` Yoshihiro Shimoda
2019-09-03  8:34       ` Yoshihiro Shimoda
2019-09-03 16:41     ` David Howells
2019-09-03 16:41       ` David Howells
2019-09-03 16:41       ` David Howells
2019-08-30 13:58   ` [PATCH 07/11] block: Add block layer notifications " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58   ` [PATCH 08/11] usb: Add USB subsystem " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-09-03  8:53     ` Yoshihiro Shimoda
2019-09-03  8:53       ` Yoshihiro Shimoda
2019-09-03  8:53       ` Yoshihiro Shimoda
2019-09-03  9:37       ` Greg Kroah-Hartman
2019-09-03  9:37         ` Greg Kroah-Hartman
2019-09-03  9:37         ` Greg Kroah-Hartman
2019-09-04  1:53         ` Yoshihiro Shimoda
2019-09-04  1:53           ` Yoshihiro Shimoda
2019-09-04  1:53           ` Yoshihiro Shimoda
2019-09-03 12:51     ` Guenter Roeck
2019-09-03 12:51       ` Guenter Roeck
2019-09-03 16:07     ` David Howells
2019-09-03 16:07       ` David Howells
2019-09-03 16:12       ` Guenter Roeck
2019-09-03 16:12         ` Guenter Roeck
2019-09-03 16:29       ` David Howells
2019-09-03 16:29         ` David Howells
2019-09-03 17:06         ` Alan Stern
2019-09-03 17:06           ` Alan Stern
2019-09-03 17:06           ` Alan Stern
2019-09-03 17:17           ` Alan Stern
2019-09-03 17:17             ` Alan Stern
2019-09-03 17:17             ` Alan Stern
2019-09-04 15:17             ` David Howells
2019-09-04 15:17               ` David Howells
2019-08-30 13:58   ` [PATCH 09/11] Add sample notification program " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58   ` [PATCH 10/11] selinux: Implement the watch_key security hook " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 14:15     ` Stephen Smalley
2019-08-30 14:15       ` Stephen Smalley
2019-08-30 14:23     ` David Howells
2019-08-30 14:23       ` David Howells
2019-08-30 14:41     ` David Howells
2019-08-30 14:41       ` David Howells
2019-08-30 15:41       ` Stephen Smalley
2019-08-30 15:41         ` Stephen Smalley
2019-08-30 13:58   ` [PATCH 11/11] smack: Implement the watch_key and post_notification hooks [untested] " David Howells
2019-08-30 13:58     ` David Howells
2019-08-30 13:58     ` David Howells
2019-09-03 15:20     ` Casey Schaufler
2019-09-03 15:20       ` Casey Schaufler
2019-09-03 15:41     ` David Howells
2019-09-03 15:41       ` David Howells
2019-09-03 17:40       ` Casey Schaufler
2019-09-03 17:40         ` Casey Schaufler
2019-09-03 18:06       ` David Howells
2019-09-03 18:06         ` David Howells
2019-09-03 22:16         ` Casey Schaufler
2019-09-03 22:16           ` Casey Schaufler
2019-09-03 22:39         ` David Howells
2019-09-03 22:39           ` David Howells
2019-09-04 12:08         ` David Howells
2019-09-04 12:08           ` David Howells
2019-09-04 14:56           ` Casey Schaufler
2019-09-04 14:56             ` Casey Schaufler
2019-08-30 14:15   ` watch_queue(7) manpage David Howells
2019-08-30 14:15     ` David Howells
2019-08-30 14:15     ` David Howells
2019-08-30 14:15   ` watch_devices(2) manpage David Howells
2019-08-30 14:15     ` David Howells
2019-08-30 14:16   ` keyctl_watch_key.3 manpage David Howells
2019-08-30 14:16     ` David Howells
2019-08-30 22:09   ` [PATCH 00/11] Keyrings, Block and USB notifications [ver #7] Casey Schaufler
2019-08-30 22:09     ` Casey Schaufler
2019-09-02 12:39   ` David Howells
2019-09-02 12:39     ` David Howells
2019-09-02 13:26   ` David Howells
2019-09-02 13:26     ` David Howells
2019-09-03 16:06   ` [PATCH 04/11] General notification queue with user mmap()'able ring buffer " David Howells
2019-09-03 16:06     ` David Howells
2019-09-03 16:37   ` David Howells
2019-09-03 16:37     ` David Howells
  -- strict thread matches above, loose matches on Subject: below --
2019-08-29 18:29 [PATCH 00/11] Keyrings, Block and USB notifications [ver #6] David Howells
2019-08-29 18:29 ` David Howells
2019-08-29 18:29 ` David Howells
2019-08-29 18:29 ` [PATCH 01/11] uapi: General notification ring definitions " David Howells
2019-08-29 18:29   ` David Howells
2019-08-29 18:29   ` David Howells
2019-08-29 18:30 ` [PATCH 02/11] security: Add hooks to rule on setting a watch " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30 ` [PATCH 03/11] security: Add a hook for the point of notification insertion " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30 ` [PATCH 04/11] General notification queue with user mmap()'able ring buffer " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30 ` [PATCH 05/11] keys: Add a notification facility " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30 ` [PATCH 06/11] Add a general, global device notification watch list " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30 ` [PATCH 07/11] block: Add block layer notifications " David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:30   ` David Howells
2019-08-29 18:31 ` [PATCH 08/11] usb: Add USB subsystem " David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31 ` [PATCH 09/11] Add sample notification program " David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31 ` [PATCH 10/11] selinux: Implement the watch_key security hook " David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:44   ` Stephen Smalley
2019-08-29 18:44     ` Stephen Smalley
2019-08-29 19:11   ` David Howells
2019-08-29 19:11     ` David Howells
2019-08-29 18:31 ` [PATCH 11/11] smack: Implement the watch_key and post_notification hooks [untested] " David Howells
2019-08-29 18:31   ` David Howells
2019-08-29 18:31   ` David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d19e8783e7fe47e51fbc12bf33c95fea16c93070.camel@redhat.com \
    --to=dlehman@redhat.com \
    --cc=christian@brauner.io \
    --cc=dhowells@redhat.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=ikent@redhat.com \
    --cc=keyrings@vger.kernel.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-security-module@vger.kernel.org \
    --cc=linux-usb@vger.kernel.org \
    --cc=nicolas.dichtel@6wind.com \
    --cc=raven@themaw.net \
    --cc=rstrode@redhat.com \
    --cc=swhiteho@redhat.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.