[LSF/MM TOPIC] [LSF/MM ATTEND] FS Management Interfaces

[LSF/MM TOPIC] [LSF/MM ATTEND] FS Management Interfaces

[Steven Whitehouse]

Hi,

This is a request both to attend LSF/MM and also a topic proposal. The 
last couple of years I've proposed a topic of the block/fs interface. 
I'm happy to do that again, if there is consensus that this would be 
useful, however this time I thought that I'd propose something a bit 
different.

I had originally thought about calling the proposal "kernel/userland 
interface", however that seemed a bit vague and management interfaces 
seems like a better title since it is I hope a bit clearer of the kind 
of thing that I'm thinking about in this case.

There are a number of possible sub-topics and I hope that I might find a 
few more before LSF too. One is that of space management (we have 
statfs, but currently no notifications for thresholds crossed etc., so 
everything is polled. That is ok sometimes, but statfs can be expensive 
in the case of distributed filesystems, if 100% accurate. We could just 
have ENOSPC notifications for 100% full, or something more generic), 
another is state transitions (is the fs running normally, or has it gone 
read only/withdrawn/etc due to I/O errors?) and a further topic would be 
working towards a common interface for fs statistics (at the moment each 
fs defines their own interface). One potential implementation, at least 
for the first two sub-topics, would be to use something along the lines 
of the quota netlink interface, but since few ideas survive first 
contact with the community at large, I'm throwing this out for further 
discussion and feedback on whether this approach is considered the right 
way to go.

Assuming the topic is accepted, my intention would be to gather together 
some additional sub-topics relating to fs management to go along with 
those I mentioned above, and I'd be very interested to hear of any other 
issues that could be usefully added to the list for discussion.

My interest in other topics is fairly wide... I'm, as usual, interested 
in all filesystem related topics and a good number of block device and 
mm topics too. Anything relating to vfs, xfs, ext*, btrfs, gfs2, 
overlayfs, NFS/CIFS, and technologies such as copy-offload, DAX, 
reflink, RDMA, NVMe(F), etc.,

Steve.

Re: [Lsf-pc] [LSF/MM TOPIC] [LSF/MM ATTEND] FS Management Interfaces

[Jan Kara]

Hi,

On Tue 10-01-17 09:44:59, Steven Whitehouse wrote:
> I had originally thought about calling the proposal "kernel/userland
> interface", however that seemed a bit vague and management interfaces seems
> like a better title since it is I hope a bit clearer of the kind of thing
> that I'm thinking about in this case.
> 
> There are a number of possible sub-topics and I hope that I might find a few
> more before LSF too. One is that of space management (we have statfs, but
> currently no notifications for thresholds crossed etc., so everything is
> polled. That is ok sometimes, but statfs can be expensive in the case of
> distributed filesystems, if 100% accurate. We could just have ENOSPC
> notifications for 100% full, or something more generic), another is state
> transitions (is the fs running normally, or has it gone read
> only/withdrawn/etc due to I/O errors?) and a further topic would be working
> towards a common interface for fs statistics (at the moment each fs defines
> their own interface). One potential implementation, at least for the first
> two sub-topics, would be to use something along the lines of the quota
> netlink interface, but since few ideas survive first contact with the
> community at large, I'm throwing this out for further discussion and
> feedback on whether this approach is considered the right way to go.
> 
> Assuming the topic is accepted, my intention would be to gather together
> some additional sub-topics relating to fs management to go along with those
> I mentioned above, and I'd be very interested to hear of any other issues
> that could be usefully added to the list for discussion.

So this topic came up last year and probably the year before as well (heh,
I can even find some patches from 2011 [1]). I think the latest attempt at
what you suggest was here [2]. So clearly there's some interest in these
interfaces but not enough to actually drive anything to completion. So for
this topic to be useful, I think you need to go at least through the
patches in [2] and comments to them and have a concrete proposal that can
be discussed and some commitment (not necessarily from yourself) that
someone is going to devote time to implement it. Because generally nobody
seems to be opposed to the abstract idea but once it gets to the
implementation details, it is non-trivial to get some wider agreement
(statx anybody? ;)).

								Honza

[1] https://lkml.org/lkml/2011/8/18/170
[2] https://lkml.org/lkml/2015/6/16/456
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [LSF/MM TOPIC] [LSF/MM ATTEND] FS Management Interfaces

[Lukas Czerner]

On Tue, Jan 10, 2017 at 09:44:59AM +0000, Steven Whitehouse wrote:
> Hi,
> 
> This is a request both to attend LSF/MM and also a topic proposal. The last
> couple of years I've proposed a topic of the block/fs interface. I'm happy
> to do that again, if there is consensus that this would be useful, however
> this time I thought that I'd propose something a bit different.
> 
> I had originally thought about calling the proposal "kernel/userland
> interface", however that seemed a bit vague and management interfaces seems
> like a better title since it is I hope a bit clearer of the kind of thing
> that I'm thinking about in this case.
> 
> There are a number of possible sub-topics and I hope that I might find a few
> more before LSF too. One is that of space management (we have statfs, but
> currently no notifications for thresholds crossed etc., so everything is
> polled. That is ok sometimes, but statfs can be expensive in the case of
> distributed filesystems, if 100% accurate. We could just have ENOSPC
> notifications for 100% full, or something more generic), another is state
> transitions (is the fs running normally, or has it gone read
> only/withdrawn/etc due to I/O errors?) and a further topic would be working
> towards a common interface for fs statistics (at the moment each fs defines
> their own interface). One potential implementation, at least for the first
> two sub-topics, would be to use something along the lines of the quota
> netlink interface, but since few ideas survive first contact with the
> community at large, I'm throwing this out for further discussion and
> feedback on whether this approach is considered the right way to go.

Quota-like netlink interface was what I propesed in the patch [1] a
while ago. There was not any real opposition to this idea and I think
that this would be acceptable for the community and widely usefull.

However the real problem is what the interface should really look like.
I wanted to make it extendable, almost generic so we can later add more
things into it. But getting people to agree on the interface like this
will be the hardest thing.

I'd me happy to discuss this on LSF as well.

Thanks!
-Lukas


> 
> Assuming the topic is accepted, my intention would be to gather together
> some additional sub-topics relating to fs management to go along with those
> I mentioned above, and I'd be very interested to hear of any other issues
> that could be usefully added to the list for discussion.
> 
> My interest in other topics is fairly wide... I'm, as usual, interested in
> all filesystem related topics and a good number of block device and mm
> topics too. Anything relating to vfs, xfs, ext*, btrfs, gfs2, overlayfs,
> NFS/CIFS, and technologies such as copy-offload, DAX, reflink, RDMA,
> NVMe(F), etc.,
> 
> Steve.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Lsf-pc] [LSF/MM TOPIC] [LSF/MM ATTEND] FS Management Interfaces

[Steven Whitehouse]

Hi,


On 10/01/17 10:14, Jan Kara wrote:
> Hi,
>
> On Tue 10-01-17 09:44:59, Steven Whitehouse wrote:
>> I had originally thought about calling the proposal "kernel/userland
>> interface", however that seemed a bit vague and management interfaces seems
>> like a better title since it is I hope a bit clearer of the kind of thing
>> that I'm thinking about in this case.
>>
>> There are a number of possible sub-topics and I hope that I might find a few
>> more before LSF too. One is that of space management (we have statfs, but
>> currently no notifications for thresholds crossed etc., so everything is
>> polled. That is ok sometimes, but statfs can be expensive in the case of
>> distributed filesystems, if 100% accurate. We could just have ENOSPC
>> notifications for 100% full, or something more generic), another is state
>> transitions (is the fs running normally, or has it gone read
>> only/withdrawn/etc due to I/O errors?) and a further topic would be working
>> towards a common interface for fs statistics (at the moment each fs defines
>> their own interface). One potential implementation, at least for the first
>> two sub-topics, would be to use something along the lines of the quota
>> netlink interface, but since few ideas survive first contact with the
>> community at large, I'm throwing this out for further discussion and
>> feedback on whether this approach is considered the right way to go.
>>
>> Assuming the topic is accepted, my intention would be to gather together
>> some additional sub-topics relating to fs management to go along with those
>> I mentioned above, and I'd be very interested to hear of any other issues
>> that could be usefully added to the list for discussion.
> So this topic came up last year and probably the year before as well (heh,
> I can even find some patches from 2011 [1]). I think the latest attempt at
> what you suggest was here [2]. So clearly there's some interest in these
> interfaces but not enough to actually drive anything to completion. So for
> this topic to be useful, I think you need to go at least through the
> patches in [2] and comments to them and have a concrete proposal that can
> be discussed and some commitment (not necessarily from yourself) that
> someone is going to devote time to implement it. Because generally nobody
> seems to be opposed to the abstract idea but once it gets to the
> implementation details, it is non-trivial to get some wider agreement
> (statx anybody? ;)).
>
> 								Honza
>
> [1] https://lkml.org/lkml/2011/8/18/170
> [2] https://lkml.org/lkml/2015/6/16/456

Yes, statx is something else I'd like to see progress too :-) Going back 
to this topic though, I agree wrt having a concrete proposal, and I'll 
try and have something ready for LSF, we have a few weeks in hand. I'll 
collect up the details of the previous efforts (including Lukas' 
suggestion) and see how far we can get in the mean time,

Steve.