Re: [proposal] making filesystem tools more machine friendly

[Theodore Ts'o <tytso@mit.edu>]

I'm only replying to linux-fsdevel@ because I think that's really the
right list for us to be having this discussion.

So a couple of comments.  First of all, this sort of thing has been
proposed before.  Specifically, this was one of the problems which
EVMS (from IBM, which ultimately lost out to device mapper) had as
part of their project.  Their userspace component has a plug-in
architecture so that file systems could provide a shared library which
could be used by GUI and CLI tools.

I had that support in e2fsprogs, and it was removed when EVMS was
ultimately killed off by device-mapper and LVM.  The commit removal
was in commit 921f4ad53: "Remove support for EVMS 1.x plugin library"
so if you look at the sources in lib/evms in the e2fsprogs git repo at
commit 921f4ad53 (aka 921f4ad53^), you can see what it looked like.

Secondly, as a file system developer and maintainer for e2fsprogs, I'm
going to be extremely hesitant to accept patches which radically
reorganize e2fsprogs and which adds dependencies on third-party
libraries that emit JSON output before I know whether or not your
project is going to be successful, or will ultimately end up getting
abandoned and left for dead, as was the case for EVMS.

That being said, I probably would be willing to accept a drop-in
library, ala lib/evms, where some of the functions were handled
directly via accessing the file system directly via the plugin
library, and where some of the changes were made by wrapping mke2fs
and e2fsck.  Over time, *if* this interface gained legs, I'd be more
willing to try to make a library version of mke2fs (and then _maybe_
e2fsck) so that the plugin library could do more of the work natively,
rather than by wrapping the mke2fs and e2fsck executables.

Third, I think you are *massively* underestimating how much work is
needed to do this in a generic fashion, especially with the file
system check tool.  In any tool where you need to ask the user for
permission to make a particular change to the file system, trying to
do this in a generic way is *hard*, and the "use JSON" is not enough.

At the same time, for the common case, where you just want to "check
to see if the file system is consistent, and return a boolean", or
"check the file system and do all of the safe things", you don't need
all of the complexity of JSON.  So I think JSON is simultaneously too
much (for the simple stuff, where the user doesn't need to see the
output anyway), and too little (for the hard cases where you are doing
a file system repair operation).

For another matter, how you would pass in mkfs parameters, which are
very file system specific, in a completely generic way, is also
completely unspecified in your proposal.  This makes me deeply
suspicious you haven't thought through the issues sufficiently, and if
you try to send patches that massively reorganize e2fsprogs and adds
JSON output in what I suspect will be a horrible, terribly ugly way,
it's most likely I will decline to accept your patches.

This probably means that you will need to do something which starts by
using screen scraping at least initially, and if the maintainers
aren't willing to adopt your code, you're going to have to maintain it
yourself.  That's how EVMS started, and then they asked if I would be
willing to integrate the ext2/3 (this was before ext4) evms plugin
into e2fsprogs, and would I be willing to start making changes to
integrate it more organically into e2fsprogs.  I suspect that kind of
intecremental approach is much more likely to be successful in the
long term.  You may also want to look at the EVMS plugin interfaces,
since the people who tried this did think about what made sense to try
to do in a file system independent way, and what would probably have
to be kept file system specific.

Cheers,

						- Ted