metadata plugins (was Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion)

[Jeff Garzik <jeff@garzik.org>]

Hans Reiser wrote:
> Jeff Garzik wrote:
>> Plugins --do not-- mean that you can just change the filesystem format
>> willy-nilly, with zero impact.

> Yes they do.....

Take off your marketing cap for a second :)

Plugins do not eliminate the going-back-to-older-kernels issue, as 
discussed at length in the recent ext4 thread.  Plugins complicate, 
rather than simplify, the issue of determining the compatibility between 
the kernel's metadata handling code and on-disk metadata.

Quite simply, there is always a compatibility impact when the filesystem 
format changes.  Enterprise customers will inevitably run several 
-generations- of your software, and they are very aware of migration 
costs when dealing with hardware+software life cycles in the 5-10 year 
range.  Power users and kernel developers bounce between kernel versions 
all the time.

This is the same problem I described in the ext4 thread, then summarized 
as "what ext3 am I talking to, today?"  You can easily rewrite that as 
"what reiser4 set of plugins am I talking to, today?"

Plugins == a ton of new software to be versioned and tracked.  Rather 
than just track "reiser4", you must now track the collection of plugin 
versions as well.  OBVIOUS increased complexity, and OBVIOUS additional 
admin learning curve for the serious IT shop.


Now prepare for the bombshell revelation...


THAT SAID, I do think fs {algorithm|metadata} plugins are a very 
interesting idea.

The plugin infrastructure is the logical result of trying to support 
multiple incompatible {inode, dir, file data, ...} object types in the 
same Linux filesystem.  Looking at in-tree Linux filesystems, one can 
see common design patterns in filesystems that hand-code support for 
multiple metadata format versions -- Al Viro's fs/sysv hacks being a 
simple and elegant example.

It is then simple to follow that train of logic:  why not make it easy 
to replace the directory algorithm [and associated metadata]?  or the 
file data space management algorithms?  or even the inode handling?  why 
not allow customers to replace a stock algorithm with an exotic, 
site-specific one?

In essence, reiser4 is not really a filesystem, but a second VFS, with a 
few stock metadata modules.

Furthermore, it completely changes the notion of what a Linux filesystem 
is.  Currently, each Linux filesystem is a tightly constrained set of 
metadata support.  reiser4 changes "tightly constrained" to "infinity". 
  While that freedom is certainly liberating, it also has obvious 
support costs due to new admin paradigms and customer configuration 
possibilities.

I don't want to be the distro support person trying to fix a crash in 
"reiser4", where the customer has secretly replaced the standard inode 
data structure with a plugin written by an intern, and secretly replaced 
the directory algorithm with a closed source plugin from PickYourVendor. 
  Trying picking through that mess with a filesystem debugger.

The reiser4 plugin system is far better implemented as the VFS level, as 
an optional library, like fs/libfs.c.  Such a library would contain 
factored-out infrastructure common to all filesystems that support 
multiple versions of the same object type, thereby shrinking the "real" 
filesystem code.  Then add a tiny bit of code that allows addition and 
removal of object types within each filesystem.

At that point, you have reduced a Linux filesystem to binding between a 
common name ("reiser4", "ext3") and a set of cooperating modules.

A very interesting train of thought.  But not one that belongs in the 
kernel in its current form.  If you hide a second VFS inside fs/reiser4, 
I GUARANTEE that you will get the locking and multi-threading wrong.  I 
guarantee your code will be under-reviewed, and NAK'd.  You lose all the 
testing that would be gained by others travelling your code paths, if 
the code was implemented generically in fs/libplugin.c rather than 
fs/reiser4/*

To move forward towards the goal of merging reiser4 into the kernel, you 
must recognize two distinctly SEPARATE ISSUES, and deal with them 
separately:

1) merging the latest whiz-bang collection of filesystem algorithms, 
under the filesystem name "reiser4"
2) adding the plugin concept to the Linux VFS

I don't know if #2 will pass consensus, but I certainly think we should 
at least experiment in that direction.  Your guys definitely have a lot 
of good ideas, and I'm glad Linux is seeing a fresh injection of new ideas.

I just don't think its a good idea to merge a filesystem called "foo" 
that can be completely redefined in the field.  Or least, not without 
thinking a lot more on the impact.  Its a VFS _and_ a filesystem, and 
should be evaluated as such.

	Jeff