VFS mediator?

VFS mediator?

[Jonathan Barker]

Dear all

In brief: a kernel module which "exported" VFS requests to a (specified) 
user-space daemon would be useful. My particular application is a daemon 
which generates files on the fly - I would like to expose this as part of the 
filesystem. Ideally, the kernel module would deal with generation of fake 
inode numbers etc and the user-space daemon would simply be asked to create a 
pipe corresponding to a "filename" and (possibly) supply a directory tree. 

Clearly, an application-specific module could do the job - cf NFS. But I am 
not (yet) skilled enough in kernel development to know how, and it seems to 
me that a "VFS mediator" module could simplify development of many 
VFS-related applications (eg mounting ftp sites). Is there a kind soul out 
there who has already done this, or is willing to undertake such a task? 

Thanks for any input

Jonathan

Dr Jonathan Barker
EMBL-EBI Hinxton
jbarker@ebi.ac.uk

 

Re: VFS mediator?

[Simon Richter]

On Thu, 14 Mar 2002, Jonathan Barker wrote:

> In brief: a kernel module which "exported" VFS requests to a (specified)
> user-space daemon would be useful. My particular application is a daemon
> which generates files on the fly - I would like to expose this as part of the
> filesystem. Ideally, the kernel module would deal with generation of fake
> inode numbers etc and the user-space daemon would simply be asked to create a
> pipe corresponding to a "filename" and (possibly) supply a directory tree.

I have experimented with using NFS for that -- start a local daemon that
exports a virtual filesystem and mount that. The great bonus is that it's
platform independent -- it works on Solaris, HP-UX and even Ultrix just as
well. Other projects have become more important, however, and I haven't
finished it. If you're interested, drop me a line.

   Simon

-- 
GPG public key available from http://phobos.fs.tum.de/pgp/Simon.Richter.asc
 Fingerprint: 040E B5F7 84F1 4FBC CEAD  ADC6 18A0 CC8D 5706 A4B4
Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!

Re: VFS mediator?

[Alan Cox]

> I have experimented with using NFS for that -- start a local daemon that
> exports a virtual filesystem and mount that. The great bonus is that it's
> platform independent -- it works on Solaris, HP-UX and even Ultrix just as
> well. Other projects have become more important, however, and I haven't
> finished it. If you're interested, drop me a line.

There are several of these and also some folks using the coda interface
to do the same work, as the coda interface is sometimes better suited. 

Re: VFS mediator?

[Alexander Viro]

On Thu, 14 Mar 2002, Alan Cox wrote:

> > I have experimented with using NFS for that -- start a local daemon that
> > exports a virtual filesystem and mount that. The great bonus is that it's
> > platform independent -- it works on Solaris, HP-UX and even Ultrix just as
> > well. Other projects have become more important, however, and I haven't
> > finished it. If you're interested, drop me a line.
> 
> There are several of these and also some folks using the coda interface
> to do the same work, as the coda interface is sometimes better suited. 

... for some kinds of work.

First of all, "VFS mediator" is simply a userland filesystem.  That's
precisely what it is - filesystem that talks to a process.  We've got
quite a few of them and which one fits the task depends on the task.

	* NFS (v2,v3):  Portable.  And that's the only good thing to say
about it - it's stateless, it has messy semantics all over the place and
implementing userland server requires a lot of glue.
	* CIFS/SMB: even worse - it has some provisions for sane cache
coherency, but its heritage shows.  Neither OS/2 nor NT (not to mention
DOS derivatives) have a sane API and that's where the thing had come
from.  Protocol is choke-full of ugly crap and stuff that doesn't map
on UNIX (and these categories overlap).
	* NFS v4: take NFS, mix with SMB, shake, evaporate and you've got
it.  Wave Of Future(tm) according to committee that had produced it.
If they are right we'll need gas masks and dramamine.
	* NCP: Creature from Black Latrine.  As elegant and sane
as its parent company - take a lot of bitty-box ad-hackery from 80s,
let it rot for 15 years and you've got it.
	* CODA: nice if you want commit-on-close semantics and basically
want a lot of regular files.  More or less portable, userland side doesn't
require much glue.  Has a nice local caching and as the result bad for any
RPC-style uses.
	* 9P: more or less sane UNIX semantics, very well suited for
RPC-style stuff, moderate amounts of glue in clients.  Standard on Plan 9
and Inferno, has a protable UNIX client (export a subtree of existing
filesystem over 9P).  There are implementations for Linux and FreeBSD, but
both need more work.  There are provisions for local caching, but AFAIK
nobody had seriously played with them.
	* 9P2000: extended version of the above.  It would be nice if
somebody who has specifications would post them.

	Sane projects: toolkit for light-weight NFS servers; cleanup of
similar toolkit for CODA (podfuk); getting 9P implementations up to date
and giving them a decent beating (possibly - with modifications needed
for 9P2000).

Re: VFS mediator?

[Simon Richter]

On Thu, 14 Mar 2002, Alexander Viro wrote:

> 	* NFS (v2,v3):  Portable.  And that's the only good thing to say
> about it - it's stateless, it has messy semantics all over the place and
> implementing userland server requires a lot of glue.

Well, you can compile the example lines from the RFC out of the box, you
just need to implement the actual RPC functions.

> 	Sane projects: toolkit for light-weight NFS servers;

This is what I have.

   Simon

-- 
GPG public key available from http://phobos.fs.tum.de/pgp/Simon.Richter.asc
 Fingerprint: 040E B5F7 84F1 4FBC CEAD  ADC6 18A0 CC8D 5706 A4B4
Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread!

Re: VFS mediator?

[Jonathan Barker]

Dear All

On Thursday 14 Mar 2002 2:52 pm, I wrote:

> In brief: a kernel module which "exported" VFS requests to a (specified)
> user-space daemon would be useful. My particular application is a daemon
> which generates files on the fly - I would like to expose this as part of
> the filesystem. Ideally, the kernel module would deal with generation of
> fake inode numbers etc and the user-space daemon would simply be asked to
> create a pipe corresponding to a "filename" and (possibly) supply a
> directory tree.

Many thanks for all your useful suggestions. I'll look at them all and (when 
I get time) compile a digest of answers. I'm particularly amused by the very 
groovy perlfs (thanks to David Golden for pointing me towards it). 

Jonathan

Re: VFS mediator?

[Pavel Machek]

Hi!

> > > I have experimented with using NFS for that -- start a local daemon that
> > > exports a virtual filesystem and mount that. The great bonus is that it's
> > > platform independent -- it works on Solaris, HP-UX and even Ultrix just as
> > > well. Other projects have become more important, however, and I haven't
> > > finished it. If you're interested, drop me a line.
> > 
> > There are several of these and also some folks using the coda interface
> > to do the same work, as the coda interface is sometimes better suited. 
> 
> ... for some kinds of work.
> 
> First of all, "VFS mediator" is simply a userland filesystem.  That's
> precisely what it is - filesystem that talks to a process.  We've got
> quite a few of them and which one fits the task depends on the task.
> 
> 	* NFS (v2,v3):  Portable.  And that's the only good thing to say
> about it - it's stateless, it has messy semantics all over the place and
> implementing userland server requires a lot of glue.

Does not work... If you mount nfs server on localhost, you can deadlock.

> 	* CODA: nice if you want commit-on-close semantics and basically
> want a lot of regular files.  More or less portable, userland side doesn't
> require much glue.  Has a nice local caching and as the result bad for any
> RPC-style uses.

And the only one that works when r/w mounted on localhost.
									Pavel
-- 
(about SSSCA) "I don't say this lightly.  However, I really think that the U.S.
no longer is classifiable as a democracy, but rather as a plutocracy." --hpa

Re: VFS mediator?

[Trond Myklebust]

>>>>> " " == Pavel Machek <pavel@suse.cz> writes:

    >> * NFS (v2,v3): Portable.  And that's the only good thing to say
    >> about it - it's stateless, it has messy semantics all over the
    >> place and implementing userland server requires a lot of glue.

     > Does not work... If you mount nfs server on localhost, you can
     > deadlock.

Huh? Examples please? A hell of a lot of work has gone into ensuring
that this cannot happen. I do most of my NFS client work on this sort
of setup, so it had bloody well better work...

Cheers,
  Trond

Re: VFS mediator?

[Alexander Viro]

On Mon, 18 Mar 2002, Pavel Machek wrote:

> > 	* CODA: nice if you want commit-on-close semantics and basically
> > want a lot of regular files.  More or less portable, userland side doesn't
> > require much glue.  Has a nice local caching and as the result bad for any
> > RPC-style uses.
> 
> And the only one that works when r/w mounted on localhost.

Wrong.  Trivial example: filesystem that doesn't cache data (and has no
mmap()).

Re: VFS mediator?

[Pavel Machek]

Hi!

> > > 	* CODA: nice if you want commit-on-close semantics and basically
> > > want a lot of regular files.  More or less portable, userland side doesn't
> > > require much glue.  Has a nice local caching and as the result bad for any
> > > RPC-style uses.
> > 
> > And the only one that works when r/w mounted on localhost.
> 
> Wrong.  Trivial example: filesystem that doesn't cache data (and has no
> mmap()).

I meant only one from the list. NFS certainly has mmap, and others
have it, too...
									Pavel

-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: VFS mediator?

[Pavel Machek]

Hi!

>     >> * NFS (v2,v3): Portable.  And that's the only good thing to say
>     >> about it - it's stateless, it has messy semantics all over the
>     >> place and implementing userland server requires a lot of glue.
> 
>      > Does not work... If you mount nfs server on localhost, you can
>      > deadlock.
> 
> Huh? Examples please? A hell of a lot of work has gone into ensuring
> that this cannot happen. I do most of my NFS client work on this sort
> of setup, so it had bloody well better work...

Okay, take userland nfs-server. (This thread was about userland
filesystems).

Then, make memory full of dirty pages. Imagine that nfs-server is
swapped-out by some bad luck. What you have is extremely nasty
deadlock, AFAICS. [To free memory you have to write out dirty data,
but you can't do that because you don't have enough memory for
nfs-server].
								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: VFS mediator?

[Trond Myklebust]

>>>>> " " == Pavel Machek <pavel@suse.cz> writes:

     > Okay, take userland nfs-server. (This thread was about userland
     > filesystems).

Yech... Nobody should be seriously considering using unfsd: it does
not even manage to follow the NFS protocol. That inability was one of
the many reasons why Olaf Kirch abandoned further develpement of unfsd
and started work on knfsd.

     > Then, make memory full of dirty pages. Imagine that nfs-server
     > is swapped-out by some bad luck. What you have is extremely
     > nasty deadlock, AFAICS. [To free memory you have to write out
     > dirty data, but you can't do that because you don't have enough
     > memory for nfs-server].

So that is another argument for using knfsd rather than unfsd. I will
agree with you that NFS is not perfect, but please judge it on its
actual merits and not on some trumped up charge...

Cheers,
  Trond

Re: VFS mediator?

[Pavel Machek]

Hi!

>      > Okay, take userland nfs-server. (This thread was about userland
>      > filesystems).
> 
> Yech... Nobody should be seriously considering using unfsd: it does
> not even manage to follow the NFS protocol. That inability was one of
> the many reasons why Olaf Kirch abandoned further develpement of unfsd
> and started work on knfsd.
> 
>      > Then, make memory full of dirty pages. Imagine that nfs-server
>      > is swapped-out by some bad luck. What you have is extremely
>      > nasty deadlock, AFAICS. [To free memory you have to write out
>      > dirty data, but you can't do that because you don't have enough
>      > memory for nfs-server].
> 
> So that is another argument for using knfsd rather than unfsd. I will
> agree with you that NFS is not perfect, but please judge it on its
> actual merits and not on some trumped up charge...

Sorry, this thread was about userland filesystems, and NFS is just not
usefull there (for read/write case).

								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: VFS mediator?

[Trond Myklebust]

>>>>> " " == Pavel Machek <pavel@suse.cz> writes:

     > Sorry, this thread was about userland filesystems, and NFS is
     > just not usefull there (for read/write case).

Nope. The point made in Alan's mail early on in the thread was that of
platform independence: the latter has nothing to do with userland
implementation or not. In fact, several of the filesystems Al
mentioned had no (known) userland implementation.

Cheers,
  Trond

Re: VFS mediator?

[Alan Cox]

>      > Does not work... If you mount nfs server on localhost, you can
>      > deadlock.
> 
> Huh? Examples please? A hell of a lot of work has gone into ensuring
> that this cannot happen. I do most of my NFS client work on this sort
> of setup, so it had bloody well better work...

At least theoretically it can. Imagine you have every other process stuck
trying to page out (or blocked on a page out) over NFS, including your
user mode nfs process. In practice it would be very hard to arrange but
the theory is real.

Alan

Re: VFS mediator?

[Pavel Machek]

Hi!

> > >      > Okay, take userland nfs-server. (This thread was about userland
> > >      > filesystems).
> > > 
> > > Yech... Nobody should be seriously considering using unfsd: it does
> > > not even manage to follow the NFS protocol. That inability was one of
> > > the many reasons why Olaf Kirch abandoned further develpement of unfsd
> > > and started work on knfsd.
> > > 
> > >      > Then, make memory full of dirty pages. Imagine that nfs-server
> > >      > is swapped-out by some bad luck. What you have is extremely
> > >      > nasty deadlock, AFAICS. [To free memory you have to write out
> > >      > dirty data, but you can't do that because you don't have enough
> > >      > memory for nfs-server].
> > > 
> > > So that is another argument for using knfsd rather than unfsd. I will
> > > agree with you that NFS is not perfect, but please judge it on its
> > > actual merits and not on some trumped up charge...
> > 
> > Sorry, this thread was about userland filesystems, and NFS is just not
> > usefull there (for read/write case).
> 
> Assuming, of course, that the daemon doesn't mprotect itself...

Even if it did, I'm not sure it would be safe. write() may need some
memory, too.

> A user mode file system is really only good at debugging a design.

Not agreed. I would not want ftpfs in kernel, yet its happy in
userland.

> All file migration style filesystems, and user mode filesystems, have this
> same problem on paging based systems:
> 
> Can't write buffer until file is migrated (file system full),

Well, filesystem full is nasty case. [I wonder how coda solves that?]
								Pavel
-- 
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

Re: VFS mediator?

[Jesse Pollard]

Pavel Machek <pavel@suse.cz>:
> Hi!
> 
> >      > Okay, take userland nfs-server. (This thread was about userland
> >      > filesystems).
> > 
> > Yech... Nobody should be seriously considering using unfsd: it does
> > not even manage to follow the NFS protocol. That inability was one of
> > the many reasons why Olaf Kirch abandoned further develpement of unfsd
> > and started work on knfsd.
> > 
> >      > Then, make memory full of dirty pages. Imagine that nfs-server
> >      > is swapped-out by some bad luck. What you have is extremely
> >      > nasty deadlock, AFAICS. [To free memory you have to write out
> >      > dirty data, but you can't do that because you don't have enough
> >      > memory for nfs-server].
> > 
> > So that is another argument for using knfsd rather than unfsd. I will
> > agree with you that NFS is not perfect, but please judge it on its
> > actual merits and not on some trumped up charge...
> 
> Sorry, this thread was about userland filesystems, and NFS is just not
> usefull there (for read/write case).

Assuming, of course, that the daemon doesn't mprotect itself...

A user mode file system is really only good at debugging a design.

All file migration style filesystems, and user mode filesystems, have this
same problem on paging based systems:

Can't write buffer until file is migrated (file system full),
Can't migrate file until buffer memory is freed....
system hung...

Although it is usually possible to detect this deadlock and abort some
process, freeing memory (and sometimes disk space at the same time).

Swapping systems can have the equivalent problem if swap space is
oversubscribed.

The problem boils down to the same solution - don't oversubscribe memory...

I know this is a bit of a troll, but memory controls are necessary to
detect, avoid, or repair the situation.
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.