Re: NFS client locking hangs for period

[Denis Vlasenko <vda@port.imtp.ilyichevsk.odessa.ua>]

On 26 January 2003 18:02, Christian Reis wrote:
> On Sat, Jan 25, 2003 at 02:54:09PM +1100, Neil Brown wrote:
> > Hmmm.  So you have several clients all mounting the same root
> > filesystem, and mounting it writable?  That doesn't sound like a
> > plan for success.  How do you make sure the clients don't tread
> > over each other when using /etc files?
>
> The truth is few (broken wrt the FHS) programs actually write to
> /etc. I have set up everything so nothing is written to in /etc, and
> it actually works very well (have to use a special init(8) that
> doesn't write to /etc/ioctl.save). This setup has been running for
> almost a year now, with the locking problem being the only one left
> to fix.

My root fs is RO. Works wonders. Clients simply CANNOT trash their
/bin, /lib etc ;)

> > I suspect that what you really want is to mount root read-only, or
> > mount separate roots for each client, and then in either case to
> > mount with the "nolock" flag.
>
> Well, mounting root read-only is a good idea but it sacrifices being
> able to administer the system from any station, and it also puts a
> lot of burden on me to fix *all* programs to not write to anywhere on
> it. This shouldn't be too hard, but we're still just working around
> the bug, which I would really like to identify and fix.

It was not really *that* difficult for me. I used devfs and symlinks.
/etc, /var, /tmp are different directories per client,
/home, /usr are shared. The rest stays on root fs readonly.
ssh to NFS server if you want to modify some files on root fs.

Separate etc/var/tmp files for each client = no concurrent rw access.

> > I suspect that your problem is related to the client trying to do
> > locking, but no having statd running on the client.
>
> I am 100% positive statd runs on every single client. This problem
> here only happens spuriously.  It goes away when I restart nfsd and
> mountd (in that order). It really does look like a bug <wink>

File locking over the network is hard to do reliably.
I have no experience with that in NFS, but presume there
can be problems in some situations (statd or portmap
crashed on a client, client hung/disconnected from the net,
etc etc etc...)

Anyway, such corner cases are painful, thank you for
your efforts to nail it down.
--
vda