NFS-mounted, read-only /dev unusable in 2.6

NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Hi,

we boot diskless clients using nfsroot. The server exports
its own / with "ro,no_root_squash" and the clients mount it via
the nfsroot parameter. Then I run my own init script to mount
client specific /dev, /var, /etc all rw.

In this scenario, the 2.6.7 kernel fails to open an initial
console, since /dev is ro.

The 2.4 kernel was able to open an initial console, and
I could also echo sth. explicitely to /dev/console, even
when it was still the ro-mounted fs from the server.

With the 2.6.7 kernel, this will fail with "permission denied",
and that's why the kernel cannot open an initial console.
It will go on silently until I mount the client-specific
/dev in my init script and redirect all output.

Similar, using /dev/ram0 for creating an initial ramdisk
will fail when / is mounted ro from the server, and server
and client use both kernel 2.6.7. If the client runs
2.4, it is possible...

Is that change between 2.4 and 2.6 desired or a bug? It sounds
correct that one cannot use a node "/dev/console" for writing
if it is mounted read-only from a NFS server, but it was very
useful in 2.4.

Or is there any other way to get an initial console or
output any messages from an init script if one boots via nfsroot
and  / (and thus, /dev) is only exported read-only from the
server?

I need that to get possible error messages from my own init
script to see what fails before I mount the client /dev.

cu,
Frank
-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: NFS-mounted, read-only /dev unusable in 2.6

[Dick Streefland]

Frank Steiner <fsteiner-mail@bio.ifi.lmu.de> wrote:
| Or is there any other way to get an initial console or
| output any messages from an init script if one boots via nfsroot
| and  / (and thus, /dev) is only exported read-only from the
| server?

You can boot with a ramdisk as root, initialized with an initrd, and
then perform all NFS mounts manually in the init script. You can use
pivot_root to switch to an NFS root to get rid of the ramdisk.

-- 
Dick Streefland                      ////                      Altium BV
dick.streefland@altium.nl           (@ @)          http://www.altium.com
--------------------------------oOO--(_)--OOo---------------------------

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Dick Streefland wrote:
> Frank Steiner <fsteiner-mail@bio.ifi.lmu.de> wrote:
> | Or is there any other way to get an initial console or
> | output any messages from an init script if one boots via nfsroot
> | and  / (and thus, /dev) is only exported read-only from the
> | server?
> 
> You can boot with a ramdisk as root, initialized with an initrd, and
> then perform all NFS mounts manually in the init script. You can use
> pivot_root to switch to an NFS root to get rid of the ramdisk.

I'm hoping for an easier solution, because it's a lot of work just
to get the console messages onto the screen. But maybe I have to go
through this :-)



-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

Re: NFS-mounted, read-only /dev unusable in 2.6

[L A Walsh]

Maybe I'm missing something, but in the 2.6 series, wasn't the ability
to mount subdirectories with different options, also, added?  Would
it be possible to export and mount /dev with rw options to a specific
client?

Or, more radical, if the roots of the clients end up being mounted RW
eventually anyway, why not specify 'rw' in the lilo option?  It's not
like it is a local filesystem that may be corrupt where one should
boot from it RO until it is checked...

But I've never tried either approach, so it's really only an 
idea/possibility.

Hope something works....the booting with a local initrd seems like it
would be a pain...

good luck,
Linda

Frank Steiner wrote:

> Dick Streefland wrote:
>
>> Frank Steiner <fsteiner-mail@bio.ifi.lmu.de> wrote:
>> | Or is there any other way to get an initial console or
>> | output any messages from an init script if one boots via nfsroot
>> | and  / (and thus, /dev) is only exported read-only from the
>> | server?
>>
>> You can boot with a ramdisk as root, initialized with an initrd, and
>> then perform all NFS mounts manually in the init script. You can use
>> pivot_root to switch to an NFS root to get rid of the ramdisk.
>
>
> I'm hoping for an easier solution, because it's a lot of work just
> to get the console messages onto the screen. But maybe I have to go
> through this :-)
>
>
>

Re: NFS-mounted, read-only /dev unusable in 2.6

[Miquel van Smoorenburg]

In article <64bf.410f9d6f.62af@altium.nl>,
Dick Streefland <dick.streefland@altium.nl> wrote:
>Frank Steiner <fsteiner-mail@bio.ifi.lmu.de> wrote:
>| Or is there any other way to get an initial console or
>| output any messages from an init script if one boots via nfsroot
>| and  / (and thus, /dev) is only exported read-only from the
>| server?
>
>You can boot with a ramdisk as root, initialized with an initrd, and
>then perform all NFS mounts manually in the init script. You can use
>pivot_root to switch to an NFS root to get rid of the ramdisk.

If having /dev mounted read-only means you cannot open devices
like /dev/console read/write then that is a bug in the NFS client
in the kernel.

On all other filesystems (ext2, ext3, xfs etc) there's no problem
opening devices r/w on a read-only filesystem.

Mike.
-- 
The question is, what is a "manamanap".
The question is, who cares ?

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

L A Walsh wrote:
> Maybe I'm missing something, but in the 2.6 series, wasn't the ability
> to mount subdirectories with different options, also, added?  Would
> it be possible to export and mount /dev with rw options to a specific
> client?

Yes, that's an option. But all this needs to be done in the init script,
since with nfsroot I only get /. So mounting /dev rw from the server is
similar to figuring out which client-specific /dev should be mounted.
But in this init process things can fail and I want to see those messages.
If I fail to mount the server /dev rw for any reason, I won't see the
error message :-(

> 
> Or, more radical, if the roots of the clients end up being mounted RW
> eventually anyway, why not specify 'rw' in the lilo option?  It's not
> like it is a local filesystem that may be corrupt where one should
> boot from it RO until it is checked...

No, the real / is exported ro from the server and stays mounted ro on
the clients. Only the client specific /dev, /var and /etc are mounted
rw...

cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Miquel van Smoorenburg wrote:

> If having /dev mounted read-only means you cannot open devices
> like /dev/console read/write then that is a bug in the NFS client
> in the kernel.

Which matches the fact the it works with server running 2.6 and
client running 2.4.

> 
> On all other filesystems (ext2, ext3, xfs etc) there's no problem
> opening devices r/w on a read-only filesystem.

Should I report that as bug to someone special?

cu,
Frank


-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: NFS-mounted, read-only /dev unusable in 2.6

[Miquel van Smoorenburg]

On 2004.08.04 08:34, Frank Steiner wrote:
> Miquel van Smoorenburg wrote:
> 
> > If having /dev mounted read-only means you cannot open devices
> > like /dev/console read/write then that is a bug in the NFS client
> > in the kernel.
> 
> Which matches the fact the it works with server running 2.6 and
> client running 2.4.
> 
> > 
> > On all other filesystems (ext2, ext3, xfs etc) there's no problem
> > opening devices r/w on a read-only filesystem.
> 
> Should I report that as bug to someone special?

Assuming you have a way to reproduce this, to the NFS client
maintainer - see the file MAINTAINERS in the kernel source.

But I just tried to reproduce this on 2.6.7-rc2 (it's what my
workstation happens to be running) and I can't. I can mount an
nfs-exported /dev from both 2.4 and 2.6 servers read-only and
I can open devices on that read-only mount just fine.

Mike.
-- 
The question is, what is a "manamanap".
The question is, who cares ?

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Miquel van Smoorenburg wrote:

> Assuming you have a way to reproduce this, to the NFS client
> maintainer - see the file MAINTAINERS in the kernel source.

Thanks! I will do so.

> 
> But I just tried to reproduce this on 2.6.7-rc2 (it's what my
> workstation happens to be running) and I can't. I can mount an
> nfs-exported /dev from both 2.4 and 2.6 servers read-only and
> I can open devices on that read-only mount just fine.

It happens only when the server exports the fs read-only. When it
exports the fs rw and you mount it ro on the client, it works. But
with exporting ro, it fails.

Before I report this, I guess I should try 2.6.8-rc3 and see if it
works there.

Thanks!
cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049

Re: NFS-mounted, read-only /dev unusable in 2.6

[Neil Brown]

On Wednesday August 4, miquels@cistron.nl wrote:
> 
> On 2004.08.04 08:34, Frank Steiner wrote:
> > Miquel van Smoorenburg wrote:
> > 
> > > If having /dev mounted read-only means you cannot open devices
> > > like /dev/console read/write then that is a bug in the NFS client
> > > in the kernel.
> > 
> > Which matches the fact the it works with server running 2.6 and
> > client running 2.4.
> > 
> > > 
> > > On all other filesystems (ext2, ext3, xfs etc) there's no problem
> > > opening devices r/w on a read-only filesystem.
> > 
> > Should I report that as bug to someone special?
> 
> Assuming you have a way to reproduce this, to the NFS client
> maintainer - see the file MAINTAINERS in the kernel source.
> 
> But I just tried to reproduce this on 2.6.7-rc2 (it's what my
> workstation happens to be running) and I can't. I can mount an
> nfs-exported /dev from both 2.4 and 2.6 servers read-only and
> I can open devices on that read-only mount just fine.

Yes, it was a bug in the NFS server in 2.6 that was fixed fairly
recently.

NeilBrown

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Neil Brown wrote:

>>But I just tried to reproduce this on 2.6.7-rc2 (it's what my
>>workstation happens to be running) and I can't. I can mount an
>>nfs-exported /dev from both 2.4 and 2.6 servers read-only and
>>I can open devices on that read-only mount just fine.
> 
> 
> Yes, it was a bug in the NFS server in 2.6 that was fixed fairly
> recently.

Hi Neil,

it still occurs in 2.6.8rc3 in some way:

Server and client both running 2.6.8rc3. The client mounts the /dev
directory from the server *read-only always*:

- when the server exports its /dev ro, the client cannot echo sth.
   to the mounted dev/console
- when the server exports its /dev rw, the client can echo to dev/console,
   although it has mounted it ro.

A client running kernel 2.4 can echo to dev/console in both cases,
even with the server exporting /dev ro.

So I'm not sure if this is a client or server issue, but I guess that
Trond is reading NFS stuff here, too...


cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049
* Rekursion kann man erst verstehen, wenn man Rekursion verstanden hat. *

Re: NFS-mounted, read-only /dev unusable in 2.6

[Neil Brown]

On Wednesday August 4, fsteiner-mail@bio.ifi.lmu.de wrote:
> Neil Brown wrote:
> 
> >>But I just tried to reproduce this on 2.6.7-rc2 (it's what my
> >>workstation happens to be running) and I can't. I can mount an
> >>nfs-exported /dev from both 2.4 and 2.6 servers read-only and
> >>I can open devices on that read-only mount just fine.
> > 
> > 
> > Yes, it was a bug in the NFS server in 2.6 that was fixed fairly
> > recently.
> 
> Hi Neil,
> 
> it still occurs in 2.6.8rc3 in some way:

.... It looks like it was *so* recent that I haven't actually
submitted the patch yet :-(

I think someone reported it, I sent them a patch, and never got a
confirmation that it worked (or missed it if I did) so the patch
didn't progres..

Try the following.

Thanks,
NeilBrown


### Diffstat output
 ./fs/nfsd/vfs.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff ./fs/nfsd/vfs.c~current~ ./fs/nfsd/vfs.c
--- ./fs/nfsd/vfs.c~current~	2004-07-26 09:54:15.000000000 +1000
+++ ./fs/nfsd/vfs.c	2004-07-23 13:37:38.000000000 +1000
@@ -376,12 +376,13 @@ static struct accessmap	nfs3_anyaccess[]
 	 * to the server to check for access for things like /dev/null
 	 * (which really, the server doesn't care about).  So
 	 * We provide simple access checking for them, looking
-	 * mainly at mode bits
+	 * mainly at mode bits, and we make sure to ignore read-only
+	 * filesystem checks
 	 */
     {	NFS3_ACCESS_READ,	MAY_READ			},
     {	NFS3_ACCESS_EXECUTE,	MAY_EXEC			},
-    {	NFS3_ACCESS_MODIFY,	MAY_WRITE			},
-    {	NFS3_ACCESS_EXTEND,	MAY_WRITE			},
+    {	NFS3_ACCESS_MODIFY,	MAY_WRITE|MAY_LOCAL_ACCESS	},
+    {	NFS3_ACCESS_EXTEND,	MAY_WRITE|MAY_LOCAL_ACCESS	},
 
     {	0,			0				}
 };

Re: NFS-mounted, read-only /dev unusable in 2.6

[Frank Steiner]

Hi Neil,

Neil Brown wrote:

> .... It looks like it was *so* recent that I haven't actually
> submitted the patch yet :-(
> 
> I think someone reported it, I sent them a patch, and never got a
> confirmation that it worked (or missed it if I did) so the patch
> didn't progres..
> 
> Try the following.

it works :-)) Especially, I got my initial console back for the diskless
clients and can again see all the debugging output from my init script,
and that really really helps.

Thanks a lot for that patch!

cu,
Frank

-- 
Dipl.-Inform. Frank Steiner   Web:  http://www.bio.ifi.lmu.de/~steiner/
Lehrstuhl f. Bioinformatik    Mail: http://www.bio.ifi.lmu.de/~steiner/m/
LMU, Amalienstr. 17           Phone: +49 89 2180-4049
80333 Muenchen, Germany       Fax:   +49 89 2180-99-4049