Re: autofs no_local_binds option (nfs <-> bind mounts)

Re: autofs no_local_binds option (nfs <-> bind mounts)

[Eric Werme USG]

hpa@zytor.com wrote:

MARX,ALEXANDER (HP-Germany,ex1) wrote:
> 
>> In some scenarios (e.g. HA), the nfs server could switch from local to
>> remote, therefore having local binds is not a desirable scenario, there
>> should always be nfs mounts.
>
>How would you expect this to work?  The local bind only happens when 
>local and destination address are the same, therefore keeping anything 
>from going across the network no matter how you slice it.
>
>Changing the DNS name of the NFS server has no effect, since once the 
>mount has happened the name was already resolved, and it can't be 
>redirected.
>
>Changing the IP address runs into the problem that local == remote.

The stop-gap cluster system in Tru64 Unix did this.  Typically pairs
of servers had system names (service names in the jargon) and bound
the IP address to a NIC on one server.  When the service was relocated
manually or on a crash, the IP address was moved to a NIC on the other
server.  Disks were on a shared SCSI bus, and the file system would also
go through a umount/mount cycle.  Note that no changes to DNS' database
are necessary, just an update to clients' arp tables.

For example, we have systems "mailhub1" and "mailhub2".  The service name
"mailhub" is where Email here winds up.  I send mail via SMTP to mailhub, and
read it via NFS from mailhub.  Normally I don't care which of mailhub1
and mailhub2 handles it.  For the most part they're just servers, but
sometimes there are reasons to login to one or both of those systems.

Several vendors have similar products.  Personally, I always hated the
weird problems we'd get into on loopback mounts, like the client
deciding to flush out some pages because memory was low.  The server,
being the same system, didn't have any more memory....

One of the benefits of the loopback mounts was that unmounting wasn't
a problem as long as local access was via NFS.  Kill the NFS server,
accesses would end, unmount.  Clients would retransmit a couple times,
but things would resume quickly.

	-Ric Werme

-- 
Eric (Ric) Werme         |  werme@zk3.dec.com
Hewlett-Packard Co.      |  http://werme.8m.net/

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

Eric Werme USG wrote:
> 
> The stop-gap cluster system in Tru64 Unix did this.  Typically pairs
> of servers had system names (service names in the jargon) and bound
> the IP address to a NIC on one server.  When the service was relocated
> manually or on a crash, the IP address was moved to a NIC on the other
> server.  Disks were on a shared SCSI bus, and the file system would also
> go through a umount/mount cycle.  Note that no changes to DNS' database
> are necessary, just an update to clients' arp tables.
> 
> For example, we have systems "mailhub1" and "mailhub2".  The service name
> "mailhub" is where Email here winds up.  I send mail via SMTP to mailhub, and
> read it via NFS from mailhub.  Normally I don't care which of mailhub1
> and mailhub2 handles it.  For the most part they're just servers, but
> sometimes there are reasons to login to one or both of those systems.

However, this doesn't address the issue of the client being *the same
system*, in which case you can't just move the IP address away from it,
since local == remote; you can no longer send packets to the server and
get a response back.  You can do it if you can get the client and the
server sides to bind to *different* IP addresses, in which case the
current autofs behaviour will correctly see them as being separate and
mount NFS.

	-hpa

Re: autofs no_local_binds option (nfs <-> bind mounts)

[Dylan]

On Tuesday 13 January 2004 20:03 pm, H. Peter Anvin wrote:

> However, this doesn't address the issue of the client being *the same
> system*, in which case you can't just move the IP address away from
> it, since local == remote; you can no longer send packets to the
> server and get a response back.  You can do it if you can get the
> client and the server sides to bind to *different* IP addresses, in
> which case the current autofs behaviour will correctly see them as
> being separate and mount NFS.

Would binding an alias address to the interface be sufficient?

Dylan

-- 
Sweet moderation
Heart of this nation
Desert us not
We are between the wars
- Billy Bragg

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

Dylan wrote:
> On Tuesday 13 January 2004 20:03 pm, H. Peter Anvin wrote:
> 
> 
>>However, this doesn't address the issue of the client being *the same
>>system*, in which case you can't just move the IP address away from
>>it, since local == remote; you can no longer send packets to the
>>server and get a response back.  You can do it if you can get the
>>client and the server sides to bind to *different* IP addresses, in
>>which case the current autofs behaviour will correctly see them as
>>being separate and mount NFS.
> 
> Would binding an alias address to the interface be sufficient?
> 

No, you have to force the local port to not be bound to the same
address.  I think this can be done with iptables rules, but I'm not
sure... I'm not a networking wizard.

	-hpa

Re: autofs no_local_binds option (nfs <-> bind mounts)

[Eric Werme USG]

   Eric Werme USG wrote:
   > 
   > The stop-gap cluster system in Tru64 Unix did this.  Typically pairs
   > of servers had system names (service names in the jargon) and bound
   > the IP address to a NIC on one server.  When the service was relocated
   > manually or on a crash, the IP address was moved to a NIC on the other
   > server.  Disks were on a shared SCSI bus, and the file system would also
   > go through a umount/mount cycle.  Note that no changes to DNS' database
   > are necessary, just an update to clients' arp tables.
   > 
   > For example, we have systems "mailhub1" and "mailhub2".  The service name
   > "mailhub" is where Email here winds up.  I send mail via SMTP to mailhub, 
  and
   > read it via NFS from mailhub.  Normally I don't care which of mailhub1
   > and mailhub2 handles it.  For the most part they're just servers, but
   > sometimes there are reasons to login to one or both of those systems.
   
   However, this doesn't address the issue of the client being *the same
   system*, in which case you can't just move the IP address away from it,
   since local == remote; you can no longer send packets to the server and
   get a response back.  You can do it if you can get the client and the
   server sides to bind to *different* IP addresses, in which case the
   current autofs behaviour will correctly see them as being separate and
   mount NFS.

*the same system* as the server?  I don't know much about Linux internals,
one reason I don't post here often, but I try to offer insight to other
systems.  Tru64 Unix has a lot of BSD heritage. If mailhub1 is providing the
mailhub service and mounts something from mailhub, messages sent to mailhub
will be caught in the routing code and directed to the loopback "NIC" lo0. 
If the mailhub service (IP address) is relocated to mailhub2, the routing
code will see that no NIC on mailhub1 has the mailhub IP address and will
give the message to a NIC that can reach it.  (And ARP resolves the MAC
address and it all runs like a normal remote mount.)

Ah.  Back to automount/autofs.  I made many fixes to Sun's old automount,
one of them was to rummage among all the NICs looking to see if the
FS was really a local mount and provide the appropriate symlink.  The
cluster folks didn't realize I also checked the alias addresses too,
so I had to add an option to disable that to force a real NFS call.

You mention "you can't just move the IP address away," is that something
Linux doesn't support yet?  No problem on Tru64.  A NIC has one permanent
address and a bunch of aliases that can come and go at the whims of the
admins or load balancing software:

  # ifconfig ee0
  ee0: flags=200c63<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST,SIMPLEX,MULTINET>
       inet 16.xx.yy.213 netmask ffffff00 broadcast 16.xx.yy.255 ipmtu 1500 
       inet 16.xx.yy.192 netmask ffffff00 broadcast 16.xx.yy.255 ipmtu 1500 

  # ifconfig ee0 -alias 16.xx.yy.192

  # ifconfig ee0
  ee0: flags=c63<UP,BROADCAST,NOTRAILERS,RUNNING,MULTICAST,SIMPLEX>
       inet 16.xx.yy.213 netmask ffffff00 broadcast 16.xx.yy.255 ipmtu 1500 

	-Ric Werme
-- 
Eric (Ric) Werme         |  werme@zk3.dec.com
Hewlett-Packard Co.      |  http://werme.8m.net/

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

Eric Werme USG wrote:
> 
> *the same system* as the server?  I don't know much about Linux internals,
> one reason I don't post here often, but I try to offer insight to other
> systems.  Tru64 Unix has a lot of BSD heritage. If mailhub1 is providing the
> mailhub service and mounts something from mailhub, messages sent to mailhub
> will be caught in the routing code and directed to the loopback "NIC" lo0. 
> If the mailhub service (IP address) is relocated to mailhub2, the routing
> code will see that no NIC on mailhub1 has the mailhub IP address and will
> give the message to a NIC that can reach it.  (And ARP resolves the MAC
> address and it all runs like a normal remote mount.)
> 

That one is not a problem.  The problem is that you either need to force
the local address of the mount explicitly at the application layer (in
this case this would require a localaddr= option to mount, or something
similar) or it needs to be done by setting up the appropriate rules in
the kernel.

> Ah.  Back to automount/autofs.  I made many fixes to Sun's old automount,
> one of them was to rummage among all the NICs looking to see if the
> FS was really a local mount and provide the appropriate symlink.  The
> cluster folks didn't realize I also checked the alias addresses too,
> so I had to add an option to disable that to force a real NFS call.
> 
> You mention "you can't just move the IP address away," is that something
> Linux doesn't support yet?  No problem on Tru64.  A NIC has one permanent
> address and a bunch of aliases that can come and go at the whims of the
> admins or load balancing software:

No, the problem is: which local address will the socket be bound to.

	-hpa

Re: autofs no_local_binds option (nfs <-> bind mounts)

[Mike Waychison]

[-- Attachment #1.1: Type: text/plain, Size: 1372 bytes --]

H. Peter Anvin wrote:

>Dylan wrote:
>  
>
>>On Tuesday 13 January 2004 20:03 pm, H. Peter Anvin wrote:
>>
>>
>>    
>>
>>>However, this doesn't address the issue of the client being *the same
>>>system*, in which case you can't just move the IP address away from
>>>it, since local == remote; you can no longer send packets to the
>>>server and get a response back.  You can do it if you can get the
>>>client and the server sides to bind to *different* IP addresses, in
>>>which case the current autofs behaviour will correctly see them as
>>>being separate and mount NFS.
>>>      
>>>
>>Would binding an alias address to the interface be sufficient?
>>
>>    
>>
>
>No, you have to force the local port to not be bound to the same
>address.  I think this can be done with iptables rules, but I'm not
>sure... I'm not a networking wizard.
>
>  
>
I know you can do this using chbind from the vserver toolset and kernel 
patch.

http://www.13thfloor.at/vserver/s_release/v1.23/

-- 
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
mailto: Michael.Waychison@Sun.COM
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me, 
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 


[-- Attachment #1.2: Type: application/pgp-signature, Size: 251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
autofs mailing list
autofs@linux.kernel.org
http://linux.kernel.org/mailman/listinfo/autofs

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

Eric Werme USG wrote:
> 
> Ah.  Back to automount/autofs.  I made many fixes to Sun's old automount,
> one of them was to rummage among all the NICs looking to see if the
> FS was really a local mount and provide the appropriate symlink.  The
> cluster folks didn't realize I also checked the alias addresses too,
> so I had to add an option to disable that to force a real NFS call.
> 

Perhaps I should clarify the algorithm used by autofs: it actually goes
through and creates a socket and connects it to each of the IP addresses
for a server (it uses a UDP socket, so it doesn't actually cause any
network traffic.)  Then it queries that socket to see what the local and
remote addresses the kernel chose for the socket.  If for any of the
possible addresses <local address == remote address> then the address is
deemed local and autofs will bind-mount.  It is thus strictly based on
what the kernel would choose as the local address.  If you can force the
local address to be something other than the remote address -- as you
need for relocatability anyway -- then autofs will quite correctly avoid
bind-mounting it.

Mike raised the at least theoretical issue of what about synthetic NFS
servers in userspace and similar issues.  I'm not convinced this is an
issue in practice, but we came up with the suggestion of making an
*explicit* -fstype=nfs force NFS mounting regardless.  This has the
advantage that it cleans up the daemon somewhat; instead of:


	parse_sun
	    |
	mount_nfs
            |
	mount_bind

one would have:

	parse_sun
	    |
	mount_default
	/	\
   mount_nfs  mount_bind


	-hpa

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

Mike Waychison wrote:
>>>
>>> Would binding an alias address to the interface be sufficient?
>>>
>> No, you have to force the local port to not be bound to the same
>> address.  I think this can be done with iptables rules, but I'm not
>> sure... I'm not a networking wizard.
>>
> I know you can do this using chbind from the vserver toolset and kernel
> patch.
> 
> http://www.13thfloor.at/vserver/s_release/v1.23/

There you go, then.  Either way, autofs should respect this rule once it
is installed in the kernel.
	
	-hpa

Re: autofs no_local_binds option (nfs <-> bind mounts)

[Mike Waychison]

[-- Attachment #1.1.1: Type: text/plain, Size: 1404 bytes --]

H. Peter Anvin wrote:

> MARX,ALEXANDER (HP-Germany,ex1) wrote:
>
>>
>> In some scenarios (e.g. HA), the nfs server could switch from local to
>> remote, therefore having local binds is not a desirable scenario, there
>> should always be nfs mounts.
>>
>
> How would you expect this to work?  The local bind only happens when 
> local and destination address are the same, therefore keeping anything 
> from going across the network no matter how you slice it.
>
This is policy made by the daemon that IMHO may be trying to be too 
smart about what the user wants. I've attached a quick patch, compiled 
not tested that shows how you can add a no_local_binds option to entries 
(written against 4.0.0pre10 because that's what I have on my laptop atm).

Doing local binds may not be the right thing to do in the first place.  
It makes assumptions that the local nfs server is serving the same 
namespace/filesystems as the current application and it will break NFS 
v4 replication and migration when it's ready.

-- 
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice
mailto: Michael.Waychison@Sun.COM
http://www.sun.com

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE:  The opinions expressed in this email are held by me, 
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 


[-- Attachment #1.1.2: nolocalbinds.patch --]
[-- Type: text/plain, Size: 919 bytes --]

diff -ru orig/modules/mount_nfs.c autofs-4.0.0pre10/modules/mount_nfs.c
--- orig/modules/mount_nfs.c	2004-01-13 09:29:27.000000000 -0800
+++ autofs-4.0.0pre10/modules/mount_nfs.c	2004-01-13 09:30:06.000000000 -0800
@@ -73,6 +73,7 @@
   struct sockaddr_in saddr, laddr;
   int sock, local, err;
   int nosymlink = 0;
+  int nolocalbinds = 0;
   size_t len;
 
   syslog(LOG_DEBUG, MODPREFIX " root=%s name=%s what=%s, fstype=%s, options=%s",
@@ -113,6 +114,8 @@
 #endif
       if (strncmp("nosymlink", cp, comma-cp-1) == 0)
 	nosymlink = 1;
+      if (strncmp("no_local_binds", cp, comma-cp-1) == 0)
+	nolocalbinds = 1;
       else {
 	memcpy(nfsp, cp, comma-cp+1);
 	nfsp += comma-cp+1;
@@ -205,7 +208,7 @@
   }
   sprintf(fullpath, "%s/%s", root, name);
   
-  if ( local ) {
+  if ( local && !nolocalbinds ) {
     /* Local host -- do a "bind" */
   
     syslog(LOG_DEBUG, MODPREFIX "%s is local, doing bind", name);

[-- Attachment #1.2: Type: application/pgp-signature, Size: 251 bytes --]

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
autofs mailing list
autofs@linux.kernel.org
http://linux.kernel.org/mailman/listinfo/autofs

Re: autofs no_local_binds option (nfs <-> bind mounts)

[H. Peter Anvin]

MARX,ALEXANDER (HP-Germany,ex1) wrote:
> 
> In some scenarios (e.g. HA), the nfs server could switch from local to
> remote, therefore having local binds is not a desirable scenario, there
> should always be nfs mounts.
> 

How would you expect this to work?  The local bind only happens when 
local and destination address are the same, therefore keeping anything 
from going across the network no matter how you slice it.

Changing the DNS name of the NFS server has no effect, since once the 
mount has happened the name was already resolved, and it can't be 
redirected.

Changing the IP address runs into the problem that local == remote.

	-hpa

autofs no_local_binds option (nfs <-> bind mounts)

[MARX,ALEXANDER (HP-Germany,ex1)]

Hi list,

autofs (v4,3.1.7-425) is too intelligent ... 

Directory 'test' is exported via the nfs server on host 'foo' and imported
again via the linux automounter locally as soon as 'test' is accessed
(corresponding entry in automounter map). The automounter recognizes that
'test' is a local directory and performs a bind mount. 

Great feature - BUT, there is no way to turn off this functionality. I would
need to always have nfs mounts regardless of having the exported directory
locally or remotely.

In some scenarios (e.g. HA), the nfs server could switch from local to
remote, therefore having local binds is not a desirable scenario, there
should always be nfs mounts.

Is there some kind of workaround or possibiliy to perform such actions, or
will the automounter always be smarter that me :-) ?

Thanks,

Alex

-
Alexander Marx