[PATCH (RESEND) 00/12] Support binding nfs/rpc to local IP address.

[PATCH (RESEND) 00/12] Support binding nfs/rpc to local IP address.

[greearb]

From: Ben Greear <greearb@candelatech.com>


This patch series allows binding the nfs and rpc logic to a specific local
IP address.

Features/Benefits:

*  Allow multiple unique mounts to the same server using unique
   source IP addresses on client system.

   Routing rules can then be created based on the source IP address for
   advanced routing of the NFS traffic.

   This could allow someone to use two 1G interfaces to access a
   server with 10G connectivity and allow aggregate 2Gbps transfer,
   for instance.

   This is also useful for load testing NFS servers as well as the
   client-side NFS logic (note the bugs & fixed found while testing
   this code!)

*  Allow using a specific IP address on multi-homed system.  This could
   increase security in some cases and in general gives the user more
   control over how the services are configured.

This code has been tested under load on 3.0 and a similar version has
been tested under load on 2.6.38 and previous kernels.

Full implementation of this feature requires patches to mount.nfs,
which have been agreed to be accepted if the kernel patches are accepted.

Ben Greear (12):
  sunrpc:  Don't attempt to bind to AF_UNSPEC address.
  nfs:  Two AF_UNSPEC addresses should always match each other.
  nfs:  Add srcaddr member to nfs_client.
  nfs:  Use request destination addr as callback source addr.
  nfs:  Pay attention to srcaddr in v4.1 callback logic.
  nfs:  Use srcaddr in nfs_match_client.
  nfs:  Add srcaddr to /proc/fs/nfsfs/servers
  nfs:  Pass srcaddr into mount request.
  nfs:  Propagate src-addr in client code.
  nfs: Bind to srcaddr in rpcb_create.
  nfs: Support srcaddr= to bind to specific IP address.
  lockd: Support binding nlm client to specific address.

 fs/lockd/clntlock.c         |    3 +-
 fs/lockd/host.c             |   17 +++++++++-
 fs/nfs/callback.h           |    3 ++
 fs/nfs/callback_proc.c      |    3 +-
 fs/nfs/callback_xdr.c       |    3 ++
 fs/nfs/client.c             |   73 ++++++++++++++++++++++++++++++++++++++++--
 fs/nfs/internal.h           |   10 +++++-
 fs/nfs/mount_clnt.c         |    1 +
 fs/nfs/super.c              |   33 +++++++++++++++++++-
 include/linux/lockd/bind.h  |    1 +
 include/linux/lockd/lockd.h |    1 +
 include/linux/nfs_fs_sb.h   |    4 ++
 include/linux/sunrpc/clnt.h |    2 +
 include/linux/sunrpc/svc.h  |    5 +++
 net/sunrpc/rpcb_clnt.c      |   28 +++++++++++++++-
 net/sunrpc/sunrpc.h         |   45 ++++++++++++++++++++++++++
 net/sunrpc/xprtsock.c       |   47 +--------------------------
 17 files changed, 223 insertions(+), 56 deletions(-)

-- 
1.7.3.4

[PATCH (RESEND) 01/12] sunrpc: Don't attempt to bind to AF_UNSPEC address.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This lets calling code simply pass AF_UNSPEC addresses
instead of adding lots of checks to conditionally pass
NULL addresses.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 72abb73... 37299c0... M	net/sunrpc/xprtsock.c
 net/sunrpc/xprtsock.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 72abb73..37299c0 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -2509,9 +2509,9 @@ static struct rpc_xprt *xs_setup_xprt(struct xprt_create *args,
 	new = container_of(xprt, struct sock_xprt, xprt);
 	memcpy(&xprt->addr, args->dstaddr, args->addrlen);
 	xprt->addrlen = args->addrlen;
-	if (args->srcaddr)
+	if (args->srcaddr && (args->srcaddr->sa_family != AF_UNSPEC)) {
 		memcpy(&new->srcaddr, args->srcaddr, args->addrlen);
-	else {
+	} else {
 		int err;
 		err = xs_init_anyaddr(args->dstaddr->sa_family,
 					(struct sockaddr *)&new->srcaddr);
-- 
1.7.3.4

[PATCH (RESEND) 02/12] nfs: Two AF_UNSPEC addresses should always match each other.

[greearb]

From: Ben Greear <greearb@candelatech.com>

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 b3dc2b8... 8ba7e1a... M	fs/nfs/client.c
 fs/nfs/client.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index b3dc2b8..8ba7e1a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -403,6 +403,8 @@ static int nfs_sockaddr_match_ipaddr(const struct sockaddr *sa1,
 		return nfs_sockaddr_match_ipaddr4(sa1, sa2);
 	case AF_INET6:
 		return nfs_sockaddr_match_ipaddr6(sa1, sa2);
+	case AF_UNSPEC:
+		return 1; /* two UNSPEC addresses match */
 	}
 	return 0;
 }
-- 
1.7.3.4

[PATCH (RESEND) 03/12] nfs: Add srcaddr member to nfs_client.

[greearb]

From: Ben Greear <greearb@candelatech.com>


Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 2a55347... 93979a9... M	fs/nfs/internal.h
:100644 100644 ce40e5c... cb41131... M	fs/nfs/super.c
:100644 100644 87694ca... 9db4473... M	include/linux/nfs_fs_sb.h
 fs/nfs/internal.h         |    5 +++++
 fs/nfs/super.c            |    2 ++
 include/linux/nfs_fs_sb.h |    4 ++++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 2a55347..93979a9 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -107,6 +107,11 @@ struct nfs_parsed_mount_data {
 	struct {
 		struct sockaddr_storage	address;
 		size_t			addrlen;
+	} srcaddr;
+
+	struct {
+		struct sockaddr_storage address;
+		size_t                  addrlen;
 		char			*hostname;
 		u32			version;
 		int			port;
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index ce40e5c..cb41131 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -904,6 +904,8 @@ static struct nfs_parsed_mount_data *nfs_alloc_parsed_mount_data(unsigned int ve
 		data->auth_flavor_len	= 1;
 		data->version		= version;
 		data->minorversion	= 0;
+		data->srcaddr.address.ss_family = AF_UNSPEC;
+		data->srcaddr.addrlen = sizeof(data->srcaddr.address);
 	}
 	return data;
 }
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 87694ca..9db4473 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -80,6 +80,10 @@ struct nfs_client {
 	struct list_head	cl_layouts;
 #endif /* CONFIG_NFS_V4 */
 
+	/* If we should bind to a local IP, it should be specified below. */
+	struct sockaddr_storage	srcaddr;
+	size_t			srcaddrlen;
+
 #ifdef CONFIG_NFS_FSCACHE
 	struct fscache_cookie	*fscache;	/* client index cache cookie */
 #endif
-- 
1.7.3.4

[PATCH (RESEND) 04/12] nfs: Use request destination addr as callback source addr.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This helps make sure that responses use the correct
source address on multi-homed machines.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 b257383... 00902ec... M	fs/nfs/callback.h
:100644 100644 c6c86a7... ed5be89... M	fs/nfs/callback_xdr.c
:100644 100644 ea29330... 45d5d7c... M	include/linux/sunrpc/svc.h
 fs/nfs/callback.h          |    3 +++
 fs/nfs/callback_xdr.c      |    3 +++
 include/linux/sunrpc/svc.h |    5 +++++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/callback.h b/fs/nfs/callback.h
index b257383..00902ec 100644
--- a/fs/nfs/callback.h
+++ b/fs/nfs/callback.h
@@ -57,6 +57,7 @@ struct cb_compound_hdr_res {
 
 struct cb_getattrargs {
 	struct sockaddr *addr;
+	struct sockaddr *srcaddr;
 	struct nfs_fh fh;
 	uint32_t bitmap[2];
 };
@@ -72,6 +73,7 @@ struct cb_getattrres {
 
 struct cb_recallargs {
 	struct sockaddr *addr;
+	struct sockaddr *srcaddr;
 	struct nfs_fh fh;
 	nfs4_stateid stateid;
 	uint32_t truncate;
@@ -92,6 +94,7 @@ struct referring_call_list {
 
 struct cb_sequenceargs {
 	struct sockaddr			*csa_addr;
+	struct sockaddr                 *csa_daddr;
 	struct nfs4_sessionid		csa_sessionid;
 	uint32_t			csa_sequenceid;
 	uint32_t			csa_slotid;
diff --git a/fs/nfs/callback_xdr.c b/fs/nfs/callback_xdr.c
index c6c86a7..ed5be89 100644
--- a/fs/nfs/callback_xdr.c
+++ b/fs/nfs/callback_xdr.c
@@ -196,6 +196,7 @@ static __be32 decode_getattr_args(struct svc_rqst *rqstp, struct xdr_stream *xdr
 	if (unlikely(status != 0))
 		goto out;
 	args->addr = svc_addr(rqstp);
+	args->srcaddr = svc_daddr(rqstp);
 	status = decode_bitmap(xdr, args->bitmap);
 out:
 	dprintk("%s: exit with status = %d\n", __func__, ntohl(status));
@@ -208,6 +209,7 @@ static __be32 decode_recall_args(struct svc_rqst *rqstp, struct xdr_stream *xdr,
 	__be32 status;
 
 	args->addr = svc_addr(rqstp);
+	args->srcaddr = svc_daddr(rqstp);
 	status = decode_stateid(xdr, &args->stateid);
 	if (unlikely(status != 0))
 		goto out;
@@ -442,6 +444,7 @@ static __be32 decode_cb_sequence_args(struct svc_rqst *rqstp,
 		goto out;
 
 	args->csa_addr = svc_addr(rqstp);
+	args->csa_daddr = svc_daddr(rqstp);
 	args->csa_sequenceid = ntohl(*p++);
 	args->csa_slotid = ntohl(*p++);
 	args->csa_highestslotid = ntohl(*p++);
diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h
index ea29330..45d5d7c 100644
--- a/include/linux/sunrpc/svc.h
+++ b/include/linux/sunrpc/svc.h
@@ -299,6 +299,11 @@ static inline struct sockaddr *svc_addr(const struct svc_rqst *rqst)
 	return (struct sockaddr *) &rqst->rq_addr;
 }
 
+static inline struct sockaddr *svc_daddr(struct svc_rqst *rqst)
+{
+	return (struct sockaddr *) &rqst->rq_daddr;
+}
+
 /*
  * Check buffer bounds after decoding arguments
  */
-- 
1.7.3.4

[PATCH (RESEND) 05/12] nfs: Pay attention to srcaddr in v4.1 callback logic.

[greearb]

From: Ben Greear <greearb@candelatech.com>

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 d4d1954... 5b20987... M	fs/nfs/callback_proc.c
:100644 100644 8ba7e1a... c571a97... M	fs/nfs/client.c
:100644 100644 93979a9... 60a36a2... M	fs/nfs/internal.h
 fs/nfs/callback_proc.c |    3 ++-
 fs/nfs/client.c        |   27 +++++++++++++++++++++++++--
 fs/nfs/internal.h      |    4 +++-
 3 files changed, 30 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index d4d1954..5b20987 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -424,7 +424,8 @@ __be32 nfs4_callback_sequence(struct cb_sequenceargs *args,
 
 	cps->clp = NULL;
 
-	clp = nfs4_find_client_sessionid(args->csa_addr, &args->csa_sessionid);
+	clp = nfs4_find_client_sessionid(args->csa_daddr, args->csa_addr,
+					 &args->csa_sessionid);
 	if (clp == NULL)
 		goto out;
 
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 8ba7e1a..c571a97 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1233,10 +1233,12 @@ nfs4_find_client_ident(int cb_ident)
  * Returns NULL if no such client
  */
 struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *addr,
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
 			   struct nfs4_sessionid *sid)
 {
 	struct nfs_client *clp;
+	struct nfs_client *ok_fit = NULL;
 
 	spin_lock(&nfs_client_lock);
 	list_for_each_entry(clp, &nfs_client_list, cl_share_link) {
@@ -1251,10 +1253,30 @@ nfs4_find_client_sessionid(const struct sockaddr *addr,
 		    sid->data, NFS4_MAX_SESSIONID_LEN) != 0)
 			continue;
 
+		if (srcaddr) {
+			const struct sockaddr *sa;
+			sa = (const struct sockaddr *)&clp->cl_addr;
+			if (!nfs_sockaddr_match_ipaddr(srcaddr, sa)) {
+				/* If clp doesn't bind to srcaddr, then
+				 * it is a potential match if we don't find
+				 * a better one.
+				 */
+				if (sa->sa_family == AF_UNSPEC && !ok_fit)
+					ok_fit = clp;
+				continue;
+			}
+		}
+found_one:
 		atomic_inc(&clp->cl_count);
 		spin_unlock(&nfs_client_lock);
 		return clp;
 	}
+
+	if (ok_fit) {
+		clp = ok_fit;
+		goto found_one;
+	}
+
 	spin_unlock(&nfs_client_lock);
 	return NULL;
 }
@@ -1262,7 +1284,8 @@ nfs4_find_client_sessionid(const struct sockaddr *addr,
 #else /* CONFIG_NFS_V4_1 */
 
 struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *addr,
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
 			   struct nfs4_sessionid *sid)
 {
 	return NULL;
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 93979a9..60a36a2 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -155,7 +155,9 @@ extern void nfs_put_client(struct nfs_client *);
 extern struct nfs_client *nfs4_find_client_no_ident(const struct sockaddr *);
 extern struct nfs_client *nfs4_find_client_ident(int);
 extern struct nfs_client *
-nfs4_find_client_sessionid(const struct sockaddr *, struct nfs4_sessionid *);
+nfs4_find_client_sessionid(const struct sockaddr *srcaddr,
+			   const struct sockaddr *addr,
+			   struct nfs4_sessionid *);
 extern struct nfs_server *nfs_create_server(
 					const struct nfs_parsed_mount_data *,
 					struct nfs_fh *);
-- 
1.7.3.4

[PATCH (RESEND) 06/12] nfs: Use srcaddr in nfs_match_client.

[greearb]

From: Ben Greear <greearb@candelatech.com>

For bound clients, we have to make sure we do not re-use
a client that is bound differently.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 c571a97... 5d8998a... M	fs/nfs/client.c
 fs/nfs/client.c |   12 ++++++++++++
 1 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index c571a97..5d8998a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -131,7 +131,9 @@ struct rpc_program		nfsacl_program = {
 struct nfs_client_initdata {
 	const char *hostname;
 	const struct sockaddr *addr;
+	const struct sockaddr *srcaddr;
 	size_t addrlen;
+	size_t srcaddrlen;
 	const struct nfs_rpc_ops *rpc_ops;
 	int proto;
 	u32 minorversion;
@@ -463,6 +465,9 @@ static struct nfs_client *nfs_match_client(const struct nfs_client_initdata *dat
 
 	list_for_each_entry(clp, &nfs_client_list, cl_share_link) {
 	        const struct sockaddr *clap = (struct sockaddr *)&clp->cl_addr;
+		const struct sockaddr *sa;
+		sa = (const struct sockaddr *)&clp->srcaddr;
+
 		/* Don't match clients that failed to initialise properly */
 		if (clp->cl_cons_state < 0)
 			continue;
@@ -480,6 +485,13 @@ static struct nfs_client *nfs_match_client(const struct nfs_client_initdata *dat
 		if (!nfs_sockaddr_cmp(sap, clap))
 			continue;
 
+		/* Check to make sure local-IP bindings match,
+		 * but just the IP-addr.
+		 */
+		if (data->srcaddr &&
+		    !nfs_sockaddr_match_ipaddr(data->srcaddr, sa))
+			continue;
+
 		atomic_inc(&clp->cl_count);
 		return clp;
 	}
-- 
1.7.3.4

[PATCH (RESEND) 07/12] nfs: Add srcaddr to /proc/fs/nfsfs/servers

[greearb]

From: Ben Greear <greearb@candelatech.com>

Helps users understand whether a server instance is
bound to a local IP or not.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 5d8998a... 223785f... M	fs/nfs/client.c
 fs/nfs/client.c |   18 ++++++++++++++++--
 1 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 5d8998a..223785f 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1895,20 +1895,34 @@ static int nfs_server_list_show(struct seq_file *m, void *v)
 
 	/* display header on line 1 */
 	if (v == &nfs_client_list) {
-		seq_puts(m, "NV SERVER   PORT USE HOSTNAME\n");
+		seq_puts(m, "NV SERVER   PORT USE HOSTNAME"
+			 "           SRCADDR\n");
 		return 0;
 	}
 
 	/* display one transport per line on subsequent lines */
 	clp = list_entry(v, struct nfs_client, cl_share_link);
 
-	seq_printf(m, "v%u %s %s %3d %s\n",
+	seq_printf(m, "v%u %s %s %3d %s",
 		   clp->rpc_ops->version,
 		   rpc_peeraddr2str(clp->cl_rpcclient, RPC_DISPLAY_HEX_ADDR),
 		   rpc_peeraddr2str(clp->cl_rpcclient, RPC_DISPLAY_HEX_PORT),
 		   atomic_read(&clp->cl_count),
 		   clp->cl_hostname);
 
+	if (clp->srcaddr.ss_family == AF_INET) {
+		const struct sockaddr_in *sin;
+		sin = (const struct sockaddr_in *)&clp->srcaddr;
+		seq_printf(m, "   %pI4\n", &sin->sin_addr.s_addr);
+	} else if (clp->srcaddr.ss_family == AF_INET6) {
+		const struct sockaddr_in6 *sin6;
+		sin6 = (const struct sockaddr_in6 *)&clp->srcaddr;
+		seq_printf(m, "   %pI6c\n", &sin6->sin6_addr);
+	} else if (clp->srcaddr.ss_family == AF_UNSPEC)
+		seq_printf(m, "   ANY\n");
+	else
+		seq_printf(m, "   UNKNOWN_%i\n", (int)(clp->srcaddr.ss_family));
+
 	return 0;
 }
 
-- 
1.7.3.4

[PATCH (RESEND) 08/12] nfs: Pass srcaddr into mount request.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This starts tying the srcaddr logic together.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 60a36a2... eaf9c66... M	fs/nfs/internal.h
:100644 100644 d4c2d6b... 5ca6e12... M	fs/nfs/mount_clnt.c
:100644 100644 cb41131... e45f616... M	fs/nfs/super.c
 fs/nfs/internal.h   |    1 +
 fs/nfs/mount_clnt.c |    1 +
 fs/nfs/super.c      |    2 ++
 3 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 60a36a2..eaf9c66 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -133,6 +133,7 @@ struct nfs_parsed_mount_data {
 /* mount_clnt.c */
 struct nfs_mount_request {
 	struct sockaddr		*sap;
+	struct sockaddr         *srcaddr;
 	size_t			salen;
 	char			*hostname;
 	char			*dirpath;
diff --git a/fs/nfs/mount_clnt.c b/fs/nfs/mount_clnt.c
index d4c2d6b..5ca6e12 100644
--- a/fs/nfs/mount_clnt.c
+++ b/fs/nfs/mount_clnt.c
@@ -155,6 +155,7 @@ int nfs_mount(struct nfs_mount_request *info)
 	struct rpc_create_args args = {
 		.net		= &init_net,
 		.protocol	= info->protocol,
+		.saddress	= info->srcaddr,
 		.address	= info->sap,
 		.addrsize	= info->salen,
 		.servername	= info->hostname,
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index cb41131..e45f616 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1600,6 +1600,8 @@ static int nfs_try_mount(struct nfs_parsed_mount_data *args,
 	struct nfs_mount_request request = {
 		.sap		= (struct sockaddr *)
 						&args->mount_server.address,
+		.salen		= args->mount_server.addrlen,
+		.srcaddr	= (struct sockaddr *)&args->srcaddr.address,
 		.dirpath	= args->nfs_server.export_path,
 		.protocol	= args->mount_server.protocol,
 		.fh		= root_fh,
-- 
1.7.3.4

[PATCH (RESEND) 09/12] nfs: Propagate src-addr in client code.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This ties the srcaddr supplied by mount to the
rpc code.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 223785f... 8b8766a... M	fs/nfs/client.c
 fs/nfs/client.c |   13 +++++++++++++
 1 files changed, 13 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 223785f..8b8766a 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -161,6 +161,8 @@ static struct nfs_client *nfs_alloc_client(const struct nfs_client_initdata *cl_
 
 	memcpy(&clp->cl_addr, cl_init->addr, cl_init->addrlen);
 	clp->cl_addrlen = cl_init->addrlen;
+	memcpy(&clp->srcaddr, cl_init->srcaddr, cl_init->srcaddrlen);
+	clp->srcaddrlen = cl_init->srcaddrlen;
 
 	if (cl_init->hostname) {
 		err = -ENOMEM;
@@ -655,6 +657,7 @@ static int nfs_create_rpc_client(struct nfs_client *clp,
 		.net		= &init_net,
 		.protocol	= clp->cl_proto,
 		.address	= (struct sockaddr *)&clp->cl_addr,
+		.saddress	= (struct sockaddr *)&clp->srcaddr,
 		.addrsize	= clp->cl_addrlen,
 		.timeout	= timeparms,
 		.servername	= clp->cl_hostname,
@@ -839,6 +842,8 @@ static int nfs_init_server(struct nfs_server *server,
 		.hostname = data->nfs_server.hostname,
 		.addr = (const struct sockaddr *)&data->nfs_server.address,
 		.addrlen = data->nfs_server.addrlen,
+		.srcaddr = (const struct sockaddr *)&data->srcaddr.address,
+		.srcaddrlen = data->srcaddr.addrlen,
 		.rpc_ops = &nfs_v2_clientops,
 		.proto = data->nfs_server.protocol,
 	};
@@ -1419,6 +1424,8 @@ static int nfs4_set_client(struct nfs_server *server,
 		const struct sockaddr *addr,
 		const size_t addrlen,
 		const char *ip_addr,
+		const struct sockaddr *srcaddr,
+		const size_t srcaddrlen,
 		rpc_authflavor_t authflavour,
 		int proto, const struct rpc_timeout *timeparms,
 		u32 minorversion)
@@ -1430,6 +1437,8 @@ static int nfs4_set_client(struct nfs_server *server,
 		.rpc_ops = &nfs_v4_clientops,
 		.proto = proto,
 		.minorversion = minorversion,
+		.srcaddr = srcaddr,
+		.srcaddrlen = srcaddrlen,
 	};
 	struct nfs_client *clp;
 	int error;
@@ -1604,6 +1613,8 @@ static int nfs4_init_server(struct nfs_server *server,
 			(const struct sockaddr *)&data->nfs_server.address,
 			data->nfs_server.addrlen,
 			data->client_address,
+			(const struct sockaddr *)&data->srcaddr.address,
+			data->srcaddr.addrlen,
 			data->auth_flavors[0],
 			data->nfs_server.protocol,
 			&timeparms,
@@ -1701,6 +1712,8 @@ struct nfs_server *nfs4_create_referral_server(struct nfs_clone_mount *data,
 				data->addr,
 				data->addrlen,
 				parent_client->cl_ipaddr,
+				(const struct sockaddr *)&parent_client->srcaddr,
+				parent_client->srcaddrlen,
 				data->authflavor,
 				parent_server->client->cl_xprt->prot,
 				parent_server->client->cl_timeout,
-- 
1.7.3.4

[PATCH (RESEND) 10/12] nfs: Bind to srcaddr in rpcb_create.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This more properly clones the existing client connection
settings, allowing rpc calls to take advantage of routing
rules based on source IP addresses.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 e45d2fb... 13dae70... M	net/sunrpc/rpcb_clnt.c
:100644 100644 90c292e... 22841e1... M	net/sunrpc/sunrpc.h
:100644 100644 37299c0... 72408c2... M	net/sunrpc/xprtsock.c
 net/sunrpc/rpcb_clnt.c |   28 ++++++++++++++++++++++++++--
 net/sunrpc/sunrpc.h    |   45 +++++++++++++++++++++++++++++++++++++++++++++
 net/sunrpc/xprtsock.c  |   43 -------------------------------------------
 3 files changed, 71 insertions(+), 45 deletions(-)

diff --git a/net/sunrpc/rpcb_clnt.c b/net/sunrpc/rpcb_clnt.c
index e45d2fb..13dae70 100644
--- a/net/sunrpc/rpcb_clnt.c
+++ b/net/sunrpc/rpcb_clnt.c
@@ -28,6 +28,7 @@
 #include <linux/sunrpc/clnt.h>
 #include <linux/sunrpc/sched.h>
 #include <linux/sunrpc/xprtsock.h>
+#include "sunrpc.h"
 
 #ifdef RPC_DEBUG
 # define RPCDBG_FACILITY	RPCDBG_BIND
@@ -292,11 +293,13 @@ out:
 }
 
 static struct rpc_clnt *rpcb_create(char *hostname, struct sockaddr *srvaddr,
-				    size_t salen, int proto, u32 version)
+				    size_t salen, int proto, u32 version,
+				    struct sockaddr *srcaddr)
 {
 	struct rpc_create_args args = {
 		.net		= &init_net,
 		.protocol	= proto,
+		.saddress	= srcaddr,
 		.address	= srvaddr,
 		.addrsize	= salen,
 		.servername	= hostname,
@@ -601,11 +604,32 @@ void rpcb_getport_async(struct rpc_task *task)
 	struct rpc_task	*child;
 	struct sockaddr_storage addr;
 	struct sockaddr *sap = (struct sockaddr *)&addr;
+	struct sockaddr_storage srcaddr;
+	struct sockaddr *srcaddra = NULL;
 	size_t salen;
 	int status;
+	struct sock_xprt *sxprt;
 
 	clnt = rpcb_find_transport_owner(task->tk_client);
 	xprt = clnt->cl_xprt;
+	sxprt = container_of(xprt, struct sock_xprt, xprt);
+
+	/* We just want to bind to the IP, not the port */
+	memset(&srcaddr, 0, sizeof(srcaddr));
+	srcaddr.ss_family = sxprt->srcaddr.ss_family;
+	if (sxprt->srcaddr.ss_family == AF_INET) {
+		struct sockaddr_in *si = (struct sockaddr_in *)(&srcaddr);
+		struct sockaddr_in *si2;
+		si2 = (struct sockaddr_in *)(&sxprt->srcaddr);
+		si->sin_addr.s_addr = si2->sin_addr.s_addr;
+		srcaddra = (struct sockaddr *)(&srcaddr);
+	} else if (sxprt->srcaddr.ss_family == AF_INET6) {
+		struct sockaddr_in6 *si = (struct sockaddr_in6 *)(&srcaddr);
+		struct sockaddr_in6 *si2;
+		si2 = (struct sockaddr_in6 *)(&sxprt->srcaddr);
+		memcpy(&si->sin6_addr, &si2->sin6_addr, sizeof(si2->sin6_addr));
+		srcaddra = (struct sockaddr *)(&srcaddr);
+	}
 
 	dprintk("RPC: %5u %s(%s, %u, %u, %d)\n",
 		task->tk_pid, __func__,
@@ -660,7 +684,7 @@ void rpcb_getport_async(struct rpc_task *task)
 		task->tk_pid, __func__, bind_version);
 
 	rpcb_clnt = rpcb_create(clnt->cl_server, sap, salen, xprt->prot,
-				bind_version);
+				bind_version, srcaddra);
 	if (IS_ERR(rpcb_clnt)) {
 		status = PTR_ERR(rpcb_clnt);
 		dprintk("RPC: %5u %s: rpcb_create failed, error %ld\n",
diff --git a/net/sunrpc/sunrpc.h b/net/sunrpc/sunrpc.h
index 90c292e..22841e1 100644
--- a/net/sunrpc/sunrpc.h
+++ b/net/sunrpc/sunrpc.h
@@ -37,6 +37,51 @@ struct rpc_buffer {
 	char	data[];
 };
 
+struct sock_xprt {
+	struct rpc_xprt		xprt;
+
+	/*
+	 * Network layer
+	 */
+	struct socket		*sock;
+	struct sock		*inet;
+
+	/*
+	 * State of TCP reply receive
+	 */
+	__be32			tcp_fraghdr,
+				tcp_xid,
+				tcp_calldir;
+
+	u32			tcp_offset,
+				tcp_reclen;
+
+	unsigned long		tcp_copied,
+				tcp_flags;
+
+	/*
+	 * Connection of transports
+	 */
+	struct delayed_work	connect_worker;
+	struct sockaddr_storage	srcaddr;
+	unsigned short		srcport;
+
+	/*
+	 * UDP socket buffer size parameters
+	 */
+	size_t			rcvsize,
+				sndsize;
+
+	/*
+	 * Saved socket callback addresses
+	 */
+	void			(*old_data_ready)(struct sock *, int);
+	void			(*old_state_change)(struct sock *);
+	void			(*old_write_space)(struct sock *);
+	void			(*old_error_report)(struct sock *);
+};
+
+
 static inline int rpc_reply_expected(struct rpc_task *task)
 {
 	return (task->tk_msg.rpc_proc != NULL) &&
diff --git a/net/sunrpc/xprtsock.c b/net/sunrpc/xprtsock.c
index 37299c0..72408c2 100644
--- a/net/sunrpc/xprtsock.c
+++ b/net/sunrpc/xprtsock.c
@@ -202,49 +202,6 @@ static inline void xs_pktdump(char *msg, u32 *packet, unsigned int count)
 }
 #endif
 
-struct sock_xprt {
-	struct rpc_xprt		xprt;
-
-	/*
-	 * Network layer
-	 */
-	struct socket *		sock;
-	struct sock *		inet;
-
-	/*
-	 * State of TCP reply receive
-	 */
-	__be32			tcp_fraghdr,
-				tcp_xid,
-				tcp_calldir;
-
-	u32			tcp_offset,
-				tcp_reclen;
-
-	unsigned long		tcp_copied,
-				tcp_flags;
-
-	/*
-	 * Connection of transports
-	 */
-	struct delayed_work	connect_worker;
-	struct sockaddr_storage	srcaddr;
-	unsigned short		srcport;
-
-	/*
-	 * UDP socket buffer size parameters
-	 */
-	size_t			rcvsize,
-				sndsize;
-
-	/*
-	 * Saved socket callback addresses
-	 */
-	void			(*old_data_ready)(struct sock *, int);
-	void			(*old_state_change)(struct sock *);
-	void			(*old_write_space)(struct sock *);
-	void			(*old_error_report)(struct sock *);
-};
 
 /*
  * TCP receive state flags
-- 
1.7.3.4

[PATCH (RESEND) 11/12] nfs: Support srcaddr= to bind to specific IP address.

[greearb]

From: Ben Greear <greearb@candelatech.com>

In order to have more control on multi-homed machines, it is
nice to be able to bind to a specific IP address.  This can aid
with interface selection, policy based routing, multiple unique
mounts to the same server, and similar things.

This patch allows srcaddr= option for NFS.  The key 'srcaddr'
was chosen to match the similar patch for cifs.

For NFSv4, if one is specifying clientaddr, it must be the same as
srcaddr or things may not work properly.

NFSv3, NFSv4 over TCP/IPv4 and TCP/IPv6 has been successfully tested.

Usage:
  mount -t nfs4 [2002::1]:/rpool/ben /mnt/foo/ben-1 -o srcaddr=2002::2,clientaddr=2002::2
  mount -t nfs4 192.168.100.3:/foo /mnt/foo/ben-2 -o srcaddr=192.168.100.174

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 e45f616... 5d88e7b... M	fs/nfs/super.c
 fs/nfs/super.c |   29 ++++++++++++++++++++++++++++-
 1 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index e45f616..5d88e7b 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -103,7 +103,7 @@ enum {
 
 	/* Mount options that take string arguments */
 	Opt_sec, Opt_proto, Opt_mountproto, Opt_mounthost,
-	Opt_addr, Opt_mountaddr, Opt_clientaddr,
+	Opt_addr, Opt_mountaddr, Opt_clientaddr, Opt_srcaddr,
 	Opt_lookupcache,
 	Opt_fscache_uniq,
 	Opt_local_lock,
@@ -173,6 +173,7 @@ static const match_table_t nfs_mount_option_tokens = {
 	{ Opt_mountproto, "mountproto=%s" },
 	{ Opt_addr, "addr=%s" },
 	{ Opt_clientaddr, "clientaddr=%s" },
+	{ Opt_srcaddr, "srcaddr=%s" },
 	{ Opt_mounthost, "mounthost=%s" },
 	{ Opt_mountaddr, "mountaddr=%s" },
 
@@ -695,6 +696,15 @@ static void nfs_show_mount_options(struct seq_file *m, struct nfs_server *nfss,
 	else
 		nfs_show_nfsv4_options(m, nfss, showdefaults);
 
+	if (clp->srcaddr.ss_family == AF_INET6) {
+		struct sockaddr_in6 *sin6;
+		sin6 = (struct sockaddr_in6 *)(&clp->srcaddr);
+		seq_printf(m, ",srcaddr=%pI6c", &sin6->sin6_addr);
+	} else if (clp->srcaddr.ss_family == AF_INET) {
+		struct sockaddr_in *sin = (struct sockaddr_in *)&clp->srcaddr;
+		seq_printf(m, ",srcaddr=%pI4", &sin->sin_addr.s_addr);
+	}
+
 	if (nfss->options & NFS_OPTION_FSCACHE)
 		seq_printf(m, ",fsc");
 
@@ -1448,6 +1458,23 @@ static int nfs_parse_mount_options(char *raw,
 				goto out_nomem;
 			mnt->options |= NFS_OPTION_FSCACHE;
 			break;
+		case Opt_srcaddr:
+			string = match_strdup(args);
+			if (string == NULL)
+				goto out_nomem;
+			mnt->srcaddr.addrlen =
+				rpc_pton(string, strlen(string),
+					 (struct sockaddr *)
+					 &mnt->srcaddr.address,
+					 sizeof(mnt->srcaddr.address));
+			kfree(string);
+			if (mnt->srcaddr.addrlen == 0) {
+				printk(KERN_WARNING
+				       "nfs: Could not parse srcaddr: %s\n",
+				       string);
+				goto out_invalid_address;
+			}
+			break;
 		case Opt_local_lock:
 			string = match_strdup(args);
 			if (string == NULL)
-- 
1.7.3.4

[PATCH (RESEND) 12/12] lockd: Support binding nlm client to specific address.

[greearb]

From: Ben Greear <greearb@candelatech.com>

This ensures that file locking messages use the proper
source IP address if the file system has been mounted
with a specified source IP address.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
:100644 100644 8d4ea83... e8dff8d... M	fs/lockd/clntlock.c
:100644 100644 b7c99bf... 140fccc... M	fs/lockd/host.c
:100644 100644 8b8766a... 39c7667... M	fs/nfs/client.c
:100644 100644 fbc48f8... 822bcc3... M	include/linux/lockd/bind.h
:100644 100644 ff9abff... b69ef3b... M	include/linux/lockd/lockd.h
:100644 100644 db7bcaf... adfe318... M	include/linux/sunrpc/clnt.h
 fs/lockd/clntlock.c         |    3 ++-
 fs/lockd/host.c             |   17 ++++++++++++++++-
 fs/nfs/client.c             |    1 +
 include/linux/lockd/bind.h  |    1 +
 include/linux/lockd/lockd.h |    1 +
 include/linux/sunrpc/clnt.h |    2 ++
 6 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/fs/lockd/clntlock.c b/fs/lockd/clntlock.c
index 8d4ea83..e8dff8d 100644
--- a/fs/lockd/clntlock.c
+++ b/fs/lockd/clntlock.c
@@ -60,7 +60,8 @@ struct nlm_host *nlmclnt_init(const struct nlmclnt_initdata *nlm_init)
 	if (status < 0)
 		return ERR_PTR(status);
 
-	host = nlmclnt_lookup_host(nlm_init->address, nlm_init->addrlen,
+	host = nlmclnt_lookup_host(nlm_init->address, nlm_init->srcaddr,
+				   nlm_init->addrlen,
 				   nlm_init->protocol, nlm_version,
 				   nlm_init->hostname, nlm_init->noresvport);
 	if (host == NULL) {
diff --git a/fs/lockd/host.c b/fs/lockd/host.c
index b7c99bf..140fccc 100644
--- a/fs/lockd/host.c
+++ b/fs/lockd/host.c
@@ -48,6 +48,7 @@ static void			nlm_gc_hosts(void);
 struct nlm_lookup_host_info {
 	const int		server;		/* search for server|client */
 	const struct sockaddr	*sap;		/* address to search for */
+	const struct sockaddr   *src_addr;      /* source address */
 	const size_t		salen;		/* it's length */
 	const unsigned short	protocol;	/* transport to search for*/
 	const u32		version;	/* NLM version to search for */
@@ -130,7 +131,12 @@ static struct nlm_host *nlm_alloc_host(struct nlm_lookup_host_info *ni,
 	memcpy(nlm_addr(host), ni->sap, ni->salen);
 	host->h_addrlen    = ni->salen;
 	rpc_set_port(nlm_addr(host), 0);
-	host->h_srcaddrlen = 0;
+	if (ni->src_addr && ni->src_addr->sa_family != AF_UNSPEC) {
+		memcpy(nlm_srcaddr(host), ni->src_addr, ni->salen);
+		host->h_srcaddrlen = ni->salen;
+	} else {
+		host->h_srcaddrlen = 0;
+	}
 
 	host->h_rpcclnt    = NULL;
 	host->h_name	   = nsm->sm_name;
@@ -202,6 +208,7 @@ static void nlm_destroy_host_locked(struct nlm_host *host)
  * created and returned.
  */
 struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
+				     const struct sockaddr *srcaddr,
 				     const size_t salen,
 				     const unsigned short protocol,
 				     const u32 version,
@@ -216,6 +223,7 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 		.version	= version,
 		.hostname	= hostname,
 		.hostname_len	= strlen(hostname),
+		.src_addr       = srcaddr,
 		.noresvport	= noresvport,
 	};
 	struct hlist_head *chain;
@@ -234,6 +242,13 @@ struct nlm_host *nlmclnt_lookup_host(const struct sockaddr *sap,
 		if (!rpc_cmp_addr(nlm_addr(host), sap))
 			continue;
 
+		/* Check for local binding match only if user
+		 * has specified the source-address.
+		 */
+		if (srcaddr && srcaddr->sa_family != AF_UNSPEC &&
+		    !rpc_cmp_addr(nlm_srcaddr(host), srcaddr))
+			continue;
+
 		/* Same address. Share an NSM handle if we already have one */
 		if (nsm == NULL)
 			nsm = host->h_nsmhandle;
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 8b8766a..39c7667 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -705,6 +705,7 @@ static int nfs_start_lockd(struct nfs_server *server)
 	struct nlmclnt_initdata nlm_init = {
 		.hostname	= clp->cl_hostname,
 		.address	= (struct sockaddr *)&clp->cl_addr,
+		.srcaddr        = (struct sockaddr *)&clp->srcaddr,
 		.addrlen	= clp->cl_addrlen,
 		.nfs_version	= clp->rpc_ops->version,
 		.noresvport	= server->flags & NFS_MOUNT_NORESVPORT ?
diff --git a/include/linux/lockd/bind.h b/include/linux/lockd/bind.h
index fbc48f8..822bcc3 100644
--- a/include/linux/lockd/bind.h
+++ b/include/linux/lockd/bind.h
@@ -38,6 +38,7 @@ extern struct nlmsvc_binding *	nlmsvc_ops;
 struct nlmclnt_initdata {
 	const char		*hostname;
 	const struct sockaddr	*address;
+	const struct sockaddr   *srcaddr;
 	size_t			addrlen;
 	unsigned short		protocol;
 	u32			nfs_version;
diff --git a/include/linux/lockd/lockd.h b/include/linux/lockd/lockd.h
index ff9abff..b69ef3b 100644
--- a/include/linux/lockd/lockd.h
+++ b/include/linux/lockd/lockd.h
@@ -218,6 +218,7 @@ void		  nlmclnt_next_cookie(struct nlm_cookie *);
  * Host cache
  */
 struct nlm_host  *nlmclnt_lookup_host(const struct sockaddr *sap,
+					const struct sockaddr *bindaddr,
 					const size_t salen,
 					const unsigned short protocol,
 					const u32 version,
diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index db7bcaf..adfe318 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -262,6 +262,8 @@ static inline bool rpc_cmp_addr(const struct sockaddr *sap1,
 			return __rpc_cmp_addr4(sap1, sap2);
 		case AF_INET6:
 			return __rpc_cmp_addr6(sap1, sap2);
+		case AF_UNSPEC:
+			return true;
 		}
 	}
 	return false;
-- 
1.7.3.4

Re: [PATCH (RESEND) 00/12] Support binding nfs/rpc to local IP address.

[Ben Greear]

On 07/15/2011 10:55 AM, greearb@candelatech.com wrote:
> From: Ben Greear<greearb@candelatech.com>
>
>
> This patch series allows binding the nfs and rpc logic to a specific local
> IP address.

Any comments on this?  I'd love to get these merged in time for 3.1!

With recent 3.0 RCU fixes, my stress test runs clean over-night, so
it has been pretty well tested....

Thanks,
Ben

>
> Features/Benefits:
>
> *  Allow multiple unique mounts to the same server using unique
>     source IP addresses on client system.
>
>     Routing rules can then be created based on the source IP address for
>     advanced routing of the NFS traffic.
>
>     This could allow someone to use two 1G interfaces to access a
>     server with 10G connectivity and allow aggregate 2Gbps transfer,
>     for instance.
>
>     This is also useful for load testing NFS servers as well as the
>     client-side NFS logic (note the bugs&  fixed found while testing
>     this code!)
>
> *  Allow using a specific IP address on multi-homed system.  This could
>     increase security in some cases and in general gives the user more
>     control over how the services are configured.
>
> This code has been tested under load on 3.0 and a similar version has
> been tested under load on 2.6.38 and previous kernels.
>
> Full implementation of this feature requires patches to mount.nfs,
> which have been agreed to be accepted if the kernel patches are accepted.
>
> Ben Greear (12):
>    sunrpc:  Don't attempt to bind to AF_UNSPEC address.
>    nfs:  Two AF_UNSPEC addresses should always match each other.
>    nfs:  Add srcaddr member to nfs_client.
>    nfs:  Use request destination addr as callback source addr.
>    nfs:  Pay attention to srcaddr in v4.1 callback logic.
>    nfs:  Use srcaddr in nfs_match_client.
>    nfs:  Add srcaddr to /proc/fs/nfsfs/servers
>    nfs:  Pass srcaddr into mount request.
>    nfs:  Propagate src-addr in client code.
>    nfs: Bind to srcaddr in rpcb_create.
>    nfs: Support srcaddr= to bind to specific IP address.
>    lockd: Support binding nlm client to specific address.
>
>   fs/lockd/clntlock.c         |    3 +-
>   fs/lockd/host.c             |   17 +++++++++-
>   fs/nfs/callback.h           |    3 ++
>   fs/nfs/callback_proc.c      |    3 +-
>   fs/nfs/callback_xdr.c       |    3 ++
>   fs/nfs/client.c             |   73 ++++++++++++++++++++++++++++++++++++++++--
>   fs/nfs/internal.h           |   10 +++++-
>   fs/nfs/mount_clnt.c         |    1 +
>   fs/nfs/super.c              |   33 +++++++++++++++++++-
>   include/linux/lockd/bind.h  |    1 +
>   include/linux/lockd/lockd.h |    1 +
>   include/linux/nfs_fs_sb.h   |    4 ++
>   include/linux/sunrpc/clnt.h |    2 +
>   include/linux/sunrpc/svc.h  |    5 +++
>   net/sunrpc/rpcb_clnt.c      |   28 +++++++++++++++-
>   net/sunrpc/sunrpc.h         |   45 ++++++++++++++++++++++++++
>   net/sunrpc/xprtsock.c       |   47 +--------------------------
>   17 files changed, 223 insertions(+), 56 deletions(-)
>


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com