[PATCH 00/15] Candidates for 3.6 [v2]

[PATCH 00/15] Candidates for 3.6 [v2]

[Chuck Lever]

Hi-

Here is a second crack at the TEST_STATEID fixes and patches to
implement UCS.  I've attempted to address the following review
comments:

  o  The changes have been reorganized to allow the meaty parts of
     the TEST_STATEID fixes to be applied to -stable.

  o  Additional in-code comments have been introduced.

  o  FREE_STATEID is now performed in all cases after TEST_STATEID
     except when the TEST_STATEID operation result is
     NFS4ERR_BAD_STATEID.

  o  The change to check clp->cl_cons_state in
     nfs4_wait_clnt_recover() is now a separate patch with its own
     justification.

As before, there is one last bug to fix: rpcauth_create() returns
-EEXIST when replacing an RPC client's rpc_auth, if both the
existing and new rpc_auth are RPC_AUTH_GSS.  I should be able to
post a solution for review next week.

The final patch in this series is a hack, posted for discussion.

---

Chuck Lever (15):
      NFS: Slow down state manager after an unhandled error
      NFS: Add nfs4_unique_id boot parameter
      NFS: Discover NFSv4 server trunking when mounting
      NFS: Use the same nfs_client_id4 for every server
      NFS: Introduce "migration" mount option
      SUNRPC: Add rpcauth_list_flavors()
      NFS: Clean up nfs4_proc_setclientid() and friends
      NFS: Treat NFS4ERR_CLID_INUSE as a fatal error
      NFS: When state recovery fails, waiting tasks should exit
      NFS: nfs_getaclargs.acl_len is a size_t
      NFS: Clean up TEST_STATEID and FREE_STATEID error reporting
      NFS: Clean up nfs41_check_expired_stateid()
      NFS: State reclaim clears OPEN and LOCK state
      NFS: Don't free a state ID the server does not recognize
      NFS: Fix up TEST_STATEID and FREE_STATEID return code handling


 Documentation/filesystems/nfs/nfs.txt |   44 +++++
 Documentation/kernel-parameters.txt   |    5 +
 fs/nfs/client.c                       |  239 +++++++++++++++++++++++++++
 fs/nfs/internal.h                     |    6 +
 fs/nfs/nfs4_fs.h                      |    8 +
 fs/nfs/nfs4proc.c                     |  293 ++++++++++++++++++++++++---------
 fs/nfs/nfs4state.c                    |  198 ++++++++++++++++++++++
 fs/nfs/super.c                        |   20 ++
 include/linux/nfs_fs_sb.h             |    5 -
 include/linux/sunrpc/auth.h           |    2 
 include/linux/sunrpc/gss_api.h        |    3 
 net/sunrpc/auth.c                     |   54 ++++++
 net/sunrpc/auth_gss/auth_gss.c        |    1 
 net/sunrpc/auth_gss/gss_mech_switch.c |   18 ++
 14 files changed, 803 insertions(+), 93 deletions(-)

-- 
Chuck Lever

[PATCH 01/15] NFS: Fix up TEST_STATEID and FREE_STATEID return code handling

[Chuck Lever]

The TEST_STATEID and FREE_STATEID operations can return
-NFS4ERR_BAD_STATEID, -NFS4ERR_OLD_STATEID, or -NFS4ERR_DEADSESSION.

nfs41_{test,free}_stateid() should not pass these errors to
nfs4_handle_exception() during state recovery, since that will
recursively kick off state recovery again, resulting in a deadlock.

In particular, when the TEST_STATEID operation returns NFS4_OK,
res.status can contain one of these errors.  _nfs41_test_stateid()
replaces NFS4_OK with the value in res.status, which is then returned
to callers.

But res.status is not passed through nfs4_stat_to_errno(), and thus is
a positive NFS4ERR value.  Currently callers are only interested in
!NFS4_OK, and nfs4_handle_exception() ignores positive values.

Thus the res.status values are currently ignored by
nfs4_handle_exception() and won't cause the deadlock above.  Thanks to
this missing negative, it is only when these operations fail (which
is very rare) that a deadlock can occur.

Bryan agrees the original intent was to return res.status as a
negative NFS4ERR value to callers of nfs41_test_stateid().

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |   24 +++++++++++++-----------
 1 files changed, 13 insertions(+), 11 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 15fc7e4..9a0397c 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6589,10 +6589,9 @@ static int _nfs41_test_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 
 	nfs41_init_sequence(&args.seq_args, &res.seq_res, 0);
 	status = nfs4_call_sync_sequence(server->client, server, &msg, &args.seq_args, &res.seq_res, 1);
-
-	if (status == NFS_OK)
-		return res.status;
-	return status;
+	if (status != NFS_OK)
+		return status;
+	return -res.status;
 }
 
 static int nfs41_test_stateid(struct nfs_server *server, nfs4_stateid *stateid)
@@ -6600,9 +6599,10 @@ static int nfs41_test_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 	struct nfs4_exception exception = { };
 	int err;
 	do {
-		err = nfs4_handle_exception(server,
-				_nfs41_test_stateid(server, stateid),
-				&exception);
+		err = _nfs41_test_stateid(server, stateid);
+		if (err != -NFS4ERR_DELAY)
+			break;
+		nfs4_handle_exception(server, err, &exception);
 	} while (exception.retry);
 	return err;
 }
@@ -6620,7 +6620,8 @@ static int _nfs4_free_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 	};
 
 	nfs41_init_sequence(&args.seq_args, &res.seq_res, 0);
-	return nfs4_call_sync_sequence(server->client, server, &msg, &args.seq_args, &res.seq_res, 1);
+	return nfs4_call_sync_sequence(server->client, server, &msg,
+					 &args.seq_args, &res.seq_res, 1);
 }
 
 static int nfs41_free_stateid(struct nfs_server *server, nfs4_stateid *stateid)
@@ -6628,9 +6629,10 @@ static int nfs41_free_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 	struct nfs4_exception exception = { };
 	int err;
 	do {
-		err = nfs4_handle_exception(server,
-				_nfs4_free_stateid(server, stateid),
-				&exception);
+		err = _nfs4_free_stateid(server, stateid);
+		if (err != -NFS4ERR_DELAY)
+			break;
+		nfs4_handle_exception(server, err, &exception);
 	} while (exception.retry);
 	return err;
 }

[PATCH 02/15] NFS: Don't free a state ID the server does not recognize

[Chuck Lever]

The result of a TEST_STATEID operation can indicate a few different
things:

  o If NFS_OK is returned, then the client can continue using the
    state ID under test, and skip recovery.

  o RFC 5661 says that if the state ID was revoked, then the client
    must perform an explicit FREE_STATEID before trying to re-open.

  o If the server doesn't recognize the state ID at all, then no
    FREE_STATEID is needed, and the client can immediately continue
    with open recovery.

Let's err on the side of caution: if the server clearly tells us the
state ID is unknown, we skip the FREE_STATEID.  For any other error,
we issue a FREE_STATEID.  Sometimes that FREE_STATEID will be
unnecessary, but leaving unused state IDs on the server needlessly
ties up resources.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 9a0397c..2986e65 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1764,7 +1764,8 @@ static int nfs41_check_expired_stateid(struct nfs4_state *state, nfs4_stateid *s
 	if (state->flags & flags) {
 		status = nfs41_test_stateid(server, stateid);
 		if (status != NFS_OK) {
-			nfs41_free_stateid(server, stateid);
+			if (status != -NFS4ERR_BAD_STATEID)
+				nfs41_free_stateid(server, stateid);
 			state->flags &= ~flags;
 		}
 	}
@@ -4708,7 +4709,9 @@ static int nfs41_check_expired_locks(struct nfs4_state *state)
 		if (lsp->ls_flags & NFS_LOCK_INITIALIZED) {
 			status = nfs41_test_stateid(server, &lsp->ls_stateid);
 			if (status != NFS_OK) {
-				nfs41_free_stateid(server, &lsp->ls_stateid);
+				if (status != -NFS4ERR_BAD_STATEID)
+					nfs41_free_stateid(server,
+							&lsp->ls_stateid);
 				lsp->ls_flags &= ~NFS_LOCK_INITIALIZED;
 				ret = status;
 			}

[PATCH 03/15] NFS: State reclaim clears OPEN and LOCK state

[Chuck Lever]

The "state->flags & flags" test in nfs41_check_expired_stateid()
allows the state manager to squelch a TEST_STATEID operation when
it is known for sure that a state ID is no longer valid.  If the
lease was purged, for example, the client already knows that state
ID is now defunct.

But open recovery is still needed for that inode.

To force a call to nfs4_open_expired(), change the default return
value for nfs41_check_expired_stateid() to force open recovery, and
the default return value for nfs41_check_locks() to force lock
recovery, if the requested flags are clear.  Fix suggested by Bryan
Schumaker.

Also, the presence of a delegation state ID must not prevent normal
open recovery.  The delegation state ID must be cleared if it was
revoked, but once cleared I don't think it's presence or absence has
any bearing on whether open recovery is still needed.  So the logic
is adjusted to ignore the TEST_STATEID result for the delegation
state ID.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |   23 ++++++++++++-----------
 1 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 2986e65..5d83f88 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1758,8 +1758,8 @@ static int nfs4_open_expired(struct nfs4_state_owner *sp, struct nfs4_state *sta
 #if defined(CONFIG_NFS_V4_1)
 static int nfs41_check_expired_stateid(struct nfs4_state *state, nfs4_stateid *stateid, unsigned int flags)
 {
-	int status = NFS_OK;
 	struct nfs_server *server = NFS_SERVER(state->inode);
+	int status = -NFS4ERR_BAD_STATEID;
 
 	if (state->flags & flags) {
 		status = nfs41_test_stateid(server, stateid);
@@ -1774,16 +1774,17 @@ static int nfs41_check_expired_stateid(struct nfs4_state *state, nfs4_stateid *s
 
 static int nfs41_open_expired(struct nfs4_state_owner *sp, struct nfs4_state *state)
 {
-	int deleg_status, open_status;
 	int deleg_flags = 1 << NFS_DELEGATED_STATE;
 	int open_flags = (1 << NFS_O_RDONLY_STATE) | (1 << NFS_O_WRONLY_STATE) | (1 << NFS_O_RDWR_STATE);
+	int status;
 
-	deleg_status = nfs41_check_expired_stateid(state, &state->stateid, deleg_flags);
-	open_status = nfs41_check_expired_stateid(state,  &state->open_stateid, open_flags);
+	nfs41_check_expired_stateid(state, &state->stateid, deleg_flags);
+	status = nfs41_check_expired_stateid(state, &state->open_stateid,
+							open_flags);
 
-	if ((deleg_status == NFS_OK) && (open_status == NFS_OK))
-		return NFS_OK;
-	return nfs4_open_expired(sp, state);
+	if (status != NFS_OK)
+		status = nfs4_open_expired(sp, state);
+	return status;
 }
 #endif
 
@@ -4701,7 +4702,7 @@ out:
 #if defined(CONFIG_NFS_V4_1)
 static int nfs41_check_expired_locks(struct nfs4_state *state)
 {
-	int status, ret = NFS_OK;
+	int status, ret = -NFS4ERR_BAD_STATEID;
 	struct nfs4_lock_state *lsp;
 	struct nfs_server *server = NFS_SERVER(state->inode);
 
@@ -4727,9 +4728,9 @@ static int nfs41_lock_expired(struct nfs4_state *state, struct file_lock *reques
 
 	if (test_bit(LK_STATE_IN_USE, &state->flags))
 		status = nfs41_check_expired_locks(state);
-	if (status == NFS_OK)
-		return status;
-	return nfs4_lock_expired(state, request);
+	if (status != NFS_OK)
+		status = nfs4_lock_expired(state, request);
+	return status;
 }
 #endif
 

[PATCH 04/15] NFS: Clean up nfs41_check_expired_stateid()

[Chuck Lever]

Clean up: Instead of open-coded flag manipulation, use test_bit() and
clear_bit() just like all other accessors of the state->flag field.
This also eliminates several unnecessary implicit integer type
conversions.

To make it absolutely clear what is going on, a number of comments
are introduced.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |   77 ++++++++++++++++++++++++++++++++++++++++++-----------
 1 files changed, 61 insertions(+), 16 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 5d83f88..c4df35b 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1756,32 +1756,67 @@ static int nfs4_open_expired(struct nfs4_state_owner *sp, struct nfs4_state *sta
 }
 
 #if defined(CONFIG_NFS_V4_1)
-static int nfs41_check_expired_stateid(struct nfs4_state *state, nfs4_stateid *stateid, unsigned int flags)
+static void nfs41_clear_delegation_stateid(struct nfs4_state *state)
 {
 	struct nfs_server *server = NFS_SERVER(state->inode);
-	int status = -NFS4ERR_BAD_STATEID;
-
-	if (state->flags & flags) {
-		status = nfs41_test_stateid(server, stateid);
-		if (status != NFS_OK) {
-			if (status != -NFS4ERR_BAD_STATEID)
-				nfs41_free_stateid(server, stateid);
-			state->flags &= ~flags;
-		}
+	nfs4_stateid *stateid = &state->stateid;
+	int status;
+
+	/* If a state reset has been done, test_stateid is unneeded */
+	if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
+		return;
+
+	status = nfs41_test_stateid(server, stateid);
+	if (status != NFS_OK) {
+		/* Free the stateid unless the server explicitly
+		 * informs us the stateid is unrecognized. */
+		if (status != -NFS4ERR_BAD_STATEID)
+			nfs41_free_stateid(server, stateid);
+
+		clear_bit(NFS_DELEGATED_STATE, &state->flags);
+	}
+}
+
+/**
+ * nfs41_check_open_stateid - possibly free an open stateid
+ *
+ * @state: NFSv4 state for an inode
+ *
+ * Returns NFS_OK if recovery for this stateid is now finished.
+ * Otherwise a negative NFS4ERR value is returned.
+ */
+static int nfs41_check_open_stateid(struct nfs4_state *state)
+{
+	struct nfs_server *server = NFS_SERVER(state->inode);
+	nfs4_stateid *stateid = &state->stateid;
+	int status;
+
+	/* If a state reset has been done, test_stateid is unneeded */
+	if ((test_bit(NFS_O_RDONLY_STATE, &state->flags) == 0) &&
+	    (test_bit(NFS_O_WRONLY_STATE, &state->flags) == 0) &&
+	    (test_bit(NFS_O_RDWR_STATE, &state->flags) == 0))
+		return -NFS4ERR_BAD_STATEID;
+
+	status = nfs41_test_stateid(server, stateid);
+	if (status != NFS_OK) {
+		/* Free the stateid unless the server explicitly
+		 * informs us the stateid is unrecognized. */
+		if (status != -NFS4ERR_BAD_STATEID)
+			nfs41_free_stateid(server, stateid);
+
+		clear_bit(NFS_O_RDONLY_STATE, &state->flags);
+		clear_bit(NFS_O_WRONLY_STATE, &state->flags);
+		clear_bit(NFS_O_RDWR_STATE, &state->flags);
 	}
 	return status;
 }
 
 static int nfs41_open_expired(struct nfs4_state_owner *sp, struct nfs4_state *state)
 {
-	int deleg_flags = 1 << NFS_DELEGATED_STATE;
-	int open_flags = (1 << NFS_O_RDONLY_STATE) | (1 << NFS_O_WRONLY_STATE) | (1 << NFS_O_RDWR_STATE);
 	int status;
 
-	nfs41_check_expired_stateid(state, &state->stateid, deleg_flags);
-	status = nfs41_check_expired_stateid(state, &state->open_stateid,
-							open_flags);
-
+	nfs41_clear_delegation_stateid(state);
+	status = nfs41_check_open_stateid(state);
 	if (status != NFS_OK)
 		status = nfs4_open_expired(sp, state);
 	return status;
@@ -4700,6 +4735,14 @@ out:
 }
 
 #if defined(CONFIG_NFS_V4_1)
+/**
+ * nfs41_check_expired_locks - possibly free a lock stateid
+ *
+ * @state: NFSv4 state for an inode
+ *
+ * Returns NFS_OK if recovery for this stateid is now finished.
+ * Otherwise a negative NFS4ERR value is returned.
+ */
 static int nfs41_check_expired_locks(struct nfs4_state *state)
 {
 	int status, ret = -NFS4ERR_BAD_STATEID;
@@ -4710,6 +4753,8 @@ static int nfs41_check_expired_locks(struct nfs4_state *state)
 		if (lsp->ls_flags & NFS_LOCK_INITIALIZED) {
 			status = nfs41_test_stateid(server, &lsp->ls_stateid);
 			if (status != NFS_OK) {
+				/* Free the stateid unless the server
+				 * informs us the stateid is unrecognized. */
 				if (status != -NFS4ERR_BAD_STATEID)
 					nfs41_free_stateid(server,
 							&lsp->ls_stateid);

[PATCH 05/15] NFS: Clean up TEST_STATEID and FREE_STATEID error reporting

[Chuck Lever]

As a finishing touch, add appropriate documenting comments and some
debugging printk's.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |   32 ++++++++++++++++++++++++++++++--
 1 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index c4df35b..0d6da38 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6636,13 +6636,28 @@ static int _nfs41_test_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 		.rpc_resp = &res,
 	};
 
+	dprintk("NFS call  test_stateid %p\n", stateid);
 	nfs41_init_sequence(&args.seq_args, &res.seq_res, 0);
 	status = nfs4_call_sync_sequence(server->client, server, &msg, &args.seq_args, &res.seq_res, 1);
-	if (status != NFS_OK)
+	if (status != NFS_OK) {
+		dprintk("NFS reply test_stateid: failed, %d\n", status);
 		return status;
+	}
+	dprintk("NFS reply test_stateid: succeeded, %d\n", -res.status);
+	return -res.status;
 	return -res.status;
 }
 
+/**
+ * nfs41_test_stateid - perform a TEST_STATEID operation
+ *
+ * @server: server / transport on which to perform the operation
+ * @stateid: state ID to test
+ *
+ * Returns NFS_OK if the server recognizes that "stateid" is valid.
+ * Otherwise a negative NFS4ERR value is returned if the operation
+ * failed or the state ID is not currently valid.
+ */
 static int nfs41_test_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 {
 	struct nfs4_exception exception = { };
@@ -6667,12 +6682,25 @@ static int _nfs4_free_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 		.rpc_argp = &args,
 		.rpc_resp = &res,
 	};
+	int status;
 
+	dprintk("NFS call  free_stateid %p\n", stateid);
 	nfs41_init_sequence(&args.seq_args, &res.seq_res, 0);
-	return nfs4_call_sync_sequence(server->client, server, &msg,
+	status = nfs4_call_sync_sequence(server->client, server, &msg,
 					 &args.seq_args, &res.seq_res, 1);
+	dprintk("NFS reply free_stateid: %d\n", status);
+	return status;
 }
 
+/**
+ * nfs41_free_stateid - perform a FREE_STATEID operation
+ *
+ * @server: server / transport on which to perform the operation
+ * @stateid: state ID to release
+ *
+ * Returns NFS_OK if the server freed "stateid".  Otherwise a
+ * negative NFS4ERR value is returned.
+ */
 static int nfs41_free_stateid(struct nfs_server *server, nfs4_stateid *stateid)
 {
 	struct nfs4_exception exception = { };

[PATCH 06/15] NFS: nfs_getaclargs.acl_len is a size_t

[Chuck Lever]

Squelch compiler warnings:

fs/nfs/nfs4proc.c: In function ‘__nfs4_get_acl_uncached’:
fs/nfs/nfs4proc.c:3811:14: warning: comparison between signed and
	unsigned integer expressions [-Wsign-compare]
fs/nfs/nfs4proc.c:3818:15: warning: comparison between signed and
	unsigned integer expressions [-Wsign-compare]

Introduced by commit bf118a34 "NFSv4: include bitmap in nfsv4 get
acl data", Dec 7, 2011.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0d6da38..6444d27 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3786,7 +3786,8 @@ static ssize_t __nfs4_get_acl_uncached(struct inode *inode, void *buf, size_t bu
 		.rpc_argp = &args,
 		.rpc_resp = &res,
 	};
-	int ret = -ENOMEM, npages, i, acl_len = 0;
+	int ret = -ENOMEM, npages, i;
+	size_t acl_len = 0;
 
 	npages = (buflen + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	/* As long as we're doing a round trip to the server anyway,

[PATCH 07/15] NFS: When state recovery fails, waiting tasks should exit

[Chuck Lever]

NFSv4 state recovery is not always successful.  Failure is signalled
by setting the nfs_client.cl_cons_state to a negative (errno) value,
then waking waiters.

Currently this can happen only during mount processing.  I'm about to
add an explicit case where state recovery failure during normal
operation should force all NFS requests waiting on that state recovery
to exit.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 6444d27..997080d 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -259,7 +259,12 @@ static int nfs4_wait_clnt_recover(struct nfs_client *clp)
 
 	res = wait_on_bit(&clp->cl_state, NFS4CLNT_MANAGER_RUNNING,
 			nfs_wait_bit_killable, TASK_KILLABLE);
-	return res;
+	if (res)
+		return res;
+
+	if (clp->cl_cons_state < 0)
+		return clp->cl_cons_state;
+	return 0;
 }
 
 static int nfs4_delay(struct rpc_clnt *clnt, long *timeout)

[PATCH 08/15] NFS: Treat NFS4ERR_CLID_INUSE as a fatal error

[Chuck Lever]

For NFSv4 minor version 0, currently the cl_id_uniquifier allows the
Linux client to generate a unique nfs_client_id4 string whenever a
server replies with NFS4ERR_CLID_INUSE.

This implementation seems to be based on a flawed reading of RFC
3530.  NFS4ERR_CLID_INUSE actually means that the client has presented
this nfs_client_id4 string with a different principal at some time in
the past, and that lease is still in use on the server.

For a Linux client this might be rather difficult to achieve: the
authentication flavor is named right in the nfs_client_id4.id
string.  If we change flavors, we change strings automatically.

So, practically speaking, NFS4ERR_CLID_INUSE means there is some other
client using our string.  There is not much that can be done to
recover automatically.  Let's make it a permanent error.

Remove the recovery logic in nfs4_proc_setclientid(), and remove the
cl_id_uniquifier field from the nfs_client data structure.  And,
remove the authentication flavor from the nfs_client_id4 string.

Keeping the authentication flavor in the nfs_client_id4.id string
means that we could have a separate lease for each authentication
flavor used by mounts on the client.  But we want just one lease for
all the mounts on this client.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c         |   47 +++++++++++++++------------------------------
 fs/nfs/nfs4state.c        |    7 ++++++-
 include/linux/nfs_fs_sb.h |    3 +--
 3 files changed, 23 insertions(+), 34 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 997080d..f5f4e6e 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4037,42 +4037,28 @@ int nfs4_proc_setclientid(struct nfs_client *clp, u32 program,
 		.rpc_resp = res,
 		.rpc_cred = cred,
 	};
-	int loop = 0;
-	int status;
 
+	/* nfs_client_id4 */
 	nfs4_init_boot_verifier(clp, &sc_verifier);
-
-	for(;;) {
-		rcu_read_lock();
-		setclientid.sc_name_len = scnprintf(setclientid.sc_name,
-				sizeof(setclientid.sc_name), "%s/%s %s %s %u",
-				clp->cl_ipaddr,
-				rpc_peeraddr2str(clp->cl_rpcclient,
-							RPC_DISPLAY_ADDR),
-				rpc_peeraddr2str(clp->cl_rpcclient,
-							RPC_DISPLAY_PROTO),
-				clp->cl_rpcclient->cl_auth->au_ops->au_name,
-				clp->cl_id_uniquifier);
-		setclientid.sc_netid_len = scnprintf(setclientid.sc_netid,
+	rcu_read_lock();
+	setclientid.sc_name_len = scnprintf(setclientid.sc_name,
+			sizeof(setclientid.sc_name), "%s/%s %s",
+			clp->cl_ipaddr,
+			rpc_peeraddr2str(clp->cl_rpcclient,
+						RPC_DISPLAY_ADDR),
+			rpc_peeraddr2str(clp->cl_rpcclient,
+						RPC_DISPLAY_PROTO));
+	/* cb_client4 */
+	setclientid.sc_netid_len = scnprintf(setclientid.sc_netid,
 				sizeof(setclientid.sc_netid),
 				rpc_peeraddr2str(clp->cl_rpcclient,
 							RPC_DISPLAY_NETID));
-		setclientid.sc_uaddr_len = scnprintf(setclientid.sc_uaddr,
+	rcu_read_unlock();
+	setclientid.sc_uaddr_len = scnprintf(setclientid.sc_uaddr,
 				sizeof(setclientid.sc_uaddr), "%s.%u.%u",
 				clp->cl_ipaddr, port >> 8, port & 255);
-		rcu_read_unlock();
 
-		status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
-		if (status != -NFS4ERR_CLID_INUSE)
-			break;
-		if (loop != 0) {
-			++clp->cl_id_uniquifier;
-			break;
-		}
-		++loop;
-		ssleep(clp->cl_lease_time / HZ + 1);
-	}
-	return status;
+	return rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
 }
 
 int nfs4_proc_setclientid_confirm(struct nfs_client *clp,
@@ -5270,10 +5256,9 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 	nfs4_init_boot_verifier(clp, &verifier);
 
 	args.id_len = scnprintf(args.id, sizeof(args.id),
-				"%s/%s/%u",
+				"%s/%s",
 				clp->cl_ipaddr,
-				clp->cl_rpcclient->cl_nodename,
-				clp->cl_rpcclient->cl_auth->au_flavor);
+				clp->cl_rpcclient->cl_nodename);
 
 	res.server_owner = kzalloc(sizeof(struct nfs41_server_owner),
 					GFP_NOFS);
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index f38300e..b83c66b 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1606,10 +1606,15 @@ static int nfs4_handle_reclaim_lease_error(struct nfs_client *clp, int status)
 			return -ESERVERFAULT;
 		/* Lease confirmation error: retry after purging the lease */
 		ssleep(1);
-	case -NFS4ERR_CLID_INUSE:
 	case -NFS4ERR_STALE_CLIENTID:
 		clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
 		break;
+	case -NFS4ERR_CLID_INUSE:
+		pr_err("NFS: Server %s reports our clientid is in use\n",
+			clp->cl_hostname);
+		nfs_mark_client_ready(clp, -EPERM);
+		clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
+		return -EPERM;
 	case -EACCES:
 		if (clp->cl_machine_cred == NULL)
 			return -EACCES;
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index f58325a..6532765 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -69,10 +69,9 @@ struct nfs_client {
 	struct idmap *		cl_idmap;
 
 	/* Our own IP address, as a null-terminated string.
-	 * This is used to generate the clientid, and the callback address.
+	 * This is used to generate the mv0 callback address.
 	 */
 	char			cl_ipaddr[48];
-	unsigned char		cl_id_uniquifier;
 	u32			cl_cb_ident;	/* v4.0 callback identifier */
 	const struct nfs4_minor_version_ops *cl_mvops;
 

[PATCH 09/15] NFS: Clean up nfs4_proc_setclientid() and friends

[Chuck Lever]

Add documenting comments and appropriate debugging messages.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c  |   45 +++++++++++++++++++++++++++++++++++++--------
 fs/nfs/nfs4state.c |    4 ++++
 2 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index f5f4e6e..43c678b 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4021,6 +4021,16 @@ static void nfs4_init_boot_verifier(const struct nfs_client *clp,
 	memcpy(bootverf->data, verf, sizeof(bootverf->data));
 }
 
+/**
+ * nfs4_proc_setclientid - Negotiate client ID
+ * @clp: state data structure
+ * @program: RPC program for NFSv4 callback service
+ * @port: IP port number for NFS4 callback service
+ * @cred: RPC credential to use for this call
+ * @res: where to place the result
+ *
+ * Returns zero, a negative errno, or a negative NFS4ERR status code.
+ */
 int nfs4_proc_setclientid(struct nfs_client *clp, u32 program,
 		unsigned short port, struct rpc_cred *cred,
 		struct nfs4_setclientid_res *res)
@@ -4037,6 +4047,7 @@ int nfs4_proc_setclientid(struct nfs_client *clp, u32 program,
 		.rpc_resp = res,
 		.rpc_cred = cred,
 	};
+	int status;
 
 	/* nfs_client_id4 */
 	nfs4_init_boot_verifier(clp, &sc_verifier);
@@ -4058,9 +4069,22 @@ int nfs4_proc_setclientid(struct nfs_client *clp, u32 program,
 				sizeof(setclientid.sc_uaddr), "%s.%u.%u",
 				clp->cl_ipaddr, port >> 8, port & 255);
 
-	return rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+	dprintk("NFS call  setclientid auth=%s, '%.*s'\n",
+		clp->cl_rpcclient->cl_auth->au_ops->au_name,
+		setclientid.sc_name_len, setclientid.sc_name);
+	status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
+	dprintk("NFS reply setclientid: %d\n", status);
+	return status;
 }
 
+/**
+ * nfs4_proc_setclientid_confirm - Confirm client ID
+ * @clp: state data structure
+ * @res: result of a previous SETCLIENTID
+ * @cred: RPC credential to use for this call
+ *
+ * Returns zero, a negative errno, or a negative NFS4ERR status code.
+ */
 int nfs4_proc_setclientid_confirm(struct nfs_client *clp,
 		struct nfs4_setclientid_res *arg,
 		struct rpc_cred *cred)
@@ -4075,6 +4099,9 @@ int nfs4_proc_setclientid_confirm(struct nfs_client *clp,
 	unsigned long now;
 	int status;
 
+	dprintk("NFS call  setclientid_confirm auth=%s, (client ID %llx)\n",
+		clp->cl_rpcclient->cl_auth->au_ops->au_name,
+		clp->cl_clientid);
 	now = jiffies;
 	status = rpc_call_sync(clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
 	if (status == 0) {
@@ -4083,6 +4110,7 @@ int nfs4_proc_setclientid_confirm(struct nfs_client *clp,
 		clp->cl_last_renewal = now;
 		spin_unlock(&clp->cl_lock);
 	}
+	dprintk("NFS reply setclientid_confirm: %d\n", status);
 	return status;
 }
 
@@ -5226,6 +5254,8 @@ out:
 /*
  * nfs4_proc_exchange_id()
  *
+ * Returns zero, a negative errno, or a negative NFS4ERR status code.
+ *
  * Since the clientid has expired, all compounds using sessions
  * associated with the stale clientid will be returning
  * NFS4ERR_BADSESSION in the sequence operation, and will therefore
@@ -5250,15 +5280,14 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 		.rpc_cred = cred,
 	};
 
-	dprintk("--> %s\n", __func__);
-	BUG_ON(clp == NULL);
-
 	nfs4_init_boot_verifier(clp, &verifier);
-
 	args.id_len = scnprintf(args.id, sizeof(args.id),
 				"%s/%s",
 				clp->cl_ipaddr,
 				clp->cl_rpcclient->cl_nodename);
+	dprintk("NFS call  exchange_id auth=%s, '%.*s'\n",
+		clp->cl_rpcclient->cl_auth->au_ops->au_name,
+		args.id_len, args.id);
 
 	res.server_owner = kzalloc(sizeof(struct nfs41_server_owner),
 					GFP_NOFS);
@@ -5321,12 +5350,12 @@ out_server_scope:
 	kfree(res.server_scope);
 out:
 	if (clp->cl_implid != NULL)
-		dprintk("%s: Server Implementation ID: "
+		dprintk("NFS reply exchange_id: Server Implementation ID: "
 			"domain: %s, name: %s, date: %llu,%u\n",
-			__func__, clp->cl_implid->domain, clp->cl_implid->name,
+			clp->cl_implid->domain, clp->cl_implid->name,
 			clp->cl_implid->date.seconds,
 			clp->cl_implid->date.nseconds);
-	dprintk("<-- %s status= %d\n", __func__, status);
+	dprintk("NFS reply exchange_id: %d\n", status);
 	return status;
 }
 
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index b83c66b..0d4c0c6 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1647,6 +1647,10 @@ static int nfs4_handle_reclaim_lease_error(struct nfs_client *clp, int status)
 	return 0;
 }
 
+/*
+ * Returns zero or a negative errno.  NFS4ERR values are converted
+ * to local errno values.
+ */
 static int nfs4_reclaim_lease(struct nfs_client *clp)
 {
 	struct rpc_cred *cred;

[PATCH 10/15] SUNRPC: Add rpcauth_list_flavors()

[Chuck Lever]

The gss_mech_list_pseudoflavors() function provides a list of
currently registered GSS pseudoflavors.  This list does not include
any non-GSS flavors that have been registered with the RPC client.
nfs4_find_root_sec() currently adds these extra flavors by hand.

Instead, nfs4_find_root_sec() should be looking at the set of flavors
that have been explicitly registered via rpcauth_register().  And,
other areas of code will soon need the same kind of list that
contains all flavors the kernel currently knows about (see below).

Rather than cloning the open-coded logic in nfs4_find_root_sec() to
those new places, introduce a generic RPC function that generates a
full list of registered auth flavors and pseudoflavors.

A new rpc_authops method is added that lists a flavor's
pseudoflavors, if it has any.  I encountered an interesting module
loader loop when I tried to get the RPC client to invoke
gss_mech_list_pseudoflavors() by name.

This patch is a pre-requisite for server trunking discovery, and a
pre-requisite for fixing up the in-kernel mount client to do better
automatic security flavor selection.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c                     |   11 ++++---
 include/linux/sunrpc/auth.h           |    2 +
 include/linux/sunrpc/gss_api.h        |    3 +-
 net/sunrpc/auth.c                     |   54 +++++++++++++++++++++++++++++++++
 net/sunrpc/auth_gss/auth_gss.c        |    1 +
 net/sunrpc/auth_gss/gss_mech_switch.c |   18 +++++++++--
 6 files changed, 80 insertions(+), 9 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 43c678b..75bff3f 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -43,7 +43,6 @@
 #include <linux/printk.h>
 #include <linux/slab.h>
 #include <linux/sunrpc/clnt.h>
-#include <linux/sunrpc/gss_api.h>
 #include <linux/nfs.h>
 #include <linux/nfs4.h>
 #include <linux/nfs_fs.h>
@@ -2417,11 +2416,15 @@ static int nfs4_find_root_sec(struct nfs_server *server, struct nfs_fh *fhandle,
 	int i, len, status = 0;
 	rpc_authflavor_t flav_array[NFS_MAX_SECFLAVORS];
 
-	len = gss_mech_list_pseudoflavors(&flav_array[0]);
-	flav_array[len] = RPC_AUTH_NULL;
-	len += 1;
+	len = rpcauth_list_flavors(flav_array, ARRAY_SIZE(flav_array));
+	BUG_ON(len < 0);
 
 	for (i = 0; i < len; i++) {
+		/* AUTH_UNIX is the default flavor if none was specified,
+		 * thus has already been tried. */
+		if (flav_array[i] == RPC_AUTH_UNIX)
+			continue;
+
 		status = nfs4_lookup_root_sec(server, fhandle, info, flav_array[i]);
 		if (status == -NFS4ERR_WRONGSEC || status == -EACCES)
 			continue;
diff --git a/include/linux/sunrpc/auth.h b/include/linux/sunrpc/auth.h
index 492a36d..f25ba92 100644
--- a/include/linux/sunrpc/auth.h
+++ b/include/linux/sunrpc/auth.h
@@ -101,6 +101,7 @@ struct rpc_authops {
 	struct rpc_cred *	(*crcreate)(struct rpc_auth*, struct auth_cred *, int);
 	int			(*pipes_create)(struct rpc_auth *);
 	void			(*pipes_destroy)(struct rpc_auth *);
+	int			(*list_pseudoflavors)(rpc_authflavor_t *, int);
 };
 
 struct rpc_credops {
@@ -135,6 +136,7 @@ int			rpcauth_register(const struct rpc_authops *);
 int			rpcauth_unregister(const struct rpc_authops *);
 struct rpc_auth *	rpcauth_create(rpc_authflavor_t, struct rpc_clnt *);
 void			rpcauth_release(struct rpc_auth *);
+int			rpcauth_list_flavors(rpc_authflavor_t *, int);
 struct rpc_cred *	rpcauth_lookup_credcache(struct rpc_auth *, struct auth_cred *, int);
 void			rpcauth_init_cred(struct rpc_cred *, const struct auth_cred *, struct rpc_auth *, const struct rpc_credops *);
 struct rpc_cred *	rpcauth_lookupcred(struct rpc_auth *, int);
diff --git a/include/linux/sunrpc/gss_api.h b/include/linux/sunrpc/gss_api.h
index 332da61..a19e254 100644
--- a/include/linux/sunrpc/gss_api.h
+++ b/include/linux/sunrpc/gss_api.h
@@ -14,6 +14,7 @@
 
 #ifdef __KERNEL__
 #include <linux/sunrpc/xdr.h>
+#include <linux/sunrpc/msg_prot.h>
 #include <linux/uio.h>
 
 /* The mechanism-independent gss-api context: */
@@ -127,7 +128,7 @@ struct gss_api_mech *gss_mech_get_by_name(const char *);
 struct gss_api_mech *gss_mech_get_by_pseudoflavor(u32);
 
 /* Fill in an array with a list of supported pseudoflavors */
-int gss_mech_list_pseudoflavors(u32 *);
+int gss_mech_list_pseudoflavors(rpc_authflavor_t *, int);
 
 /* Just increments the mechanism's reference count and returns its input: */
 struct gss_api_mech * gss_mech_get(struct gss_api_mech *);
diff --git a/net/sunrpc/auth.c b/net/sunrpc/auth.c
index 727e506..b5c067b 100644
--- a/net/sunrpc/auth.c
+++ b/net/sunrpc/auth.c
@@ -13,6 +13,7 @@
 #include <linux/errno.h>
 #include <linux/hash.h>
 #include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/gss_api.h>
 #include <linux/spinlock.h>
 
 #ifdef RPC_DEBUG
@@ -122,6 +123,59 @@ rpcauth_unregister(const struct rpc_authops *ops)
 }
 EXPORT_SYMBOL_GPL(rpcauth_unregister);
 
+/**
+ * rpcauth_list_flavors - discover registered flavors and pseudoflavors
+ * @array: array to fill in
+ * @size: size of "array"
+ *
+ * Returns the number of array items filled in, or a negative errno.
+ *
+ * The returned array is not sorted by any policy.  Callers should not
+ * rely on the order of the items in the returned array.
+ */
+int
+rpcauth_list_flavors(rpc_authflavor_t *array, int size)
+{
+	rpc_authflavor_t flavor;
+	int result = 0;
+
+	spin_lock(&rpc_authflavor_lock);
+	for (flavor = 0; flavor < RPC_AUTH_MAXFLAVOR; flavor++) {
+		const struct rpc_authops *ops = auth_flavors[flavor];
+		rpc_authflavor_t pseudos[4];
+		int i, len;
+
+		if (result >= size) {
+			result = -ENOMEM;
+			break;
+		}
+
+		if (ops == NULL)
+			continue;
+		if (ops->list_pseudoflavors == NULL) {
+			array[result++] = ops->au_flavor;
+			continue;
+		}
+		len = ops->list_pseudoflavors(pseudos, ARRAY_SIZE(pseudos));
+		if (len < 0) {
+			result = len;
+			break;
+		}
+		for (i = 0; i < len; i++) {
+			if (result >= size) {
+				result = -ENOMEM;
+				break;
+			}
+			array[result++] = pseudos[i];
+		}
+	}
+	spin_unlock(&rpc_authflavor_lock);
+
+	dprintk("RPC:       %s returns %d\n", __func__, result);
+	return result;
+}
+EXPORT_SYMBOL_GPL(rpcauth_list_flavors);
+
 struct rpc_auth *
 rpcauth_create(rpc_authflavor_t pseudoflavor, struct rpc_clnt *clnt)
 {
diff --git a/net/sunrpc/auth_gss/auth_gss.c b/net/sunrpc/auth_gss/auth_gss.c
index d3ad81f..34c5220 100644
--- a/net/sunrpc/auth_gss/auth_gss.c
+++ b/net/sunrpc/auth_gss/auth_gss.c
@@ -1619,6 +1619,7 @@ static const struct rpc_authops authgss_ops = {
 	.crcreate	= gss_create_cred,
 	.pipes_create	= gss_pipes_dentries_create,
 	.pipes_destroy	= gss_pipes_dentries_destroy,
+	.list_pseudoflavors = gss_mech_list_pseudoflavors,
 };
 
 static const struct rpc_credops gss_credops = {
diff --git a/net/sunrpc/auth_gss/gss_mech_switch.c b/net/sunrpc/auth_gss/gss_mech_switch.c
index 782bfe1..6ac5dfc 100644
--- a/net/sunrpc/auth_gss/gss_mech_switch.c
+++ b/net/sunrpc/auth_gss/gss_mech_switch.c
@@ -239,14 +239,26 @@ gss_mech_get_by_pseudoflavor(u32 pseudoflavor)
 
 EXPORT_SYMBOL_GPL(gss_mech_get_by_pseudoflavor);
 
-int gss_mech_list_pseudoflavors(rpc_authflavor_t *array_ptr)
+/**
+ * gss_mech_list_pseudoflavors - Discover registered GSS pseudoflavors
+ * @array: array to fill in
+ * @size: size of "array"
+ *
+ * Returns the number of array items filled in, or a negative errno.
+ *
+ * The returned array is not sorted by any policy.  Callers should not
+ * rely on the order of the items in the returned array.
+ */
+int gss_mech_list_pseudoflavors(rpc_authflavor_t *array_ptr, int size)
 {
 	struct gss_api_mech *pos = NULL;
 	int j, i = 0;
 
 	spin_lock(&registered_mechs_lock);
 	list_for_each_entry(pos, &registered_mechs, gm_list) {
-		for (j=0; j < pos->gm_pf_num; j++) {
+		for (j = 0; j < pos->gm_pf_num; j++) {
+			if (i >= size)
+				return -ENOMEM;
 			array_ptr[i++] = pos->gm_pfs[j].pseudoflavor;
 		}
 	}
@@ -254,8 +266,6 @@ int gss_mech_list_pseudoflavors(rpc_authflavor_t *array_ptr)
 	return i;
 }
 
-EXPORT_SYMBOL_GPL(gss_mech_list_pseudoflavors);
-
 u32
 gss_svc_to_pseudoflavor(struct gss_api_mech *gm, u32 service)
 {

[PATCH 11/15] NFS: Introduce "migration" mount option

[Chuck Lever]

Currently, the Linux client uses a unique nfs_client_id4.id string
when identifying itself to distinct NFS servers.

To support transparent state migration, the Linux client will have to
use the same nfs_client_id4 string for all servers it communicates
with (also known as the "uniform client string" approach).  Otherwise
NFS servers can not recognize that open and lock state need to be
merged after a file system transition.

Unfortunately, there are some NFSv4.0 servers currently in the field
that do not tolerate the uniform client string approach.

Thus, by default, our NFSv4.0 mounts will continue to use the current
approach, and we introduce a mount option that switches them to use
the uniform model.  Client administrators must identify which servers
can be mounted with this option.  Eventually most NFSv4.0 servers will
be able to handle the uniform approach, and we can change the default.

The first mount of a server controls the behavior for all subsequent
mounts for the lifetime of that set of mounts of that server.  After
the last mount of that server is gone, the client erases the data
structure that tracks the lease.  A subsequent lease may then honor
a different "migration" setting.

This patch adds only the infrastructure for parsing the new mount
option.  Support for uniform client strings is added in a subsequent
patch.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/client.c           |    2 ++
 fs/nfs/super.c            |   20 ++++++++++++++++++++
 include/linux/nfs_fs_sb.h |    2 ++
 3 files changed, 24 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index f005b5b..5bad694 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -1451,6 +1451,8 @@ static int nfs4_set_client(struct nfs_server *server,
 
 	if (server->flags & NFS_MOUNT_NORESVPORT)
 		set_bit(NFS_CS_NORESVPORT, &cl_init.init_flags);
+	if (server->options & NFS_OPTION_MIGRATION)
+		set_bit(NFS_CS_MIGRATION, &cl_init.init_flags);
 
 	/* Allocate or find a client reference we can use */
 	clp = nfs_get_client(&cl_init, timeparms, ip_addr, authflavour);
diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 906f09c..63a75eb 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -87,6 +87,7 @@ enum {
 	Opt_sharecache, Opt_nosharecache,
 	Opt_resvport, Opt_noresvport,
 	Opt_fscache, Opt_nofscache,
+	Opt_migration, Opt_nomigration,
 
 	/* Mount options that take integer arguments */
 	Opt_port,
@@ -146,6 +147,8 @@ static const match_table_t nfs_mount_option_tokens = {
 	{ Opt_noresvport, "noresvport" },
 	{ Opt_fscache, "fsc" },
 	{ Opt_nofscache, "nofsc" },
+	{ Opt_migration, "migration" },
+	{ Opt_nomigration, "nomigration" },
 
 	{ Opt_port, "port=%s" },
 	{ Opt_rsize, "rsize=%s" },
@@ -734,6 +737,9 @@ static void nfs_show_mount_options(struct seq_file *m, struct nfs_server *nfss,
 	if (nfss->options & NFS_OPTION_FSCACHE)
 		seq_printf(m, ",fsc");
 
+	if (nfss->options & NFS_OPTION_MIGRATION)
+		seq_printf(m, ",migration");
+
 	if (nfss->flags & NFS_MOUNT_LOOKUP_CACHE_NONEG) {
 		if (nfss->flags & NFS_MOUNT_LOOKUP_CACHE_NONE)
 			seq_printf(m, ",lookupcache=none");
@@ -1296,6 +1302,12 @@ static int nfs_parse_mount_options(char *raw,
 			kfree(mnt->fscache_uniq);
 			mnt->fscache_uniq = NULL;
 			break;
+		case Opt_migration:
+			mnt->options |= NFS_OPTION_MIGRATION;
+			break;
+		case Opt_nomigration:
+			mnt->options &= NFS_OPTION_MIGRATION;
+			break;
 
 		/*
 		 * options that take numeric values
@@ -1588,6 +1600,10 @@ static int nfs_parse_mount_options(char *raw,
 	if (mnt->minorversion && mnt->version != 4)
 		goto out_minorversion_mismatch;
 
+	if (mnt->options & NFS_OPTION_MIGRATION &&
+	    mnt->version != 4 && mnt->minorversion != 0)
+		goto out_migration_misuse;
+
 	/*
 	 * verify that any proto=/mountproto= options match the address
 	 * familiies in the addr=/mountaddr= options.
@@ -1625,6 +1641,10 @@ out_minorversion_mismatch:
 	printk(KERN_INFO "NFS: mount option vers=%u does not support "
 			 "minorversion=%u\n", mnt->version, mnt->minorversion);
 	return 0;
+out_migration_misuse:
+	printk(KERN_INFO
+		"NFS: 'migration' not supported for this NFS version\n");
+	return 0;
 out_nomem:
 	printk(KERN_INFO "NFS: not enough memory to parse option\n");
 	return 0;
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 6532765..abaa47a 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -39,6 +39,7 @@ struct nfs_client {
 	unsigned long		cl_flags;	/* behavior switches */
 #define NFS_CS_NORESVPORT	0		/* - use ephemeral src port */
 #define NFS_CS_DISCRTRY		1		/* - disconnect on RPC retry */
+#define NFS_CS_MIGRATION	2		/* - transparent state migr */
 	struct sockaddr_storage	cl_addr;	/* server identifier */
 	size_t			cl_addrlen;
 	char *			cl_hostname;	/* hostname of server */
@@ -124,6 +125,7 @@ struct nfs_server {
 	unsigned int		namelen;
 	unsigned int		options;	/* extra options enabled by mount */
 #define NFS_OPTION_FSCACHE	0x00000001	/* - local caching enabled */
+#define NFS_OPTION_MIGRATION	0x00000002	/* - NFSv4 migration enabled */
 
 	struct nfs_fsid		fsid;
 	__u64			maxfilesize;	/* maximum file size */

[PATCH 12/15] NFS: Use the same nfs_client_id4 for every server

[Chuck Lever]

Currently, when identifying itself to NFS servers, the Linux NFS
client uses a unique nfs_client_id4.id string for each server IP
address it talks with.  For example, when client A talks to server X,
the client identifies itself using a string like "AX".  The
requirements for these strings are specified in detail by RFC 3530
(and bis).

This form of client identification presents a problem for Transparent
State Migration.  When client A's state on server X is migrated to
server Y, it continues to be associated with string "AX."  But,
according to the rules of client string construction above, client
A will present string "AY" when communicating with server Y.

Server Y thus has no way to know that client A should be associated
with the state migrated from server X.  "AX" is all but abandoned,
interfering with establishing fresh state for client A on server Y.

To support transparent state migration, then, NFSv4.0 clients must
instead use the same nfs_client_id4.id string to identify themselves
to every NFS server; something like "A".

Now a client identifies itself as "A" to server X.  When a file
system on server X transitions to server Y, and client A identifies
itself as "A" to server Y, Y will know immediately that the state
associated with "A," whether it is native or migrated, is owned by
the client, and can merge both into a single lease.

As a pre-requisite to adding support for NFSv4 migration to the Linux
NFS client, this patch changes the way Linux identifies itself to NFS
servers via the SETCLIENTID (NFSv4 minor version 0) and EXCHANGE_ID
(NFSv4 minor version 1) operations.

In addition to removing the server's IP address from nfs_client_id4,
the Linux NFS client will also no longer use its own source IP address
as part of the nfs_client_id4 string.  On multi-homed clients, the
value of this address depends on the address family and network
routing used to contact the server, thus it can be different for each
server.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/nfs4proc.c |   51 +++++++++++++++++++++++++++++++++++++++------------
 1 files changed, 39 insertions(+), 12 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 75bff3f..b09b849 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4024,6 +4024,32 @@ static void nfs4_init_boot_verifier(const struct nfs_client *clp,
 	memcpy(bootverf->data, verf, sizeof(bootverf->data));
 }
 
+static unsigned int
+nfs4_init_nonuniform_client_string(const struct nfs_client *clp,
+				   char *buf, size_t len)
+{
+	unsigned int result;
+
+	rcu_read_lock();
+	result = scnprintf(buf, len, "Linux NFSv4.0 %s/%s %s",
+				clp->cl_ipaddr,
+				rpc_peeraddr2str(clp->cl_rpcclient,
+							RPC_DISPLAY_ADDR),
+				rpc_peeraddr2str(clp->cl_rpcclient,
+							RPC_DISPLAY_PROTO));
+	rcu_read_unlock();
+	return result;
+}
+
+static unsigned int
+nfs4_init_uniform_client_string(const struct nfs_client *clp,
+				char *buf, size_t len)
+{
+	return scnprintf(buf, len, "Linux NFSv%u.%u %s",
+				clp->rpc_ops->version, clp->cl_minorversion,
+				clp->cl_rpcclient->cl_nodename);
+}
+
 /**
  * nfs4_proc_setclientid - Negotiate client ID
  * @clp: state data structure
@@ -4054,15 +4080,18 @@ int nfs4_proc_setclientid(struct nfs_client *clp, u32 program,
 
 	/* nfs_client_id4 */
 	nfs4_init_boot_verifier(clp, &sc_verifier);
-	rcu_read_lock();
-	setclientid.sc_name_len = scnprintf(setclientid.sc_name,
-			sizeof(setclientid.sc_name), "%s/%s %s",
-			clp->cl_ipaddr,
-			rpc_peeraddr2str(clp->cl_rpcclient,
-						RPC_DISPLAY_ADDR),
-			rpc_peeraddr2str(clp->cl_rpcclient,
-						RPC_DISPLAY_PROTO));
+	if (test_bit(NFS_CS_MIGRATION, &clp->cl_flags))
+		setclientid.sc_name_len =
+				nfs4_init_uniform_client_string(clp,
+						setclientid.sc_name,
+						sizeof(setclientid.sc_name));
+	else
+		setclientid.sc_name_len =
+				nfs4_init_nonuniform_client_string(clp,
+						setclientid.sc_name,
+						sizeof(setclientid.sc_name));
 	/* cb_client4 */
+	rcu_read_lock();
 	setclientid.sc_netid_len = scnprintf(setclientid.sc_netid,
 				sizeof(setclientid.sc_netid),
 				rpc_peeraddr2str(clp->cl_rpcclient,
@@ -5284,10 +5313,8 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 	};
 
 	nfs4_init_boot_verifier(clp, &verifier);
-	args.id_len = scnprintf(args.id, sizeof(args.id),
-				"%s/%s",
-				clp->cl_ipaddr,
-				clp->cl_rpcclient->cl_nodename);
+	args.id_len = nfs4_init_uniform_client_string(clp, args.id,
+							sizeof(args.id));
 	dprintk("NFS call  exchange_id auth=%s, '%.*s'\n",
 		clp->cl_rpcclient->cl_auth->au_ops->au_name,
 		args.id_len, args.id);

[PATCH 13/15] NFS: Discover NFSv4 server trunking when mounting

[Chuck Lever]

"Server trunking" is a fancy named for a multi-homed NFS server.
Trunking might occur if a client sends NFS requests for a single
workload to multiple network interfaces on the same server.  There
are some implications for NFSv4 state management that make it useful
for a client to know if a single NFSv4 server instance is
multi-homed.  (Note this is only a consideration for NFSv4, not for
legacy versions of NFS, which are stateless).

If a client cares about server trunking, no NFSv4 operations can
proceed until that client determines who it is talking to.  Thus
server IP trunking discovery must be done when the client first
encounters an unfamiliar server IP address.

The nfs_get_client() function walks the nfs_client_list and matches
on server IP address.  The outcome of that walk tells us immediately
if we have an unfamiliar server IP address.  It invokes
nfs_init_client() in this case.  Thus, nfs4_init_client() is a good
spot to perform trunking discovery.

Discovery requires a client to establish a fresh client ID, so our
client will now send SETCLIENTID or EXCHANGE_ID as the first NFS
operation after a successful ping, rather than waiting for an
application to perform an operation that requires NFSv4 state.

The exact process for detecting trunking is different for NFSv4.0 and
NFSv4.1, so a minorversion-specific init_client callout method is
introduced.

CLID_INUSE recovery is important for the trunking discovery process.
CLID_INUSE is a sign the server recognizes the client's nfs_client_id4
id string, but the client is using the wrong principal this time for
the SETCLIENTID operation.  The SETCLIENTID must be retried with a
series of different principals until one works, and the rest of
trunking discovery can proceed.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 fs/nfs/client.c    |  237 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/internal.h  |    6 +
 fs/nfs/nfs4_fs.h   |    8 ++
 fs/nfs/nfs4proc.c  |    2 
 fs/nfs/nfs4state.c |  186 ++++++++++++++++++++++++++++++++++++++++-
 5 files changed, 433 insertions(+), 6 deletions(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 5bad694..ea1e423 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -576,7 +576,8 @@ nfs_get_client(const struct nfs_client_initdata *cl_init,
 			return nfs_found_client(cl_init, clp);
 		}
 		if (new) {
-			list_add(&new->cl_share_link, &nn->nfs_client_list);
+			list_add_tail(&new->cl_share_link,
+					&nn->nfs_client_list);
 			spin_unlock(&nn->nfs_client_lock);
 			new->cl_flags = cl_init->init_flags;
 			return cl_init->rpc_ops->init_client(new,
@@ -594,6 +595,228 @@ nfs_get_client(const struct nfs_client_initdata *cl_init,
 	return new;
 }
 
+#ifdef CONFIG_NFS_V4
+/*
+ * Returns true if the client IDs match
+ */
+static bool nfs4_match_clientids(struct nfs_client *a, struct nfs_client *b)
+{
+	if (a->cl_clientid != b->cl_clientid) {
+		dprintk("NFS: --> %s client ID %llx does not match %llx\n",
+			__func__, a->cl_clientid, b->cl_clientid);
+		return false;
+	}
+	dprintk("NFS: --> %s client ID %llx matches %llx\n",
+		__func__, a->cl_clientid, b->cl_clientid);
+	return true;
+}
+
+/*
+ * SETCLIENTID just did a callback update with the callback ident in
+ * "drop," but server trunking discovery claims "drop" and "keep" are
+ * actually the same server.  Swap the callback IDs so that "keep"
+ * will continue to use the callback ident the server now knows about,
+ * and so that "keep"'s original callback ident is destroyed when
+ * "drop" is freed.
+ */
+static void nfs4_swap_callback_idents(struct nfs_client *keep,
+				      struct nfs_client *drop)
+{
+	struct nfs_net *nn = net_generic(keep->cl_net, nfs_net_id);
+	unsigned int save = keep->cl_cb_ident;
+
+	if (keep->cl_cb_ident == drop->cl_cb_ident)
+		return;
+
+	dprintk("%s: keeping callback ident %u and dropping ident %u\n",
+		__func__, keep->cl_cb_ident, drop->cl_cb_ident);
+
+	spin_lock(&nn->nfs_client_lock);
+
+	idr_replace(&nn->cb_ident_idr, keep, drop->cl_cb_ident);
+	keep->cl_cb_ident = drop->cl_cb_ident;
+
+	idr_replace(&nn->cb_ident_idr, drop, save);
+	drop->cl_cb_ident = save;
+
+	spin_unlock(&nn->nfs_client_lock);
+}
+
+/**
+ * nfs40_walk_client_list - Find server that recognizes a client ID
+ *
+ * @new: nfs_client with client ID to test
+ * @result: OUT: found nfs_client, or new
+ * @cred: credential to use for trunking test
+ *
+ * Returns zero, a negative errno, or a negative NFS4ERR status.
+ * If zero is returned, an nfs_client pointer is planted in "result."
+ *
+ * NB: nfs40_walk_client_list() relies on the new nfs_client being
+ *     the last nfs_client on the list.
+ */
+int nfs40_walk_client_list(struct nfs_client *new,
+			   struct nfs_client **result,
+			   struct rpc_cred *cred)
+{
+	struct nfs_net *nn = net_generic(new->cl_net, nfs_net_id);
+	struct nfs_client *pos, *prev = NULL;
+	struct nfs4_setclientid_res clid = {
+		.clientid	= new->cl_clientid,
+		.confirm	= new->cl_confirm,
+	};
+	int status;
+
+	spin_lock(&nn->nfs_client_lock);
+
+	list_for_each_entry(pos, &nn->nfs_client_list, cl_share_link) {
+		if (pos->cl_cons_state < 0)
+			continue;
+
+		if (pos->rpc_ops != new->rpc_ops)
+			continue;
+
+		if (pos->cl_proto != new->cl_proto)
+			continue;
+
+		if (pos->cl_minorversion != new->cl_minorversion)
+			continue;
+
+		if (pos->cl_clientid != new->cl_clientid)
+			continue;
+
+		atomic_inc(&pos->cl_count);
+		spin_unlock(&nn->nfs_client_lock);
+
+		if (prev)
+			nfs_put_client(prev);
+
+		status = nfs4_proc_setclientid_confirm(pos, &clid, cred);
+		if (status == 0) {
+			nfs4_swap_callback_idents(pos, new);
+
+			nfs_put_client(pos);
+			*result = pos;
+			dprintk("NFS: <-- %s using nfs_client = %p ({%d})\n",
+				__func__, pos, atomic_read(&pos->cl_count));
+			return 0;
+		}
+		if (status != -NFS4ERR_STALE_CLIENTID) {
+			nfs_put_client(pos);
+			dprintk("NFS: <-- %s status = %d, no result\n",
+				__func__, status);
+			return status;
+		}
+
+		spin_lock(&nn->nfs_client_lock);
+		prev = pos;
+	}
+
+	/*
+	 * No matching nfs_client found.  This should be impossible,
+	 * because the new nfs_client has already been added to
+	 * nfs_client_list by nfs_get_client().
+	 *
+	 * Don't BUG(), since the caller is holding a mutex.
+	 */
+	spin_unlock(&nn->nfs_client_lock);
+	pr_err("NFS: %s Error: no matching nfs_client found\n", __func__);
+	return -NFS4ERR_STALE_CLIENTID;
+}
+
+#ifdef CONFIG_NFS_V4_1
+/*
+ * Returns true if the server owners match
+ */
+static bool
+nfs4_match_serverowners(struct nfs_client *a, struct nfs_client *b)
+{
+	struct nfs41_server_owner *o1 = a->cl_serverowner;
+	struct nfs41_server_owner *o2 = b->cl_serverowner;
+
+	if (o1->minor_id != o2->minor_id) {
+		dprintk("NFS: --> %s server owner minor IDs do not match\n",
+			__func__);
+		return false;
+	}
+
+	if (o1->major_id_sz != o2->major_id_sz)
+		goto out_major_mismatch;
+	if (memcmp(o1->major_id, o2->major_id, o1->major_id_sz) != 0)
+		goto out_major_mismatch;
+
+	dprintk("NFS: --> %s server owners match\n", __func__);
+	return true;
+
+out_major_mismatch:
+	dprintk("NFS: --> %s server owner major IDs do not match\n",
+		__func__);
+	return false;
+}
+
+/**
+ * nfs41_walk_client_list - Find nfs_client that matches a client/server owner
+ *
+ * @new: nfs_client with client ID to test
+ * @result: OUT: found nfs_client, or new
+ * @cred: credential to use for trunking test
+ *
+ * Returns zero, a negative errno, or a negative NFS4ERR status.
+ * If zero is returned, an nfs_client pointer is planted in "result."
+ *
+ * NB: nfs41_walk_client_list() relies on the new nfs_client being
+ *     the last nfs_client on the list.
+ */
+int nfs41_walk_client_list(struct nfs_client *new,
+			   struct nfs_client **result,
+			   struct rpc_cred *cred)
+{
+	struct nfs_net *nn = net_generic(new->cl_net, nfs_net_id);
+	struct nfs_client *pos;
+
+	spin_lock(&nn->nfs_client_lock);
+
+	list_for_each_entry(pos, &nn->nfs_client_list, cl_share_link) {
+		if (pos->cl_cons_state < 0)
+			continue;
+
+		if (pos->rpc_ops != new->rpc_ops)
+			continue;
+
+		if (pos->cl_proto != new->cl_proto)
+			continue;
+
+		if (pos->cl_minorversion != new->cl_minorversion)
+			continue;
+
+		if (!nfs4_match_clientids(pos, new))
+			continue;
+
+		if (!nfs4_match_serverowners(pos, new))
+			continue;
+
+		spin_unlock(&nn->nfs_client_lock);
+		dprintk("NFS: <-- %s using nfs_client = %p ({%d})\n",
+			__func__, pos, atomic_read(&pos->cl_count));
+
+		*result = pos;
+		return 0;
+	}
+
+	/*
+	 * No matching nfs_client found.  This should be impossible,
+	 * because the new nfs_client has already been added to
+	 * nfs_client_list by nfs_get_client().
+	 *
+	 * Don't BUG(), since the caller is holding a mutex.
+	 */
+	spin_unlock(&nn->nfs_client_lock);
+	pr_err("NFS: %s Error: no matching nfs_client found\n", __func__);
+	return -NFS4ERR_STALE_CLIENTID;
+}
+#endif	/* CONFIG_NFS_V4_1 */
+#endif	/* CONFIG_NFS_V4 */
+
 /*
  * Mark a server as ready or failed
  */
@@ -1369,6 +1592,7 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
 				    rpc_authflavor_t authflavour)
 {
 	char buf[INET6_ADDRSTRLEN + 1];
+	struct nfs_client *old;
 	int error;
 
 	if (clp->cl_cons_state == NFS_CS_READY) {
@@ -1414,6 +1638,17 @@ struct nfs_client *nfs4_init_client(struct nfs_client *clp,
 
 	if (!nfs4_has_session(clp))
 		nfs_mark_client_ready(clp, NFS_CS_READY);
+
+	error = nfs4_discover_server_trunking(clp, &old);
+	if (error < 0)
+		goto error;
+	if (clp != old) {
+		nfs_mark_client_ready(clp, NFS_CS_READY);
+		nfs_put_client(clp);
+		clp = old;
+		atomic_inc(&clp->cl_count);
+	}
+
 	return clp;
 
 error:
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 92d1a5e..305a549 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -155,6 +155,12 @@ extern struct nfs_client *nfs4_find_client_ident(struct net *, int);
 extern struct nfs_client *
 nfs4_find_client_sessionid(struct net *, const struct sockaddr *,
 				struct nfs4_sessionid *);
+extern int nfs40_walk_client_list(struct nfs_client *clp,
+				struct nfs_client **result,
+				struct rpc_cred *cred);
+extern int nfs41_walk_client_list(struct nfs_client *clp,
+				struct nfs_client **result,
+				struct rpc_cred *cred);
 extern struct nfs_server *nfs_create_server(
 					const struct nfs_parsed_mount_data *,
 					struct nfs_fh *);
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index cc5900a..2df6668 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -191,6 +191,8 @@ struct nfs4_state_recovery_ops {
 	int (*establish_clid)(struct nfs_client *, struct rpc_cred *);
 	struct rpc_cred * (*get_clid_cred)(struct nfs_client *);
 	int (*reclaim_complete)(struct nfs_client *);
+	int (*detect_trunking)(struct nfs_client *, struct nfs_client **,
+		struct rpc_cred *);
 };
 
 struct nfs4_state_maintenance_ops {
@@ -310,9 +312,15 @@ extern void nfs4_renew_state(struct work_struct *);
 /* nfs4state.c */
 struct rpc_cred *nfs4_get_setclientid_cred(struct nfs_client *clp);
 struct rpc_cred *nfs4_get_renew_cred_locked(struct nfs_client *clp);
+int nfs4_discover_server_trunking(struct nfs_client *clp,
+			struct nfs_client **);
+int nfs40_discover_server_trunking(struct nfs_client *clp,
+			struct nfs_client **, struct rpc_cred *);
 #if defined(CONFIG_NFS_V4_1)
 struct rpc_cred *nfs4_get_machine_cred_locked(struct nfs_client *clp);
 struct rpc_cred *nfs4_get_exchange_id_cred(struct nfs_client *clp);
+int nfs41_discover_server_trunking(struct nfs_client *clp,
+			struct nfs_client **, struct rpc_cred *);
 extern void nfs4_schedule_session_recovery(struct nfs4_session *, int);
 #else
 static inline void nfs4_schedule_session_recovery(struct nfs4_session *session, int err)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index b09b849..bb46ed1 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6794,6 +6794,7 @@ static const struct nfs4_state_recovery_ops nfs40_reboot_recovery_ops = {
 	.recover_lock	= nfs4_lock_reclaim,
 	.establish_clid = nfs4_init_clientid,
 	.get_clid_cred	= nfs4_get_setclientid_cred,
+	.detect_trunking = nfs40_discover_server_trunking,
 };
 
 #if defined(CONFIG_NFS_V4_1)
@@ -6805,6 +6806,7 @@ static const struct nfs4_state_recovery_ops nfs41_reboot_recovery_ops = {
 	.establish_clid = nfs41_init_clientid,
 	.get_clid_cred	= nfs4_get_exchange_id_cred,
 	.reclaim_complete = nfs41_proc_reclaim_complete,
+	.detect_trunking = nfs41_discover_server_trunking,
 };
 #endif /* CONFIG_NFS_V4_1 */
 
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 0d4c0c6..5e3bebc 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -51,6 +51,8 @@
 #include <linux/bitops.h>
 #include <linux/jiffies.h>
 
+#include <linux/sunrpc/clnt.h>
+
 #include "nfs4_fs.h"
 #include "callback.h"
 #include "delegation.h"
@@ -62,7 +64,7 @@
 #define OPENOWNER_POOL_SIZE	8
 
 const nfs4_stateid zero_stateid;
-
+static DEFINE_MUTEX(nfs_clid_init_mutex);
 static LIST_HEAD(nfs4_clientid_list);
 
 int nfs4_init_clientid(struct nfs_client *clp, struct rpc_cred *cred)
@@ -96,6 +98,52 @@ out:
 	return status;
 }
 
+/**
+ * nfs40_discover_server_trunking - Detect server IP address trunking (mv0)
+ *
+ * @clp: nfs_client under test
+ * @result: OUT: found nfs_client, or clp
+ * @cred: credential to use for trunking test
+ *
+ * Returns zero, a negative errno, or a negative NFS4ERR status.
+ * If zero is returned, an nfs_client pointer is planted in
+ * "result".
+ */
+int nfs40_discover_server_trunking(struct nfs_client *clp,
+				   struct nfs_client **result,
+				   struct rpc_cred *cred)
+{
+	struct nfs4_setclientid_res clid = {
+		.clientid = clp->cl_clientid,
+		.confirm = clp->cl_confirm,
+	};
+	unsigned short port;
+	int status;
+
+	port = nfs_callback_tcpport;
+	if (clp->cl_addr.ss_family == AF_INET6)
+		port = nfs_callback_tcpport6;
+
+	status = nfs4_proc_setclientid(clp, NFS4_CALLBACK, port, cred, &clid);
+	if (status != 0)
+		goto out;
+	clp->cl_clientid = clid.clientid;
+	clp->cl_confirm = clid.confirm;
+
+	status = nfs40_walk_client_list(clp, result, cred);
+	if (status == -NFS4ERR_STALE_CLIENTID)
+		set_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
+
+	/*
+	 * Sustain the lease, even if it's empty.  If the clientid4
+	 * goes stale it's of no use for trunking discovery.
+	 */
+	nfs4_schedule_state_renewal(*result);
+
+out:
+	return status;
+}
+
 struct rpc_cred *nfs4_get_machine_cred_locked(struct nfs_client *clp)
 {
 	struct rpc_cred *cred = NULL;
@@ -275,6 +323,46 @@ out:
 	return status;
 }
 
+/**
+ * nfs41_discover_server_trunking - Detect server IP address trunking (mv1)
+ *
+ * @clp: nfs_client under test
+ * @result: OUT: found nfs_client, or clp
+ * @cred: credential to use for trunking test
+ *
+ * Returns NFS4_OK, a negative errno, or a negative NFS4ERR status.
+ * If NFS4_OK is returned, an nfs_client pointer is planted in
+ * "result".
+ */
+int nfs41_discover_server_trunking(struct nfs_client *clp,
+				   struct nfs_client **result,
+				   struct rpc_cred *cred)
+{
+	struct nfs_client *trunked;
+	int status;
+
+	nfs4_begin_drain_session(clp);
+	status = nfs4_proc_exchange_id(clp, cred);
+	if (status != NFS4_OK)
+		goto out;
+
+	status = nfs41_walk_client_list(clp, &trunked, cred);
+	if (status != NFS4_OK)
+		goto out;
+
+	set_bit(NFS4CLNT_LEASE_CONFIRM, &trunked->cl_state);
+	status = nfs4_proc_create_session(trunked, cred);
+	if (status != NFS4_OK)
+		goto out;
+	clear_bit(NFS4CLNT_LEASE_CONFIRM, &trunked->cl_state);
+	nfs41_setup_state_renewal(trunked);
+	nfs_mark_client_ready(trunked, NFS_CS_READY);
+	*result = trunked;
+out:
+	nfs4_end_drain_session(clp);
+	return status;
+}
+
 struct rpc_cred *nfs4_get_exchange_id_cred(struct nfs_client *clp)
 {
 	struct rpc_cred *cred;
@@ -1656,16 +1744,104 @@ static int nfs4_reclaim_lease(struct nfs_client *clp)
 	struct rpc_cred *cred;
 	const struct nfs4_state_recovery_ops *ops =
 		clp->cl_mvops->reboot_recovery_ops;
-	int status;
+	int status = -ENOENT;
 
+	mutex_lock(&nfs_clid_init_mutex);
 	cred = ops->get_clid_cred(clp);
 	if (cred == NULL)
-		return -ENOENT;
+		goto out;
 	status = ops->establish_clid(clp, cred);
 	put_rpccred(cred);
 	if (status != 0)
-		return nfs4_handle_reclaim_lease_error(clp, status);
-	return 0;
+		status = nfs4_handle_reclaim_lease_error(clp, status);
+out:
+	mutex_unlock(&nfs_clid_init_mutex);
+	return status;
+}
+
+/**
+ * nfs4_discover_server_trunking - Detect server IP address trunking
+ *
+ * @clp: nfs_client under test
+ * @result: OUT: found nfs_client, or clp
+ *
+ * Returns zero or a negative errno.  If zero is returned,
+ * an nfs_client pointer is planted in "result".
+ *
+ * Note: since we are invoked in process context, and
+ * not from inside the state manager, we cannot use
+ * nfs4_handle_reclaim_lease_error().
+ */
+int nfs4_discover_server_trunking(struct nfs_client *clp,
+				  struct nfs_client **result)
+{
+	const struct nfs4_state_recovery_ops *ops =
+				clp->cl_mvops->reboot_recovery_ops;
+	rpc_authflavor_t flav, flavors[NFS_MAX_SECFLAVORS];
+	struct rpc_auth *auth;
+	struct rpc_cred *cred;
+	int i, len, status;
+
+	dprintk("NFS: %s: testing '%s'\n", __func__, clp->cl_hostname);
+	mutex_lock(&nfs_clid_init_mutex);
+
+	i = 0;
+	len = rpcauth_list_flavors(flavors, ARRAY_SIZE(flavors));
+
+	status  = -ENOENT;
+again:
+	cred = ops->get_clid_cred(clp);
+	if (cred == NULL)
+		goto out;
+
+	status = ops->detect_trunking(clp, result, cred);
+	put_rpccred(cred);
+	switch (status) {
+	case 0:
+		break;
+
+	case -EACCES:
+		if (clp->cl_machine_cred == NULL)
+			break;
+		/* Handle case where the user hasn't set up machine creds */
+		nfs4_clear_machine_cred(clp);
+	case -NFS4ERR_DELAY:
+	case -ETIMEDOUT:
+	case -EAGAIN:
+		ssleep(1);
+		dprintk("NFS: %s after status %d, retrying\n",
+			__func__, status);
+		goto again;
+
+	case -NFS4ERR_CLID_INUSE:
+	case -NFS4ERR_WRONGSEC:
+		status = -EPERM;
+		if (i >= len)
+			break;
+
+		flav = flavors[i++];
+		auth = rpcauth_create(flav, clp->cl_rpcclient);
+		if (IS_ERR(auth)) {
+			status = PTR_ERR(auth);
+			break;
+		}
+		goto again;
+
+	case -NFS4ERR_MINOR_VERS_MISMATCH:
+		status = -EPROTONOSUPPORT;
+		break;
+
+	case -EKEYEXPIRED:
+		nfs4_warn_keyexpired(clp->cl_hostname);
+	case -NFS4ERR_NOT_SAME: /* FixMe: implement recovery
+				 * in nfs4_exchange_id */
+		status = -EKEYEXPIRED;
+	}
+
+out:
+	mutex_unlock(&nfs_clid_init_mutex);
+	dprintk("NFS: %s: status = %d\n", __func__, status);
+	return status;
 }
 
 #ifdef CONFIG_NFS_V4_1

[PATCH 14/15] NFS: Add nfs4_unique_id boot parameter

[Chuck Lever]

An optional boot parameter is introduced to allow client
administrators to specify a string that the Linux NFS client can
insert into its nfs_client_id4 id string, to make it both more
globally unique, and to ensure that it doesn't change even if the
client's nodename changes.

If this boot parameter is not specified, the client's nodename is
used, as before.

Client installation procedures can create a unique string (typically,
a UUID) which remains unchanged during the lifetime of that client
instance.  This works just like creating a UUID for the label of the
system's root and boot volumes.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---

 Documentation/filesystems/nfs/nfs.txt |   44 +++++++++++++++++++++++++++++++--
 Documentation/kernel-parameters.txt   |    5 ++++
 fs/nfs/nfs4proc.c                     |   12 ++++++++-
 3 files changed, 57 insertions(+), 4 deletions(-)

diff --git a/Documentation/filesystems/nfs/nfs.txt b/Documentation/filesystems/nfs/nfs.txt
index f50f26c..f2571c8 100644
--- a/Documentation/filesystems/nfs/nfs.txt
+++ b/Documentation/filesystems/nfs/nfs.txt
@@ -12,9 +12,47 @@ and work is in progress on adding support for minor version 1 of the NFSv4
 protocol.
 
 The purpose of this document is to provide information on some of the
-upcall interfaces that are used in order to provide the NFS client with
-some of the information that it requires in order to fully comply with
-the NFS spec.
+special features of the NFS client that can be configured by system
+administrators.
+
+
+The nfs4_unique_id parameter
+============================
+
+NFSv4 requires clients to identify themselves to servers with a unique
+string.  File open and lock state shared between one client and one server
+is associated with this identity.  To support robust NFSv4 state recovery
+and transparent state migration, this identity string must not change
+across client reboots.
+
+Without any other intervention, the Linux client uses a string that contains
+the local system's node name.  System administrators, however, often do not
+take care to ensure that node names are fully qualified and do not change
+over the lifetime of a client system.  Node names can have other
+administrative requirements that require particular behavior that does not
+work well as part of an nfs_client_id4 string.
+
+The nfs.nfs4_unique_id boot parameter specifies a unique string that can be
+used instead of a system's node name when an NFS client identifies itself to
+a server.  Thus, if the system's node name is not unique, or it changes, its
+nfs.nfs4_unique_id stays the same, preventing collision with other clients
+or loss of state during NFS reboot recovery or transparent state migration.
+
+The nfs.nfs4_unique_id string is typically a UUID, though it can contain
+anything that is believed to be unique across all NFS clients.  An
+nfs4_unique_id string should be chosen when a client system is installed,
+just as a system's root file system gets a fresh UUID in its label at
+install time.
+
+The string should remain fixed for the lifetime of the client.  It can be
+changed safely if care is taken that the client shuts down cleanly and all
+outstanding NFSv4 state has expired, to prevent loss of NFSv4 state.
+
+This string can be stored in an NFS client's grub.conf, or it can be provided
+via a net boot facility such as PXE.  It may also be specified as an nfs.ko
+module parameter.  Specifying a uniquifier string is not support for NFS
+clients running in containers.
+
 
 The DNS resolver
 ================
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index a92c5eb..733b5ab 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1720,6 +1720,11 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			will be autodetected by the client, and it will fall
 			back to using the idmapper.
 			To turn off this behaviour, set the value to '0'.
+	nfs.nfs4_unique_id=
+			[NFS4] Specify an additional fixed unique ident-
+			ification string that NFSv4 clients can insert into
+			their nfs_client_id4 string.  This is typically a
+			UUID that is generated at system install time.
 
 	nfs.send_implementation_id =
 			[NFSv4.1] Send client implementation identification
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index bb46ed1..9cc2477 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -74,6 +74,12 @@
 
 static unsigned short max_session_slots = NFS4_DEF_SLOT_TABLE_SIZE;
 
+#define NFS4_CLIENT_ID_UNIQ_LEN		(64)
+static char nfs4_client_id_uniquifier[NFS4_CLIENT_ID_UNIQ_LEN] = "";
+module_param_string(nfs4_unique_id, nfs4_client_id_uniquifier,
+			NFS4_CLIENT_ID_UNIQ_LEN, 0600);
+MODULE_PARM_DESC(nfs4_unique_id, "nfs_client_id4 uniquifier string");
+
 struct nfs4_opendata;
 static int _nfs4_proc_open(struct nfs4_opendata *data);
 static int _nfs4_recover_proc_open(struct nfs4_opendata *data);
@@ -4045,9 +4051,13 @@ static unsigned int
 nfs4_init_uniform_client_string(const struct nfs_client *clp,
 				char *buf, size_t len)
 {
+	char *nodename = clp->cl_rpcclient->cl_nodename;
+
+	if (nfs4_client_id_uniquifier[0] != '\0')
+		nodename = nfs4_client_id_uniquifier;
 	return scnprintf(buf, len, "Linux NFSv%u.%u %s",
 				clp->rpc_ops->version, clp->cl_minorversion,
-				clp->cl_rpcclient->cl_nodename);
+				nodename);
 }
 
 /**

[PATCH 15/15] NFS: Slow down state manager after an unhandled error

[Chuck Lever]

If the state manager thread is not actually able to fully recover from
some situation, it wakes up waiters, who kick off a new state manager
thread.  Quite often the fresh invocation of the state manager is just
as successful.

This results in a livelock as the client dumps thousands of NFS
requests a second on the network in a vain attempt to recover.  Not
very friendly.

To mitigate this situation, add a delay in the state manager after
an unhandled error, so that the client sends just a few requests
every second in this case.
---

 fs/nfs/nfs4state.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 5e3bebc..38959eb 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -2151,6 +2151,7 @@ static void nfs4_state_manager(struct nfs_client *clp)
 out_error:
 	pr_warn_ratelimited("NFS: state manager failed on NFSv4 server %s"
 			" with error %d\n", clp->cl_hostname, -status);
+	ssleep(1);
 	nfs4_end_drain_session(clp);
 	nfs4_clear_state_manager_bit(clp);
 }

Re: [PATCH 13/15] NFS: Discover NFSv4 server trunking when mounting

[Myklebust, Trond]

T24gV2VkLCAyMDEyLTA3LTExIGF0IDE2OjMxIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
IlNlcnZlciB0cnVua2luZyIgaXMgYSBmYW5jeSBuYW1lZCBmb3IgYSBtdWx0aS1ob21lZCBORlMg
c2VydmVyLg0KPiBUcnVua2luZyBtaWdodCBvY2N1ciBpZiBhIGNsaWVudCBzZW5kcyBORlMgcmVx
dWVzdHMgZm9yIGEgc2luZ2xlDQo+IHdvcmtsb2FkIHRvIG11bHRpcGxlIG5ldHdvcmsgaW50ZXJm
YWNlcyBvbiB0aGUgc2FtZSBzZXJ2ZXIuICBUaGVyZQ0KPiBhcmUgc29tZSBpbXBsaWNhdGlvbnMg
Zm9yIE5GU3Y0IHN0YXRlIG1hbmFnZW1lbnQgdGhhdCBtYWtlIGl0IHVzZWZ1bA0KPiBmb3IgYSBj
bGllbnQgdG8ga25vdyBpZiBhIHNpbmdsZSBORlN2NCBzZXJ2ZXIgaW5zdGFuY2UgaXMNCj4gbXVs
dGktaG9tZWQuICAoTm90ZSB0aGlzIGlzIG9ubHkgYSBjb25zaWRlcmF0aW9uIGZvciBORlN2NCwg
bm90IGZvcg0KPiBsZWdhY3kgdmVyc2lvbnMgb2YgTkZTLCB3aGljaCBhcmUgc3RhdGVsZXNzKS4N
Cj4gDQo+IElmIGEgY2xpZW50IGNhcmVzIGFib3V0IHNlcnZlciB0cnVua2luZywgbm8gTkZTdjQg
b3BlcmF0aW9ucyBjYW4NCj4gcHJvY2VlZCB1bnRpbCB0aGF0IGNsaWVudCBkZXRlcm1pbmVzIHdo
byBpdCBpcyB0YWxraW5nIHRvLiAgVGh1cw0KPiBzZXJ2ZXIgSVAgdHJ1bmtpbmcgZGlzY292ZXJ5
IG11c3QgYmUgZG9uZSB3aGVuIHRoZSBjbGllbnQgZmlyc3QNCj4gZW5jb3VudGVycyBhbiB1bmZh
bWlsaWFyIHNlcnZlciBJUCBhZGRyZXNzLg0KPiANCj4gVGhlIG5mc19nZXRfY2xpZW50KCkgZnVu
Y3Rpb24gd2Fsa3MgdGhlIG5mc19jbGllbnRfbGlzdCBhbmQgbWF0Y2hlcw0KPiBvbiBzZXJ2ZXIg
SVAgYWRkcmVzcy4gIFRoZSBvdXRjb21lIG9mIHRoYXQgd2FsayB0ZWxscyB1cyBpbW1lZGlhdGVs
eQ0KPiBpZiB3ZSBoYXZlIGFuIHVuZmFtaWxpYXIgc2VydmVyIElQIGFkZHJlc3MuICBJdCBpbnZv
a2VzDQo+IG5mc19pbml0X2NsaWVudCgpIGluIHRoaXMgY2FzZS4gIFRodXMsIG5mczRfaW5pdF9j
bGllbnQoKSBpcyBhIGdvb2QNCj4gc3BvdCB0byBwZXJmb3JtIHRydW5raW5nIGRpc2NvdmVyeS4N
Cj4gDQo+IERpc2NvdmVyeSByZXF1aXJlcyBhIGNsaWVudCB0byBlc3RhYmxpc2ggYSBmcmVzaCBj
bGllbnQgSUQsIHNvIG91cg0KPiBjbGllbnQgd2lsbCBub3cgc2VuZCBTRVRDTElFTlRJRCBvciBF
WENIQU5HRV9JRCBhcyB0aGUgZmlyc3QgTkZTDQo+IG9wZXJhdGlvbiBhZnRlciBhIHN1Y2Nlc3Nm
dWwgcGluZywgcmF0aGVyIHRoYW4gd2FpdGluZyBmb3IgYW4NCj4gYXBwbGljYXRpb24gdG8gcGVy
Zm9ybSBhbiBvcGVyYXRpb24gdGhhdCByZXF1aXJlcyBORlN2NCBzdGF0ZS4NCj4gDQo+IFRoZSBl
eGFjdCBwcm9jZXNzIGZvciBkZXRlY3RpbmcgdHJ1bmtpbmcgaXMgZGlmZmVyZW50IGZvciBORlN2
NC4wIGFuZA0KPiBORlN2NC4xLCBzbyBhIG1pbm9ydmVyc2lvbi1zcGVjaWZpYyBpbml0X2NsaWVu
dCBjYWxsb3V0IG1ldGhvZCBpcw0KPiBpbnRyb2R1Y2VkLg0KPiANCj4gQ0xJRF9JTlVTRSByZWNv
dmVyeSBpcyBpbXBvcnRhbnQgZm9yIHRoZSB0cnVua2luZyBkaXNjb3ZlcnkgcHJvY2Vzcy4NCj4g
Q0xJRF9JTlVTRSBpcyBhIHNpZ24gdGhlIHNlcnZlciByZWNvZ25pemVzIHRoZSBjbGllbnQncyBu
ZnNfY2xpZW50X2lkNA0KPiBpZCBzdHJpbmcsIGJ1dCB0aGUgY2xpZW50IGlzIHVzaW5nIHRoZSB3
cm9uZyBwcmluY2lwYWwgdGhpcyB0aW1lIGZvcg0KPiB0aGUgU0VUQ0xJRU5USUQgb3BlcmF0aW9u
LiAgVGhlIFNFVENMSUVOVElEIG11c3QgYmUgcmV0cmllZCB3aXRoIGENCj4gc2VyaWVzIG9mIGRp
ZmZlcmVudCBwcmluY2lwYWxzIHVudGlsIG9uZSB3b3JrcywgYW5kIHRoZSByZXN0IG9mDQo+IHRy
dW5raW5nIGRpc2NvdmVyeSBjYW4gcHJvY2VlZC4NCj4gDQo+IFNpZ25lZC1vZmYtYnk6IENodWNr
IExldmVyIDxjaHVjay5sZXZlckBvcmFjbGUuY29tPg0KPiAtLS0NCj4gDQo+ICBmcy9uZnMvY2xp
ZW50LmMgICAgfCAgMjM3ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysNCj4gIGZzL25mcy9pbnRlcm5hbC5oICB8ICAgIDYgKw0KPiAgZnMvbmZzL25m
czRfZnMuaCAgIHwgICAgOCArKw0KPiAgZnMvbmZzL25mczRwcm9jLmMgIHwgICAgMiANCj4gIGZz
L25mcy9uZnM0c3RhdGUuYyB8ICAxODYgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKy0NCj4gIDUgZmlsZXMgY2hhbmdlZCwgNDMzIGluc2VydGlvbnMoKyksIDYgZGVsZXRp
b25zKC0pDQoNCkFwcGx5aW5nOiBORlM6IERpc2NvdmVyIE5GU3Y0IHNlcnZlciB0cnVua2luZyB3
aGVuIG1vdW50aW5nDQplcnJvcjogcGF0Y2ggZmFpbGVkOiBmcy9uZnMvbmZzNHN0YXRlLmM6MTY1
Ng0KZXJyb3I6IGZzL25mcy9uZnM0c3RhdGUuYzogcGF0Y2ggZG9lcyBub3QgYXBwbHkNClBhdGNo
IGZhaWxlZCBhdCAwMDAxIE5GUzogRGlzY292ZXIgTkZTdjQgc2VydmVyIHRydW5raW5nIHdoZW4g
bW91bnRpbmcNCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRh
aW5lcg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNv
bQ0KDQo=

Re: [PATCH 05/15] NFS: Clean up TEST_STATEID and FREE_STATEID error reporting

[Myklebust, Trond]

T24gV2VkLCAyMDEyLTA3LTExIGF0IDE2OjMwIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
QXMgYSBmaW5pc2hpbmcgdG91Y2gsIGFkZCBhcHByb3ByaWF0ZSBkb2N1bWVudGluZyBjb21tZW50
cyBhbmQgc29tZQ0KPiBkZWJ1Z2dpbmcgcHJpbnRrJ3MuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBD
aHVjayBMZXZlciA8Y2h1Y2subGV2ZXJAb3JhY2xlLmNvbT4NCj4gLS0tDQo+IA0KPiAgZnMvbmZz
L25mczRwcm9jLmMgfCAgIDMyICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tDQo+ICAx
IGZpbGVzIGNoYW5nZWQsIDMwIGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pDQo+IA0KPiBk
aWZmIC0tZ2l0IGEvZnMvbmZzL25mczRwcm9jLmMgYi9mcy9uZnMvbmZzNHByb2MuYw0KPiBpbmRl
eCBjNGRmMzViLi4wZDZkYTM4IDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvbmZzNHByb2MuYw0KPiAr
KysgYi9mcy9uZnMvbmZzNHByb2MuYw0KPiBAQCAtNjYzNiwxMyArNjYzNiwyOCBAQCBzdGF0aWMg
aW50IF9uZnM0MV90ZXN0X3N0YXRlaWQoc3RydWN0IG5mc19zZXJ2ZXIgKnNlcnZlciwgbmZzNF9z
dGF0ZWlkICpzdGF0ZWlkKQ0KPiAgCQkucnBjX3Jlc3AgPSAmcmVzLA0KPiAgCX07DQo+ICANCj4g
KwlkcHJpbnRrKCJORlMgY2FsbCAgdGVzdF9zdGF0ZWlkICVwXG4iLCBzdGF0ZWlkKTsNCj4gIAlu
ZnM0MV9pbml0X3NlcXVlbmNlKCZhcmdzLnNlcV9hcmdzLCAmcmVzLnNlcV9yZXMsIDApOw0KPiAg
CXN0YXR1cyA9IG5mczRfY2FsbF9zeW5jX3NlcXVlbmNlKHNlcnZlci0+Y2xpZW50LCBzZXJ2ZXIs
ICZtc2csICZhcmdzLnNlcV9hcmdzLCAmcmVzLnNlcV9yZXMsIDEpOw0KPiAtCWlmIChzdGF0dXMg
IT0gTkZTX09LKQ0KPiArCWlmIChzdGF0dXMgIT0gTkZTX09LKSB7DQo+ICsJCWRwcmludGsoIk5G
UyByZXBseSB0ZXN0X3N0YXRlaWQ6IGZhaWxlZCwgJWRcbiIsIHN0YXR1cyk7DQo+ICAJCXJldHVy
biBzdGF0dXM7DQo+ICsJfQ0KPiArCWRwcmludGsoIk5GUyByZXBseSB0ZXN0X3N0YXRlaWQ6IHN1
Y2NlZWRlZCwgJWRcbiIsIC1yZXMuc3RhdHVzKTsNCj4gKwlyZXR1cm4gLXJlcy5zdGF0dXM7DQo+
ICAJcmV0dXJuIC1yZXMuc3RhdHVzOw0KDQpUaGlzIGJpdCBkaWRuJ3QgbG9vayBjbGVhbmVyLCBz
byBJIGxvcGVkIG9mZiB0aGUgZXh0cmEgcmV0dXJuDQpzdGF0ZW1lbnQuLi4NCg0KLS0gDQpUcm9u
ZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25k
Lk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0KDQo=

Re: [PATCH 13/15] NFS: Discover NFSv4 server trunking when mounting

[Chuck Lever]

On Jul 16, 2012, at 3:05 PM, Myklebust, Trond wrote:

> On Wed, 2012-07-11 at 16:31 -0400, Chuck Lever wrote:
>> "Server trunking" is a fancy named for a multi-homed NFS server.
>> Trunking might occur if a client sends NFS requests for a single
>> workload to multiple network interfaces on the same server.  There
>> are some implications for NFSv4 state management that make it useful
>> for a client to know if a single NFSv4 server instance is
>> multi-homed.  (Note this is only a consideration for NFSv4, not for
>> legacy versions of NFS, which are stateless).
>> 
>> If a client cares about server trunking, no NFSv4 operations can
>> proceed until that client determines who it is talking to.  Thus
>> server IP trunking discovery must be done when the client first
>> encounters an unfamiliar server IP address.
>> 
>> The nfs_get_client() function walks the nfs_client_list and matches
>> on server IP address.  The outcome of that walk tells us immediately
>> if we have an unfamiliar server IP address.  It invokes
>> nfs_init_client() in this case.  Thus, nfs4_init_client() is a good
>> spot to perform trunking discovery.
>> 
>> Discovery requires a client to establish a fresh client ID, so our
>> client will now send SETCLIENTID or EXCHANGE_ID as the first NFS
>> operation after a successful ping, rather than waiting for an
>> application to perform an operation that requires NFSv4 state.
>> 
>> The exact process for detecting trunking is different for NFSv4.0 and
>> NFSv4.1, so a minorversion-specific init_client callout method is
>> introduced.
>> 
>> CLID_INUSE recovery is important for the trunking discovery process.
>> CLID_INUSE is a sign the server recognizes the client's nfs_client_id4
>> id string, but the client is using the wrong principal this time for
>> the SETCLIENTID operation.  The SETCLIENTID must be retried with a
>> series of different principals until one works, and the rest of
>> trunking discovery can proceed.
>> 
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>> 
>> fs/nfs/client.c    |  237 ++++++++++++++++++++++++++++++++++++++++++++++++++++
>> fs/nfs/internal.h  |    6 +
>> fs/nfs/nfs4_fs.h   |    8 ++
>> fs/nfs/nfs4proc.c  |    2 
>> fs/nfs/nfs4state.c |  186 ++++++++++++++++++++++++++++++++++++++++-
>> 5 files changed, 433 insertions(+), 6 deletions(-)
> 
> Applying: NFS: Discover NFSv4 server trunking when mounting
> error: patch failed: fs/nfs/nfs4state.c:1656
> error: fs/nfs/nfs4state.c: patch does not apply
> Patch failed at 0001 NFS: Discover NFSv4 server trunking when mounting

I can take a look.  What are you applying it to, exactly?

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH 13/15] NFS: Discover NFSv4 server trunking when mounting

[Myklebust, Trond]

T24gTW9uLCAyMDEyLTA3LTE2IGF0IDE1OjA5IC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
T24gSnVsIDE2LCAyMDEyLCBhdCAzOjA1IFBNLCBNeWtsZWJ1c3QsIFRyb25kIHdyb3RlOg0KPiAN
Cj4gPiBPbiBXZWQsIDIwMTItMDctMTEgYXQgMTY6MzEgLTA0MDAsIENodWNrIExldmVyIHdyb3Rl
Og0KPiA+PiAiU2VydmVyIHRydW5raW5nIiBpcyBhIGZhbmN5IG5hbWVkIGZvciBhIG11bHRpLWhv
bWVkIE5GUyBzZXJ2ZXIuDQo+ID4+IFRydW5raW5nIG1pZ2h0IG9jY3VyIGlmIGEgY2xpZW50IHNl
bmRzIE5GUyByZXF1ZXN0cyBmb3IgYSBzaW5nbGUNCj4gPj4gd29ya2xvYWQgdG8gbXVsdGlwbGUg
bmV0d29yayBpbnRlcmZhY2VzIG9uIHRoZSBzYW1lIHNlcnZlci4gIFRoZXJlDQo+ID4+IGFyZSBz
b21lIGltcGxpY2F0aW9ucyBmb3IgTkZTdjQgc3RhdGUgbWFuYWdlbWVudCB0aGF0IG1ha2UgaXQg
dXNlZnVsDQo+ID4+IGZvciBhIGNsaWVudCB0byBrbm93IGlmIGEgc2luZ2xlIE5GU3Y0IHNlcnZl
ciBpbnN0YW5jZSBpcw0KPiA+PiBtdWx0aS1ob21lZC4gIChOb3RlIHRoaXMgaXMgb25seSBhIGNv
bnNpZGVyYXRpb24gZm9yIE5GU3Y0LCBub3QgZm9yDQo+ID4+IGxlZ2FjeSB2ZXJzaW9ucyBvZiBO
RlMsIHdoaWNoIGFyZSBzdGF0ZWxlc3MpLg0KPiA+PiANCj4gPj4gSWYgYSBjbGllbnQgY2FyZXMg
YWJvdXQgc2VydmVyIHRydW5raW5nLCBubyBORlN2NCBvcGVyYXRpb25zIGNhbg0KPiA+PiBwcm9j
ZWVkIHVudGlsIHRoYXQgY2xpZW50IGRldGVybWluZXMgd2hvIGl0IGlzIHRhbGtpbmcgdG8uICBU
aHVzDQo+ID4+IHNlcnZlciBJUCB0cnVua2luZyBkaXNjb3ZlcnkgbXVzdCBiZSBkb25lIHdoZW4g
dGhlIGNsaWVudCBmaXJzdA0KPiA+PiBlbmNvdW50ZXJzIGFuIHVuZmFtaWxpYXIgc2VydmVyIElQ
IGFkZHJlc3MuDQo+ID4+IA0KPiA+PiBUaGUgbmZzX2dldF9jbGllbnQoKSBmdW5jdGlvbiB3YWxr
cyB0aGUgbmZzX2NsaWVudF9saXN0IGFuZCBtYXRjaGVzDQo+ID4+IG9uIHNlcnZlciBJUCBhZGRy
ZXNzLiAgVGhlIG91dGNvbWUgb2YgdGhhdCB3YWxrIHRlbGxzIHVzIGltbWVkaWF0ZWx5DQo+ID4+
IGlmIHdlIGhhdmUgYW4gdW5mYW1pbGlhciBzZXJ2ZXIgSVAgYWRkcmVzcy4gIEl0IGludm9rZXMN
Cj4gPj4gbmZzX2luaXRfY2xpZW50KCkgaW4gdGhpcyBjYXNlLiAgVGh1cywgbmZzNF9pbml0X2Ns
aWVudCgpIGlzIGEgZ29vZA0KPiA+PiBzcG90IHRvIHBlcmZvcm0gdHJ1bmtpbmcgZGlzY292ZXJ5
Lg0KPiA+PiANCj4gPj4gRGlzY292ZXJ5IHJlcXVpcmVzIGEgY2xpZW50IHRvIGVzdGFibGlzaCBh
IGZyZXNoIGNsaWVudCBJRCwgc28gb3VyDQo+ID4+IGNsaWVudCB3aWxsIG5vdyBzZW5kIFNFVENM
SUVOVElEIG9yIEVYQ0hBTkdFX0lEIGFzIHRoZSBmaXJzdCBORlMNCj4gPj4gb3BlcmF0aW9uIGFm
dGVyIGEgc3VjY2Vzc2Z1bCBwaW5nLCByYXRoZXIgdGhhbiB3YWl0aW5nIGZvciBhbg0KPiA+PiBh
cHBsaWNhdGlvbiB0byBwZXJmb3JtIGFuIG9wZXJhdGlvbiB0aGF0IHJlcXVpcmVzIE5GU3Y0IHN0
YXRlLg0KPiA+PiANCj4gPj4gVGhlIGV4YWN0IHByb2Nlc3MgZm9yIGRldGVjdGluZyB0cnVua2lu
ZyBpcyBkaWZmZXJlbnQgZm9yIE5GU3Y0LjAgYW5kDQo+ID4+IE5GU3Y0LjEsIHNvIGEgbWlub3J2
ZXJzaW9uLXNwZWNpZmljIGluaXRfY2xpZW50IGNhbGxvdXQgbWV0aG9kIGlzDQo+ID4+IGludHJv
ZHVjZWQuDQo+ID4+IA0KPiA+PiBDTElEX0lOVVNFIHJlY292ZXJ5IGlzIGltcG9ydGFudCBmb3Ig
dGhlIHRydW5raW5nIGRpc2NvdmVyeSBwcm9jZXNzLg0KPiA+PiBDTElEX0lOVVNFIGlzIGEgc2ln
biB0aGUgc2VydmVyIHJlY29nbml6ZXMgdGhlIGNsaWVudCdzIG5mc19jbGllbnRfaWQ0DQo+ID4+
IGlkIHN0cmluZywgYnV0IHRoZSBjbGllbnQgaXMgdXNpbmcgdGhlIHdyb25nIHByaW5jaXBhbCB0
aGlzIHRpbWUgZm9yDQo+ID4+IHRoZSBTRVRDTElFTlRJRCBvcGVyYXRpb24uICBUaGUgU0VUQ0xJ
RU5USUQgbXVzdCBiZSByZXRyaWVkIHdpdGggYQ0KPiA+PiBzZXJpZXMgb2YgZGlmZmVyZW50IHBy
aW5jaXBhbHMgdW50aWwgb25lIHdvcmtzLCBhbmQgdGhlIHJlc3Qgb2YNCj4gPj4gdHJ1bmtpbmcg
ZGlzY292ZXJ5IGNhbiBwcm9jZWVkLg0KPiA+PiANCj4gPj4gU2lnbmVkLW9mZi1ieTogQ2h1Y2sg
TGV2ZXIgPGNodWNrLmxldmVyQG9yYWNsZS5jb20+DQo+ID4+IC0tLQ0KPiA+PiANCj4gPj4gZnMv
bmZzL2NsaWVudC5jICAgIHwgIDIzNyArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrDQo+ID4+IGZzL25mcy9pbnRlcm5hbC5oICB8ICAgIDYgKw0KPiA+
PiBmcy9uZnMvbmZzNF9mcy5oICAgfCAgICA4ICsrDQo+ID4+IGZzL25mcy9uZnM0cHJvYy5jICB8
ICAgIDIgDQo+ID4+IGZzL25mcy9uZnM0c3RhdGUuYyB8ICAxODYgKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKy0NCj4gPj4gNSBmaWxlcyBjaGFuZ2VkLCA0MzMgaW5zZXJ0
aW9ucygrKSwgNiBkZWxldGlvbnMoLSkNCj4gPiANCj4gPiBBcHBseWluZzogTkZTOiBEaXNjb3Zl
ciBORlN2NCBzZXJ2ZXIgdHJ1bmtpbmcgd2hlbiBtb3VudGluZw0KPiA+IGVycm9yOiBwYXRjaCBm
YWlsZWQ6IGZzL25mcy9uZnM0c3RhdGUuYzoxNjU2DQo+ID4gZXJyb3I6IGZzL25mcy9uZnM0c3Rh
dGUuYzogcGF0Y2ggZG9lcyBub3QgYXBwbHkNCj4gPiBQYXRjaCBmYWlsZWQgYXQgMDAwMSBORlM6
IERpc2NvdmVyIE5GU3Y0IHNlcnZlciB0cnVua2luZyB3aGVuIG1vdW50aW5nDQo+IA0KPiBJIGNh
biB0YWtlIGEgbG9vay4gIFdoYXQgYXJlIHlvdSBhcHBseWluZyBpdCB0bywgZXhhY3RseT8NCg0K
SGF2ZSBhIGxvb2sgYXQgbXkgJ25mcy1mb3ItY2VsJyBicmFuY2guLi4NCg0KLS0gDQpUcm9uZCBN
eWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15
a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0KDQo=

Re: [PATCH 15/15] NFS: Slow down state manager after an unhandled error

[Myklebust, Trond]

T24gV2VkLCAyMDEyLTA3LTExIGF0IDE2OjMxIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
SWYgdGhlIHN0YXRlIG1hbmFnZXIgdGhyZWFkIGlzIG5vdCBhY3R1YWxseSBhYmxlIHRvIGZ1bGx5
IHJlY292ZXIgZnJvbQ0KPiBzb21lIHNpdHVhdGlvbiwgaXQgd2FrZXMgdXAgd2FpdGVycywgd2hv
IGtpY2sgb2ZmIGEgbmV3IHN0YXRlIG1hbmFnZXINCj4gdGhyZWFkLiAgUXVpdGUgb2Z0ZW4gdGhl
IGZyZXNoIGludm9jYXRpb24gb2YgdGhlIHN0YXRlIG1hbmFnZXIgaXMganVzdA0KPiBhcyBzdWNj
ZXNzZnVsLg0KPiANCj4gVGhpcyByZXN1bHRzIGluIGEgbGl2ZWxvY2sgYXMgdGhlIGNsaWVudCBk
dW1wcyB0aG91c2FuZHMgb2YgTkZTDQo+IHJlcXVlc3RzIGEgc2Vjb25kIG9uIHRoZSBuZXR3b3Jr
IGluIGEgdmFpbiBhdHRlbXB0IHRvIHJlY292ZXIuICBOb3QNCj4gdmVyeSBmcmllbmRseS4NCj4g
DQo+IFRvIG1pdGlnYXRlIHRoaXMgc2l0dWF0aW9uLCBhZGQgYSBkZWxheSBpbiB0aGUgc3RhdGUg
bWFuYWdlciBhZnRlcg0KPiBhbiB1bmhhbmRsZWQgZXJyb3IsIHNvIHRoYXQgdGhlIGNsaWVudCBz
ZW5kcyBqdXN0IGEgZmV3IHJlcXVlc3RzDQo+IGV2ZXJ5IHNlY29uZCBpbiB0aGlzIGNhc2UuDQoN
Cg0KSSBhc3N1bWUgdGhhdCB0aGlzIHdhcyBpbnRlbmRlZCB0byBoYXZlIGEgcy1vLWIgbGluZT8N
Cg0KPiAtLS0NCj4gDQo+ICBmcy9uZnMvbmZzNHN0YXRlLmMgfCAgICAxICsNCj4gIDEgZmlsZXMg
Y2hhbmdlZCwgMSBpbnNlcnRpb25zKCspLCAwIGRlbGV0aW9ucygtKQ0KPiANCj4gZGlmZiAtLWdp
dCBhL2ZzL25mcy9uZnM0c3RhdGUuYyBiL2ZzL25mcy9uZnM0c3RhdGUuYw0KPiBpbmRleCA1ZTNi
ZWJjLi4zODk1OWViIDEwMDY0NA0KPiAtLS0gYS9mcy9uZnMvbmZzNHN0YXRlLmMNCj4gKysrIGIv
ZnMvbmZzL25mczRzdGF0ZS5jDQo+IEBAIC0yMTUxLDYgKzIxNTEsNyBAQCBzdGF0aWMgdm9pZCBu
ZnM0X3N0YXRlX21hbmFnZXIoc3RydWN0IG5mc19jbGllbnQgKmNscCkNCj4gIG91dF9lcnJvcjoN
Cj4gIAlwcl93YXJuX3JhdGVsaW1pdGVkKCJORlM6IHN0YXRlIG1hbmFnZXIgZmFpbGVkIG9uIE5G
U3Y0IHNlcnZlciAlcyINCj4gIAkJCSIgd2l0aCBlcnJvciAlZFxuIiwgY2xwLT5jbF9ob3N0bmFt
ZSwgLXN0YXR1cyk7DQo+ICsJc3NsZWVwKDEpOw0KPiAgCW5mczRfZW5kX2RyYWluX3Nlc3Npb24o
Y2xwKTsNCj4gIAluZnM0X2NsZWFyX3N0YXRlX21hbmFnZXJfYml0KGNscCk7DQo+ICB9DQo+IA0K
DQotLSANClRyb25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5l
dEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQoNCg==

Re: [PATCH 15/15] NFS: Slow down state manager after an unhandled error

[Chuck Lever]

On Jul 17, 2012, at 2:06 PM, Myklebust, Trond wrote:

> On Wed, 2012-07-11 at 16:31 -0400, Chuck Lever wrote:
>> If the state manager thread is not actually able to fully recover from
>> some situation, it wakes up waiters, who kick off a new state manager
>> thread.  Quite often the fresh invocation of the state manager is just
>> as successful.
>> 
>> This results in a livelock as the client dumps thousands of NFS
>> requests a second on the network in a vain attempt to recover.  Not
>> very friendly.
>> 
>> To mitigate this situation, add a delay in the state manager after
>> an unhandled error, so that the client sends just a few requests
>> every second in this case.
> 
> 
> I assume that this was intended to have a s-o-b line?

Yes.  Actually it was submitted mainly for discussion, which is why the S-O-B was accidentally omitted.  But if you feel it is appropriate for mainline now, I'm happy to see it merged.

> 
>> ---
>> 
>> fs/nfs/nfs4state.c |    1 +
>> 1 files changed, 1 insertions(+), 0 deletions(-)
>> 
>> diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
>> index 5e3bebc..38959eb 100644
>> --- a/fs/nfs/nfs4state.c
>> +++ b/fs/nfs/nfs4state.c
>> @@ -2151,6 +2151,7 @@ static void nfs4_state_manager(struct nfs_client *clp)
>> out_error:
>> 	pr_warn_ratelimited("NFS: state manager failed on NFSv4 server %s"
>> 			" with error %d\n", clp->cl_hostname, -status);
>> +	ssleep(1);
>> 	nfs4_end_drain_session(clp);
>> 	nfs4_clear_state_manager_bit(clp);
>> }
>> 
> 
> -- 
> Trond Myklebust
> Linux NFS client maintainer
> 
> NetApp
> Trond.Myklebust@netapp.com
> www.netapp.com
> 

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com