[PATCH v2 00/33] Proposed migration patches for 3.12

[PATCH v2 00/33] Proposed migration patches for 3.12

[Chuck Lever]

This patch series implements NFSv4 migration.  I hope to get this
series merged into 3.12.  See also this topic branch:

 http://git.linux-nfs.org/?p=cel/cel-2.6.git;a=shortlog;\
		h=refs/heads/for-3.12


- Changes since v1 -

 o  Rebased on 3.11-rc4
 o  Review comments addressed
 o  Tighter integration with struct nfs4_slot_table
 o  Graceful failure when migration does not succeed
 o  Temporary kernel config option to disable NFSv4.1 migration


- Patch roadmap -

After a dprintk clean-up patch, patches "NFS: Rename
nfs41_call_sync_data as a common data structure" through "NFS:
Update session draining barriers for NFSv4.0 transport blocking"
implement a transport blocking scheme for NFSv4.0 based on
struct nfs4_slot_table.

Patches "SUNRPC: Modify synopsis of rpc_client_register()" through
"NFS: Add functions to swap transports during migration recovery"
provide a method of swapping an rpc_clnt's rpc_xprt by choosing
a location in an array.

Patches "NFS: Introduce a vector of migration recovery ops" through
"NFS: Add migration recovery callouts in nfs4proc.c" implement
support for recovering from NFS4ERR_MOVED.

Patches "NFS: Add method to detect whether an FSID is still on the
server" through "NFS: Implement NFSv4.1 support for
NFS4ERR_LEASE_MOVED" implement support for recovering from
NFS4ERR_LEASE_MOVED.

The final patch "NFS: Set EXCHGID4_FLAG_SUPP_MOVED_MIGR" adds a
provisional advertisement to NFSv4.1 servers that our client now
supports migration.


- Merge criteria -

I have three criteria for determining whether I believe this work is
ready to merge:

 1.  The series doesn't regress any non-migration related
     functionality

 2.  Everyone is happy with the architecture and APIs, such that
     subsequent patches will amount to bug fixes and enhancements
     rather than reworking anything

 3.  The series is ported to the tip of Linus' master (3.11-rc4
     as of this writing)

I'd like to merge sooner rather than later so others can start to
build on this functionality, and so I don't have to keep forward
porting for another two years.

In addition, this is something that distributions want yesterday.
Including it now and then fixing it makes it easier for them to
include it in their kernels.

I expect bug fixes and enhancements after merge.  Enhancements might
include:

 Optimize LEASE_MOVED recovery by performing the fsid_present
 checks asynchronously

 Introduce support for fs_locations_info

 Finish NFSv4.1 migration support

 Finish support for non-TSM migration

NFSv4.1 migration is almost sure not to work yet, so I've left the
default setting for the EXCHANGE_ID flags as "don't advertise".  We
may have a prototype server to test with soon.  I expect there are
some protocol issues to work out as well.  Once NFSv4.1 migration
is working, the new CONFIG option will be removed.

Migration without TSM showed our prototype server is not behaving
optimally.  However, the Linux client was able to recover open and
lock state successfully.  More testing is needed.


- Test plan -

This implementation still needs a lot of testing.  I've done some of
this list, plan to do more.

  x Builds and runs non-migration workload with CONFIG_NFS_V4_1
    disabled
  x Builds and runs non-migration workload with CONFIG_NFS_V4_1
    enabled
  x Single filesystem migration under load with TSM
  x Single filesystem migration under load without TSM
  x Single filesystem under load migrated twice
  * Active filesystem A while B is migrated (lease-moved recovery)
  * Active filesystem A while active filesystem B is migrated
  * Migrate filesystem A, reboot destination server, migrate B
  * Migration during lock-intensive workload
  x What happens when migration recovery fails?
  * How about migration with a Kerberos 5 mount?

---

Chuck Lever (32):
      NFS: When displaying session slot numbers, use "%u" consistently
      NFS: Rename nfs41_call_sync_data as a common data structure
      NFS: Clean up nfs4_setup_sequence()
      NFS: Common versions of sequence helper functions
      NFS: Add RPC callouts to start NFSv4.0 synchronous requests
      NFS: Remove unused call_sync minor version op
      NFS: Enable slot table helpers for NFSv4.0
      NFS: Add global helper to set up a stand-along nfs4_slot_table
      NFS: Add global helper for releasing slot table resources
      NFS: Add a slot table to struct nfs_client for NFSv4.0 transport blocking
      NFS: NFSv4.0 transport blocking
      NFS: Enable nfs4_setup_sequence() for DELEGRETURN
      NFS: Add nfs4_sequence calls for RELEASE_LOCKOWNER
      NFS: Add nfs4_sequence calls for OPEN_CONFIRM
      NFS: Update session draining barriers for NFSv4.0 transport blocking
      SUNRPC: Modify synopsis of rpc_client_register()
      NFS: Add nfs4_update_server
      NFS: Add functions to swap transports during migration recovery
      NFS: Introduce a vector of migration recovery ops
      NFS: Export _nfs_display_fhandle()
      NFS: Add method to retrieve fs_locations during migration recovery
      NFS: Add a super_block backpointer to the nfs_server struct
      NFS: Add basic migration support to state manager thread
      NFS: Re-use exit code in nfs4_async_handle_error()
      NFS: Rename "stateid_invalid" label
      NFS: Add migration recovery callouts in nfs4proc.c
      NFS: Add method to detect whether an FSID is still on the server
      NFS: Implement support for NFS4ERR_LEASE_MOVED in state manager
      NFS: Implement support for NFS4ERR_LEASE_MOVED
      NFS: Implement support for NFS4ERR_LEASE_MOVED
      NFS: Implement NFSv4.1 support for NFS4ERR_LEASE_MOVED
      NFS: Set EXCHGID4_FLAG_SUPP_MOVED_MIGR

Trond Myklebust (1):
      SUNRPC: Add a helper to switch the transport of an rpc_clnt


 fs/nfs/Kconfig              |   11 +
 fs/nfs/Makefile             |    4 
 fs/nfs/callback_proc.c      |    8 -
 fs/nfs/client.c             |    3 
 fs/nfs/inode.c              |    2 
 fs/nfs/internal.h           |    3 
 fs/nfs/nfs4_fs.h            |   39 ++-
 fs/nfs/nfs4client.c         |  191 +++++++++++--
 fs/nfs/nfs4namespace.c      |  101 +++++++
 fs/nfs/nfs4proc.c           |  641 ++++++++++++++++++++++++++++++++++++-------
 fs/nfs/nfs4session.c        |   84 ++++--
 fs/nfs/nfs4session.h        |   31 +-
 fs/nfs/nfs4state.c          |  289 +++++++++++++++++--
 fs/nfs/nfs4xdr.c            |  126 +++++++-
 fs/nfs/super.c              |    1 
 include/linux/nfs4.h        |    1 
 include/linux/nfs_fs_sb.h   |   11 +
 include/linux/nfs_xdr.h     |   17 +
 include/linux/sunrpc/clnt.h |    4 
 net/sunrpc/clnt.c           |  102 ++++++-
 20 files changed, 1399 insertions(+), 270 deletions(-)

-- 
Chuck Lever

[PATCH v2 01/33] NFS: When displaying session slot numbers, use "%u" consistently

[Chuck Lever]

Clean up, since slot and sequence numbers are all unsigned anyway.

Among other things, squelch compiler warnings:

linux/fs/nfs/nfs4proc.c: In function ‘nfs4_setup_sequence’:
linux/fs/nfs/nfs4proc.c:703:2: warning: signed and unsigned type in
	conditional expression [-Wsign-compare]

and

linux/fs/nfs/nfs4session.c: In function ‘nfs4_alloc_slot’:
linux/fs/nfs/nfs4session.c:151:31: warning: signed and unsigned type in
	conditional expression [-Wsign-compare]

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/callback_proc.c |    8 ++++----
 fs/nfs/nfs4proc.c      |    6 +++---
 fs/nfs/nfs4session.c   |   10 +++++-----
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index e6ebc4c..e6f3b8f 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -301,14 +301,14 @@ validate_seqid(struct nfs4_slot_table *tbl, struct cb_sequenceargs * args)
 {
 	struct nfs4_slot *slot;
 
-	dprintk("%s enter. slotid %d seqid %d\n",
+	dprintk("%s enter. slotid %u seqid %u\n",
 		__func__, args->csa_slotid, args->csa_sequenceid);
 
 	if (args->csa_slotid >= NFS41_BC_MAX_CALLBACKS)
 		return htonl(NFS4ERR_BADSLOT);
 
 	slot = tbl->slots + args->csa_slotid;
-	dprintk("%s slot table seqid: %d\n", __func__, slot->seq_nr);
+	dprintk("%s slot table seqid: %u\n", __func__, slot->seq_nr);
 
 	/* Normal */
 	if (likely(args->csa_sequenceid == slot->seq_nr + 1)) {
@@ -318,7 +318,7 @@ validate_seqid(struct nfs4_slot_table *tbl, struct cb_sequenceargs * args)
 
 	/* Replay */
 	if (args->csa_sequenceid == slot->seq_nr) {
-		dprintk("%s seqid %d is a replay\n",
+		dprintk("%s seqid %u is a replay\n",
 			__func__, args->csa_sequenceid);
 		/* Signal process_op to set this error on next op */
 		if (args->csa_cachethis == 0)
@@ -518,7 +518,7 @@ __be32 nfs4_callback_recallslot(struct cb_recallslotargs *args, void *dummy,
 	if (!cps->clp) /* set in cb_sequence */
 		goto out;
 
-	dprintk_rcu("NFS: CB_RECALL_SLOT request from %s target highest slotid %d\n",
+	dprintk_rcu("NFS: CB_RECALL_SLOT request from %s target highest slotid %u\n",
 		rpc_peeraddr2str(cps->clp->cl_rpcclient, RPC_DISPLAY_ADDR),
 		args->crsa_target_highest_slotid);
 
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 28ed79e..f7a8f53 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -647,7 +647,7 @@ int nfs41_setup_sequence(struct nfs4_session *session,
 
 	args->sa_slot = slot;
 
-	dprintk("<-- %s slotid=%d seqid=%d\n", __func__,
+	dprintk("<-- %s slotid=%u seqid=%u\n", __func__,
 			slot->slot_nr, slot->seq_nr);
 
 	res->sr_slot = slot;
@@ -686,9 +686,9 @@ int nfs4_setup_sequence(const struct nfs_server *server,
 		goto out;
 	}
 
-	dprintk("--> %s clp %p session %p sr_slot %d\n",
+	dprintk("--> %s clp %p session %p sr_slot %u\n",
 		__func__, session->clp, session, res->sr_slot ?
-			res->sr_slot->slot_nr : -1);
+			res->sr_slot->slot_nr : NFS4_NO_SLOT);
 
 	ret = nfs41_setup_sequence(session, args, res, task);
 out:
diff --git a/fs/nfs/nfs4session.c b/fs/nfs/nfs4session.c
index 202e363..0161ad2 100644
--- a/fs/nfs/nfs4session.c
+++ b/fs/nfs/nfs4session.c
@@ -76,7 +76,7 @@ void nfs4_free_slot(struct nfs4_slot_table *tbl, struct nfs4_slot *slot)
 			nfs4_slot_tbl_drain_complete(tbl);
 		}
 	}
-	dprintk("%s: slotid %u highest_used_slotid %d\n", __func__,
+	dprintk("%s: slotid %u highest_used_slotid %u\n", __func__,
 		slotid, tbl->highest_used_slotid);
 }
 
@@ -146,9 +146,9 @@ struct nfs4_slot *nfs4_alloc_slot(struct nfs4_slot_table *tbl)
 	ret->generation = tbl->generation;
 
 out:
-	dprintk("<-- %s used_slots=%04lx highest_used=%d slotid=%d \n",
+	dprintk("<-- %s used_slots=%04lx highest_used=%u slotid=%u\n",
 		__func__, tbl->used_slots[0], tbl->highest_used_slotid,
-		!IS_ERR(ret) ? ret->slot_nr : -1);
+		!IS_ERR(ret) ? ret->slot_nr : NFS4_NO_SLOT);
 	return ret;
 }
 
@@ -191,7 +191,7 @@ static int nfs4_realloc_slot_table(struct nfs4_slot_table *tbl,
 {
 	int ret;
 
-	dprintk("--> %s: max_reqs=%u, tbl->max_slots %d\n", __func__,
+	dprintk("--> %s: max_reqs=%u, tbl->max_slots %u\n", __func__,
 		max_reqs, tbl->max_slots);
 
 	if (max_reqs > NFS4_MAX_SLOT_TABLE)
@@ -205,7 +205,7 @@ static int nfs4_realloc_slot_table(struct nfs4_slot_table *tbl,
 	nfs4_reset_slot_table(tbl, max_reqs - 1, ivalue);
 	spin_unlock(&tbl->slot_tbl_lock);
 
-	dprintk("%s: tbl=%p slots=%p max_slots=%d\n", __func__,
+	dprintk("%s: tbl=%p slots=%p max_slots=%u\n", __func__,
 		tbl, tbl->slots, tbl->max_slots);
 out:
 	dprintk("<-- %s: return %d\n", __func__, ret);

[PATCH v2 02/33] NFS: Rename nfs41_call_sync_data as a common data structure

[Chuck Lever]

Clean up: rename nfs41_call_sync_data for use as a data structure
common to all NFSv4 minor versions.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index f7a8f53..63e2279 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -447,6 +447,12 @@ static void renew_lease(const struct nfs_server *server, unsigned long timestamp
 	do_renew_lease(server->nfs_client, timestamp);
 }
 
+struct nfs4_call_sync_data {
+	const struct nfs_server *seq_server;
+	struct nfs4_sequence_args *seq_args;
+	struct nfs4_sequence_res *seq_res;
+};
+
 #if defined(CONFIG_NFS_V4_1)
 
 static void nfs41_sequence_free_slot(struct nfs4_sequence_res *res)
@@ -696,15 +702,9 @@ out:
 	return ret;
 }
 
-struct nfs41_call_sync_data {
-	const struct nfs_server *seq_server;
-	struct nfs4_sequence_args *seq_args;
-	struct nfs4_sequence_res *seq_res;
-};
-
 static void nfs41_call_sync_prepare(struct rpc_task *task, void *calldata)
 {
-	struct nfs41_call_sync_data *data = calldata;
+	struct nfs4_call_sync_data *data = calldata;
 	struct nfs4_session *session = nfs4_get_session(data->seq_server);
 
 	dprintk("--> %s data->seq_server %p\n", __func__, data->seq_server);
@@ -714,7 +714,7 @@ static void nfs41_call_sync_prepare(struct rpc_task *task, void *calldata)
 
 static void nfs41_call_sync_done(struct rpc_task *task, void *calldata)
 {
-	struct nfs41_call_sync_data *data = calldata;
+	struct nfs4_call_sync_data *data = calldata;
 
 	nfs41_sequence_done(task, data->seq_res);
 }
@@ -732,7 +732,7 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 {
 	int ret;
 	struct rpc_task *task;
-	struct nfs41_call_sync_data data = {
+	struct nfs4_call_sync_data data = {
 		.seq_server = server,
 		.seq_args = args,
 		.seq_res = res,

[PATCH v2 03/33] NFS: Clean up nfs4_setup_sequence()

[Chuck Lever]

Clean up: Both the NFSv4.0 and NFSv4.1 version of
nfs4_setup_sequence() are used only in fs/nfs/nfs4proc.c.  No need
to keep global header declarations for either version.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h  |   11 -----------
 fs/nfs/nfs4proc.c |   16 ++++++++++++----
 2 files changed, 12 insertions(+), 15 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index d7bb59d..0feb238 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -247,9 +247,6 @@ static inline struct nfs4_session *nfs4_get_session(const struct nfs_server *ser
 	return server->nfs_client->cl_session;
 }
 
-extern int nfs4_setup_sequence(const struct nfs_server *server,
-		struct nfs4_sequence_args *args, struct nfs4_sequence_res *res,
-		struct rpc_task *task);
 extern int nfs41_setup_sequence(struct nfs4_session *session,
 		struct nfs4_sequence_args *args, struct nfs4_sequence_res *res,
 		struct rpc_task *task);
@@ -278,14 +275,6 @@ static inline struct nfs4_session *nfs4_get_session(const struct nfs_server *ser
 	return NULL;
 }
 
-static inline int nfs4_setup_sequence(const struct nfs_server *server,
-		struct nfs4_sequence_args *args, struct nfs4_sequence_res *res,
-		struct rpc_task *task)
-{
-	rpc_call_start(task);
-	return 0;
-}
-
 static inline bool
 is_ds_only_client(struct nfs_client *clp)
 {
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 63e2279..275d17d 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -679,10 +679,10 @@ out_sleep:
 }
 EXPORT_SYMBOL_GPL(nfs41_setup_sequence);
 
-int nfs4_setup_sequence(const struct nfs_server *server,
-			struct nfs4_sequence_args *args,
-			struct nfs4_sequence_res *res,
-			struct rpc_task *task)
+static int nfs4_setup_sequence(const struct nfs_server *server,
+			       struct nfs4_sequence_args *args,
+			       struct nfs4_sequence_res *res,
+			       struct rpc_task *task)
 {
 	struct nfs4_session *session = nfs4_get_session(server);
 	int ret = 0;
@@ -765,6 +765,14 @@ static void nfs4_set_sequence_privileged(struct nfs4_sequence_args *args)
 {
 }
 
+static int nfs4_setup_sequence(const struct nfs_server *server,
+			       struct nfs4_sequence_args *args,
+			       struct nfs4_sequence_res *res,
+			       struct rpc_task *task)
+{
+	rpc_call_start(task);
+	return 0;
+}
 
 static int nfs4_sequence_done(struct rpc_task *task,
 			       struct nfs4_sequence_res *res)

[PATCH v2 04/33] NFS: Common versions of sequence helper functions

[Chuck Lever]

NFSv4.0 will have need for this functionality when I add the ability
to block NFSv4.0 traffic before migration recovery.

I'm not really clear on why nfs4_set_sequence_privileged() gets a
generic name, but nfs41_init_sequence() gets a minor
version-specific name.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   79 +++++++++++++++++++++++------------------------------
 1 file changed, 34 insertions(+), 45 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 275d17d..bc8f5d7 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -453,6 +453,21 @@ struct nfs4_call_sync_data {
 	struct nfs4_sequence_res *seq_res;
 };
 
+static void nfs4_init_sequence(struct nfs4_sequence_args *args,
+			       struct nfs4_sequence_res *res, int cache_reply)
+{
+	args->sa_slot = NULL;
+	args->sa_cache_this = cache_reply;
+	args->sa_privileged = 0;
+
+	res->sr_slot = NULL;
+}
+
+static void nfs4_set_sequence_privileged(struct nfs4_sequence_args *args)
+{
+	args->sa_privileged = 1;
+}
+
 #if defined(CONFIG_NFS_V4_1)
 
 static void nfs41_sequence_free_slot(struct nfs4_sequence_res *res)
@@ -600,22 +615,6 @@ static int nfs4_sequence_done(struct rpc_task *task,
 	return nfs41_sequence_done(task, res);
 }
 
-static void nfs41_init_sequence(struct nfs4_sequence_args *args,
-		struct nfs4_sequence_res *res, int cache_reply)
-{
-	args->sa_slot = NULL;
-	args->sa_cache_this = 0;
-	args->sa_privileged = 0;
-	if (cache_reply)
-		args->sa_cache_this = 1;
-	res->sr_slot = NULL;
-}
-
-static void nfs4_set_sequence_privileged(struct nfs4_sequence_args *args)
-{
-	args->sa_privileged = 1;
-}
-
 int nfs41_setup_sequence(struct nfs4_session *session,
 				struct nfs4_sequence_args *args,
 				struct nfs4_sequence_res *res,
@@ -755,16 +754,6 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 }
 
 #else
-static
-void nfs41_init_sequence(struct nfs4_sequence_args *args,
-		struct nfs4_sequence_res *res, int cache_reply)
-{
-}
-
-static void nfs4_set_sequence_privileged(struct nfs4_sequence_args *args)
-{
-}
-
 static int nfs4_setup_sequence(const struct nfs_server *server,
 			       struct nfs4_sequence_args *args,
 			       struct nfs4_sequence_res *res,
@@ -799,7 +788,7 @@ int nfs4_call_sync(struct rpc_clnt *clnt,
 		   struct nfs4_sequence_res *res,
 		   int cache_reply)
 {
-	nfs41_init_sequence(args, res, cache_reply);
+	nfs4_init_sequence(args, res, cache_reply);
 	return server->nfs_client->cl_mvops->call_sync(clnt, server, msg,
 						args, res);
 }
@@ -1750,7 +1739,7 @@ static int nfs4_run_open_task(struct nfs4_opendata *data, int isrecover)
 	};
 	int status;
 
-	nfs41_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1);
+	nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1);
 	kref_get(&data->kref);
 	data->rpc_done = 0;
 	data->rpc_status = 0;
@@ -2522,7 +2511,7 @@ int nfs4_do_close(struct nfs4_state *state, gfp_t gfp_mask, int wait)
 	calldata = kzalloc(sizeof(*calldata), gfp_mask);
 	if (calldata == NULL)
 		goto out;
-	nfs41_init_sequence(&calldata->arg.seq_args, &calldata->res.seq_res, 1);
+	nfs4_init_sequence(&calldata->arg.seq_args, &calldata->res.seq_res, 1);
 	calldata->inode = state->inode;
 	calldata->state = state;
 	calldata->arg.fh = NFS_FH(state->inode);
@@ -3278,7 +3267,7 @@ static void nfs4_proc_unlink_setup(struct rpc_message *msg, struct inode *dir)
 
 	res->server = server;
 	msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_REMOVE];
-	nfs41_init_sequence(&args->seq_args, &res->seq_res, 1);
+	nfs4_init_sequence(&args->seq_args, &res->seq_res, 1);
 
 	nfs_fattr_init(res->dir_attr);
 }
@@ -3311,7 +3300,7 @@ static void nfs4_proc_rename_setup(struct rpc_message *msg, struct inode *dir)
 
 	msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_RENAME];
 	res->server = server;
-	nfs41_init_sequence(&arg->seq_args, &res->seq_res, 1);
+	nfs4_init_sequence(&arg->seq_args, &res->seq_res, 1);
 }
 
 static void nfs4_proc_rename_rpc_prepare(struct rpc_task *task, struct nfs_renamedata *data)
@@ -3912,7 +3901,7 @@ static void nfs4_proc_read_setup(struct nfs_read_data *data, struct rpc_message
 	data->timestamp   = jiffies;
 	data->read_done_cb = nfs4_read_done_cb;
 	msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_READ];
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 0);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 0);
 }
 
 static void nfs4_proc_read_rpc_prepare(struct rpc_task *task, struct nfs_read_data *data)
@@ -3995,7 +3984,7 @@ static void nfs4_proc_write_setup(struct nfs_write_data *data, struct rpc_messag
 	data->timestamp   = jiffies;
 
 	msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_WRITE];
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 }
 
 static void nfs4_proc_write_rpc_prepare(struct rpc_task *task, struct nfs_write_data *data)
@@ -4043,7 +4032,7 @@ static void nfs4_proc_commit_setup(struct nfs_commit_data *data, struct rpc_mess
 		data->commit_done_cb = nfs4_commit_done_cb;
 	data->res.server = server;
 	msg->rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_COMMIT];
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 }
 
 struct nfs4_renewdata {
@@ -4845,7 +4834,7 @@ static int _nfs4_proc_delegreturn(struct inode *inode, struct rpc_cred *cred, co
 	data = kzalloc(sizeof(*data), GFP_NOFS);
 	if (data == NULL)
 		return -ENOMEM;
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 	data->args.fhandle = &data->fh;
 	data->args.stateid = &data->stateid;
 	data->args.bitmask = server->cache_consistency_bitmask;
@@ -5108,7 +5097,7 @@ static struct rpc_task *nfs4_do_unlck(struct file_lock *fl,
 		return ERR_PTR(-ENOMEM);
 	}
 
-	nfs41_init_sequence(&data->arg.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->arg.seq_args, &data->res.seq_res, 1);
 	msg.rpc_argp = &data->arg;
 	msg.rpc_resp = &data->res;
 	task_setup_data.callback_data = data;
@@ -5343,7 +5332,7 @@ static int _nfs4_do_setlk(struct nfs4_state *state, int cmd, struct file_lock *f
 		return -ENOMEM;
 	if (IS_SETLKW(cmd))
 		data->arg.block = 1;
-	nfs41_init_sequence(&data->arg.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->arg.seq_args, &data->res.seq_res, 1);
 	msg.rpc_argp = &data->arg;
 	msg.rpc_resp = &data->res;
 	task_setup_data.callback_data = data;
@@ -6165,7 +6154,7 @@ int nfs4_proc_get_lease_time(struct nfs_client *clp, struct nfs_fsinfo *fsinfo)
 	};
 	int status;
 
-	nfs41_init_sequence(&args.la_seq_args, &res.lr_seq_res, 0);
+	nfs4_init_sequence(&args.la_seq_args, &res.lr_seq_res, 0);
 	nfs4_set_sequence_privileged(&args.la_seq_args);
 	dprintk("--> %s\n", __func__);
 	task = rpc_run_task(&task_setup);
@@ -6468,7 +6457,7 @@ static struct rpc_task *_nfs41_proc_sequence(struct nfs_client *clp,
 		nfs_put_client(clp);
 		return ERR_PTR(-ENOMEM);
 	}
-	nfs41_init_sequence(&calldata->args, &calldata->res, 0);
+	nfs4_init_sequence(&calldata->args, &calldata->res, 0);
 	if (is_privileged)
 		nfs4_set_sequence_privileged(&calldata->args);
 	msg.rpc_argp = &calldata->args;
@@ -6610,7 +6599,7 @@ static int nfs41_proc_reclaim_complete(struct nfs_client *clp,
 	calldata->clp = clp;
 	calldata->arg.one_fs = 0;
 
-	nfs41_init_sequence(&calldata->arg.seq_args, &calldata->res.seq_res, 0);
+	nfs4_init_sequence(&calldata->arg.seq_args, &calldata->res.seq_res, 0);
 	nfs4_set_sequence_privileged(&calldata->arg.seq_args);
 	msg.rpc_argp = &calldata->arg;
 	msg.rpc_resp = &calldata->res;
@@ -6801,7 +6790,7 @@ nfs4_proc_layoutget(struct nfs4_layoutget *lgp, gfp_t gfp_flags)
 
 	lgp->res.layoutp = &lgp->args.layout;
 	lgp->res.seq_res.sr_slot = NULL;
-	nfs41_init_sequence(&lgp->args.seq_args, &lgp->res.seq_res, 0);
+	nfs4_init_sequence(&lgp->args.seq_args, &lgp->res.seq_res, 0);
 
 	/* nfs4_layoutget_release calls pnfs_put_layout_hdr */
 	pnfs_get_layout_hdr(NFS_I(inode)->layout);
@@ -6892,7 +6881,7 @@ int nfs4_proc_layoutreturn(struct nfs4_layoutreturn *lrp)
 	int status;
 
 	dprintk("--> %s\n", __func__);
-	nfs41_init_sequence(&lrp->args.seq_args, &lrp->res.seq_res, 1);
+	nfs4_init_sequence(&lrp->args.seq_args, &lrp->res.seq_res, 1);
 	task = rpc_run_task(&task_setup_data);
 	if (IS_ERR(task))
 		return PTR_ERR(task);
@@ -7073,7 +7062,7 @@ nfs4_proc_layoutcommit(struct nfs4_layoutcommit_data *data, bool sync)
 		data->args.lastbytewritten,
 		data->args.inode->i_ino);
 
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 1);
 	task = rpc_run_task(&task_setup_data);
 	if (IS_ERR(task))
 		return PTR_ERR(task);
@@ -7186,7 +7175,7 @@ static int _nfs41_test_stateid(struct nfs_server *server,
 	};
 
 	dprintk("NFS call  test_stateid %p\n", stateid);
-	nfs41_init_sequence(&args.seq_args, &res.seq_res, 0);
+	nfs4_init_sequence(&args.seq_args, &res.seq_res, 0);
 	nfs4_set_sequence_privileged(&args.seq_args);
 	status = nfs4_call_sync_sequence(server->client, server, &msg,
 			&args.seq_args, &res.seq_res);
@@ -7291,7 +7280,7 @@ static struct rpc_task *_nfs41_free_stateid(struct nfs_server *server,
 
 	msg.rpc_argp = &data->args;
 	msg.rpc_resp = &data->res;
-	nfs41_init_sequence(&data->args.seq_args, &data->res.seq_res, 0);
+	nfs4_init_sequence(&data->args.seq_args, &data->res.seq_res, 0);
 	if (privileged)
 		nfs4_set_sequence_privileged(&data->args.seq_args);
 

[PATCH v2 05/33] NFS: Add RPC callouts to start NFSv4.0 synchronous requests

[Chuck Lever]

Refactor nfs4_call_sync_sequence() so it is used for NFSv4.0 now.
The RPC callouts will house transport blocking logic similar to
NFSv4.1 sessions.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h  |    1 +
 fs/nfs/nfs4proc.c |   61 ++++++++++++++++++++++++++++++++++++-----------------
 2 files changed, 42 insertions(+), 20 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 0feb238..bf2184e 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -49,6 +49,7 @@ struct nfs4_minor_version_ops {
 			struct nfs_fsinfo *);
 	int	(*free_lock_state)(struct nfs_server *,
 			struct nfs4_lock_state *);
+	const struct rpc_call_ops *call_sync_ops;
 	const struct nfs4_state_recovery_ops *reboot_recovery_ops;
 	const struct nfs4_state_recovery_ops *nograce_recovery_ops;
 	const struct nfs4_state_maintenance_ops *state_renewal_ops;
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index bc8f5d7..940a39e 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -723,6 +723,41 @@ static const struct rpc_call_ops nfs41_call_sync_ops = {
 	.rpc_call_done = nfs41_call_sync_done,
 };
 
+#else
+static int nfs4_setup_sequence(const struct nfs_server *server,
+			       struct nfs4_sequence_args *args,
+			       struct nfs4_sequence_res *res,
+			       struct rpc_task *task)
+{
+	rpc_call_start(task);
+	return 0;
+}
+
+static int nfs4_sequence_done(struct rpc_task *task,
+			       struct nfs4_sequence_res *res)
+{
+	return 1;
+}
+#endif /* CONFIG_NFS_V4_1 */
+
+static void nfs40_call_sync_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_call_sync_data *data = calldata;
+	nfs4_setup_sequence(data->seq_server,
+				data->seq_args, data->seq_res, task);
+}
+
+static void nfs40_call_sync_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_call_sync_data *data = calldata;
+	nfs4_sequence_done(task, data->seq_res);
+}
+
+static const struct rpc_call_ops nfs40_call_sync_ops = {
+	.rpc_call_prepare = nfs40_call_sync_prepare,
+	.rpc_call_done = nfs40_call_sync_done,
+};
+
 static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 				   struct nfs_server *server,
 				   struct rpc_message *msg,
@@ -731,6 +766,7 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 {
 	int ret;
 	struct rpc_task *task;
+	struct nfs_client *clp = server->nfs_client;
 	struct nfs4_call_sync_data data = {
 		.seq_server = server,
 		.seq_args = args,
@@ -739,7 +775,7 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 	struct rpc_task_setup task_setup = {
 		.rpc_client = clnt,
 		.rpc_message = msg,
-		.callback_ops = &nfs41_call_sync_ops,
+		.callback_ops = clp->cl_mvops->call_sync_ops,
 		.callback_data = &data
 	};
 
@@ -753,23 +789,6 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 	return ret;
 }
 
-#else
-static int nfs4_setup_sequence(const struct nfs_server *server,
-			       struct nfs4_sequence_args *args,
-			       struct nfs4_sequence_res *res,
-			       struct rpc_task *task)
-{
-	rpc_call_start(task);
-	return 0;
-}
-
-static int nfs4_sequence_done(struct rpc_task *task,
-			       struct nfs4_sequence_res *res)
-{
-	return 1;
-}
-#endif /* CONFIG_NFS_V4_1 */
-
 static
 int _nfs4_call_sync(struct rpc_clnt *clnt,
 		    struct nfs_server *server,
@@ -789,8 +808,7 @@ int nfs4_call_sync(struct rpc_clnt *clnt,
 		   int cache_reply)
 {
 	nfs4_init_sequence(args, res, cache_reply);
-	return server->nfs_client->cl_mvops->call_sync(clnt, server, msg,
-						args, res);
+	return nfs4_call_sync_sequence(clnt, server, msg, args, res);
 }
 
 static void update_changeattr(struct inode *dir, struct nfs4_change_info *cinfo)
@@ -7413,6 +7431,7 @@ static const struct nfs4_minor_version_ops nfs_v4_0_minor_ops = {
 	.match_stateid = nfs4_match_stateid,
 	.find_root_sec = nfs4_find_root_sec,
 	.free_lock_state = nfs4_release_lockowner,
+	.call_sync_ops = &nfs40_call_sync_ops,
 	.reboot_recovery_ops = &nfs40_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs40_nograce_recovery_ops,
 	.state_renewal_ops = &nfs40_state_renewal_ops,
@@ -7431,6 +7450,7 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
+	.call_sync_ops = &nfs41_call_sync_ops,
 	.reboot_recovery_ops = &nfs41_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs41_nograce_recovery_ops,
 	.state_renewal_ops = &nfs41_state_renewal_ops,
@@ -7450,6 +7470,7 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = {
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
+	.call_sync_ops = &nfs41_call_sync_ops,
 	.reboot_recovery_ops = &nfs41_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs41_nograce_recovery_ops,
 	.state_renewal_ops = &nfs41_state_renewal_ops,

[PATCH v2 06/33] NFS: Remove unused call_sync minor version op

[Chuck Lever]

Clean up.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h  |    5 -----
 fs/nfs/nfs4proc.c |   13 -------------
 2 files changed, 18 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index bf2184e..8de9b93 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -38,11 +38,6 @@ struct nfs4_minor_version_ops {
 	u32	minor_version;
 	unsigned init_caps;
 
-	int	(*call_sync)(struct rpc_clnt *clnt,
-			struct nfs_server *server,
-			struct rpc_message *msg,
-			struct nfs4_sequence_args *args,
-			struct nfs4_sequence_res *res);
 	bool	(*match_stateid)(const nfs4_stateid *,
 			const nfs4_stateid *);
 	int	(*find_root_sec)(struct nfs_server *, struct nfs_fh *,
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 940a39e..0ba7e35 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -790,16 +790,6 @@ static int nfs4_call_sync_sequence(struct rpc_clnt *clnt,
 }
 
 static
-int _nfs4_call_sync(struct rpc_clnt *clnt,
-		    struct nfs_server *server,
-		    struct rpc_message *msg,
-		    struct nfs4_sequence_args *args,
-		    struct nfs4_sequence_res *res)
-{
-	return rpc_call_sync(clnt, msg, 0);
-}
-
-static
 int nfs4_call_sync(struct rpc_clnt *clnt,
 		   struct nfs_server *server,
 		   struct rpc_message *msg,
@@ -7427,7 +7417,6 @@ static const struct nfs4_minor_version_ops nfs_v4_0_minor_ops = {
 		| NFS_CAP_ATOMIC_OPEN
 		| NFS_CAP_CHANGE_ATTR
 		| NFS_CAP_POSIX_LOCK,
-	.call_sync = _nfs4_call_sync,
 	.match_stateid = nfs4_match_stateid,
 	.find_root_sec = nfs4_find_root_sec,
 	.free_lock_state = nfs4_release_lockowner,
@@ -7446,7 +7435,6 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
 		| NFS_CAP_ATOMIC_OPEN_V1,
-	.call_sync = nfs4_call_sync_sequence,
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
@@ -7466,7 +7454,6 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = {
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
 		| NFS_CAP_ATOMIC_OPEN_V1,
-	.call_sync = nfs4_call_sync_sequence,
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,

[PATCH v2 07/33] NFS: Enable slot table helpers for NFSv4.0

[Chuck Lever]

I'd like to re-use NFSv4.1's slot table machinery for NFSv4.0
transport blocking.  Re-organize some of nfs4session.c so the slot
table code is built even when NFS_V4_1 is disabled.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/Makefile      |    4 ++--
 fs/nfs/nfs4session.c |   29 +++++++++++++++++++++--------
 fs/nfs/nfs4session.h |   28 +++++++++++++---------------
 fs/nfs/nfs4state.c   |    9 ---------
 4 files changed, 36 insertions(+), 34 deletions(-)

diff --git a/fs/nfs/Makefile b/fs/nfs/Makefile
index e0bb048..2839bfe 100644
--- a/fs/nfs/Makefile
+++ b/fs/nfs/Makefile
@@ -21,10 +21,10 @@ nfsv3-$(CONFIG_NFS_V3_ACL) += nfs3acl.o
 obj-$(CONFIG_NFS_V4) += nfsv4.o
 nfsv4-y := nfs4proc.o nfs4xdr.o nfs4state.o nfs4renewd.o nfs4super.o nfs4file.o \
 	  delegation.o idmap.o callback.o callback_xdr.o callback_proc.o \
-	  nfs4namespace.o nfs4getroot.o nfs4client.o dns_resolve.o
+	  nfs4namespace.o nfs4getroot.o nfs4client.o dns_resolve.o nfs4session.o
 nfsv4-$(CONFIG_NFS_USE_LEGACY_DNS) += cache_lib.o
 nfsv4-$(CONFIG_SYSCTL)	+= nfs4sysctl.o
-nfsv4-$(CONFIG_NFS_V4_1)	+= nfs4session.o pnfs.o pnfs_dev.o
+nfsv4-$(CONFIG_NFS_V4_1)	+= pnfs.o pnfs_dev.o
 
 obj-$(CONFIG_PNFS_FILE_LAYOUT) += nfs_layout_nfsv41_files.o
 nfs_layout_nfsv41_files-y := nfs4filelayout.o nfs4filelayoutdev.o
diff --git a/fs/nfs/nfs4session.c b/fs/nfs/nfs4session.c
index 0161ad2..746675b 100644
--- a/fs/nfs/nfs4session.c
+++ b/fs/nfs/nfs4session.c
@@ -44,6 +44,17 @@ static void nfs4_shrink_slot_table(struct nfs4_slot_table  *tbl, u32 newsize)
 	}
 }
 
+/**
+ * nfs4_slot_tbl_drain_complete - wake waiters when drain is complete
+ * @tbl - controlling slot table
+ *
+ */
+void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl)
+{
+	if (nfs4_slot_tbl_draining(tbl))
+		complete(&tbl->complete);
+}
+
 /*
  * nfs4_free_slot - free a slot and efficiently update slot table.
  *
@@ -212,13 +223,6 @@ out:
 	return ret;
 }
 
-/* Destroy the slot table */
-static void nfs4_destroy_slot_tables(struct nfs4_session *session)
-{
-	nfs4_shrink_slot_table(&session->fc_slot_table, 0);
-	nfs4_shrink_slot_table(&session->bc_slot_table, 0);
-}
-
 static bool nfs41_assign_slot(struct rpc_task *task, void *pslot)
 {
 	struct nfs4_sequence_args *args = task->tk_msg.rpc_argp;
@@ -383,6 +387,15 @@ void nfs41_update_target_slotid(struct nfs4_slot_table *tbl,
 	spin_unlock(&tbl->slot_tbl_lock);
 }
 
+#if defined(CONFIG_NFS_V4_1)
+
+/* Destroy the slot table */
+static void nfs4_destroy_slot_tables(struct nfs4_session *session)
+{
+	nfs4_shrink_slot_table(&session->fc_slot_table, 0);
+	nfs4_shrink_slot_table(&session->bc_slot_table, 0);
+}
+
 /*
  * Initialize or reset the forechannel and backchannel tables
  */
@@ -513,4 +526,4 @@ int nfs4_init_ds_session(struct nfs_client *clp, unsigned long lease_time)
 }
 EXPORT_SYMBOL_GPL(nfs4_init_ds_session);
 
-
+#endif	/* defined(CONFIG_NFS_V4_1) */
diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
index 3a153d8..23d26b4 100644
--- a/fs/nfs/nfs4session.h
+++ b/fs/nfs/nfs4session.h
@@ -72,15 +72,24 @@ enum nfs4_session_state {
 	NFS4_SESSION_INITING,
 };
 
-#if defined(CONFIG_NFS_V4_1)
 extern struct nfs4_slot *nfs4_alloc_slot(struct nfs4_slot_table *tbl);
 extern void nfs4_free_slot(struct nfs4_slot_table *tbl, struct nfs4_slot *slot);
-
-extern void nfs41_set_target_slotid(struct nfs4_slot_table *tbl,
-		u32 target_highest_slotid);
+extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);
 extern void nfs41_update_target_slotid(struct nfs4_slot_table *tbl,
 		struct nfs4_slot *slot,
 		struct nfs4_sequence_res *res);
+bool nfs41_wake_and_assign_slot(struct nfs4_slot_table *tbl,
+		struct nfs4_slot *slot);
+void nfs41_wake_slot_table(struct nfs4_slot_table *tbl);
+
+static inline bool nfs4_slot_tbl_draining(struct nfs4_slot_table *tbl)
+{
+	return !!test_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);
+}
+
+#if defined(CONFIG_NFS_V4_1)
+extern void nfs41_set_target_slotid(struct nfs4_slot_table *tbl,
+		u32 target_highest_slotid);
 
 extern int nfs4_setup_session_slot_tables(struct nfs4_session *ses);
 
@@ -89,17 +98,6 @@ extern void nfs4_destroy_session(struct nfs4_session *session);
 extern int nfs4_init_session(struct nfs_client *clp);
 extern int nfs4_init_ds_session(struct nfs_client *, unsigned long);
 
-extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);
-
-static inline bool nfs4_slot_tbl_draining(struct nfs4_slot_table *tbl)
-{
-	return !!test_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);
-}
-
-bool nfs41_wake_and_assign_slot(struct nfs4_slot_table *tbl,
-		struct nfs4_slot *slot);
-void nfs41_wake_slot_table(struct nfs4_slot_table *tbl);
-
 /*
  * Determine if sessions are in use.
  */
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 6818964..8826df3 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -260,15 +260,6 @@ static void nfs4_end_drain_session(struct nfs_client *clp)
 	}
 }
 
-/*
- * Signal state manager thread if session fore channel is drained
- */
-void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl)
-{
-	if (nfs4_slot_tbl_draining(tbl))
-		complete(&tbl->complete);
-}
-
 static int nfs4_drain_slot_tbl(struct nfs4_slot_table *tbl)
 {
 	set_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);

[PATCH v2 08/33] NFS: Add global helper to set up a stand-along nfs4_slot_table

[Chuck Lever]

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4session.c |   38 +++++++++++++++++++++++++-------------
 fs/nfs/nfs4session.h |    2 ++
 2 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/fs/nfs/nfs4session.c b/fs/nfs/nfs4session.c
index 746675b..be273c5 100644
--- a/fs/nfs/nfs4session.c
+++ b/fs/nfs/nfs4session.c
@@ -23,6 +23,14 @@
 
 #define NFSDBG_FACILITY		NFSDBG_STATE
 
+static void nfs4_init_slot_table(struct nfs4_slot_table *tbl, const char *queue)
+{
+	tbl->highest_used_slotid = NFS4_NO_SLOT;
+	spin_lock_init(&tbl->slot_tbl_lock);
+	rpc_init_priority_wait_queue(&tbl->slot_tbl_waitq, queue);
+	init_completion(&tbl->complete);
+}
+
 /*
  * nfs4_shrink_slot_table - free retired slots from the slot table
  */
@@ -223,6 +231,21 @@ out:
 	return ret;
 }
 
+/**
+ * nfs4_setup_slot_table - prepare a stand-alone slot table for use
+ * @tbl: slot table to set up
+ * @max_reqs: maximum number of requests allowed
+ * @queue: name to give RPC wait queue
+ *
+ * Returns zero on success, or a negative errno.
+ */
+int nfs4_setup_slot_table(struct nfs4_slot_table *tbl, unsigned int max_reqs,
+		const char *queue)
+{
+	nfs4_init_slot_table(tbl, queue);
+	return nfs4_realloc_slot_table(tbl, max_reqs, 0);
+}
+
 static bool nfs41_assign_slot(struct rpc_task *task, void *pslot)
 {
 	struct nfs4_sequence_args *args = task->tk_msg.rpc_argp;
@@ -425,24 +448,13 @@ int nfs4_setup_session_slot_tables(struct nfs4_session *ses)
 struct nfs4_session *nfs4_alloc_session(struct nfs_client *clp)
 {
 	struct nfs4_session *session;
-	struct nfs4_slot_table *tbl;
 
 	session = kzalloc(sizeof(struct nfs4_session), GFP_NOFS);
 	if (!session)
 		return NULL;
 
-	tbl = &session->fc_slot_table;
-	tbl->highest_used_slotid = NFS4_NO_SLOT;
-	spin_lock_init(&tbl->slot_tbl_lock);
-	rpc_init_priority_wait_queue(&tbl->slot_tbl_waitq, "ForeChannel Slot table");
-	init_completion(&tbl->complete);
-
-	tbl = &session->bc_slot_table;
-	tbl->highest_used_slotid = NFS4_NO_SLOT;
-	spin_lock_init(&tbl->slot_tbl_lock);
-	rpc_init_wait_queue(&tbl->slot_tbl_waitq, "BackChannel Slot table");
-	init_completion(&tbl->complete);
-
+	nfs4_init_slot_table(&session->fc_slot_table, "ForeChannel Slot table");
+	nfs4_init_slot_table(&session->bc_slot_table, "BackChannel Slot table");
 	session->session_state = 1<<NFS4_SESSION_INITING;
 
 	session->clp = clp;
diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
index 23d26b4..7781838 100644
--- a/fs/nfs/nfs4session.h
+++ b/fs/nfs/nfs4session.h
@@ -72,6 +72,8 @@ enum nfs4_session_state {
 	NFS4_SESSION_INITING,
 };
 
+extern int nfs4_setup_slot_table(struct nfs4_slot_table *tbl,
+		unsigned int max_reqs, const char *queue);
 extern struct nfs4_slot *nfs4_alloc_slot(struct nfs4_slot_table *tbl);
 extern void nfs4_free_slot(struct nfs4_slot_table *tbl, struct nfs4_slot *slot);
 extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);

[PATCH v2 09/33] NFS: Add global helper for releasing slot table resources

[Chuck Lever]

The nfs4_destroy_slot_tables() function is renamed to avoid
confusion with the new helper.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4session.c |   21 +++++++++++++++------
 fs/nfs/nfs4session.h |    1 +
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/fs/nfs/nfs4session.c b/fs/nfs/nfs4session.c
index be273c5..e0bf666 100644
--- a/fs/nfs/nfs4session.c
+++ b/fs/nfs/nfs4session.c
@@ -232,6 +232,16 @@ out:
 }
 
 /**
+ * nfs4_release_slot_table - release resources attached to a slot table
+ * @tbl: slot table to shut down
+ *
+ */
+void nfs4_release_slot_table(struct nfs4_slot_table *tbl)
+{
+	nfs4_shrink_slot_table(tbl, 0);
+}
+
+/**
  * nfs4_setup_slot_table - prepare a stand-alone slot table for use
  * @tbl: slot table to set up
  * @max_reqs: maximum number of requests allowed
@@ -412,11 +422,10 @@ void nfs41_update_target_slotid(struct nfs4_slot_table *tbl,
 
 #if defined(CONFIG_NFS_V4_1)
 
-/* Destroy the slot table */
-static void nfs4_destroy_slot_tables(struct nfs4_session *session)
+static void nfs4_destroy_session_slot_tables(struct nfs4_session *session)
 {
-	nfs4_shrink_slot_table(&session->fc_slot_table, 0);
-	nfs4_shrink_slot_table(&session->bc_slot_table, 0);
+	nfs4_release_slot_table(&session->fc_slot_table);
+	nfs4_release_slot_table(&session->bc_slot_table);
 }
 
 /*
@@ -441,7 +450,7 @@ int nfs4_setup_session_slot_tables(struct nfs4_session *ses)
 	if (status && tbl->slots == NULL)
 		/* Fore and back channel share a connection so get
 		 * both slot tables or neither */
-		nfs4_destroy_slot_tables(ses);
+		nfs4_destroy_session_slot_tables(ses);
 	return status;
 }
 
@@ -477,7 +486,7 @@ void nfs4_destroy_session(struct nfs4_session *session)
 	dprintk("%s Destroy backchannel for xprt %p\n",
 		__func__, xprt);
 	xprt_destroy_backchannel(xprt, NFS41_BC_MIN_CALLBACKS);
-	nfs4_destroy_slot_tables(session);
+	nfs4_destroy_session_slot_tables(session);
 	kfree(session);
 }
 
diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
index 7781838..92c27f4 100644
--- a/fs/nfs/nfs4session.h
+++ b/fs/nfs/nfs4session.h
@@ -74,6 +74,7 @@ enum nfs4_session_state {
 
 extern int nfs4_setup_slot_table(struct nfs4_slot_table *tbl,
 		unsigned int max_reqs, const char *queue);
+extern void nfs4_release_slot_table(struct nfs4_slot_table *tbl);
 extern struct nfs4_slot *nfs4_alloc_slot(struct nfs4_slot_table *tbl);
 extern void nfs4_free_slot(struct nfs4_slot_table *tbl, struct nfs4_slot *slot);
 extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);

[PATCH v2 10/33] NFS: Add a slot table to struct nfs_client for NFSv4.0 transport blocking

[Chuck Lever]

Anchor an nfs4_slot_table in the nfs_client for use with NFSv4.0
transport blocking.  It is initialized only for NFSv4.0 nfs_client's.

Introduce appropriate minor version ops to handle nfs_client
initialization and shutdown requirements that differ for each minor
version.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h          |    6 +++
 fs/nfs/nfs4client.c       |  101 +++++++++++++++++++++++++++++++++------------
 fs/nfs/nfs4proc.c         |    6 +++
 include/linux/nfs_fs_sb.h |    3 +
 4 files changed, 89 insertions(+), 27 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 8de9b93..af2d5bf 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -38,6 +38,8 @@ struct nfs4_minor_version_ops {
 	u32	minor_version;
 	unsigned init_caps;
 
+	int	(*init_client)(struct nfs_client *);
+	void	(*shutdown_client)(struct nfs_client *);
 	bool	(*match_stateid)(const nfs4_stateid *,
 			const nfs4_stateid *);
 	int	(*find_root_sec)(struct nfs_server *, struct nfs_fh *,
@@ -292,6 +294,10 @@ extern const u32 nfs4_pathconf_bitmap[3];
 extern const u32 nfs4_fsinfo_bitmap[3];
 extern const u32 nfs4_fs_locations_bitmap[3];
 
+void nfs40_shutdown_client(struct nfs_client *);
+void nfs41_shutdown_client(struct nfs_client *);
+int nfs40_init_client(struct nfs_client *);
+int nfs41_init_client(struct nfs_client *);
 void nfs4_free_client(struct nfs_client *);
 
 struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *);
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index 90dce91..eedd686 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -41,7 +41,7 @@ static int nfs_get_cb_ident_idr(struct nfs_client *clp, int minorversion)
 }
 
 #ifdef CONFIG_NFS_V4_1
-static void nfs4_shutdown_session(struct nfs_client *clp)
+void nfs41_shutdown_client(struct nfs_client *clp)
 {
 	if (nfs4_has_session(clp)) {
 		nfs4_destroy_session(clp->cl_session);
@@ -49,11 +49,15 @@ static void nfs4_shutdown_session(struct nfs_client *clp)
 	}
 
 }
-#else /* CONFIG_NFS_V4_1 */
-static void nfs4_shutdown_session(struct nfs_client *clp)
+#endif	/* CONFIG_NFS_V4_1 */
+
+void nfs40_shutdown_client(struct nfs_client *clp)
 {
+	if (clp->cl_slot_tbl) {
+		nfs4_release_slot_table(clp->cl_slot_tbl);
+		kfree(clp->cl_slot_tbl);
+	}
 }
-#endif /* CONFIG_NFS_V4_1 */
 
 struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init)
 {
@@ -97,7 +101,7 @@ static void nfs4_shutdown_client(struct nfs_client *clp)
 {
 	if (__test_and_clear_bit(NFS_CS_RENEWD, &clp->cl_res_state))
 		nfs4_kill_renewd(clp);
-	nfs4_shutdown_session(clp);
+	clp->cl_mvops->shutdown_client(clp);
 	nfs4_destroy_callback(clp);
 	if (__test_and_clear_bit(NFS_CS_IDMAP, &clp->cl_res_state))
 		nfs_idmap_delete(clp);
@@ -144,34 +148,77 @@ static int nfs4_init_callback(struct nfs_client *clp)
 	return 0;
 }
 
+/**
+ * nfs40_init_client - nfs_client initialization tasks for NFSv4.0
+ * @clp - nfs_client to initialize
+ *
+ * Returns zero on success, or a negative errno if some error occurred.
+ */
+int nfs40_init_client(struct nfs_client *clp)
+{
+	struct nfs4_slot_table *tbl;
+	int ret;
+
+	tbl = kzalloc(sizeof(*tbl), GFP_NOFS);
+	if (tbl == NULL)
+		return -ENOMEM;
+
+	ret = nfs4_setup_slot_table(tbl, NFS4_MAX_SLOT_TABLE,
+					"NFSv4.0 transport Slot table");
+	if (ret) {
+		kfree(tbl);
+		return ret;
+	}
+
+	clp->cl_slot_tbl = tbl;
+	return 0;
+}
+
+#if defined(CONFIG_NFS_V4_1)
+
+/**
+ * nfs41_init_client - nfs_client initialization tasks for NFSv4.1+
+ * @clp - nfs_client to initialize
+ *
+ * Returns zero on success, or a negative errno if some error occurred.
+ */
+int nfs41_init_client(struct nfs_client *clp)
+{
+	struct nfs4_session *session = NULL;
+
+	/*
+	 * Create the session and mark it expired.
+	 * When a SEQUENCE operation encounters the expired session
+	 * it will do session recovery to initialize it.
+	 */
+	session = nfs4_alloc_session(clp);
+	if (!session)
+		return -ENOMEM;
+
+	clp->cl_session = session;
+
+	/*
+	 * The create session reply races with the server back
+	 * channel probe. Mark the client NFS_CS_SESSION_INITING
+	 * so that the client back channel can find the
+	 * nfs_client struct
+	 */
+	nfs_mark_client_ready(clp, NFS_CS_SESSION_INITING);
+	return 0;
+}
+
+#endif	/* CONFIG_NFS_V4_1 */
+
 /*
  * Initialize the minor version specific parts of an NFS4 client record
  */
 static int nfs4_init_client_minor_version(struct nfs_client *clp)
 {
-#if defined(CONFIG_NFS_V4_1)
-	if (clp->cl_mvops->minor_version) {
-		struct nfs4_session *session = NULL;
-		/*
-		 * Create the session and mark it expired.
-		 * When a SEQUENCE operation encounters the expired session
-		 * it will do session recovery to initialize it.
-		 */
-		session = nfs4_alloc_session(clp);
-		if (!session)
-			return -ENOMEM;
-
-		clp->cl_session = session;
-		/*
-		 * The create session reply races with the server back
-		 * channel probe. Mark the client NFS_CS_SESSION_INITING
-		 * so that the client back channel can find the
-		 * nfs_client struct
-		 */
-		nfs_mark_client_ready(clp, NFS_CS_SESSION_INITING);
-	}
-#endif /* CONFIG_NFS_V4_1 */
+	int ret;
 
+	ret = clp->cl_mvops->init_client(clp);
+	if (ret)
+		return ret;
 	return nfs4_init_callback(clp);
 }
 
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0ba7e35..4352d51 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -7417,6 +7417,8 @@ static const struct nfs4_minor_version_ops nfs_v4_0_minor_ops = {
 		| NFS_CAP_ATOMIC_OPEN
 		| NFS_CAP_CHANGE_ATTR
 		| NFS_CAP_POSIX_LOCK,
+	.init_client = nfs40_init_client,
+	.shutdown_client = nfs40_shutdown_client,
 	.match_stateid = nfs4_match_stateid,
 	.find_root_sec = nfs4_find_root_sec,
 	.free_lock_state = nfs4_release_lockowner,
@@ -7435,6 +7437,8 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
 		| NFS_CAP_ATOMIC_OPEN_V1,
+	.init_client = nfs41_init_client,
+	.shutdown_client = nfs41_shutdown_client,
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
@@ -7454,6 +7458,8 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = {
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
 		| NFS_CAP_ATOMIC_OPEN_V1,
+	.init_client = nfs41_init_client,
+	.shutdown_client = nfs41_shutdown_client,
 	.match_stateid = nfs41_match_stateid,
 	.find_root_sec = nfs41_find_root_sec,
 	.free_lock_state = nfs41_free_lock_state,
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index d221243..fc83d3d 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -78,6 +78,9 @@ struct nfs_client {
 	u32			cl_cb_ident;	/* v4.0 callback identifier */
 	const struct nfs4_minor_version_ops *cl_mvops;
 
+	/* NFSv4.0 transport blocking */
+	struct nfs4_slot_table	*cl_slot_tbl;
+
 	/* The sequence id to use for the next CREATE_SESSION */
 	u32			cl_seqid;
 	/* The flags used for obtaining the clientid during EXCHANGE_ID */

[PATCH v2 11/33] NFS: NFSv4.0 transport blocking

[Chuck Lever]

Plumb in a mechanism for plugging an NFSv4.0 mount, using the
same infrastructure as NFSv4.1 sessions.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   82 +++++++++++++++++++++++++++++++++++++++++++++++------
 fs/nfs/nfs4xdr.c  |   13 +++++---
 2 files changed, 79 insertions(+), 16 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 4352d51..51f4272 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -468,6 +468,67 @@ static void nfs4_set_sequence_privileged(struct nfs4_sequence_args *args)
 	args->sa_privileged = 1;
 }
 
+static int nfs40_setup_sequence(const struct nfs_server *server,
+				struct nfs4_sequence_args *args,
+				struct nfs4_sequence_res *res,
+				struct rpc_task *task)
+{
+	struct nfs4_slot_table *tbl = server->nfs_client->cl_slot_tbl;
+	struct nfs4_slot *slot;
+
+	/* slot already allocated? */
+	if (res->sr_slot != NULL)
+		goto out_start;
+
+	spin_lock(&tbl->slot_tbl_lock);
+	if (nfs4_slot_tbl_draining(tbl) && !args->sa_privileged)
+		goto out_sleep;
+
+	slot = nfs4_alloc_slot(tbl);
+	if (IS_ERR(slot)) {
+		if (slot == ERR_PTR(-ENOMEM))
+			task->tk_timeout = HZ >> 2;
+		goto out_sleep;
+	}
+	spin_unlock(&tbl->slot_tbl_lock);
+
+	args->sa_slot = slot;
+	res->sr_slot = slot;
+
+out_start:
+	rpc_call_start(task);
+	return 0;
+
+out_sleep:
+	if (args->sa_privileged)
+		rpc_sleep_on_priority(&tbl->slot_tbl_waitq, task,
+				NULL, RPC_PRIORITY_PRIVILEGED);
+	else
+		rpc_sleep_on(&tbl->slot_tbl_waitq, task, NULL);
+	spin_unlock(&tbl->slot_tbl_lock);
+	return -EAGAIN;
+}
+
+static int nfs40_sequence_done(struct rpc_task *task,
+			       struct nfs4_sequence_res *res)
+{
+	struct nfs4_slot *slot = res->sr_slot;
+	struct nfs4_slot_table *tbl;
+
+	if (!RPC_WAS_SENT(task))
+		goto out;
+
+	tbl = slot->table;
+	spin_lock(&tbl->slot_tbl_lock);
+	if (!nfs41_wake_and_assign_slot(tbl, slot))
+		nfs4_free_slot(tbl, slot);
+	spin_unlock(&tbl->slot_tbl_lock);
+
+	res->sr_slot = NULL;
+out:
+	return 1;
+}
+
 #if defined(CONFIG_NFS_V4_1)
 
 static void nfs41_sequence_free_slot(struct nfs4_sequence_res *res)
@@ -612,6 +673,8 @@ static int nfs4_sequence_done(struct rpc_task *task,
 {
 	if (res->sr_slot == NULL)
 		return 1;
+	if (!res->sr_slot->table->session)
+		return nfs40_sequence_done(task, res);
 	return nfs41_sequence_done(task, res);
 }
 
@@ -686,17 +749,15 @@ static int nfs4_setup_sequence(const struct nfs_server *server,
 	struct nfs4_session *session = nfs4_get_session(server);
 	int ret = 0;
 
-	if (session == NULL) {
-		rpc_call_start(task);
-		goto out;
-	}
+	if (!session)
+		return nfs40_setup_sequence(server, args, res, task);
 
 	dprintk("--> %s clp %p session %p sr_slot %u\n",
 		__func__, session->clp, session, res->sr_slot ?
 			res->sr_slot->slot_nr : NFS4_NO_SLOT);
 
 	ret = nfs41_setup_sequence(session, args, res, task);
-out:
+
 	dprintk("<-- %s status=%d\n", __func__, ret);
 	return ret;
 }
@@ -723,22 +784,23 @@ static const struct rpc_call_ops nfs41_call_sync_ops = {
 	.rpc_call_done = nfs41_call_sync_done,
 };
 
-#else
+#else	/* !CONFIG_NFS_V4_1 */
+
 static int nfs4_setup_sequence(const struct nfs_server *server,
 			       struct nfs4_sequence_args *args,
 			       struct nfs4_sequence_res *res,
 			       struct rpc_task *task)
 {
-	rpc_call_start(task);
-	return 0;
+	return nfs40_setup_sequence(server, args, res, task);
 }
 
 static int nfs4_sequence_done(struct rpc_task *task,
 			       struct nfs4_sequence_res *res)
 {
-	return 1;
+	return nfs40_sequence_done(task, res);
 }
-#endif /* CONFIG_NFS_V4_1 */
+
+#endif	/* !CONFIG_NFS_V4_1 */
 
 static void nfs40_call_sync_prepare(struct rpc_task *task, void *calldata)
 {
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 3850b01..9b195fb 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1877,11 +1877,10 @@ static void encode_sequence(struct xdr_stream *xdr,
 	struct nfs4_slot *slot = args->sa_slot;
 	__be32 *p;
 
-	if (slot == NULL)
-		return;
-
 	tp = slot->table;
 	session = tp->session;
+	if (!session)
+		return;
 
 	encode_op_hdr(xdr, OP_SEQUENCE, decode_sequence_maxsz, hdr);
 
@@ -2062,9 +2061,9 @@ static void encode_free_stateid(struct xdr_stream *xdr,
 static u32 nfs4_xdr_minorversion(const struct nfs4_sequence_args *args)
 {
 #if defined(CONFIG_NFS_V4_1)
-
-	if (args->sa_slot)
-		return args->sa_slot->table->session->clp->cl_mvops->minor_version;
+	struct nfs4_session *session = args->sa_slot->table->session;
+	if (session)
+		return session->clp->cl_mvops->minor_version;
 #endif /* CONFIG_NFS_V4_1 */
 	return 0;
 }
@@ -5614,6 +5613,8 @@ static int decode_sequence(struct xdr_stream *xdr,
 
 	if (res->sr_slot == NULL)
 		return 0;
+	if (!res->sr_slot->table->session)
+		return 0;
 
 	status = decode_op_hdr(xdr, OP_SEQUENCE);
 	if (!status)

[PATCH v2 12/33] NFS: Enable nfs4_setup_sequence() for DELEGRETURN

[Chuck Lever]

When CONFIG_NFS_V4_1 is disabled, the calls to nfs4_setup_sequence()
and nfs4_sequence_done() are compiled out for the DELEGRETURN
operation.  To allow NFSv4.0 transport blocking to work for
DELEGRETURN, these call sites have to be present all the time.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |    4 ----
 1 file changed, 4 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 51f4272..b915c7d 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4862,7 +4862,6 @@ static void nfs4_delegreturn_release(void *calldata)
 	kfree(calldata);
 }
 
-#if defined(CONFIG_NFS_V4_1)
 static void nfs4_delegreturn_prepare(struct rpc_task *task, void *data)
 {
 	struct nfs4_delegreturndata *d_data;
@@ -4874,12 +4873,9 @@ static void nfs4_delegreturn_prepare(struct rpc_task *task, void *data)
 			&d_data->res.seq_res,
 			task);
 }
-#endif /* CONFIG_NFS_V4_1 */
 
 static const struct rpc_call_ops nfs4_delegreturn_ops = {
-#if defined(CONFIG_NFS_V4_1)
 	.rpc_call_prepare = nfs4_delegreturn_prepare,
-#endif /* CONFIG_NFS_V4_1 */
 	.rpc_call_done = nfs4_delegreturn_done,
 	.rpc_release = nfs4_delegreturn_release,
 };

[PATCH v2 13/33] NFS: Add nfs4_sequence calls for RELEASE_LOCKOWNER

[Chuck Lever]

Ensure RELEASE_LOCKOWNER is not emitted while the transport is
plugged.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index b915c7d..9104467 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5662,8 +5662,23 @@ struct nfs_release_lockowner_data {
 	struct nfs4_lock_state *lsp;
 	struct nfs_server *server;
 	struct nfs_release_lockowner_args args;
+	struct nfs4_sequence_args seq_args;
+	struct nfs4_sequence_res seq_res;
 };
 
+static void nfs4_release_lockowner_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs_release_lockowner_data *data = calldata;
+	nfs40_setup_sequence(data->server,
+				&data->seq_args, &data->seq_res, task);
+}
+
+static void nfs4_release_lockowner_done(struct rpc_task *task, void *calldata)
+{
+	struct nfs_release_lockowner_data *data = calldata;
+	nfs40_sequence_done(task, &data->seq_res);
+}
+
 static void nfs4_release_lockowner_release(void *calldata)
 {
 	struct nfs_release_lockowner_data *data = calldata;
@@ -5672,6 +5687,8 @@ static void nfs4_release_lockowner_release(void *calldata)
 }
 
 static const struct rpc_call_ops nfs4_release_lockowner_ops = {
+	.rpc_call_prepare = nfs4_release_lockowner_prepare,
+	.rpc_call_done = nfs4_release_lockowner_done,
 	.rpc_release = nfs4_release_lockowner_release,
 };
 
@@ -5684,14 +5701,17 @@ static int nfs4_release_lockowner(struct nfs_server *server, struct nfs4_lock_st
 
 	if (server->nfs_client->cl_mvops->minor_version != 0)
 		return -EINVAL;
+
 	data = kmalloc(sizeof(*data), GFP_NOFS);
 	if (!data)
 		return -ENOMEM;
+	nfs4_init_sequence(&data->seq_args, &data->seq_res, 0);
 	data->lsp = lsp;
 	data->server = server;
 	data->args.lock_owner.clientid = server->nfs_client->cl_clientid;
 	data->args.lock_owner.id = lsp->ls_seqid.owner_id;
 	data->args.lock_owner.s_dev = server->s_dev;
+
 	msg.rpc_argp = &data->args;
 	rpc_call_async(server->client, &msg, 0, &nfs4_release_lockowner_ops, data);
 	return 0;

[PATCH v2 14/33] NFS: Add nfs4_sequence calls for OPEN_CONFIRM

[Chuck Lever]

Ensure OPEN_CONFIRM is not emitted while the transport is plugged.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 9104467..5640dc0 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1591,10 +1591,20 @@ int nfs4_open_delegation_recall(struct nfs_open_context *ctx, struct nfs4_state
 	return nfs4_handle_delegation_recall_error(server, state, stateid, err);
 }
 
+static void nfs4_open_confirm_prepare(struct rpc_task *task, void *calldata)
+{
+	struct nfs4_opendata *data = calldata;
+
+	nfs40_setup_sequence(data->o_arg.server, &data->o_arg.seq_args,
+				&data->o_res.seq_res, task);
+}
+
 static void nfs4_open_confirm_done(struct rpc_task *task, void *calldata)
 {
 	struct nfs4_opendata *data = calldata;
 
+	nfs40_sequence_done(task, &data->o_res.seq_res);
+
 	data->rpc_status = task->tk_status;
 	if (data->rpc_status == 0) {
 		nfs4_stateid_copy(&data->o_res.stateid, &data->c_res.stateid);
@@ -1623,6 +1633,7 @@ out_free:
 }
 
 static const struct rpc_call_ops nfs4_open_confirm_ops = {
+	.rpc_call_prepare = nfs4_open_confirm_prepare,
 	.rpc_call_done = nfs4_open_confirm_done,
 	.rpc_release = nfs4_open_confirm_release,
 };
@@ -1650,6 +1661,7 @@ static int _nfs4_proc_open_confirm(struct nfs4_opendata *data)
 	};
 	int status;
 
+	nfs4_init_sequence(&data->o_arg.seq_args, &data->o_res.seq_res, 1);
 	kref_get(&data->kref);
 	data->rpc_done = 0;
 	data->rpc_status = 0;

[PATCH v2 15/33] NFS: Update session draining barriers for NFSv4.0 transport blocking

[Chuck Lever]

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4state.c |   61 +++++++++++++++++++++++++++++-----------------------
 1 file changed, 34 insertions(+), 27 deletions(-)

diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 8826df3..d816d93 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -215,32 +215,6 @@ out:
 	return cred;
 }
 
-#if defined(CONFIG_NFS_V4_1)
-
-static int nfs41_setup_state_renewal(struct nfs_client *clp)
-{
-	int status;
-	struct nfs_fsinfo fsinfo;
-
-	if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
-		nfs4_schedule_state_renewal(clp);
-		return 0;
-	}
-
-	status = nfs4_proc_get_lease_time(clp, &fsinfo);
-	if (status == 0) {
-		/* Update lease time and schedule renewal */
-		spin_lock(&clp->cl_lock);
-		clp->cl_lease_time = fsinfo.lease_time * HZ;
-		clp->cl_last_renewal = jiffies;
-		spin_unlock(&clp->cl_lock);
-
-		nfs4_schedule_state_renewal(clp);
-	}
-
-	return status;
-}
-
 static void nfs4_end_drain_slot_table(struct nfs4_slot_table *tbl)
 {
 	if (test_and_clear_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state)) {
@@ -254,6 +228,11 @@ static void nfs4_end_drain_session(struct nfs_client *clp)
 {
 	struct nfs4_session *ses = clp->cl_session;
 
+	if (clp->cl_slot_tbl) {
+		nfs4_end_drain_slot_table(clp->cl_slot_tbl);
+		return;
+	}
+
 	if (ses != NULL) {
 		nfs4_end_drain_slot_table(&ses->bc_slot_table);
 		nfs4_end_drain_slot_table(&ses->fc_slot_table);
@@ -278,6 +257,9 @@ static int nfs4_begin_drain_session(struct nfs_client *clp)
 	struct nfs4_session *ses = clp->cl_session;
 	int ret = 0;
 
+	if (clp->cl_slot_tbl)
+		return nfs4_drain_slot_tbl(clp->cl_slot_tbl);
+
 	/* back channel */
 	ret = nfs4_drain_slot_tbl(&ses->bc_slot_table);
 	if (ret)
@@ -286,6 +268,32 @@ static int nfs4_begin_drain_session(struct nfs_client *clp)
 	return nfs4_drain_slot_tbl(&ses->fc_slot_table);
 }
 
+#if defined(CONFIG_NFS_V4_1)
+
+static int nfs41_setup_state_renewal(struct nfs_client *clp)
+{
+	int status;
+	struct nfs_fsinfo fsinfo;
+
+	if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
+		nfs4_schedule_state_renewal(clp);
+		return 0;
+	}
+
+	status = nfs4_proc_get_lease_time(clp, &fsinfo);
+	if (status == 0) {
+		/* Update lease time and schedule renewal */
+		spin_lock(&clp->cl_lock);
+		clp->cl_lease_time = fsinfo.lease_time * HZ;
+		clp->cl_last_renewal = jiffies;
+		spin_unlock(&clp->cl_lock);
+
+		nfs4_schedule_state_renewal(clp);
+	}
+
+	return status;
+}
+
 static void nfs41_finish_session_reset(struct nfs_client *clp)
 {
 	clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
@@ -2083,7 +2091,6 @@ static int nfs4_bind_conn_to_session(struct nfs_client *clp)
 }
 #else /* CONFIG_NFS_V4_1 */
 static int nfs4_reset_session(struct nfs_client *clp) { return 0; }
-static void nfs4_end_drain_session(struct nfs_client *clp) { }
 
 static int nfs4_bind_conn_to_session(struct nfs_client *clp)
 {

[PATCH v2 16/33] SUNRPC: Modify synopsis of rpc_client_register()

[Chuck Lever]

The rpc_client_register() helper was added in commit e73f4cc0,
"SUNRPC: split client creation routine into setup and registration,"
Mon Jun 24 11:52:52 2013.

I'd like to invoke rpc_client_register() from a context where a
struct rpc_create_args is not available.  There appear to be only
two fields of interest in struct rpc_create_args: .program and
.authflavor .

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Cc: Stanislav Kinsbursky <skinsbursky@parallels.com>
---
 net/sunrpc/clnt.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index 74f6a70..f9e3926 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -279,10 +279,10 @@ static void rpc_clnt_set_nodename(struct rpc_clnt *clnt, const char *nodename)
 	memcpy(clnt->cl_nodename, nodename, clnt->cl_nodelen);
 }
 
-static int rpc_client_register(const struct rpc_create_args *args,
-			       struct rpc_clnt *clnt)
+static int rpc_client_register(struct rpc_clnt *clnt,
+			       rpc_authflavor_t pseudoflavor)
 {
-	const struct rpc_program *program = args->program;
+	const struct rpc_program *program = clnt->cl_program;
 	struct rpc_auth *auth;
 	struct net *net = rpc_net_ns(clnt);
 	struct super_block *pipefs_sb;
@@ -299,10 +299,10 @@ static int rpc_client_register(const struct rpc_create_args *args,
 	if (pipefs_sb)
 		rpc_put_sb_net(net);
 
-	auth = rpcauth_create(args->authflavor, clnt);
+	auth = rpcauth_create(pseudoflavor, clnt);
 	if (IS_ERR(auth)) {
 		dprintk("RPC:       Couldn't create auth handle (flavor %u)\n",
-				args->authflavor);
+				pseudoflavor);
 		err = PTR_ERR(auth);
 		goto err_auth;
 	}
@@ -384,7 +384,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, stru
 	/* save the nodename */
 	rpc_clnt_set_nodename(clnt, utsname()->nodename);
 
-	err = rpc_client_register(args, clnt);
+	err = rpc_client_register(clnt, args->authflavor);
 	if (err)
 		goto out_no_path;
 	return clnt;

[PATCH v2 17/33] SUNRPC: Add a helper to switch the transport of an rpc_clnt

[Chuck Lever]

From: Trond Myklebust <Trond.Myklebust@netapp.com>

Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>
[ cel: forward ported to 3.11 ]
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 include/linux/sunrpc/clnt.h |    4 ++
 net/sunrpc/clnt.c           |   90 ++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 87 insertions(+), 7 deletions(-)

diff --git a/include/linux/sunrpc/clnt.h b/include/linux/sunrpc/clnt.h
index bfe11be..29bb018 100644
--- a/include/linux/sunrpc/clnt.h
+++ b/include/linux/sunrpc/clnt.h
@@ -134,6 +134,10 @@ void rpc_task_reset_client(struct rpc_task *task, struct rpc_clnt *clnt);
 struct rpc_clnt *rpc_clone_client(struct rpc_clnt *);
 struct rpc_clnt *rpc_clone_client_set_auth(struct rpc_clnt *,
 				rpc_authflavor_t);
+int		rpc_switch_client_transport(struct rpc_clnt *,
+				struct xprt_create *,
+				const struct rpc_timeout *);
+
 void		rpc_shutdown_client(struct rpc_clnt *);
 void		rpc_release_client(struct rpc_clnt *);
 void		rpc_task_release_client(struct rpc_task *);
diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
index f9e3926..ebacc63 100644
--- a/net/sunrpc/clnt.c
+++ b/net/sunrpc/clnt.c
@@ -25,6 +25,7 @@
 #include <linux/namei.h>
 #include <linux/mount.h>
 #include <linux/slab.h>
+#include <linux/rcupdate.h>
 #include <linux/utsname.h>
 #include <linux/workqueue.h>
 #include <linux/in.h>
@@ -271,6 +272,25 @@ void rpc_clients_notifier_unregister(void)
 	return rpc_pipefs_notifier_unregister(&rpc_clients_block);
 }
 
+static struct rpc_xprt *rpc_clnt_set_transport(struct rpc_clnt *clnt,
+		struct rpc_xprt *xprt,
+		const struct rpc_timeout *timeout)
+{
+	struct rpc_xprt *old;
+
+	spin_lock(&clnt->cl_lock);
+	old = clnt->cl_xprt;
+
+	if (!xprt_bound(xprt))
+		clnt->cl_autobind = 1;
+
+	clnt->cl_timeout = timeout;
+	rcu_assign_pointer(clnt->cl_xprt, xprt);
+	spin_unlock(&clnt->cl_lock);
+
+	return old;
+}
+
 static void rpc_clnt_set_nodename(struct rpc_clnt *clnt, const char *nodename)
 {
 	clnt->cl_nodelen = strlen(nodename);
@@ -321,7 +341,8 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, stru
 {
 	const struct rpc_program *program = args->program;
 	const struct rpc_version *version;
-	struct rpc_clnt		*clnt = NULL;
+	struct rpc_clnt *clnt = NULL;
+	const struct rpc_timeout *timeout;
 	int err;
 
 	/* sanity check the name before trying to print it */
@@ -345,7 +366,6 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, stru
 		goto out_err;
 	clnt->cl_parent = clnt;
 
-	rcu_assign_pointer(clnt->cl_xprt, xprt);
 	clnt->cl_procinfo = version->procs;
 	clnt->cl_maxproc  = version->nrprocs;
 	clnt->cl_protname = program->name;
@@ -360,16 +380,15 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, stru
 	INIT_LIST_HEAD(&clnt->cl_tasks);
 	spin_lock_init(&clnt->cl_lock);
 
-	if (!xprt_bound(xprt))
-		clnt->cl_autobind = 1;
-
-	clnt->cl_timeout = xprt->timeout;
+	timeout = xprt->timeout;
 	if (args->timeout != NULL) {
 		memcpy(&clnt->cl_timeout_default, args->timeout,
 				sizeof(clnt->cl_timeout_default));
-		clnt->cl_timeout = &clnt->cl_timeout_default;
+		timeout = &clnt->cl_timeout_default;
 	}
 
+	rpc_clnt_set_transport(clnt, xprt, timeout);
+
 	clnt->cl_rtt = &clnt->cl_rtt_default;
 	rpc_init_rtt(&clnt->cl_rtt_default, clnt->cl_timeout->to_initval);
 	clnt->cl_principal = NULL;
@@ -589,6 +608,63 @@ rpc_clone_client_set_auth(struct rpc_clnt *clnt, rpc_authflavor_t flavor)
 }
 EXPORT_SYMBOL_GPL(rpc_clone_client_set_auth);
 
+/**
+ * rpc_switch_client_transport: switch the RPC transport on the fly
+ * @clnt: pointer to a struct rpc_clnt
+ * @args: pointer to the new transport arguments
+ * @timeout: pointer to the new timeout parameters
+ *
+ * This function allows the caller to switch the RPC transport for the
+ * rpc_clnt structure 'clnt' to allow it to connect to a mirrored NFS
+ * server, for instance.  It assumes that the caller has ensured that
+ * there are no active tasks by using some form of locking.
+ *
+ * Returns zero if "clnt" is now using the new xprt.  Otherwise a
+ * negative errno is returned, and "clnt" continues to use the old
+ * xprt.
+ */
+int rpc_switch_client_transport(struct rpc_clnt *clnt,
+		struct xprt_create *args,
+		const struct rpc_timeout *timeout)
+{
+	const struct rpc_timeout *old_timeo;
+	rpc_authflavor_t pseudoflavor;
+	struct rpc_xprt *xprt, *old;
+	int err;
+
+	xprt = xprt_create_transport(args);
+	if (IS_ERR(xprt)) {
+		dprintk("RPC:      failed to switch xprt for clnt %p\n", clnt);
+		return PTR_ERR(xprt);
+	}
+
+	pseudoflavor = clnt->cl_auth->au_flavor;
+
+	old_timeo = clnt->cl_timeout;
+	old = rpc_clnt_set_transport(clnt, xprt, timeout);
+
+	/*
+	 * Note: we must always create a new rpc_auth cache
+	 * when switching to a different server!  GSS contexts
+	 * in particular are between a single client and
+	 * server.
+	 */
+	rpc_unregister_client(clnt);
+	__rpc_clnt_remove_pipedir(clnt);
+	err = rpc_client_register(clnt, pseudoflavor);
+	if (err) {
+		rpc_clnt_set_transport(clnt, old, old_timeo);
+		rpc_client_register(clnt, pseudoflavor);
+		xprt_put(xprt);
+		return err;
+	}
+
+	synchronize_rcu();
+	xprt_put(old);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(rpc_switch_client_transport);
+
 /*
  * Kill all tasks for the given client.
  * XXX: kill their descendants as well?

[PATCH v2 18/33] NFS: Add nfs4_update_server

[Chuck Lever]

New function nfs4_update_server() moves an nfs_server to a different
nfs_client.  This is done as part of migration recovery.

Though it may be appealing to think of them as the same thing, a
migration recovery is not the same as following a referral.

For a referral, the client has not descended into the file system
yet: it has no nfs_server, no super block, no inodes or open state.
It is enough to simply instantiate the nfs_server and super block.

For a migration, however, we have all of those things already, and
they have to be moved to a different nfs_client.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/client.c     |    3 +-
 fs/nfs/internal.h   |    3 ++
 fs/nfs/nfs4client.c |   87 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 92 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 340b1ef..783b868 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -945,7 +945,7 @@ void nfs_server_insert_lists(struct nfs_server *server)
 }
 EXPORT_SYMBOL_GPL(nfs_server_insert_lists);
 
-static void nfs_server_remove_lists(struct nfs_server *server)
+void nfs_server_remove_lists(struct nfs_server *server)
 {
 	struct nfs_client *clp = server->nfs_client;
 	struct nfs_net *nn;
@@ -962,6 +962,7 @@ static void nfs_server_remove_lists(struct nfs_server *server)
 
 	synchronize_rcu();
 }
+EXPORT_SYMBOL_GPL(nfs_server_remove_lists);
 
 /*
  * Allocate and initialise a server record
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 3c8373f..8392278 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -153,6 +153,7 @@ struct nfs_client *nfs_get_client(const struct nfs_client_initdata *,
 				  rpc_authflavor_t);
 int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *, struct nfs_fattr *);
 void nfs_server_insert_lists(struct nfs_server *);
+void nfs_server_remove_lists(struct nfs_server *);
 void nfs_init_timeout_values(struct rpc_timeout *, int, unsigned int, unsigned int);
 int nfs_init_server_rpcclient(struct nfs_server *, const struct rpc_timeout *t,
 		rpc_authflavor_t);
@@ -173,6 +174,8 @@ extern struct nfs_server *nfs4_create_server(
 					struct nfs_subversion *);
 extern struct nfs_server *nfs4_create_referral_server(struct nfs_clone_mount *,
 						      struct nfs_fh *);
+extern int nfs4_update_server(struct nfs_server *server, const char *hostname,
+					struct sockaddr *sap, size_t salen);
 extern void nfs_free_server(struct nfs_server *server);
 extern struct nfs_server *nfs_clone_server(struct nfs_server *,
 					   struct nfs_fh *,
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index eedd686..d462e40 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -968,3 +968,90 @@ error:
 	dprintk("<-- nfs4_create_referral_server() = error %d\n", error);
 	return ERR_PTR(error);
 }
+
+/**
+ * nfs4_update_server - Replace transitioned nfs_server's transport
+ *
+ * @server: FSID whose transport will be replaced
+ * @hostname: new end-point's hostname
+ * @sap: new end-point's socket address
+ * @salen: size of "sap"
+ *
+ * The nfs_server must be quiescent before this function is invoked.
+ * Either its session is drained, or the transport is plugged and
+ * drained.
+ *
+ * Returns zero on success, or a negative errno value.
+ */
+int nfs4_update_server(struct nfs_server *server, const char *hostname,
+		       struct sockaddr *sap, size_t salen)
+{
+	struct nfs_client *clp = server->nfs_client;
+	struct rpc_clnt *clnt = server->client;
+	struct xprt_create xargs = {
+		.ident		= clp->cl_proto,
+		.net		= &init_net,
+		.dstaddr	= sap,
+		.addrlen	= salen,
+		.servername	= hostname,
+	};
+	char buf[INET6_ADDRSTRLEN + 1];
+	struct sockaddr_storage address;
+	struct sockaddr *localaddr = (struct sockaddr *)&address;
+	struct nfs_fh *mntfh;
+	int error;
+
+	dprintk("--> %s: move FSID %llx:%llx to \"%s\")\n", __func__,
+			(unsigned long long)server->fsid.major,
+			(unsigned long long)server->fsid.minor,
+			hostname);
+
+	error = -ENOMEM;
+	mntfh = nfs_alloc_fhandle();
+	if (mntfh == NULL)
+		goto out;
+
+	error = rpc_switch_client_transport(clnt, &xargs, clnt->cl_timeout);
+	if (error != 0) {
+		dprintk("<-- %s(): rpc_switch_client_transport returned %d\n",
+			__func__, error);
+		goto out;
+	}
+
+	error = rpc_localaddr(clnt, localaddr, sizeof(address));
+	if (error != 0) {
+		dprintk("<-- %s(): rpc_localaddr returned %d\n",
+			__func__, error);
+		goto out;
+	}
+
+	error = -EAFNOSUPPORT;
+	if (rpc_ntop(localaddr, buf, sizeof(buf)) == 0) {
+		dprintk("<-- %s(): rpc_ntop returned %d\n",
+			__func__, error);
+		goto out;
+	}
+
+	nfs_server_remove_lists(server);
+	error = nfs4_set_client(server, hostname, sap, salen, buf,
+				clp->cl_rpcclient->cl_auth->au_flavor,
+				clp->cl_proto, clnt->cl_timeout,
+				clp->cl_minorversion, clp->cl_net);
+	if (error != 0) {
+		dprintk("<-- %s(): nfs4_set_client returned %d\n",
+			__func__, error);
+		goto out;
+	}
+
+	if (server->nfs_client->cl_hostname == NULL)
+		server->nfs_client->cl_hostname = kstrdup(hostname, GFP_KERNEL);
+
+	error = nfs4_server_common_setup(server, mntfh);
+	if (error < 0)
+		goto out;
+
+	dprintk("<-- %s() succeeded\n", __func__);
+out:
+	nfs_free_fhandle(mntfh);
+	return error;
+}

[PATCH v2 19/33] NFS: Add functions to swap transports during migration recovery

[Chuck Lever]

Introduce functions that can walk through an array of returned
fs_locations information and connect a transport to one of the
destination servers listed therein.

Note that NFS minor version 1 introduces "fs_locations_info" which
extends the locations array sorting criteria available to clients.
This is not supported yet.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h       |    2 +
 fs/nfs/nfs4namespace.c |  101 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 103 insertions(+)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index af2d5bf..75b2335 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -216,6 +216,8 @@ rpc_authflavor_t nfs_find_best_sec(struct nfs4_secinfo_flavors *);
 struct rpc_clnt *nfs4_create_sec_client(struct rpc_clnt *, struct inode *, struct qstr *);
 struct vfsmount *nfs4_submount(struct nfs_server *, struct dentry *,
 			       struct nfs_fh *, struct nfs_fattr *);
+int nfs4_replace_transport(struct nfs_server *server,
+				const struct nfs4_fs_locations *locations);
 
 /* nfs4proc.c */
 extern int nfs4_proc_setclientid(struct nfs_client *, u32, unsigned short, struct rpc_cred *, struct nfs4_setclientid_res *);
diff --git a/fs/nfs/nfs4namespace.c b/fs/nfs/nfs4namespace.c
index cdb0b41..faa8b80 100644
--- a/fs/nfs/nfs4namespace.c
+++ b/fs/nfs/nfs4namespace.c
@@ -387,3 +387,104 @@ struct vfsmount *nfs4_submount(struct nfs_server *server, struct dentry *dentry,
 	rpc_shutdown_client(client);
 	return mnt;
 }
+
+/*
+ * Try one location from the fs_locations array.
+ *
+ * Returns zero on success, or a negative errno value.
+ */
+static int nfs4_try_replacing_one_location(struct nfs_server *server,
+		char *page, char *page2,
+		const struct nfs4_fs_location *location)
+{
+	const size_t addr_bufsize = sizeof(struct sockaddr_storage);
+	struct sockaddr *sap;
+	unsigned int s;
+	size_t salen;
+	int error;
+
+	sap = kmalloc(addr_bufsize, GFP_KERNEL);
+	if (sap == NULL)
+		return -ENOMEM;
+
+	error = -ENOENT;
+	for (s = 0; s < location->nservers; s++) {
+		const struct nfs4_string *buf = &location->servers[s];
+		char *hostname;
+
+		if (buf->len <= 0 || buf->len > PAGE_SIZE)
+			continue;
+
+		if (memchr(buf->data, IPV6_SCOPE_DELIMITER, buf->len) != NULL)
+			continue;
+
+		salen = nfs_parse_server_name(buf->data, buf->len,
+						sap, addr_bufsize, server);
+		if (salen == 0)
+			continue;
+		rpc_set_port(sap, NFS_PORT);
+
+		error = -ENOMEM;
+		hostname = kstrndup(buf->data, buf->len, GFP_KERNEL);
+		if (hostname == NULL)
+			break;
+
+		error = nfs4_update_server(server, hostname, sap, salen);
+		kfree(hostname);
+		if (error == 0)
+			break;
+	}
+
+	kfree(sap);
+	return error;
+}
+
+/**
+ * nfs4_replace_transport - set up transport to destination server
+ *
+ * @server: export being migrated
+ * @locations: fs_locations array
+ *
+ * Returns zero on success, or a negative errno value.
+ *
+ * The client tries all the entries in the "locations" array, in the
+ * order returned by the server, until one works or the end of the
+ * array is reached.
+ */
+int nfs4_replace_transport(struct nfs_server *server,
+			   const struct nfs4_fs_locations *locations)
+{
+	char *page = NULL, *page2 = NULL;
+	int loc, error;
+
+	error = -ENOENT;
+	if (locations == NULL || locations->nlocations <= 0)
+		goto out;
+
+	error = -ENOMEM;
+	page = (char *) __get_free_page(GFP_USER);
+	if (!page)
+		goto out;
+	page2 = (char *) __get_free_page(GFP_USER);
+	if (!page2)
+		goto out;
+
+	for (loc = 0; loc < locations->nlocations; loc++) {
+		const struct nfs4_fs_location *location =
+						&locations->locations[loc];
+
+		if (location == NULL || location->nservers <= 0 ||
+		    location->rootpath.ncomponents == 0)
+			continue;
+
+		error = nfs4_try_replacing_one_location(server, page,
+							page2, location);
+		if (error == 0)
+			break;
+	}
+
+out:
+	free_page((unsigned long)page);
+	free_page((unsigned long)page2);
+	return error;
+}

[PATCH v2 20/33] NFS: Introduce a vector of migration recovery ops

[Chuck Lever]

The differences between minor version 0 and minor version 1
migration will be abstracted by the addition of a set of migration
recovery ops.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h  |    4 ++++
 fs/nfs/nfs4proc.c |   10 ++++++++++
 2 files changed, 14 insertions(+)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 75b2335..02af354 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -50,6 +50,7 @@ struct nfs4_minor_version_ops {
 	const struct nfs4_state_recovery_ops *reboot_recovery_ops;
 	const struct nfs4_state_recovery_ops *nograce_recovery_ops;
 	const struct nfs4_state_maintenance_ops *state_renewal_ops;
+	const struct nfs4_mig_recovery_ops *mig_recovery_ops;
 };
 
 #define NFS_SEQID_CONFIRMED 1
@@ -202,6 +203,9 @@ struct nfs4_state_maintenance_ops {
 	int (*renew_lease)(struct nfs_client *, struct rpc_cred *);
 };
 
+struct nfs4_mig_recovery_ops {
+};
+
 extern const struct dentry_operations nfs4_dentry_operations;
 
 /* dir.c */
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 5640dc0..513de09 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -7501,6 +7501,14 @@ static const struct nfs4_state_maintenance_ops nfs41_state_renewal_ops = {
 };
 #endif
 
+static const struct nfs4_mig_recovery_ops nfs40_mig_recovery_ops = {
+};
+
+#if defined(CONFIG_NFS_V4_1)
+static const struct nfs4_mig_recovery_ops nfs41_mig_recovery_ops = {
+};
+#endif	/* CONFIG_NFS_V4_1 */
+
 static const struct nfs4_minor_version_ops nfs_v4_0_minor_ops = {
 	.minor_version = 0,
 	.init_caps = NFS_CAP_READDIRPLUS
@@ -7516,6 +7524,7 @@ static const struct nfs4_minor_version_ops nfs_v4_0_minor_ops = {
 	.reboot_recovery_ops = &nfs40_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs40_nograce_recovery_ops,
 	.state_renewal_ops = &nfs40_state_renewal_ops,
+	.mig_recovery_ops = &nfs40_mig_recovery_ops,
 };
 
 #if defined(CONFIG_NFS_V4_1)
@@ -7536,6 +7545,7 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 	.reboot_recovery_ops = &nfs41_reboot_recovery_ops,
 	.nograce_recovery_ops = &nfs41_nograce_recovery_ops,
 	.state_renewal_ops = &nfs41_state_renewal_ops,
+	.mig_recovery_ops = &nfs41_mig_recovery_ops,
 };
 #endif
 

[PATCH v2 21/33] NFS: Export _nfs_display_fhandle()

[Chuck Lever]

Allow code in nfsv4.ko to use _nfs_display_fhandle().

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/inode.c      |    2 ++
 fs/nfs/nfs4client.c |    2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index af6e806..7d52385 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1192,6 +1192,7 @@ u32 _nfs_display_fhandle_hash(const struct nfs_fh *fh)
 	 * not on the result */
 	return ~crc32(0xFFFFFFFF, &fh->data[0], fh->size);
 }
+EXPORT_SYMBOL_GPL(_nfs_display_fhandle_hash);
 
 /*
  * _nfs_display_fhandle - display an NFS file handle on the console
@@ -1236,6 +1237,7 @@ void _nfs_display_fhandle(const struct nfs_fh *fh, const char *caption)
 		}
 	}
 }
+EXPORT_SYMBOL_GPL(_nfs_display_fhandle);
 #endif
 
 /**
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index d462e40..55d4ff5 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -809,7 +809,7 @@ static int nfs4_server_common_setup(struct nfs_server *server,
 	dprintk("Server FSID: %llx:%llx\n",
 			(unsigned long long) server->fsid.major,
 			(unsigned long long) server->fsid.minor);
-	dprintk("Mount FH: %d\n", mntfh->size);
+	nfs_display_fhandle(mntfh, "Pseudo-fs root FH");
 
 	nfs4_session_set_rwsize(server);
 

[PATCH v2 22/33] NFS: Add method to retrieve fs_locations during migration recovery

[Chuck Lever]

The nfs4_proc_fs_locations() function is invoked during referral
processing to perform a GETATTR(fs_locations) on an object's parent
directory in order to discover the target of the referral.  It
performs a LOOKUP in the compound, so the client needs to know the
parent's file handle a priori.

Unfortunately this function is not adequate for handling migration
recovery.  We need to probe fs_locations information on an FSID, but
there's no parent directory available for many operations that
can return NFS4ERR_MOVED.

Another subtlety: recovering from NFS4ERR_LEASE_MOVED is a process
of walking over a list of known FSIDs that reside on the server, and
probing whether they have migrated.  Once the server has detected
that the client has probed all migrated file systems, it stops
returning NFS4ERR_LEASE_MOVED.

A minor version zero server needs to know what client ID is
requesting fs_locations information so it can clear the flag that
forces it to continue returning NFS4ERR_LEASE_MOVED.  This flag is
set per client ID and per FSID.  However, the client ID is not an
argument of either the PUTFH or GETATTR operations.  Later minor
versions have client ID information embedded in the compound's
SEQUENCE operation.

Therefore, by convention, minor version zero clients send a RENEW
operation in the same compound as the GETATTR(fs_locations), since
RENEW's one argument is a clientid4.  This allows a minor version
zero server to identify correctly the client that is probing for a
migration.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h        |    4 +
 fs/nfs/nfs4proc.c       |  150 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4xdr.c        |   46 +++++++++++---
 include/linux/nfs_xdr.h |    4 +
 4 files changed, 193 insertions(+), 11 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 02af354..3495c66 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -204,6 +204,8 @@ struct nfs4_state_maintenance_ops {
 };
 
 struct nfs4_mig_recovery_ops {
+	int (*get_locations)(struct inode *, struct nfs4_fs_locations *,
+		struct page *);
 };
 
 extern const struct dentry_operations nfs4_dentry_operations;
@@ -236,6 +238,8 @@ extern int nfs4_do_close(struct nfs4_state *state, gfp_t gfp_mask, int wait);
 extern int nfs4_server_capabilities(struct nfs_server *server, struct nfs_fh *fhandle);
 extern int nfs4_proc_fs_locations(struct rpc_clnt *, struct inode *, const struct qstr *,
 				  struct nfs4_fs_locations *, struct page *);
+extern int nfs4_proc_get_locations(struct inode *, struct nfs4_fs_locations *,
+		struct page *page);
 extern struct rpc_clnt *nfs4_proc_lookup_mountpoint(struct inode *, struct qstr *,
 			    struct nfs_fh *, struct nfs_fattr *);
 extern int nfs4_proc_secinfo(struct inode *, const struct qstr *, struct nfs4_secinfo_flavors *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 513de09..3e9b8a3 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5885,6 +5885,154 @@ int nfs4_proc_fs_locations(struct rpc_clnt *client, struct inode *dir,
 	return err;
 }
 
+/*
+ * This operation also signals the server that this client is
+ * performing migration recovery.  The server can stop returning
+ * NFS4ERR_LEASE_MOVED to this client.  A RENEW operation is
+ * appended to this compound to identify the client ID which is
+ * performing recovery.
+ */
+static int _nfs40_proc_get_locations(struct inode *inode,
+				     struct nfs4_fs_locations *locations,
+				     struct page *page)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct rpc_clnt *clnt = server->nfs_client->cl_rpcclient;
+	u32 bitmask[2] = {
+		[0] = FATTR4_WORD0_FSID | FATTR4_WORD0_FS_LOCATIONS,
+	};
+	struct nfs4_fs_locations_arg args = {
+		.clientid	= server->nfs_client->cl_clientid,
+		.fh		= NFS_FH(inode),
+		.page		= page,
+		.bitmask	= bitmask,
+		.migration	= 1,		/* skip LOOKUP */
+		.renew		= 1,		/* append RENEW */
+	};
+	struct nfs4_fs_locations_res res = {
+		.fs_locations	= locations,
+		.migration	= 1,
+		.renew		= 1,
+	};
+	struct rpc_message msg = {
+		.rpc_proc	= &nfs4_procedures[NFSPROC4_CLNT_FS_LOCATIONS],
+		.rpc_argp	= &args,
+		.rpc_resp	= &res,
+	};
+	unsigned long now = jiffies;
+	int status;
+
+	nfs_fattr_init(&locations->fattr);
+	locations->server = server;
+	locations->nlocations = 0;
+
+	nfs4_init_sequence(&args.seq_args, &res.seq_res, 0);
+	nfs4_set_sequence_privileged(&args.seq_args);
+	status = nfs4_call_sync_sequence(clnt, server, &msg,
+					&args.seq_args, &res.seq_res);
+	if (status)
+		return status;
+
+	renew_lease(server, now);
+	return 0;
+}
+
+#ifdef CONFIG_NFS_V4_1
+
+/*
+ * This operation also signals the server that this client is
+ * performing migration recovery.  The server can stop asserting
+ * SEQ4_STATUS_LEASE_MOVED for this client.  The client ID
+ * performing this operation is identified in the SEQUENCE
+ * operation in this compound.
+ *
+ * When the client supports GETATTR(fs_locations_info), it can
+ * be plumbed in here.
+ */
+static int _nfs41_proc_get_locations(struct inode *inode,
+				     struct nfs4_fs_locations *locations,
+				     struct page *page)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct rpc_clnt *clnt = server->nfs_client->cl_rpcclient;
+	u32 bitmask[2] = {
+		[0] = FATTR4_WORD0_FSID | FATTR4_WORD0_FS_LOCATIONS,
+	};
+	struct nfs4_fs_locations_arg args = {
+		.fh		= NFS_FH(inode),
+		.page		= page,
+		.bitmask	= bitmask,
+		.migration	= 1,		/* skip LOOKUP */
+	};
+	struct nfs4_fs_locations_res res = {
+		.fs_locations	= locations,
+		.migration	= 1,
+	};
+	struct rpc_message msg = {
+		.rpc_proc	= &nfs4_procedures[NFSPROC4_CLNT_FS_LOCATIONS],
+		.rpc_argp	= &args,
+		.rpc_resp	= &res,
+	};
+	int status;
+
+	nfs_fattr_init(&locations->fattr);
+	locations->server = server;
+	locations->nlocations = 0;
+
+	nfs4_init_sequence(&args.seq_args, &res.seq_res, 0);
+	nfs4_set_sequence_privileged(&args.seq_args);
+	status = nfs4_call_sync_sequence(clnt, server, &msg,
+					&args.seq_args, &res.seq_res);
+	if (status == NFS4_OK &&
+	    res.seq_res.sr_status_flags & SEQ4_STATUS_LEASE_MOVED)
+		status = -NFS4ERR_LEASE_MOVED;
+	return status;
+}
+
+#endif	/* CONFIG_NFS_V4_1 */
+
+/**
+ * nfs4_proc_get_locations - discover locations for a migrated FSID
+ * @inode: inode on FSID that is migrating
+ * @locations: result of query
+ * @page: buffer
+ *
+ * Returns NFS4_OK on success, a negative NFS4ERR status code if the
+ * operation failed, or a negative errno if a local error occurred.
+ *
+ * On success, "locations" is filled in, but if the server has
+ * no locations information, NFS_ATTR_FATTR_V4_LOCATIONS is not
+ * asserted.
+ *
+ * -NFS4ERR_LEASE_MOVED is returned if the server still has leases
+ * from this client that require migration recovery.
+ */
+int nfs4_proc_get_locations(struct inode *inode,
+			    struct nfs4_fs_locations *locations,
+			    struct page *page)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct nfs_client *clp = server->nfs_client;
+	const struct nfs4_mig_recovery_ops *ops =
+					clp->cl_mvops->mig_recovery_ops;
+	struct nfs4_exception exception = { };
+	int status;
+
+	dprintk("%s: FSID %llx:%llx on \"%s\"\n", __func__,
+		(unsigned long long)server->fsid.major,
+		(unsigned long long)server->fsid.minor,
+		clp->cl_hostname);
+	nfs_display_fhandle(NFS_FH(inode), __func__);
+
+	do {
+		status = ops->get_locations(inode, locations, page);
+		if (status != -NFS4ERR_DELAY)
+			break;
+		nfs4_handle_exception(server, status, &exception);
+	} while (exception.retry);
+	return status;
+}
+
 static int _nfs4_proc_secinfo(struct inode *dir, const struct qstr *name, struct nfs4_secinfo_flavors *flavors)
 {
 	int status;
@@ -7502,10 +7650,12 @@ static const struct nfs4_state_maintenance_ops nfs41_state_renewal_ops = {
 #endif
 
 static const struct nfs4_mig_recovery_ops nfs40_mig_recovery_ops = {
+	.get_locations = _nfs40_proc_get_locations,
 };
 
 #if defined(CONFIG_NFS_V4_1)
 static const struct nfs4_mig_recovery_ops nfs41_mig_recovery_ops = {
+	.get_locations = _nfs41_proc_get_locations,
 };
 #endif	/* CONFIG_NFS_V4_1 */
 
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 9b195fb..d098831 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -732,13 +732,15 @@ static int nfs4_stat_to_errno(int);
 				 encode_sequence_maxsz + \
 				 encode_putfh_maxsz + \
 				 encode_lookup_maxsz + \
-				 encode_fs_locations_maxsz)
+				 encode_fs_locations_maxsz + \
+				 encode_renew_maxsz)
 #define NFS4_dec_fs_locations_sz \
 				(compound_decode_hdr_maxsz + \
 				 decode_sequence_maxsz + \
 				 decode_putfh_maxsz + \
 				 decode_lookup_maxsz + \
-				 decode_fs_locations_maxsz)
+				 decode_fs_locations_maxsz + \
+				 decode_renew_maxsz)
 #define NFS4_enc_secinfo_sz 	(compound_encode_hdr_maxsz + \
 				encode_sequence_maxsz + \
 				encode_putfh_maxsz + \
@@ -2683,11 +2685,20 @@ static void nfs4_xdr_enc_fs_locations(struct rpc_rqst *req,
 
 	encode_compound_hdr(xdr, req, &hdr);
 	encode_sequence(xdr, &args->seq_args, &hdr);
-	encode_putfh(xdr, args->dir_fh, &hdr);
-	encode_lookup(xdr, args->name, &hdr);
-	replen = hdr.replen;	/* get the attribute into args->page */
-	encode_fs_locations(xdr, args->bitmask, &hdr);
+	if (args->migration) {
+		encode_putfh(xdr, args->fh, &hdr);
+		replen = hdr.replen;
+		encode_fs_locations(xdr, args->bitmask, &hdr);
+		if (args->renew)
+			encode_renew(xdr, args->clientid, &hdr);
+	} else {
+		encode_putfh(xdr, args->dir_fh, &hdr);
+		encode_lookup(xdr, args->name, &hdr);
+		replen = hdr.replen;
+		encode_fs_locations(xdr, args->bitmask, &hdr);
+	}
 
+	/* Set up reply kvec to capture returned fs_locations array. */
 	xdr_inline_pages(&req->rq_rcv_buf, replen << 2, &args->page,
 			0, PAGE_SIZE);
 	encode_nops(&hdr);
@@ -6804,13 +6815,26 @@ static int nfs4_xdr_dec_fs_locations(struct rpc_rqst *req,
 	status = decode_putfh(xdr);
 	if (status)
 		goto out;
-	status = decode_lookup(xdr);
-	if (status)
-		goto out;
-	xdr_enter_page(xdr, PAGE_SIZE);
-	status = decode_getfattr_generic(xdr, &res->fs_locations->fattr,
+	if (res->migration) {
+		xdr_enter_page(xdr, PAGE_SIZE);
+		status = decode_getfattr_generic(xdr,
+					&res->fs_locations->fattr,
 					 NULL, res->fs_locations,
 					 NULL, res->fs_locations->server);
+		if (status)
+			goto out;
+		if (res->renew)
+			status = decode_renew(xdr);
+	} else {
+		status = decode_lookup(xdr);
+		if (status)
+			goto out;
+		xdr_enter_page(xdr, PAGE_SIZE);
+		status = decode_getfattr_generic(xdr,
+					&res->fs_locations->fattr,
+					 NULL, res->fs_locations,
+					 NULL, res->fs_locations->server);
+	}
 out:
 	return status;
 }
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 8651574..9a6b776 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1053,14 +1053,18 @@ struct nfs4_fs_locations {
 struct nfs4_fs_locations_arg {
 	struct nfs4_sequence_args	seq_args;
 	const struct nfs_fh *dir_fh;
+	const struct nfs_fh *fh;
 	const struct qstr *name;
 	struct page *page;
 	const u32 *bitmask;
+	clientid4 clientid;
+	unsigned char migration:1, renew:1;
 };
 
 struct nfs4_fs_locations_res {
 	struct nfs4_sequence_res	seq_res;
 	struct nfs4_fs_locations       *fs_locations;
+	unsigned char			migration:1, renew:1;
 };
 
 struct nfs4_secinfo4 {

[PATCH v2 23/33] NFS: Add a super_block backpointer to the nfs_server struct

[Chuck Lever]

NFS_SB() returns the pointer to an nfs_server struct, given a
pointer to a super_block.  But we have no way to go back the other
way.

Add a super_block backpointer field so that, given an nfs_server
struct, it is easy to get to the filesystem's root dentry.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/super.c            |    1 +
 include/linux/nfs_fs_sb.h |    1 +
 2 files changed, 2 insertions(+)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index 71fdc0d..f530ee5 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -2494,6 +2494,7 @@ struct dentry *nfs_fs_mount_common(struct nfs_server *server,
 			mntroot = ERR_PTR(error);
 			goto error_splat_bdi;
 		}
+		server->super = s;
 	}
 
 	if (!s->s_root) {
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index fc83d3d..8064980 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -137,6 +137,7 @@ struct nfs_server {
 	__u64			maxfilesize;	/* maximum file size */
 	struct timespec		time_delta;	/* smallest time granularity */
 	unsigned long		mount_time;	/* when this fs was mounted */
+	struct super_block	*super;		/* VFS super block */
 	dev_t			s_dev;		/* superblock dev numbers */
 
 #ifdef CONFIG_NFS_FSCACHE

[PATCH v2 24/33] NFS: Add basic migration support to state manager thread

[Chuck Lever]

Migration recovery and state recovery must be serialized, so handle
both in the state manager thread.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h          |    2 +
 fs/nfs/nfs4client.c       |    1 
 fs/nfs/nfs4state.c        |  150 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/nfs_fs_sb.h |    7 ++
 4 files changed, 160 insertions(+)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 3495c66..2920746 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -29,6 +29,7 @@ enum nfs4_client_state {
 	NFS4CLNT_SERVER_SCOPE_MISMATCH,
 	NFS4CLNT_PURGE_STATE,
 	NFS4CLNT_BIND_CONN_TO_SESSION,
+	NFS4CLNT_MOVED,
 };
 
 #define NFS4_RENEW_TIMEOUT		0x01
@@ -355,6 +356,7 @@ extern int nfs4_client_recover_expired_lease(struct nfs_client *clp);
 extern void nfs4_schedule_state_manager(struct nfs_client *);
 extern void nfs4_schedule_path_down_recovery(struct nfs_client *clp);
 extern int nfs4_schedule_stateid_recovery(const struct nfs_server *, struct nfs4_state *);
+extern int nfs4_schedule_migration_recovery(const struct nfs_server *);
 extern void nfs41_handle_sequence_flag_errors(struct nfs_client *clp, u32 flags);
 extern void nfs41_handle_server_scope(struct nfs_client *,
 				      struct nfs41_server_scope **);
diff --git a/fs/nfs/nfs4client.c b/fs/nfs/nfs4client.c
index 55d4ff5..f6eacb4 100644
--- a/fs/nfs/nfs4client.c
+++ b/fs/nfs/nfs4client.c
@@ -81,6 +81,7 @@ struct nfs_client *nfs4_alloc_client(const struct nfs_client_initdata *cl_init)
 	clp->cl_state = 1 << NFS4CLNT_LEASE_EXPIRED;
 	clp->cl_minorversion = cl_init->minorversion;
 	clp->cl_mvops = nfs_v4_minor_ops[cl_init->minorversion];
+	clp->cl_mig_gen = 1;
 	return clp;
 
 error:
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index d816d93..9443b67 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1189,6 +1189,48 @@ void nfs4_schedule_lease_recovery(struct nfs_client *clp)
 }
 EXPORT_SYMBOL_GPL(nfs4_schedule_lease_recovery);
 
+/**
+ * nfs4_schedule_migration_recovery - trigger migration recovery
+ *
+ * @server: FSID that is migrating
+ *
+ * Returns zero if recovery has started, otherwise a negative NFS4ERR
+ * value is returned.
+ */
+int nfs4_schedule_migration_recovery(const struct nfs_server *server)
+{
+	struct nfs_client *clp = server->nfs_client;
+
+	if (test_bit(NFS_CS_MIGRATION, &clp->cl_flags) == 0) {
+		pr_err("NFS: migration not supported (server %s)\n",
+				clp->cl_hostname);
+		return -NFS4ERR_IO;
+	}
+
+	if (server->fh_expire_type != NFS4_FH_PERSISTENT) {
+		pr_err("NFS: volatile file handles not supported (server %s)\n",
+				clp->cl_hostname);
+		return -NFS4ERR_IO;
+	}
+
+	if (test_bit(NFS_MIG_FAILED, &server->mig_status))
+		return -NFS4ERR_IO;
+
+	dprintk("%s: scheduling migration recovery for (%llx:%llx) on %s\n",
+			__func__,
+			(unsigned long long)server->fsid.major,
+			(unsigned long long)server->fsid.minor,
+			clp->cl_hostname);
+
+	set_bit(NFS_MIG_IN_TRANSITION,
+			&((struct nfs_server *)server)->mig_status);
+	set_bit(NFS4CLNT_MOVED, &clp->cl_state);
+
+	nfs4_schedule_state_manager(clp);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(nfs4_schedule_migration_recovery);
+
 int nfs4_wait_clnt_recover(struct nfs_client *clp)
 {
 	int res;
@@ -1819,6 +1861,107 @@ static int nfs4_purge_lease(struct nfs_client *clp)
 	return 0;
 }
 
+/*
+ * Try remote migration of one FSID from a source server to a
+ * destination server.  The source server provides a list of
+ * potential destinations.
+ *
+ * Returns zero or a negative NFS4ERR status code.
+ */
+static int nfs4_try_migration(struct nfs_server *server)
+{
+	struct nfs_client *clp = server->nfs_client;
+	struct nfs4_fs_locations *locations = NULL;
+	struct inode *inode;
+	struct page *page;
+	int status, result;
+
+	dprintk("--> %s: FSID %llx:%llx on \"%s\"\n", __func__,
+			(unsigned long long)server->fsid.major,
+			(unsigned long long)server->fsid.minor,
+			clp->cl_hostname);
+
+	result = 0;
+	page = alloc_page(GFP_KERNEL);
+	locations = kmalloc(sizeof(struct nfs4_fs_locations), GFP_KERNEL);
+	if (page == NULL || locations == NULL) {
+		dprintk("<-- %s: no memory\n", __func__);
+		goto out;
+	}
+
+	inode = server->super->s_root->d_inode;
+	result = nfs4_proc_get_locations(inode, locations, page);
+	if (result) {
+		dprintk("<-- %s: failed to retrieve fs_locations: %d\n",
+			__func__, result);
+		goto out;
+	}
+
+	result = -NFS4ERR_NXIO;
+	if (!(locations->fattr.valid & NFS_ATTR_FATTR_V4_LOCATIONS)) {
+		dprintk("<-- %s: No fs_locations data, migration skipped\n",
+			__func__);
+		goto out;
+	}
+
+	nfs4_begin_drain_session(clp);
+
+	status = nfs4_replace_transport(server, locations);
+	if (status != 0) {
+		dprintk("<-- %s: failed to replace transport: %d\n",
+			__func__, status);
+		goto out;
+	}
+
+	result = 0;
+	dprintk("<-- %s: migration succeeded\n", __func__);
+
+out:
+	if (page != NULL)
+		__free_page(page);
+	kfree(locations);
+	if (result) {
+		pr_err("NFS: migration recovery failed (server %s)\n",
+				clp->cl_hostname);
+		set_bit(NFS_MIG_FAILED, &server->mig_status);
+	}
+	return result;
+}
+
+/*
+ * Returns zero or a negative NFS4ERR status code.
+ */
+static int nfs4_handle_migration(struct nfs_client *clp)
+{
+	struct nfs_server *server;
+
+	dprintk("%s: migration reported on \"%s\"\n", __func__,
+			clp->cl_hostname);
+
+	clp->cl_mig_gen++;
+restart:
+	rcu_read_lock();
+	list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) {
+		int status;
+
+		if (server->mig_gen == clp->cl_mig_gen)
+			continue;
+		server->mig_gen = clp->cl_mig_gen;
+
+		if (!test_and_clear_bit(NFS_MIG_IN_TRANSITION,
+						&server->mig_status))
+			continue;
+
+		rcu_read_unlock();
+		status = nfs4_try_migration(server);
+		if (status < 0)
+			return status;
+		goto restart;
+	}
+	rcu_read_unlock();
+	return 0;
+}
+
 /**
  * nfs4_discover_server_trunking - Detect server IP address trunking
  *
@@ -2147,6 +2290,13 @@ static void nfs4_state_manager(struct nfs_client *clp)
 			status = nfs4_check_lease(clp);
 			if (status < 0)
 				goto out_error;
+		}
+
+		if (test_and_clear_bit(NFS4CLNT_MOVED, &clp->cl_state)) {
+			section = "migration";
+			status = nfs4_handle_migration(clp);
+			if (status < 0)
+				goto out_error;
 			continue;
 		}
 
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 8064980..557f96c 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -77,6 +77,7 @@ struct nfs_client {
 	char			cl_ipaddr[48];
 	u32			cl_cb_ident;	/* v4.0 callback identifier */
 	const struct nfs4_minor_version_ops *cl_mvops;
+	unsigned long		cl_mig_gen;
 
 	/* NFSv4.0 transport blocking */
 	struct nfs4_slot_table	*cl_slot_tbl;
@@ -178,6 +179,12 @@ struct nfs_server {
 	struct list_head	state_owners_lru;
 	struct list_head	layouts;
 	struct list_head	delegations;
+
+	unsigned long		mig_gen;
+	unsigned long		mig_status;
+#define NFS_MIG_IN_TRANSITION		(1)
+#define NFS_MIG_FAILED			(2)
+
 	void (*destroy)(struct nfs_server *);
 
 	atomic_t active; /* Keep trace of any activity to this server */

[PATCH v2 25/33] NFS: Re-use exit code in nfs4_async_handle_error()

[Chuck Lever]

Clean up.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |    9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 3e9b8a3..4cc5181 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4675,19 +4675,15 @@ nfs4_async_handle_error(struct rpc_task *task, const struct nfs_server *server,
 			dprintk("%s ERROR %d, Reset session\n", __func__,
 				task->tk_status);
 			nfs4_schedule_session_recovery(clp->cl_session, task->tk_status);
-			task->tk_status = 0;
-			return -EAGAIN;
+			goto restart_call;
 #endif /* CONFIG_NFS_V4_1 */
 		case -NFS4ERR_DELAY:
 			nfs_inc_server_stats(server, NFSIOS_DELAY);
 		case -NFS4ERR_GRACE:
 			rpc_delay(task, NFS4_POLL_RETRY_MAX);
-			task->tk_status = 0;
-			return -EAGAIN;
 		case -NFS4ERR_RETRY_UNCACHED_REP:
 		case -NFS4ERR_OLD_STATEID:
-			task->tk_status = 0;
-			return -EAGAIN;
+			goto restart_call;
 	}
 	task->tk_status = nfs4_map_errors(task->tk_status);
 	return 0;
@@ -4698,6 +4694,7 @@ wait_on_recovery:
 	rpc_sleep_on(&clp->cl_rpcwaitq, task, NULL);
 	if (test_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state) == 0)
 		rpc_wake_up_queued_task(&clp->cl_rpcwaitq, task);
+restart_call:
 	task->tk_status = 0;
 	return -EAGAIN;
 }

[PATCH v2 26/33] NFS: Rename "stateid_invalid" label

[Chuck Lever]

I'm going to use this exit label also for migration recovery
failures.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 4cc5181..93eeac3 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4653,12 +4653,12 @@ nfs4_async_handle_error(struct rpc_task *task, const struct nfs_server *server,
 			if (state == NULL)
 				break;
 			if (nfs4_schedule_stateid_recovery(server, state) < 0)
-				goto stateid_invalid;
+				goto recovery_failed;
 			goto wait_on_recovery;
 		case -NFS4ERR_EXPIRED:
 			if (state != NULL) {
 				if (nfs4_schedule_stateid_recovery(server, state) < 0)
-					goto stateid_invalid;
+					goto recovery_failed;
 			}
 		case -NFS4ERR_STALE_STATEID:
 		case -NFS4ERR_STALE_CLIENTID:
@@ -4687,7 +4687,7 @@ nfs4_async_handle_error(struct rpc_task *task, const struct nfs_server *server,
 	}
 	task->tk_status = nfs4_map_errors(task->tk_status);
 	return 0;
-stateid_invalid:
+recovery_failed:
 	task->tk_status = -EIO;
 	return 0;
 wait_on_recovery:

[PATCH v2 27/33] NFS: Add migration recovery callouts in nfs4proc.c

[Chuck Lever]

When a server returns NFS4ERR_MOVED, trigger the new migration
recovery logic in the state manager.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |   28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 93eeac3..ae19cd5 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -381,6 +381,11 @@ static int nfs4_handle_exception(struct nfs_server *server, int errorcode, struc
 		case -NFS4ERR_STALE_CLIENTID:
 			nfs4_schedule_lease_recovery(clp);
 			goto wait_on_recovery;
+		case -NFS4ERR_MOVED:
+			ret = nfs4_schedule_migration_recovery(server);
+			if (ret < 0)
+				break;
+			goto wait_on_recovery;
 #if defined(CONFIG_NFS_V4_1)
 		case -NFS4ERR_BADSESSION:
 		case -NFS4ERR_BADSLOT:
@@ -428,6 +433,8 @@ static int nfs4_handle_exception(struct nfs_server *server, int errorcode, struc
 	return nfs4_map_errors(ret);
 wait_on_recovery:
 	ret = nfs4_wait_clnt_recover(clp);
+	if (test_bit(NFS_MIG_FAILED, &server->mig_status))
+		return -EIO;
 	if (ret == 0)
 		exception->retry = 1;
 	return ret;
@@ -2928,11 +2935,16 @@ static int nfs4_get_referral(struct rpc_clnt *client, struct inode *dir,
 	status = nfs4_proc_fs_locations(client, dir, name, locations, page);
 	if (status != 0)
 		goto out;
-	/* Make sure server returned a different fsid for the referral */
+
+	/*
+	 * If the fsid didn't change, this is a migration event, not a
+	 * referral.  Cause us to drop into the exception handler, which
+	 * will kick off migration recovery.
+	 */
 	if (nfs_fsid_equal(&NFS_SERVER(dir)->fsid, &locations->fattr.fsid)) {
 		dprintk("%s: server did not return a different fsid for"
 			" a referral at %s\n", __func__, name->name);
-		status = -EIO;
+		status = -NFS4ERR_MOVED;
 		goto out;
 	}
 	/* Fixup attributes for the nfs_lookup() call to nfs_fhget() */
@@ -4664,6 +4676,16 @@ nfs4_async_handle_error(struct rpc_task *task, const struct nfs_server *server,
 		case -NFS4ERR_STALE_CLIENTID:
 			nfs4_schedule_lease_recovery(clp);
 			goto wait_on_recovery;
+		case -NFS4ERR_MOVED:
+			if (state == NULL) {
+				pr_err("%s: no inode for migration recovery\n",
+					__func__);
+				WARN_ON(1);
+				goto recovery_failed;
+			}
+			if (nfs4_schedule_migration_recovery(server) < 0)
+				goto recovery_failed;
+			goto wait_on_recovery;
 #if defined(CONFIG_NFS_V4_1)
 		case -NFS4ERR_BADSESSION:
 		case -NFS4ERR_BADSLOT:
@@ -4694,6 +4716,8 @@ wait_on_recovery:
 	rpc_sleep_on(&clp->cl_rpcwaitq, task, NULL);
 	if (test_bit(NFS4CLNT_MANAGER_RUNNING, &clp->cl_state) == 0)
 		rpc_wake_up_queued_task(&clp->cl_rpcwaitq, task);
+	if (test_bit(NFS_MIG_FAILED, &server->mig_status))
+		goto recovery_failed;
 restart_call:
 	task->tk_status = 0;
 	return -EAGAIN;

[PATCH v2 28/33] NFS: Add method to detect whether an FSID is still on the server

[Chuck Lever]

Introduce a mechanism for probing a server to determine if an FSID
is present or absent.

The on-the-wire compound is different between minor version 0 and 1.
Minor version 0 appends a RENEW operation to identify which client
ID is probing.  Minor version 1 has a SEQUENCE operation in the
compound which effectively carries the same information.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h        |    2 +
 fs/nfs/nfs4proc.c       |  125 +++++++++++++++++++++++++++++++++++++++++++++++
 fs/nfs/nfs4xdr.c        |   67 ++++++++++++++++++++++++-
 include/linux/nfs4.h    |    1 
 include/linux/nfs_xdr.h |   13 +++++
 5 files changed, 206 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 2920746..b4d7559 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -207,6 +207,7 @@ struct nfs4_state_maintenance_ops {
 struct nfs4_mig_recovery_ops {
 	int (*get_locations)(struct inode *, struct nfs4_fs_locations *,
 		struct page *);
+	int (*fsid_present)(struct inode *);
 };
 
 extern const struct dentry_operations nfs4_dentry_operations;
@@ -241,6 +242,7 @@ extern int nfs4_proc_fs_locations(struct rpc_clnt *, struct inode *, const struc
 				  struct nfs4_fs_locations *, struct page *);
 extern int nfs4_proc_get_locations(struct inode *, struct nfs4_fs_locations *,
 		struct page *page);
+extern int nfs4_proc_fsid_present(struct inode *);
 extern struct rpc_clnt *nfs4_proc_lookup_mountpoint(struct inode *, struct qstr *,
 			    struct nfs_fh *, struct nfs_fattr *);
 extern int nfs4_proc_secinfo(struct inode *, const struct qstr *, struct nfs4_secinfo_flavors *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ae19cd5..36cd989 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6054,6 +6054,129 @@ int nfs4_proc_get_locations(struct inode *inode,
 	return status;
 }
 
+/*
+ * This operation also signals the server that this client is
+ * performing "lease moved" recovery.  The server can stop
+ * returning NFS4ERR_LEASE_MOVED to this client.  A RENEW operation
+ * is appended to this compound to identify the client ID which is
+ * performing recovery.
+ */
+static int _nfs40_proc_fsid_present(struct inode *inode)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct nfs_client *clp = NFS_SERVER(inode)->nfs_client;
+	struct rpc_clnt *clnt = clp->cl_rpcclient;
+	struct nfs4_fsid_present_arg args = {
+		.fh		= NFS_FH(inode),
+		.clientid	= clp->cl_clientid,
+		.renew		= 1,		/* append RENEW */
+	};
+	struct nfs4_fsid_present_res res = {
+		.renew		= 1,
+	};
+	struct rpc_message msg = {
+		.rpc_proc	= &nfs4_procedures[NFSPROC4_CLNT_FSID_PRESENT],
+		.rpc_argp	= &args,
+		.rpc_resp	= &res,
+	};
+	unsigned long now = jiffies;
+	int status;
+
+	res.fh = nfs_alloc_fhandle();
+	if (res.fh == NULL)
+		return -ENOMEM;
+
+	nfs4_init_sequence(&args.seq_args, &res.seq_res, 0);
+	nfs4_set_sequence_privileged(&args.seq_args);
+	status = nfs4_call_sync_sequence(clnt, server, &msg,
+						&args.seq_args, &res.seq_res);
+	nfs_free_fhandle(res.fh);
+	if (status)
+		return status;
+
+	do_renew_lease(clp, now);
+	return 0;
+}
+
+#ifdef CONFIG_NFS_V4_1
+
+/*
+ * This operation also signals the server that this client is
+ * performing "lease moved" recovery.  The server can stop asserting
+ * SEQ4_STATUS_LEASE_MOVED for this client.  The client ID performing
+ * this operation is identified in the SEQUENCE operation in this
+ * compound.
+ */
+static int _nfs41_proc_fsid_present(struct inode *inode)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct rpc_clnt *clnt = server->nfs_client->cl_rpcclient;
+	struct nfs4_fsid_present_arg args = {
+		.fh		= NFS_FH(inode),
+	};
+	struct nfs4_fsid_present_res res = {
+	};
+	struct rpc_message msg = {
+		.rpc_proc	= &nfs4_procedures[NFSPROC4_CLNT_FSID_PRESENT],
+		.rpc_argp	= &args,
+		.rpc_resp	= &res,
+	};
+	int status;
+
+	res.fh = nfs_alloc_fhandle();
+	if (res.fh == NULL)
+		return -ENOMEM;
+
+	nfs4_init_sequence(&args.seq_args, &res.seq_res, 0);
+	nfs4_set_sequence_privileged(&args.seq_args);
+	status = nfs4_call_sync_sequence(clnt, server, &msg,
+						&args.seq_args, &res.seq_res);
+	nfs_free_fhandle(res.fh);
+	if (status == NFS4_OK &&
+	    res.seq_res.sr_status_flags & SEQ4_STATUS_LEASE_MOVED)
+		status = -NFS4ERR_LEASE_MOVED;
+	return status;
+}
+
+#endif	/* CONFIG_NFS_V4_1 */
+
+/**
+ * nfs4_proc_fsid_present - Is this FSID present or absent on server?
+ * @inode: inode on FSID to check
+ *
+ * Server indicates whether the FSID is present, moved, or not
+ * recognized.  This operation is necessary to clear a LEASE_MOVED
+ * condition for this client ID.
+ *
+ * Returns NFS4_OK if the FSID is present on this server,
+ * -NFS4ERR_MOVED if the FSID is no longer present, a negative
+ *  NFS4ERR code if some error occurred on the server, or a
+ *  negative errno if a local failure occurred.
+ */
+int nfs4_proc_fsid_present(struct inode *inode)
+{
+	struct nfs_server *server = NFS_SERVER(inode);
+	struct nfs_client *clp = server->nfs_client;
+	const struct nfs4_mig_recovery_ops *ops =
+					clp->cl_mvops->mig_recovery_ops;
+	struct nfs4_exception exception = { };
+	int status;
+
+	dprintk("%s: FSID %llx:%llx on \"%s\"\n", __func__,
+		(unsigned long long)server->fsid.major,
+		(unsigned long long)server->fsid.minor,
+		clp->cl_hostname);
+	nfs_display_fhandle(NFS_FH(inode), __func__);
+
+	do {
+		status = ops->fsid_present(inode);
+		if (status != -NFS4ERR_DELAY)
+			break;
+		nfs4_handle_exception(server, status, &exception);
+	} while (exception.retry);
+	return status;
+}
+
 static int _nfs4_proc_secinfo(struct inode *dir, const struct qstr *name, struct nfs4_secinfo_flavors *flavors)
 {
 	int status;
@@ -7672,11 +7795,13 @@ static const struct nfs4_state_maintenance_ops nfs41_state_renewal_ops = {
 
 static const struct nfs4_mig_recovery_ops nfs40_mig_recovery_ops = {
 	.get_locations = _nfs40_proc_get_locations,
+	.fsid_present = _nfs40_proc_fsid_present,
 };
 
 #if defined(CONFIG_NFS_V4_1)
 static const struct nfs4_mig_recovery_ops nfs41_mig_recovery_ops = {
 	.get_locations = _nfs41_proc_get_locations,
+	.fsid_present = _nfs41_proc_fsid_present,
 };
 #endif	/* CONFIG_NFS_V4_1 */
 
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index d098831..47e60c6 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -591,11 +591,13 @@ static int nfs4_stat_to_errno(int);
 #define NFS4_enc_getattr_sz	(compound_encode_hdr_maxsz + \
 				encode_sequence_maxsz + \
 				encode_putfh_maxsz + \
-				encode_getattr_maxsz)
+				encode_getattr_maxsz + \
+				encode_renew_maxsz)
 #define NFS4_dec_getattr_sz	(compound_decode_hdr_maxsz + \
 				decode_sequence_maxsz + \
 				decode_putfh_maxsz + \
-				decode_getattr_maxsz)
+				decode_getattr_maxsz + \
+				decode_renew_maxsz)
 #define NFS4_enc_lookup_sz	(compound_encode_hdr_maxsz + \
 				encode_sequence_maxsz + \
 				encode_putfh_maxsz + \
@@ -749,6 +751,18 @@ static int nfs4_stat_to_errno(int);
 				decode_sequence_maxsz + \
 				decode_putfh_maxsz + \
 				decode_secinfo_maxsz)
+#define NFS4_enc_fsid_present_sz \
+				(compound_encode_hdr_maxsz + \
+				 encode_sequence_maxsz + \
+				 encode_putfh_maxsz + \
+				 encode_getfh_maxsz + \
+				 encode_renew_maxsz)
+#define NFS4_dec_fsid_present_sz \
+				(compound_decode_hdr_maxsz + \
+				 decode_sequence_maxsz + \
+				 decode_putfh_maxsz + \
+				 decode_getfh_maxsz + \
+				 decode_renew_maxsz)
 #if defined(CONFIG_NFS_V4_1)
 #define NFS4_enc_bind_conn_to_session_sz \
 				(compound_encode_hdr_maxsz + \
@@ -2722,6 +2736,26 @@ static void nfs4_xdr_enc_secinfo(struct rpc_rqst *req,
 	encode_nops(&hdr);
 }
 
+/*
+ * Encode FSID_PRESENT request
+ */
+static void nfs4_xdr_enc_fsid_present(struct rpc_rqst *req,
+				      struct xdr_stream *xdr,
+				      struct nfs4_fsid_present_arg *args)
+{
+	struct compound_hdr hdr = {
+		.minorversion = nfs4_xdr_minorversion(&args->seq_args),
+	};
+
+	encode_compound_hdr(xdr, req, &hdr);
+	encode_sequence(xdr, &args->seq_args, &hdr);
+	encode_putfh(xdr, args->fh, &hdr);
+	encode_getfh(xdr, &hdr);
+	if (args->renew)
+		encode_renew(xdr, args->clientid, &hdr);
+	encode_nops(&hdr);
+}
+
 #if defined(CONFIG_NFS_V4_1)
 /*
  * BIND_CONN_TO_SESSION request
@@ -6863,6 +6897,34 @@ out:
 	return status;
 }
 
+/*
+ * Decode FSID_PRESENT response
+ */
+static int nfs4_xdr_dec_fsid_present(struct rpc_rqst *rqstp,
+				     struct xdr_stream *xdr,
+				     struct nfs4_fsid_present_res *res)
+{
+	struct compound_hdr hdr;
+	int status;
+
+	status = decode_compound_hdr(xdr, &hdr);
+	if (status)
+		goto out;
+	status = decode_sequence(xdr, &res->seq_res, rqstp);
+	if (status)
+		goto out;
+	status = decode_putfh(xdr);
+	if (status)
+		goto out;
+	status = decode_getfh(xdr, res->fh);
+	if (status)
+		goto out;
+	if (res->renew)
+		status = decode_renew(xdr);
+out:
+	return status;
+}
+
 #if defined(CONFIG_NFS_V4_1)
 /*
  * Decode BIND_CONN_TO_SESSION response
@@ -7377,6 +7439,7 @@ struct rpc_procinfo	nfs4_procedures[] = {
 	PROC(FS_LOCATIONS,	enc_fs_locations,	dec_fs_locations),
 	PROC(RELEASE_LOCKOWNER,	enc_release_lockowner,	dec_release_lockowner),
 	PROC(SECINFO,		enc_secinfo,		dec_secinfo),
+	PROC(FSID_PRESENT,	enc_fsid_present,	dec_fsid_present),
 #if defined(CONFIG_NFS_V4_1)
 	PROC(EXCHANGE_ID,	enc_exchange_id,	dec_exchange_id),
 	PROC(CREATE_SESSION,	enc_create_session,	dec_create_session),
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index e36dee5..c56fa8f 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -460,6 +460,7 @@ enum {
 	NFSPROC4_CLNT_FS_LOCATIONS,
 	NFSPROC4_CLNT_RELEASE_LOCKOWNER,
 	NFSPROC4_CLNT_SECINFO,
+	NFSPROC4_CLNT_FSID_PRESENT,
 
 	/* nfs41 */
 	NFSPROC4_CLNT_EXCHANGE_ID,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 9a6b776..89d6d3d 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -1088,6 +1088,19 @@ struct nfs4_secinfo_res {
 	struct nfs4_secinfo_flavors	*flavors;
 };
 
+struct nfs4_fsid_present_arg {
+	struct nfs4_sequence_args	seq_args;
+	const struct nfs_fh		*fh;
+	clientid4			clientid;
+	unsigned char			renew:1;
+};
+
+struct nfs4_fsid_present_res {
+	struct nfs4_sequence_res	seq_res;
+	struct nfs_fh			*fh;
+	unsigned char			renew:1;
+};
+
 #endif /* CONFIG_NFS_V4 */
 
 struct nfstime4 {

[PATCH v2 29/33] NFS: Implement support for NFS4ERR_LEASE_MOVED in state manager

[Chuck Lever]

To recover from NFS4ERR_LEASE_MOVED, check each FSID for that server
to see if it is still present.  Invoke nfs4_try_migration() if the
FSID is no longer present on the server.

The FSID presence check also informs servers that this client has
recognized the LEASE_MOVED condition and that therefore the server
should no longer assert LEASE_MOVED for that client.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4_fs.h   |    2 ++
 fs/nfs/nfs4state.c |   64 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 65 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index b4d7559..5c14712 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -30,6 +30,7 @@ enum nfs4_client_state {
 	NFS4CLNT_PURGE_STATE,
 	NFS4CLNT_BIND_CONN_TO_SESSION,
 	NFS4CLNT_MOVED,
+	NFS4CLNT_LEASE_MOVED,
 };
 
 #define NFS4_RENEW_TIMEOUT		0x01
@@ -359,6 +360,7 @@ extern void nfs4_schedule_state_manager(struct nfs_client *);
 extern void nfs4_schedule_path_down_recovery(struct nfs_client *clp);
 extern int nfs4_schedule_stateid_recovery(const struct nfs_server *, struct nfs4_state *);
 extern int nfs4_schedule_migration_recovery(const struct nfs_server *);
+extern void nfs4_schedule_lease_moved_recovery(struct nfs_client *);
 extern void nfs41_handle_sequence_flag_errors(struct nfs_client *clp, u32 flags);
 extern void nfs41_handle_server_scope(struct nfs_client *,
 				      struct nfs41_server_scope **);
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 9443b67..175c0c7 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1231,6 +1231,22 @@ int nfs4_schedule_migration_recovery(const struct nfs_server *server)
 }
 EXPORT_SYMBOL_GPL(nfs4_schedule_migration_recovery);
 
+/**
+ * nfs4_schedule_lease_moved_recovery - start lease-moved recovery
+ *
+ * @clp: server to check for moved leases
+ *
+ */
+void nfs4_schedule_lease_moved_recovery(struct nfs_client *clp)
+{
+	dprintk("%s: scheduling lease-moved recovery for client ID %llx on %s\n",
+		__func__, clp->cl_clientid, clp->cl_hostname);
+
+	set_bit(NFS4CLNT_LEASE_MOVED, &clp->cl_state);
+	nfs4_schedule_state_manager(clp);
+}
+EXPORT_SYMBOL_GPL(nfs4_schedule_lease_moved_recovery);
+
 int nfs4_wait_clnt_recover(struct nfs_client *clp)
 {
 	int res;
@@ -1660,7 +1676,6 @@ static int nfs4_recovery_handle_error(struct nfs_client *clp, int error)
 			nfs4_state_end_reclaim_reboot(clp);
 			break;
 		case -NFS4ERR_STALE_CLIENTID:
-		case -NFS4ERR_LEASE_MOVED:
 			set_bit(NFS4CLNT_LEASE_EXPIRED, &clp->cl_state);
 			nfs4_state_clear_reclaim_reboot(clp);
 			nfs4_state_start_reclaim_reboot(clp);
@@ -1962,6 +1977,45 @@ restart:
 	return 0;
 }
 
+/*
+ * Test each nfs_server on the clp's cl_superblocks list to see
+ * if it's moved to another server.  Stop when the server no longer
+ * returns NFS4ERR_LEASE_MOVED.
+ */
+static int nfs4_handle_lease_moved(struct nfs_client *clp)
+{
+	struct nfs_server *server;
+
+	dprintk("%s: lease moved reported on \"%s\"\n", __func__,
+			clp->cl_hostname);
+
+	clp->cl_mig_gen++;
+restart:
+	rcu_read_lock();
+	list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) {
+		struct inode *inode;
+		int status;
+
+		if (server->mig_gen == clp->cl_mig_gen)
+			continue;
+		server->mig_gen = clp->cl_mig_gen;
+
+		rcu_read_unlock();
+
+		inode = server->super->s_root->d_inode;
+		status = nfs4_proc_fsid_present(inode);
+		if (status != -NFS4ERR_MOVED)
+			goto restart;	/* wasn't this one */
+		if (nfs4_try_migration(server) == -NFS4ERR_LEASE_MOVED)
+			goto restart;	/* there are more */
+		goto out;
+	}
+	rcu_read_unlock();
+
+out:
+	return 0;
+}
+
 /**
  * nfs4_discover_server_trunking - Detect server IP address trunking
  *
@@ -2300,6 +2354,14 @@ static void nfs4_state_manager(struct nfs_client *clp)
 			continue;
 		}
 
+		if (test_and_clear_bit(NFS4CLNT_LEASE_MOVED, &clp->cl_state)) {
+			section = "lease moved";
+			status = nfs4_handle_lease_moved(clp);
+			if (status < 0)
+				goto out_error;
+			continue;
+		}
+
 		/* First recover reboot state... */
 		if (test_bit(NFS4CLNT_RECLAIM_REBOOT, &clp->cl_state)) {
 			section = "reclaim reboot";

[PATCH v2 30/33] NFS: Implement support for NFS4ERR_LEASE_MOVED

[Chuck Lever]

Trigger lease-moved recovery when a request returns
NFS4ERR_LEASE_MOVED.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 36cd989..11283da 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -386,6 +386,9 @@ static int nfs4_handle_exception(struct nfs_server *server, int errorcode, struc
 			if (ret < 0)
 				break;
 			goto wait_on_recovery;
+		case -NFS4ERR_LEASE_MOVED:
+			nfs4_schedule_lease_moved_recovery(clp);
+			goto wait_on_recovery;
 #if defined(CONFIG_NFS_V4_1)
 		case -NFS4ERR_BADSESSION:
 		case -NFS4ERR_BADSLOT:
@@ -4686,6 +4689,9 @@ nfs4_async_handle_error(struct rpc_task *task, const struct nfs_server *server,
 			if (nfs4_schedule_migration_recovery(server) < 0)
 				goto recovery_failed;
 			goto wait_on_recovery;
+		case -NFS4ERR_LEASE_MOVED:
+			nfs4_schedule_lease_moved_recovery(clp);
+			goto wait_on_recovery;
 #if defined(CONFIG_NFS_V4_1)
 		case -NFS4ERR_BADSESSION:
 		case -NFS4ERR_BADSLOT:

[PATCH v2 31/33] NFS: Implement support for NFS4ERR_LEASE_MOVED

[Chuck Lever]

With NFSv4 minor version 0, the asynchronous lease RENEW
heartbeat can return NFS4ERR_LEASE_MOVED.  Error recovery logic for
async RENEW is a separate code path from the generic NFS proc paths,
so it must be updated to handle LEASE_MOVED as well.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4proc.c |    8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 11283da..ecf8e96 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -4158,7 +4158,13 @@ static void nfs4_renew_done(struct rpc_task *task, void *calldata)
 	struct nfs_client *clp = data->client;
 	unsigned long timestamp = data->timestamp;
 
-	if (task->tk_status < 0) {
+	switch (task->tk_status) {
+	case 0:
+		break;
+	case -NFS4ERR_LEASE_MOVED:
+		nfs4_schedule_lease_moved_recovery(clp);
+		break;
+	default:
 		/* Unless we're shutting down, schedule state recovery! */
 		if (test_bit(NFS_CS_RENEWD, &clp->cl_res_state) == 0)
 			return;

[PATCH v2 32/33] NFS: Implement NFSv4.1 support for NFS4ERR_LEASE_MOVED

[Chuck Lever]

NFSv4 minor version 0 servers return an error code,
NFS4ERR_LEASE_MOVED, to signal LEASE_MOVED.  This error causes the
whole compound to fail.

Minor version 1+ servers set a bit flag in the reply to a compound's
SEQUENCE operation to signal LEASE_MOVED.  The compound succeeds.

After the server sets one of the sr_status_flags in a typical
compound operation, nfs41_sequence_done() initiates standard lease
recovery.  A stand-alone SEQUENCE operation is performed to
discover what recovery is needed.

If the LEASE_MOVED status flag is set in this SEQUENCE operation,
our client kick offs LEASE_MOVED recovery.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/nfs4state.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 175c0c7..5510ab2 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -2204,9 +2204,10 @@ void nfs41_handle_sequence_flag_errors(struct nfs_client *clp, u32 flags)
 		nfs41_handle_server_reboot(clp);
 	if (flags & (SEQ4_STATUS_EXPIRED_ALL_STATE_REVOKED |
 			    SEQ4_STATUS_EXPIRED_SOME_STATE_REVOKED |
-			    SEQ4_STATUS_ADMIN_STATE_REVOKED |
-			    SEQ4_STATUS_LEASE_MOVED))
+			    SEQ4_STATUS_ADMIN_STATE_REVOKED))
 		nfs41_handle_state_revoked(clp);
+	if (flags & SEQ4_STATUS_LEASE_MOVED)
+		nfs4_schedule_lease_moved_recovery(clp);
 	if (flags & SEQ4_STATUS_RECALLABLE_STATE_REVOKED)
 		nfs41_handle_recallable_state_revoked(clp);
 	if (flags & SEQ4_STATUS_BACKCHANNEL_FAULT)

[PATCH v2 33/33] NFS: Set EXCHGID4_FLAG_SUPP_MOVED_MIGR

[Chuck Lever]

Broadly speaking, v4.1 migration is untested.  There are no servers
in the wild that support NFSv4.1 migration.  However, as server
implementations become available, we do want to enable testing by
developers, while leaving it disabled for environments for which
broken migration support would be an unpleasant surprise.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 fs/nfs/Kconfig    |   11 +++++++++++
 fs/nfs/nfs4proc.c |    8 +++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/Kconfig b/fs/nfs/Kconfig
index b5e80b0..38c1768 100644
--- a/fs/nfs/Kconfig
+++ b/fs/nfs/Kconfig
@@ -140,6 +140,17 @@ config NFS_V4_1_IMPLEMENTATION_ID_DOMAIN
 	  If the NFS client is unchanged from the upstream kernel, this
 	  option should be set to the default "kernel.org".
 
+config NFS_V4_1_MIGRATION
+	bool "NFSv4.1 client support for migration"
+	depends on NFS_V4_1
+	default n
+	help
+	  This option makes the NFS client advertise to NFSv4.1 servers that
+          it can support NFSv4 migration.
+
+          The NFSv4.1 pieces of the Linux NFSv4 migration implementation are
+          still experimental.  If you are not an NFSv4 developer, say N here.
+
 config NFS_V4_SECURITY_LABEL
 	bool
 	depends on NFS_V4_2 && SECURITY
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ecf8e96..0ded41b 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -6325,8 +6325,14 @@ int nfs4_proc_exchange_id(struct nfs_client *clp, struct rpc_cred *cred)
 	struct nfs41_exchange_id_args args = {
 		.verifier = &verifier,
 		.client = clp,
+#ifdef CONFIG_NFS_V4_1_MIGRATION
 		.flags = EXCHGID4_FLAG_SUPP_MOVED_REFER |
-			EXCHGID4_FLAG_BIND_PRINC_STATEID,
+			 EXCHGID4_FLAG_BIND_PRINC_STATEID |
+			 EXCHGID4_FLAG_SUPP_MOVED_MIGR,
+#else
+		.flags = EXCHGID4_FLAG_SUPP_MOVED_REFER |
+			 EXCHGID4_FLAG_BIND_PRINC_STATEID,
+#endif
 	};
 	struct nfs41_exchange_id_res res = {
 		0

Re: [PATCH v2 07/33] NFS: Enable slot table helpers for NFSv4.0

[Myklebust, Trond]

T24gRnJpLCAyMDEzLTA4LTA5IGF0IDEyOjQ4IC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
SSdkIGxpa2UgdG8gcmUtdXNlIE5GU3Y0LjEncyBzbG90IHRhYmxlIG1hY2hpbmVyeSBmb3IgTkZT
djQuMA0KPiB0cmFuc3BvcnQgYmxvY2tpbmcuICBSZS1vcmdhbml6ZSBzb21lIG9mIG5mczRzZXNz
aW9uLmMgc28gdGhlIHNsb3QNCj4gdGFibGUgY29kZSBpcyBidWlsdCBldmVuIHdoZW4gTkZTX1Y0
XzEgaXMgZGlzYWJsZWQuDQo+IA0KPiBTaWduZWQtb2ZmLWJ5OiBDaHVjayBMZXZlciA8Y2h1Y2su
bGV2ZXJAb3JhY2xlLmNvbT4NCj4gLS0tDQo+ICBmcy9uZnMvTWFrZWZpbGUgICAgICB8ICAgIDQg
KystLQ0KPiAgZnMvbmZzL25mczRzZXNzaW9uLmMgfCAgIDI5ICsrKysrKysrKysrKysrKysrKysr
Ky0tLS0tLS0tDQo+ICBmcy9uZnMvbmZzNHNlc3Npb24uaCB8ICAgMjggKysrKysrKysrKysrKy0t
LS0tLS0tLS0tLS0tLQ0KPiAgZnMvbmZzL25mczRzdGF0ZS5jICAgfCAgICA5IC0tLS0tLS0tLQ0K
PiAgNCBmaWxlcyBjaGFuZ2VkLCAzNiBpbnNlcnRpb25zKCspLCAzNCBkZWxldGlvbnMoLSkNCj4g
DQo+IGRpZmYgLS1naXQgYS9mcy9uZnMvTWFrZWZpbGUgYi9mcy9uZnMvTWFrZWZpbGUNCj4gaW5k
ZXggZTBiYjA0OC4uMjgzOWJmZSAxMDA2NDQNCj4gLS0tIGEvZnMvbmZzL01ha2VmaWxlDQo+ICsr
KyBiL2ZzL25mcy9NYWtlZmlsZQ0KPiBAQCAtMjEsMTAgKzIxLDEwIEBAIG5mc3YzLSQoQ09ORklH
X05GU19WM19BQ0wpICs9IG5mczNhY2wubw0KPiAgb2JqLSQoQ09ORklHX05GU19WNCkgKz0gbmZz
djQubw0KPiAgbmZzdjQteSA6PSBuZnM0cHJvYy5vIG5mczR4ZHIubyBuZnM0c3RhdGUubyBuZnM0
cmVuZXdkLm8gbmZzNHN1cGVyLm8gbmZzNGZpbGUubyBcDQo+ICAJICBkZWxlZ2F0aW9uLm8gaWRt
YXAubyBjYWxsYmFjay5vIGNhbGxiYWNrX3hkci5vIGNhbGxiYWNrX3Byb2MubyBcDQo+IC0JICBu
ZnM0bmFtZXNwYWNlLm8gbmZzNGdldHJvb3QubyBuZnM0Y2xpZW50Lm8gZG5zX3Jlc29sdmUubw0K
PiArCSAgbmZzNG5hbWVzcGFjZS5vIG5mczRnZXRyb290Lm8gbmZzNGNsaWVudC5vIGRuc19yZXNv
bHZlLm8gbmZzNHNlc3Npb24ubw0KPiAgbmZzdjQtJChDT05GSUdfTkZTX1VTRV9MRUdBQ1lfRE5T
KSArPSBjYWNoZV9saWIubw0KPiAgbmZzdjQtJChDT05GSUdfU1lTQ1RMKQkrPSBuZnM0c3lzY3Rs
Lm8NCj4gLW5mc3Y0LSQoQ09ORklHX05GU19WNF8xKQkrPSBuZnM0c2Vzc2lvbi5vIHBuZnMubyBw
bmZzX2Rldi5vDQo+ICtuZnN2NC0kKENPTkZJR19ORlNfVjRfMSkJKz0gcG5mcy5vIHBuZnNfZGV2
Lm8NCj4gIA0KPiAgb2JqLSQoQ09ORklHX1BORlNfRklMRV9MQVlPVVQpICs9IG5mc19sYXlvdXRf
bmZzdjQxX2ZpbGVzLm8NCj4gIG5mc19sYXlvdXRfbmZzdjQxX2ZpbGVzLXkgOj0gbmZzNGZpbGVs
YXlvdXQubyBuZnM0ZmlsZWxheW91dGRldi5vDQo+IGRpZmYgLS1naXQgYS9mcy9uZnMvbmZzNHNl
c3Npb24uYyBiL2ZzL25mcy9uZnM0c2Vzc2lvbi5jDQo+IGluZGV4IDAxNjFhZDIuLjc0NjY3NWIg
MTAwNjQ0DQo+IC0tLSBhL2ZzL25mcy9uZnM0c2Vzc2lvbi5jDQo+ICsrKyBiL2ZzL25mcy9uZnM0
c2Vzc2lvbi5jDQo+IEBAIC00NCw2ICs0NCwxNyBAQCBzdGF0aWMgdm9pZCBuZnM0X3Nocmlua19z
bG90X3RhYmxlKHN0cnVjdCBuZnM0X3Nsb3RfdGFibGUgICp0YmwsIHUzMiBuZXdzaXplKQ0KPiAg
CX0NCj4gIH0NCj4gIA0KPiArLyoqDQo+ICsgKiBuZnM0X3Nsb3RfdGJsX2RyYWluX2NvbXBsZXRl
IC0gd2FrZSB3YWl0ZXJzIHdoZW4gZHJhaW4gaXMgY29tcGxldGUNCj4gKyAqIEB0YmwgLSBjb250
cm9sbGluZyBzbG90IHRhYmxlDQo+ICsgKg0KPiArICovDQo+ICt2b2lkIG5mczRfc2xvdF90Ymxf
ZHJhaW5fY29tcGxldGUoc3RydWN0IG5mczRfc2xvdF90YWJsZSAqdGJsKQ0KPiArew0KPiArCWlm
IChuZnM0X3Nsb3RfdGJsX2RyYWluaW5nKHRibCkpDQo+ICsJCWNvbXBsZXRlKCZ0YmwtPmNvbXBs
ZXRlKTsNCj4gK30NCj4gKw0KPiAgLyoNCj4gICAqIG5mczRfZnJlZV9zbG90IC0gZnJlZSBhIHNs
b3QgYW5kIGVmZmljaWVudGx5IHVwZGF0ZSBzbG90IHRhYmxlLg0KPiAgICoNCj4gQEAgLTIxMiwx
MyArMjIzLDYgQEAgb3V0Og0KPiAgCXJldHVybiByZXQ7DQo+ICB9DQo+ICANCj4gLS8qIERlc3Ry
b3kgdGhlIHNsb3QgdGFibGUgKi8NCj4gLXN0YXRpYyB2b2lkIG5mczRfZGVzdHJveV9zbG90X3Rh
YmxlcyhzdHJ1Y3QgbmZzNF9zZXNzaW9uICpzZXNzaW9uKQ0KPiAtew0KPiAtCW5mczRfc2hyaW5r
X3Nsb3RfdGFibGUoJnNlc3Npb24tPmZjX3Nsb3RfdGFibGUsIDApOw0KPiAtCW5mczRfc2hyaW5r
X3Nsb3RfdGFibGUoJnNlc3Npb24tPmJjX3Nsb3RfdGFibGUsIDApOw0KPiAtfQ0KPiAtDQo+ICBz
dGF0aWMgYm9vbCBuZnM0MV9hc3NpZ25fc2xvdChzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2ssIHZvaWQg
KnBzbG90KQ0KPiAgew0KPiAgCXN0cnVjdCBuZnM0X3NlcXVlbmNlX2FyZ3MgKmFyZ3MgPSB0YXNr
LT50a19tc2cucnBjX2FyZ3A7DQo+IEBAIC0zODMsNiArMzg3LDE1IEBAIHZvaWQgbmZzNDFfdXBk
YXRlX3RhcmdldF9zbG90aWQoc3RydWN0IG5mczRfc2xvdF90YWJsZSAqdGJsLA0KPiAgCXNwaW5f
dW5sb2NrKCZ0YmwtPnNsb3RfdGJsX2xvY2spOw0KPiAgfQ0KPiAgDQo+ICsjaWYgZGVmaW5lZChD
T05GSUdfTkZTX1Y0XzEpDQo+ICsNCj4gKy8qIERlc3Ryb3kgdGhlIHNsb3QgdGFibGUgKi8NCj4g
K3N0YXRpYyB2b2lkIG5mczRfZGVzdHJveV9zbG90X3RhYmxlcyhzdHJ1Y3QgbmZzNF9zZXNzaW9u
ICpzZXNzaW9uKQ0KPiArew0KPiArCW5mczRfc2hyaW5rX3Nsb3RfdGFibGUoJnNlc3Npb24tPmZj
X3Nsb3RfdGFibGUsIDApOw0KPiArCW5mczRfc2hyaW5rX3Nsb3RfdGFibGUoJnNlc3Npb24tPmJj
X3Nsb3RfdGFibGUsIDApOw0KPiArfQ0KPiArDQo+ICAvKg0KPiAgICogSW5pdGlhbGl6ZSBvciBy
ZXNldCB0aGUgZm9yZWNoYW5uZWwgYW5kIGJhY2tjaGFubmVsIHRhYmxlcw0KPiAgICovDQo+IEBA
IC01MTMsNCArNTI2LDQgQEAgaW50IG5mczRfaW5pdF9kc19zZXNzaW9uKHN0cnVjdCBuZnNfY2xp
ZW50ICpjbHAsIHVuc2lnbmVkIGxvbmcgbGVhc2VfdGltZSkNCj4gIH0NCj4gIEVYUE9SVF9TWU1C
T0xfR1BMKG5mczRfaW5pdF9kc19zZXNzaW9uKTsNCj4gIA0KPiAtDQo+ICsjZW5kaWYJLyogZGVm
aW5lZChDT05GSUdfTkZTX1Y0XzEpICovDQo+IGRpZmYgLS1naXQgYS9mcy9uZnMvbmZzNHNlc3Np
b24uaCBiL2ZzL25mcy9uZnM0c2Vzc2lvbi5oDQo+IGluZGV4IDNhMTUzZDguLjIzZDI2YjQgMTAw
NjQ0DQo+IC0tLSBhL2ZzL25mcy9uZnM0c2Vzc2lvbi5oDQo+ICsrKyBiL2ZzL25mcy9uZnM0c2Vz
c2lvbi5oDQo+IEBAIC03MiwxNSArNzIsMjQgQEAgZW51bSBuZnM0X3Nlc3Npb25fc3RhdGUgew0K
PiAgCU5GUzRfU0VTU0lPTl9JTklUSU5HLA0KPiAgfTsNCj4gIA0KPiAtI2lmIGRlZmluZWQoQ09O
RklHX05GU19WNF8xKQ0KPiAgZXh0ZXJuIHN0cnVjdCBuZnM0X3Nsb3QgKm5mczRfYWxsb2Nfc2xv
dChzdHJ1Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwpOw0KPiAgZXh0ZXJuIHZvaWQgbmZzNF9mcmVl
X3Nsb3Qoc3RydWN0IG5mczRfc2xvdF90YWJsZSAqdGJsLCBzdHJ1Y3QgbmZzNF9zbG90ICpzbG90
KTsNCj4gLQ0KPiAtZXh0ZXJuIHZvaWQgbmZzNDFfc2V0X3RhcmdldF9zbG90aWQoc3RydWN0IG5m
czRfc2xvdF90YWJsZSAqdGJsLA0KPiAtCQl1MzIgdGFyZ2V0X2hpZ2hlc3Rfc2xvdGlkKTsNCj4g
K2V4dGVybiB2b2lkIG5mczRfc2xvdF90YmxfZHJhaW5fY29tcGxldGUoc3RydWN0IG5mczRfc2xv
dF90YWJsZSAqdGJsKTsNCj4gIGV4dGVybiB2b2lkIG5mczQxX3VwZGF0ZV90YXJnZXRfc2xvdGlk
KHN0cnVjdCBuZnM0X3Nsb3RfdGFibGUgKnRibCwNCj4gIAkJc3RydWN0IG5mczRfc2xvdCAqc2xv
dCwNCj4gIAkJc3RydWN0IG5mczRfc2VxdWVuY2VfcmVzICpyZXMpOw0KPiArYm9vbCBuZnM0MV93
YWtlX2FuZF9hc3NpZ25fc2xvdChzdHJ1Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwsDQo+ICsJCXN0
cnVjdCBuZnM0X3Nsb3QgKnNsb3QpOw0KPiArdm9pZCBuZnM0MV93YWtlX3Nsb3RfdGFibGUoc3Ry
dWN0IG5mczRfc2xvdF90YWJsZSAqdGJsKTsNCj4gKw0KPiArc3RhdGljIGlubGluZSBib29sIG5m
czRfc2xvdF90YmxfZHJhaW5pbmcoc3RydWN0IG5mczRfc2xvdF90YWJsZSAqdGJsKQ0KPiArew0K
PiArCXJldHVybiAhIXRlc3RfYml0KE5GUzRfU0xPVF9UQkxfRFJBSU5JTkcsICZ0YmwtPnNsb3Rf
dGJsX3N0YXRlKTsNCj4gK30NCj4gKw0KPiArI2lmIGRlZmluZWQoQ09ORklHX05GU19WNF8xKQ0K
PiArZXh0ZXJuIHZvaWQgbmZzNDFfc2V0X3RhcmdldF9zbG90aWQoc3RydWN0IG5mczRfc2xvdF90
YWJsZSAqdGJsLA0KPiArCQl1MzIgdGFyZ2V0X2hpZ2hlc3Rfc2xvdGlkKTsNCj4gIA0KPiAgZXh0
ZXJuIGludCBuZnM0X3NldHVwX3Nlc3Npb25fc2xvdF90YWJsZXMoc3RydWN0IG5mczRfc2Vzc2lv
biAqc2VzKTsNCj4gIA0KPiBAQCAtODksMTcgKzk4LDYgQEAgZXh0ZXJuIHZvaWQgbmZzNF9kZXN0
cm95X3Nlc3Npb24oc3RydWN0IG5mczRfc2Vzc2lvbiAqc2Vzc2lvbik7DQo+ICBleHRlcm4gaW50
IG5mczRfaW5pdF9zZXNzaW9uKHN0cnVjdCBuZnNfY2xpZW50ICpjbHApOw0KPiAgZXh0ZXJuIGlu
dCBuZnM0X2luaXRfZHNfc2Vzc2lvbihzdHJ1Y3QgbmZzX2NsaWVudCAqLCB1bnNpZ25lZCBsb25n
KTsNCj4gIA0KPiAtZXh0ZXJuIHZvaWQgbmZzNF9zbG90X3RibF9kcmFpbl9jb21wbGV0ZShzdHJ1
Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwpOw0KPiAtDQo+IC1zdGF0aWMgaW5saW5lIGJvb2wgbmZz
NF9zbG90X3RibF9kcmFpbmluZyhzdHJ1Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwpDQo+IC17DQo+
IC0JcmV0dXJuICEhdGVzdF9iaXQoTkZTNF9TTE9UX1RCTF9EUkFJTklORywgJnRibC0+c2xvdF90
Ymxfc3RhdGUpOw0KPiAtfQ0KPiAtDQo+IC1ib29sIG5mczQxX3dha2VfYW5kX2Fzc2lnbl9zbG90
KHN0cnVjdCBuZnM0X3Nsb3RfdGFibGUgKnRibCwNCj4gLQkJc3RydWN0IG5mczRfc2xvdCAqc2xv
dCk7DQo+IC12b2lkIG5mczQxX3dha2Vfc2xvdF90YWJsZShzdHJ1Y3QgbmZzNF9zbG90X3RhYmxl
ICp0YmwpOw0KPiAtDQo+ICAvKg0KPiAgICogRGV0ZXJtaW5lIGlmIHNlc3Npb25zIGFyZSBpbiB1
c2UuDQo+ICAgKi8NCj4gZGlmZiAtLWdpdCBhL2ZzL25mcy9uZnM0c3RhdGUuYyBiL2ZzL25mcy9u
ZnM0c3RhdGUuYw0KPiBpbmRleCA2ODE4OTY0Li44ODI2ZGYzIDEwMDY0NA0KPiAtLS0gYS9mcy9u
ZnMvbmZzNHN0YXRlLmMNCj4gKysrIGIvZnMvbmZzL25mczRzdGF0ZS5jDQo+IEBAIC0yNjAsMTUg
KzI2MCw2IEBAIHN0YXRpYyB2b2lkIG5mczRfZW5kX2RyYWluX3Nlc3Npb24oc3RydWN0IG5mc19j
bGllbnQgKmNscCkNCj4gIAl9DQo+ICB9DQo+ICANCj4gLS8qDQo+IC0gKiBTaWduYWwgc3RhdGUg
bWFuYWdlciB0aHJlYWQgaWYgc2Vzc2lvbiBmb3JlIGNoYW5uZWwgaXMgZHJhaW5lZA0KPiAtICov
DQo+IC12b2lkIG5mczRfc2xvdF90YmxfZHJhaW5fY29tcGxldGUoc3RydWN0IG5mczRfc2xvdF90
YWJsZSAqdGJsKQ0KPiAtew0KPiAtCWlmIChuZnM0X3Nsb3RfdGJsX2RyYWluaW5nKHRibCkpDQo+
IC0JCWNvbXBsZXRlKCZ0YmwtPmNvbXBsZXRlKTsNCj4gLX0NCj4gLQ0KPiAgc3RhdGljIGludCBu
ZnM0X2RyYWluX3Nsb3RfdGJsKHN0cnVjdCBuZnM0X3Nsb3RfdGFibGUgKnRibCkNCj4gIHsNCj4g
IAlzZXRfYml0KE5GUzRfU0xPVF9UQkxfRFJBSU5JTkcsICZ0YmwtPnNsb3RfdGJsX3N0YXRlKTsN
Cj4gDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5l
ICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiIGluDQo+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBt
YWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6
Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0KDQp3aGVuIGNvbXBpbGVkIHdp
dGhvdXQgQ09ORklHX05GU19WNF8xLCBzcGFyc2UgY29tcGxhaW5zOg0KDQpbdHJvbmRteUBsZWly
YSBsaW51eC1uZnNdJCBtYWtlIE89Li4vb2JqZWN0Lng4Nl82NCBXPTEgQz0yIGZzL25mcy9uZnM0
c2Vzc2lvbi5vDQptYWtlWzJdOiBOb3RoaW5nIHRvIGJlIGRvbmUgZm9yIGBhbGwnLg0KbWFrZVsy
XTogTm90aGluZyB0byBiZSBkb25lIGZvciBgcmVsb2NzJy4NCiAgVXNpbmcgL2hvbWUvdHJvbmRt
eS9kZXZlbC9rZXJuZWwvbGludXgtbmZzIGFzIHNvdXJjZSBmb3Iga2VybmVsDQogIEdFTiAgICAg
L2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvb2JqZWN0Lng4Nl82NC9NYWtlZmlsZQ0KICBDSEsg
ICAgIGluY2x1ZGUvZ2VuZXJhdGVkL3VhcGkvbGludXgvdmVyc2lvbi5oDQogIENISyAgICAgaW5j
bHVkZS9nZW5lcmF0ZWQvdXRzcmVsZWFzZS5oDQogIENBTEwgICAgL2hvbWUvdHJvbmRteS9kZXZl
bC9rZXJuZWwvbGludXgtbmZzL3NjcmlwdHMvY2hlY2tzeXNjYWxscy5zaA0KICBDSEVDSyAgIC9o
b21lL3Ryb25kbXkvZGV2ZWwva2VybmVsL2xpbnV4LW5mcy9zY3JpcHRzL21vZC9lbXB0eS5jDQog
IENIRUNLICAgL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL2ZzL25mcy9uZnM0
c2Vzc2lvbi5jDQovaG9tZS90cm9uZG15L2RldmVsL2tlcm5lbC9saW51eC1uZnMvZnMvbmZzL25m
czRzZXNzaW9uLmM6MzA0OjY6IHdhcm5pbmc6IHN5bWJvbCAnbmZzNDFfc2V0X3RhcmdldF9zbG90
aWQnIHdhcyBub3QgZGVjbGFyZWQuIFNob3VsZCBpdCBiZSBzdGF0aWM/DQovaG9tZS90cm9uZG15
L2RldmVsL2tlcm5lbC9saW51eC1uZnMvaW5jbHVkZS9saW51eC9lcnIuaDozNDoxNjogd2Fybmlu
ZzogZGVyZWZlcmVuY2Ugb2Ygbm9kZXJlZiBleHByZXNzaW9uDQovaG9tZS90cm9uZG15L2RldmVs
L2tlcm5lbC9saW51eC1uZnMvaW5jbHVkZS9saW51eC9lcnIuaDozNDoxNjogd2FybmluZzogZGVy
ZWZlcmVuY2Ugb2Ygbm9kZXJlZiBleHByZXNzaW9uDQovaG9tZS90cm9uZG15L2RldmVsL2tlcm5l
bC9saW51eC1uZnMvaW5jbHVkZS9saW51eC9lcnIuaDozNDoxNjogd2FybmluZzogZGVyZWZlcmVu
Y2Ugb2Ygbm9kZXJlZiBleHByZXNzaW9uDQovaG9tZS90cm9uZG15L2RldmVsL2tlcm5lbC9saW51
eC1uZnMvaW5jbHVkZS9saW51eC9lcnIuaDozNDoxNjogd2FybmluZzogZGVyZWZlcmVuY2Ugb2Yg
bm9kZXJlZiBleHByZXNzaW9uDQogIENDIFtNXSAgZnMvbmZzL25mczRzZXNzaW9uLm8NCi9ob21l
L3Ryb25kbXkvZGV2ZWwva2VybmVsL2xpbnV4LW5mcy9mcy9uZnMvbmZzNHNlc3Npb24uYzozMDQ6
Njogd2FybmluZzogbm8gcHJldmlvdXMgcHJvdG90eXBlIGZvciDigJhuZnM0MV9zZXRfdGFyZ2V0
X3Nsb3RpZOKAmSBbLVdtaXNzaW5nLXByb3RvdHlwZXNdDQogdm9pZCBuZnM0MV9zZXRfdGFyZ2V0
X3Nsb3RpZChzdHJ1Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwsDQogICAgICBeDQovaG9tZS90cm9u
ZG15L2RldmVsL2tlcm5lbC9saW51eC1uZnMvZnMvbmZzL25mczRzZXNzaW9uLmM6MjAwOjEyOiB3
YXJuaW5nOiDigJhuZnM0X3JlYWxsb2Nfc2xvdF90YWJsZeKAmSBkZWZpbmVkIGJ1dCBub3QgdXNl
ZCBbLVd1bnVzZWQtZnVuY3Rpb25dDQogc3RhdGljIGludCBuZnM0X3JlYWxsb2Nfc2xvdF90YWJs
ZShzdHJ1Y3QgbmZzNF9zbG90X3RhYmxlICp0YmwsDQoNCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QN
CkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpOZXRBcHANClRyb25kLk15a2xlYnVzdEBu
ZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K

Re: [PATCH v2 15/33] NFS: Update session draining barriers for NFSv4.0 transport blocking

[Myklebust, Trond]

T24gRnJpLCAyMDEzLTA4LTA5IGF0IDEyOjQ5IC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
U2lnbmVkLW9mZi1ieTogQ2h1Y2sgTGV2ZXIgPGNodWNrLmxldmVyQG9yYWNsZS5jb20+DQo+IC0t
LQ0KPiAgZnMvbmZzL25mczRzdGF0ZS5jIHwgICA2MSArKysrKysrKysrKysrKysrKysrKysrKysr
KysrKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQo+ICAxIGZpbGUgY2hhbmdlZCwgMzQgaW5zZXJ0
aW9ucygrKSwgMjcgZGVsZXRpb25zKC0pDQo+IA0KPiBkaWZmIC0tZ2l0IGEvZnMvbmZzL25mczRz
dGF0ZS5jIGIvZnMvbmZzL25mczRzdGF0ZS5jDQo+IGluZGV4IDg4MjZkZjMuLmQ4MTZkOTMgMTAw
NjQ0DQo+IC0tLSBhL2ZzL25mcy9uZnM0c3RhdGUuYw0KPiArKysgYi9mcy9uZnMvbmZzNHN0YXRl
LmMNCj4gQEAgLTIxNSwzMiArMjE1LDYgQEAgb3V0Og0KPiAgCXJldHVybiBjcmVkOw0KPiAgfQ0K
PiAgDQo+IC0jaWYgZGVmaW5lZChDT05GSUdfTkZTX1Y0XzEpDQo+IC0NCj4gLXN0YXRpYyBpbnQg
bmZzNDFfc2V0dXBfc3RhdGVfcmVuZXdhbChzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwKQ0KPiAtew0K
PiAtCWludCBzdGF0dXM7DQo+IC0Jc3RydWN0IG5mc19mc2luZm8gZnNpbmZvOw0KPiAtDQo+IC0J
aWYgKCF0ZXN0X2JpdChORlNfQ1NfQ0hFQ0tfTEVBU0VfVElNRSwgJmNscC0+Y2xfcmVzX3N0YXRl
KSkgew0KPiAtCQluZnM0X3NjaGVkdWxlX3N0YXRlX3JlbmV3YWwoY2xwKTsNCj4gLQkJcmV0dXJu
IDA7DQo+IC0JfQ0KPiAtDQo+IC0Jc3RhdHVzID0gbmZzNF9wcm9jX2dldF9sZWFzZV90aW1lKGNs
cCwgJmZzaW5mbyk7DQo+IC0JaWYgKHN0YXR1cyA9PSAwKSB7DQo+IC0JCS8qIFVwZGF0ZSBsZWFz
ZSB0aW1lIGFuZCBzY2hlZHVsZSByZW5ld2FsICovDQo+IC0JCXNwaW5fbG9jaygmY2xwLT5jbF9s
b2NrKTsNCj4gLQkJY2xwLT5jbF9sZWFzZV90aW1lID0gZnNpbmZvLmxlYXNlX3RpbWUgKiBIWjsN
Cj4gLQkJY2xwLT5jbF9sYXN0X3JlbmV3YWwgPSBqaWZmaWVzOw0KPiAtCQlzcGluX3VubG9jaygm
Y2xwLT5jbF9sb2NrKTsNCj4gLQ0KPiAtCQluZnM0X3NjaGVkdWxlX3N0YXRlX3JlbmV3YWwoY2xw
KTsNCj4gLQl9DQo+IC0NCj4gLQlyZXR1cm4gc3RhdHVzOw0KPiAtfQ0KPiAtDQo+ICBzdGF0aWMg
dm9pZCBuZnM0X2VuZF9kcmFpbl9zbG90X3RhYmxlKHN0cnVjdCBuZnM0X3Nsb3RfdGFibGUgKnRi
bCkNCj4gIHsNCj4gIAlpZiAodGVzdF9hbmRfY2xlYXJfYml0KE5GUzRfU0xPVF9UQkxfRFJBSU5J
TkcsICZ0YmwtPnNsb3RfdGJsX3N0YXRlKSkgew0KPiBAQCAtMjU0LDYgKzIyOCwxMSBAQCBzdGF0
aWMgdm9pZCBuZnM0X2VuZF9kcmFpbl9zZXNzaW9uKHN0cnVjdCBuZnNfY2xpZW50ICpjbHApDQo+
ICB7DQo+ICAJc3RydWN0IG5mczRfc2Vzc2lvbiAqc2VzID0gY2xwLT5jbF9zZXNzaW9uOw0KPiAg
DQo+ICsJaWYgKGNscC0+Y2xfc2xvdF90YmwpIHsNCj4gKwkJbmZzNF9lbmRfZHJhaW5fc2xvdF90
YWJsZShjbHAtPmNsX3Nsb3RfdGJsKTsNCj4gKwkJcmV0dXJuOw0KPiArCX0NCj4gKw0KPiAgCWlm
IChzZXMgIT0gTlVMTCkgew0KPiAgCQluZnM0X2VuZF9kcmFpbl9zbG90X3RhYmxlKCZzZXMtPmJj
X3Nsb3RfdGFibGUpOw0KPiAgCQluZnM0X2VuZF9kcmFpbl9zbG90X3RhYmxlKCZzZXMtPmZjX3Ns
b3RfdGFibGUpOw0KPiBAQCAtMjc4LDYgKzI1Nyw5IEBAIHN0YXRpYyBpbnQgbmZzNF9iZWdpbl9k
cmFpbl9zZXNzaW9uKHN0cnVjdCBuZnNfY2xpZW50ICpjbHApDQo+ICAJc3RydWN0IG5mczRfc2Vz
c2lvbiAqc2VzID0gY2xwLT5jbF9zZXNzaW9uOw0KPiAgCWludCByZXQgPSAwOw0KPiAgDQo+ICsJ
aWYgKGNscC0+Y2xfc2xvdF90YmwpDQo+ICsJCXJldHVybiBuZnM0X2RyYWluX3Nsb3RfdGJsKGNs
cC0+Y2xfc2xvdF90YmwpOw0KPiArDQo+ICAJLyogYmFjayBjaGFubmVsICovDQo+ICAJcmV0ID0g
bmZzNF9kcmFpbl9zbG90X3RibCgmc2VzLT5iY19zbG90X3RhYmxlKTsNCj4gIAlpZiAocmV0KQ0K
PiBAQCAtMjg2LDYgKzI2OCwzMiBAQCBzdGF0aWMgaW50IG5mczRfYmVnaW5fZHJhaW5fc2Vzc2lv
bihzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwKQ0KPiAgCXJldHVybiBuZnM0X2RyYWluX3Nsb3RfdGJs
KCZzZXMtPmZjX3Nsb3RfdGFibGUpOw0KPiAgfQ0KPiAgDQo+ICsjaWYgZGVmaW5lZChDT05GSUdf
TkZTX1Y0XzEpDQo+ICsNCj4gK3N0YXRpYyBpbnQgbmZzNDFfc2V0dXBfc3RhdGVfcmVuZXdhbChz
dHJ1Y3QgbmZzX2NsaWVudCAqY2xwKQ0KPiArew0KPiArCWludCBzdGF0dXM7DQo+ICsJc3RydWN0
IG5mc19mc2luZm8gZnNpbmZvOw0KPiArDQo+ICsJaWYgKCF0ZXN0X2JpdChORlNfQ1NfQ0hFQ0tf
TEVBU0VfVElNRSwgJmNscC0+Y2xfcmVzX3N0YXRlKSkgew0KPiArCQluZnM0X3NjaGVkdWxlX3N0
YXRlX3JlbmV3YWwoY2xwKTsNCj4gKwkJcmV0dXJuIDA7DQo+ICsJfQ0KPiArDQo+ICsJc3RhdHVz
ID0gbmZzNF9wcm9jX2dldF9sZWFzZV90aW1lKGNscCwgJmZzaW5mbyk7DQo+ICsJaWYgKHN0YXR1
cyA9PSAwKSB7DQo+ICsJCS8qIFVwZGF0ZSBsZWFzZSB0aW1lIGFuZCBzY2hlZHVsZSByZW5ld2Fs
ICovDQo+ICsJCXNwaW5fbG9jaygmY2xwLT5jbF9sb2NrKTsNCj4gKwkJY2xwLT5jbF9sZWFzZV90
aW1lID0gZnNpbmZvLmxlYXNlX3RpbWUgKiBIWjsNCj4gKwkJY2xwLT5jbF9sYXN0X3JlbmV3YWwg
PSBqaWZmaWVzOw0KPiArCQlzcGluX3VubG9jaygmY2xwLT5jbF9sb2NrKTsNCj4gKw0KPiArCQlu
ZnM0X3NjaGVkdWxlX3N0YXRlX3JlbmV3YWwoY2xwKTsNCj4gKwl9DQo+ICsNCj4gKwlyZXR1cm4g
c3RhdHVzOw0KPiArfQ0KPiArDQo+ICBzdGF0aWMgdm9pZCBuZnM0MV9maW5pc2hfc2Vzc2lvbl9y
ZXNldChzdHJ1Y3QgbmZzX2NsaWVudCAqY2xwKQ0KPiAgew0KPiAgCWNsZWFyX2JpdChORlM0Q0xO
VF9MRUFTRV9DT05GSVJNLCAmY2xwLT5jbF9zdGF0ZSk7DQo+IEBAIC0yMDgzLDcgKzIwOTEsNiBA
QCBzdGF0aWMgaW50IG5mczRfYmluZF9jb25uX3RvX3Nlc3Npb24oc3RydWN0IG5mc19jbGllbnQg
KmNscCkNCj4gIH0NCj4gICNlbHNlIC8qIENPTkZJR19ORlNfVjRfMSAqLw0KPiAgc3RhdGljIGlu
dCBuZnM0X3Jlc2V0X3Nlc3Npb24oc3RydWN0IG5mc19jbGllbnQgKmNscCkgeyByZXR1cm4gMDsg
fQ0KPiAtc3RhdGljIHZvaWQgbmZzNF9lbmRfZHJhaW5fc2Vzc2lvbihzdHJ1Y3QgbmZzX2NsaWVu
dCAqY2xwKSB7IH0NCj4gIA0KPiAgc3RhdGljIGludCBuZnM0X2JpbmRfY29ubl90b19zZXNzaW9u
KHN0cnVjdCBuZnNfY2xpZW50ICpjbHApDQo+ICB7DQo+IA0KPiAtLQ0KPiBUbyB1bnN1YnNjcmli
ZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3JpYmUgbGludXgtbmZzIiBp
bg0KPiB0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIua2VybmVsLm9yZw0K
PiBNb3JlIG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9yZG9t
by1pbmZvLmh0bWwNCg0KV2hlbiBjb21waWxpbmcgd2l0aG91dCBDT05GSUdfTkZTX1Y0XzE6DQoN
Clt0cm9uZG15QGxlaXJhIGxpbnV4LW5mc10kIG1ha2UgTz0uLi9vYmplY3QueDg2XzY0IFc9MSBD
PTIgZnMvbmZzL25mczRzdGF0ZS5vDQptYWtlWzJdOiBOb3RoaW5nIHRvIGJlIGRvbmUgZm9yIGBh
bGwnLg0KbWFrZVsyXTogTm90aGluZyB0byBiZSBkb25lIGZvciBgcmVsb2NzJy4NCiAgVXNpbmcg
L2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzIGFzIHNvdXJjZSBmb3Iga2VybmVs
DQogIEdFTiAgICAgL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvb2JqZWN0Lng4Nl82NC9NYWtl
ZmlsZQ0KICBDSEsgICAgIGluY2x1ZGUvZ2VuZXJhdGVkL3VhcGkvbGludXgvdmVyc2lvbi5oDQog
IENISyAgICAgaW5jbHVkZS9nZW5lcmF0ZWQvdXRzcmVsZWFzZS5oDQogIENBTEwgICAgL2hvbWUv
dHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL3NjcmlwdHMvY2hlY2tzeXNjYWxscy5zaA0K
ICBDSEVDSyAgIC9ob21lL3Ryb25kbXkvZGV2ZWwva2VybmVsL2xpbnV4LW5mcy9zY3JpcHRzL21v
ZC9lbXB0eS5jDQogIENIRUNLICAgL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZz
L2ZzL25mcy9uZnM0c3RhdGUuYw0KL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZz
L2luY2x1ZGUvbGludXgvZXJyLmg6MzQ6MTY6IHdhcm5pbmc6IGRlcmVmZXJlbmNlIG9mIG5vZGVy
ZWYgZXhwcmVzc2lvbg0KL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL2luY2x1
ZGUvbGludXgvZXJyLmg6MzQ6MTY6IHdhcm5pbmc6IGRlcmVmZXJlbmNlIG9mIG5vZGVyZWYgZXhw
cmVzc2lvbg0KL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL2luY2x1ZGUvbGlu
dXgvZXJyLmg6Mjk6MjM6IHdhcm5pbmc6IGRlcmVmZXJlbmNlIG9mIG5vZGVyZWYgZXhwcmVzc2lv
bg0KL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL2luY2x1ZGUvbGludXgvZXJy
Lmg6MzQ6MTY6IHdhcm5pbmc6IGRlcmVmZXJlbmNlIG9mIG5vZGVyZWYgZXhwcmVzc2lvbg0KL2hv
bWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZzL2luY2x1ZGUvbGludXgvZXJyLmg6Mjk6
MjM6IHdhcm5pbmc6IGRlcmVmZXJlbmNlIG9mIG5vZGVyZWYgZXhwcmVzc2lvbg0KICBDQyBbTV0g
IGZzL25mcy9uZnM0c3RhdGUubw0KL2hvbWUvdHJvbmRteS9kZXZlbC9rZXJuZWwvbGludXgtbmZz
L2ZzL25mcy9uZnM0c3RhdGUuYzoyNTU6MTI6IHdhcm5pbmc6IOKAmG5mczRfYmVnaW5fZHJhaW5f
c2Vzc2lvbuKAmSBkZWZpbmVkIGJ1dCBub3QgdXNlZCBbLVd1bnVzZWQtZnVuY3Rpb25dDQogc3Rh
dGljIGludCBuZnM0X2JlZ2luX2RyYWluX3Nlc3Npb24oc3RydWN0IG5mc19jbGllbnQgKmNscCkN
Cg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lcg0KDQpO
ZXRBcHANClRyb25kLk15a2xlYnVzdEBuZXRhcHAuY29tDQp3d3cubmV0YXBwLmNvbQ0K

Re: [PATCH v2 15/33] NFS: Update session draining barriers for NFSv4.0 transport blocking

[Chuck Lever]

On Sep 3, 2013, at 1:52 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Fri, 2013-08-09 at 12:49 -0400, Chuck Lever wrote:
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>> fs/nfs/nfs4state.c |   61 +++++++++++++++++++++++++++++-----------------------
>> 1 file changed, 34 insertions(+), 27 deletions(-)
>> 
>> diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
>> index 8826df3..d816d93 100644
>> --- a/fs/nfs/nfs4state.c
>> +++ b/fs/nfs/nfs4state.c
>> @@ -215,32 +215,6 @@ out:
>> 	return cred;
>> }
>> 
>> -#if defined(CONFIG_NFS_V4_1)
>> -
>> -static int nfs41_setup_state_renewal(struct nfs_client *clp)
>> -{
>> -	int status;
>> -	struct nfs_fsinfo fsinfo;
>> -
>> -	if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
>> -		nfs4_schedule_state_renewal(clp);
>> -		return 0;
>> -	}
>> -
>> -	status = nfs4_proc_get_lease_time(clp, &fsinfo);
>> -	if (status == 0) {
>> -		/* Update lease time and schedule renewal */
>> -		spin_lock(&clp->cl_lock);
>> -		clp->cl_lease_time = fsinfo.lease_time * HZ;
>> -		clp->cl_last_renewal = jiffies;
>> -		spin_unlock(&clp->cl_lock);
>> -
>> -		nfs4_schedule_state_renewal(clp);
>> -	}
>> -
>> -	return status;
>> -}
>> -
>> static void nfs4_end_drain_slot_table(struct nfs4_slot_table *tbl)
>> {
>> 	if (test_and_clear_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state)) {
>> @@ -254,6 +228,11 @@ static void nfs4_end_drain_session(struct nfs_client *clp)
>> {
>> 	struct nfs4_session *ses = clp->cl_session;
>> 
>> +	if (clp->cl_slot_tbl) {
>> +		nfs4_end_drain_slot_table(clp->cl_slot_tbl);
>> +		return;
>> +	}
>> +
>> 	if (ses != NULL) {
>> 		nfs4_end_drain_slot_table(&ses->bc_slot_table);
>> 		nfs4_end_drain_slot_table(&ses->fc_slot_table);
>> @@ -278,6 +257,9 @@ static int nfs4_begin_drain_session(struct nfs_client *clp)
>> 	struct nfs4_session *ses = clp->cl_session;
>> 	int ret = 0;
>> 
>> +	if (clp->cl_slot_tbl)
>> +		return nfs4_drain_slot_tbl(clp->cl_slot_tbl);
>> +
>> 	/* back channel */
>> 	ret = nfs4_drain_slot_tbl(&ses->bc_slot_table);
>> 	if (ret)
>> @@ -286,6 +268,32 @@ static int nfs4_begin_drain_session(struct nfs_client *clp)
>> 	return nfs4_drain_slot_tbl(&ses->fc_slot_table);
>> }
>> 
>> +#if defined(CONFIG_NFS_V4_1)
>> +
>> +static int nfs41_setup_state_renewal(struct nfs_client *clp)
>> +{
>> +	int status;
>> +	struct nfs_fsinfo fsinfo;
>> +
>> +	if (!test_bit(NFS_CS_CHECK_LEASE_TIME, &clp->cl_res_state)) {
>> +		nfs4_schedule_state_renewal(clp);
>> +		return 0;
>> +	}
>> +
>> +	status = nfs4_proc_get_lease_time(clp, &fsinfo);
>> +	if (status == 0) {
>> +		/* Update lease time and schedule renewal */
>> +		spin_lock(&clp->cl_lock);
>> +		clp->cl_lease_time = fsinfo.lease_time * HZ;
>> +		clp->cl_last_renewal = jiffies;
>> +		spin_unlock(&clp->cl_lock);
>> +
>> +		nfs4_schedule_state_renewal(clp);
>> +	}
>> +
>> +	return status;
>> +}
>> +
>> static void nfs41_finish_session_reset(struct nfs_client *clp)
>> {
>> 	clear_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
>> @@ -2083,7 +2091,6 @@ static int nfs4_bind_conn_to_session(struct nfs_client *clp)
>> }
>> #else /* CONFIG_NFS_V4_1 */
>> static int nfs4_reset_session(struct nfs_client *clp) { return 0; }
>> -static void nfs4_end_drain_session(struct nfs_client *clp) { }
>> 
>> static int nfs4_bind_conn_to_session(struct nfs_client *clp)
>> {
>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> When compiling without CONFIG_NFS_V4_1:
> 
> [trondmy@leira linux-nfs]$ make O=../object.x86_64 W=1 C=2 fs/nfs/nfs4state.o
> make[2]: Nothing to be done for `all'.
> make[2]: Nothing to be done for `relocs'.
>  Using /home/trondmy/devel/kernel/linux-nfs as source for kernel
>  GEN     /home/trondmy/devel/kernel/object.x86_64/Makefile
>  CHK     include/generated/uapi/linux/version.h
>  CHK     include/generated/utsrelease.h
>  CALL    /home/trondmy/devel/kernel/linux-nfs/scripts/checksyscalls.sh
>  CHECK   /home/trondmy/devel/kernel/linux-nfs/scripts/mod/empty.c
>  CHECK   /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4state.c
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:29:23: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:29:23: warning: dereference of noderef expression
>  CC [M]  fs/nfs/nfs4state.o
> /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4state.c:255:12: warning: ‘nfs4_begin_drain_session’ defined but not used [-Wunused-function]
> static int nfs4_begin_drain_session(struct nfs_client *clp)

That warning should be gone after "NFS: Add basic migration support to state manager thread".

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH v2 16/33] SUNRPC: Modify synopsis of rpc_client_register()

[Myklebust, Trond]

T24gRnJpLCAyMDEzLTA4LTA5IGF0IDEyOjUwIC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
VGhlIHJwY19jbGllbnRfcmVnaXN0ZXIoKSBoZWxwZXIgd2FzIGFkZGVkIGluIGNvbW1pdCBlNzNm
NGNjMCwNCj4gIlNVTlJQQzogc3BsaXQgY2xpZW50IGNyZWF0aW9uIHJvdXRpbmUgaW50byBzZXR1
cCBhbmQgcmVnaXN0cmF0aW9uLCINCj4gTW9uIEp1biAyNCAxMTo1Mjo1MiAyMDEzLg0KPiANCj4g
SSdkIGxpa2UgdG8gaW52b2tlIHJwY19jbGllbnRfcmVnaXN0ZXIoKSBmcm9tIGEgY29udGV4dCB3
aGVyZSBhDQo+IHN0cnVjdCBycGNfY3JlYXRlX2FyZ3MgaXMgbm90IGF2YWlsYWJsZS4gIFRoZXJl
IGFwcGVhciB0byBiZSBvbmx5DQo+IHR3byBmaWVsZHMgb2YgaW50ZXJlc3QgaW4gc3RydWN0IHJw
Y19jcmVhdGVfYXJnczogLnByb2dyYW0gYW5kDQo+IC5hdXRoZmxhdm9yIC4NCj4gDQo+IFNpZ25l
ZC1vZmYtYnk6IENodWNrIExldmVyIDxjaHVjay5sZXZlckBvcmFjbGUuY29tPg0KPiBDYzogU3Rh
bmlzbGF2IEtpbnNidXJza3kgPHNraW5zYnVyc2t5QHBhcmFsbGVscy5jb20+DQo+IC0tLQ0KPiAg
bmV0L3N1bnJwYy9jbG50LmMgfCAgIDEyICsrKysrKy0tLS0tLQ0KPiAgMSBmaWxlIGNoYW5nZWQs
IDYgaW5zZXJ0aW9ucygrKSwgNiBkZWxldGlvbnMoLSkNCj4gDQo+IGRpZmYgLS1naXQgYS9uZXQv
c3VucnBjL2NsbnQuYyBiL25ldC9zdW5ycGMvY2xudC5jDQo+IGluZGV4IDc0ZjZhNzAuLmY5ZTM5
MjYgMTAwNjQ0DQo+IC0tLSBhL25ldC9zdW5ycGMvY2xudC5jDQo+ICsrKyBiL25ldC9zdW5ycGMv
Y2xudC5jDQo+IEBAIC0yNzksMTAgKzI3OSwxMCBAQCBzdGF0aWMgdm9pZCBycGNfY2xudF9zZXRf
bm9kZW5hbWUoc3RydWN0IHJwY19jbG50ICpjbG50LCBjb25zdCBjaGFyICpub2RlbmFtZSkNCj4g
IAltZW1jcHkoY2xudC0+Y2xfbm9kZW5hbWUsIG5vZGVuYW1lLCBjbG50LT5jbF9ub2RlbGVuKTsN
Cj4gIH0NCj4gIA0KPiAtc3RhdGljIGludCBycGNfY2xpZW50X3JlZ2lzdGVyKGNvbnN0IHN0cnVj
dCBycGNfY3JlYXRlX2FyZ3MgKmFyZ3MsDQo+IC0JCQkgICAgICAgc3RydWN0IHJwY19jbG50ICpj
bG50KQ0KPiArc3RhdGljIGludCBycGNfY2xpZW50X3JlZ2lzdGVyKHN0cnVjdCBycGNfY2xudCAq
Y2xudCwNCj4gKwkJCSAgICAgICBycGNfYXV0aGZsYXZvcl90IHBzZXVkb2ZsYXZvcikNCj4gIHsN
Cj4gLQljb25zdCBzdHJ1Y3QgcnBjX3Byb2dyYW0gKnByb2dyYW0gPSBhcmdzLT5wcm9ncmFtOw0K
PiArCWNvbnN0IHN0cnVjdCBycGNfcHJvZ3JhbSAqcHJvZ3JhbSA9IGNsbnQtPmNsX3Byb2dyYW07
DQo+ICAJc3RydWN0IHJwY19hdXRoICphdXRoOw0KPiAgCXN0cnVjdCBuZXQgKm5ldCA9IHJwY19u
ZXRfbnMoY2xudCk7DQo+ICAJc3RydWN0IHN1cGVyX2Jsb2NrICpwaXBlZnNfc2I7DQo+IEBAIC0y
OTksMTAgKzI5OSwxMCBAQCBzdGF0aWMgaW50IHJwY19jbGllbnRfcmVnaXN0ZXIoY29uc3Qgc3Ry
dWN0IHJwY19jcmVhdGVfYXJncyAqYXJncywNCj4gIAlpZiAocGlwZWZzX3NiKQ0KPiAgCQlycGNf
cHV0X3NiX25ldChuZXQpOw0KPiAgDQo+IC0JYXV0aCA9IHJwY2F1dGhfY3JlYXRlKGFyZ3MtPmF1
dGhmbGF2b3IsIGNsbnQpOw0KPiArCWF1dGggPSBycGNhdXRoX2NyZWF0ZShwc2V1ZG9mbGF2b3Is
IGNsbnQpOw0KPiAgCWlmIChJU19FUlIoYXV0aCkpIHsNCj4gIAkJZHByaW50aygiUlBDOiAgICAg
ICBDb3VsZG4ndCBjcmVhdGUgYXV0aCBoYW5kbGUgKGZsYXZvciAldSlcbiIsDQo+IC0JCQkJYXJn
cy0+YXV0aGZsYXZvcik7DQo+ICsJCQkJcHNldWRvZmxhdm9yKTsNCj4gIAkJZXJyID0gUFRSX0VS
UihhdXRoKTsNCj4gIAkJZ290byBlcnJfYXV0aDsNCj4gIAl9DQo+IEBAIC0zODQsNyArMzg0LDcg
QEAgc3RhdGljIHN0cnVjdCBycGNfY2xudCAqIHJwY19uZXdfY2xpZW50KGNvbnN0IHN0cnVjdCBy
cGNfY3JlYXRlX2FyZ3MgKmFyZ3MsIHN0cnUNCj4gIAkvKiBzYXZlIHRoZSBub2RlbmFtZSAqLw0K
PiAgCXJwY19jbG50X3NldF9ub2RlbmFtZShjbG50LCB1dHNuYW1lKCktPm5vZGVuYW1lKTsNCj4g
IA0KPiAtCWVyciA9IHJwY19jbGllbnRfcmVnaXN0ZXIoYXJncywgY2xudCk7DQo+ICsJZXJyID0g
cnBjX2NsaWVudF9yZWdpc3RlcihjbG50LCBhcmdzLT5hdXRoZmxhdm9yKTsNCj4gIAlpZiAoZXJy
KQ0KPiAgCQlnb3RvIG91dF9ub19wYXRoOw0KPiAgCXJldHVybiBjbG50Ow0KPiANCg0KVGhpcyBj
b25mbGljdHMgYSBiaXQgd2l0aCB0aGUgbmV3IG1hbmFnZW1lbnQgY29kZSBmb3IgdGhlIHJwY19j
bG50J3MNCnJwY19waXBlZnMuIFRoZSBmaXh1cCBhYm92ZSBpcyB0cml2aWFsLCBidXQgSSB0aGlu
ayB0aGF0IHlvdSBtaWdodCB3YW50DQp0byBsb29rIGludG8gd2hldGhlciBvciBub3QgdGhlIG5l
dyBjb2RlIGNoYW5nZXMgYW55dGhpbmcgZm9yIHlvdXINCnJwY19hdXRoIHN0cmF0ZWd5Lg0KSW4g
cGFydGljdWxhciBub3RlIHRoYXQgdGhlIG5ldyBjb2RlIHVzZXMgdGhlIHJwY19jbG50LT5jbF9w
YXJlbnQgY2hhaW4NCmluIG9yZGVyIHRvIGltcGxlbWVudCBzaGFyaW5nIG9mIHRoZSBSUENTRUNf
R1NTIGNhY2hlcyBhbmQgcGlwZSB1cGNhbGxzLg0KDQotLSANClRyb25kIE15a2xlYnVzdA0KTGlu
dXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0QG5ldGFw
cC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: [PATCH v2 16/33] SUNRPC: Modify synopsis of rpc_client_register()

[Chuck Lever]

On Sep 3, 2013, at 2:04 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Fri, 2013-08-09 at 12:50 -0400, Chuck Lever wrote:
>> The rpc_client_register() helper was added in commit e73f4cc0,
>> "SUNRPC: split client creation routine into setup and registration,"
>> Mon Jun 24 11:52:52 2013.
>> 
>> I'd like to invoke rpc_client_register() from a context where a
>> struct rpc_create_args is not available.  There appear to be only
>> two fields of interest in struct rpc_create_args: .program and
>> .authflavor .
>> 
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> Cc: Stanislav Kinsbursky <skinsbursky@parallels.com>
>> ---
>> net/sunrpc/clnt.c |   12 ++++++------
>> 1 file changed, 6 insertions(+), 6 deletions(-)
>> 
>> diff --git a/net/sunrpc/clnt.c b/net/sunrpc/clnt.c
>> index 74f6a70..f9e3926 100644
>> --- a/net/sunrpc/clnt.c
>> +++ b/net/sunrpc/clnt.c
>> @@ -279,10 +279,10 @@ static void rpc_clnt_set_nodename(struct rpc_clnt *clnt, const char *nodename)
>> 	memcpy(clnt->cl_nodename, nodename, clnt->cl_nodelen);
>> }
>> 
>> -static int rpc_client_register(const struct rpc_create_args *args,
>> -			       struct rpc_clnt *clnt)
>> +static int rpc_client_register(struct rpc_clnt *clnt,
>> +			       rpc_authflavor_t pseudoflavor)
>> {
>> -	const struct rpc_program *program = args->program;
>> +	const struct rpc_program *program = clnt->cl_program;
>> 	struct rpc_auth *auth;
>> 	struct net *net = rpc_net_ns(clnt);
>> 	struct super_block *pipefs_sb;
>> @@ -299,10 +299,10 @@ static int rpc_client_register(const struct rpc_create_args *args,
>> 	if (pipefs_sb)
>> 		rpc_put_sb_net(net);
>> 
>> -	auth = rpcauth_create(args->authflavor, clnt);
>> +	auth = rpcauth_create(pseudoflavor, clnt);
>> 	if (IS_ERR(auth)) {
>> 		dprintk("RPC:       Couldn't create auth handle (flavor %u)\n",
>> -				args->authflavor);
>> +				pseudoflavor);
>> 		err = PTR_ERR(auth);
>> 		goto err_auth;
>> 	}
>> @@ -384,7 +384,7 @@ static struct rpc_clnt * rpc_new_client(const struct rpc_create_args *args, stru
>> 	/* save the nodename */
>> 	rpc_clnt_set_nodename(clnt, utsname()->nodename);
>> 
>> -	err = rpc_client_register(args, clnt);
>> +	err = rpc_client_register(clnt, args->authflavor);
>> 	if (err)
>> 		goto out_no_path;
>> 	return clnt;
>> 
> 
> This conflicts a bit with the new management code for the rpc_clnt's
> rpc_pipefs. The fixup above is trivial, but I think that you might want
> to look into whether or not the new code changes anything for your
> rpc_auth strategy.

Agreed.

> In particular note that the new code uses the rpc_clnt->cl_parent chain
> in order to implement sharing of the RPCSEC_GSS caches and pipe upcalls.

This might be a problem when swapping in a transport to a destination server, for example.

How would you like to proceed with merging these?  Hold off until 3.13, take up to 15/33 now and the rest in 3.13, or take the whole series now and fix it up after the window closes?

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH v2 07/33] NFS: Enable slot table helpers for NFSv4.0

[Chuck Lever]

On Sep 3, 2013, at 12:50 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Fri, 2013-08-09 at 12:48 -0400, Chuck Lever wrote:
>> I'd like to re-use NFSv4.1's slot table machinery for NFSv4.0
>> transport blocking.  Re-organize some of nfs4session.c so the slot
>> table code is built even when NFS_V4_1 is disabled.
>> 
>> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
>> ---
>> fs/nfs/Makefile      |    4 ++--
>> fs/nfs/nfs4session.c |   29 +++++++++++++++++++++--------
>> fs/nfs/nfs4session.h |   28 +++++++++++++---------------
>> fs/nfs/nfs4state.c   |    9 ---------
>> 4 files changed, 36 insertions(+), 34 deletions(-)
>> 
>> diff --git a/fs/nfs/Makefile b/fs/nfs/Makefile
>> index e0bb048..2839bfe 100644
>> --- a/fs/nfs/Makefile
>> +++ b/fs/nfs/Makefile
>> @@ -21,10 +21,10 @@ nfsv3-$(CONFIG_NFS_V3_ACL) += nfs3acl.o
>> obj-$(CONFIG_NFS_V4) += nfsv4.o
>> nfsv4-y := nfs4proc.o nfs4xdr.o nfs4state.o nfs4renewd.o nfs4super.o nfs4file.o \
>> 	  delegation.o idmap.o callback.o callback_xdr.o callback_proc.o \
>> -	  nfs4namespace.o nfs4getroot.o nfs4client.o dns_resolve.o
>> +	  nfs4namespace.o nfs4getroot.o nfs4client.o dns_resolve.o nfs4session.o
>> nfsv4-$(CONFIG_NFS_USE_LEGACY_DNS) += cache_lib.o
>> nfsv4-$(CONFIG_SYSCTL)	+= nfs4sysctl.o
>> -nfsv4-$(CONFIG_NFS_V4_1)	+= nfs4session.o pnfs.o pnfs_dev.o
>> +nfsv4-$(CONFIG_NFS_V4_1)	+= pnfs.o pnfs_dev.o
>> 
>> obj-$(CONFIG_PNFS_FILE_LAYOUT) += nfs_layout_nfsv41_files.o
>> nfs_layout_nfsv41_files-y := nfs4filelayout.o nfs4filelayoutdev.o
>> diff --git a/fs/nfs/nfs4session.c b/fs/nfs/nfs4session.c
>> index 0161ad2..746675b 100644
>> --- a/fs/nfs/nfs4session.c
>> +++ b/fs/nfs/nfs4session.c
>> @@ -44,6 +44,17 @@ static void nfs4_shrink_slot_table(struct nfs4_slot_table  *tbl, u32 newsize)
>> 	}
>> }
>> 
>> +/**
>> + * nfs4_slot_tbl_drain_complete - wake waiters when drain is complete
>> + * @tbl - controlling slot table
>> + *
>> + */
>> +void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl)
>> +{
>> +	if (nfs4_slot_tbl_draining(tbl))
>> +		complete(&tbl->complete);
>> +}
>> +
>> /*
>>  * nfs4_free_slot - free a slot and efficiently update slot table.
>>  *
>> @@ -212,13 +223,6 @@ out:
>> 	return ret;
>> }
>> 
>> -/* Destroy the slot table */
>> -static void nfs4_destroy_slot_tables(struct nfs4_session *session)
>> -{
>> -	nfs4_shrink_slot_table(&session->fc_slot_table, 0);
>> -	nfs4_shrink_slot_table(&session->bc_slot_table, 0);
>> -}
>> -
>> static bool nfs41_assign_slot(struct rpc_task *task, void *pslot)
>> {
>> 	struct nfs4_sequence_args *args = task->tk_msg.rpc_argp;
>> @@ -383,6 +387,15 @@ void nfs41_update_target_slotid(struct nfs4_slot_table *tbl,
>> 	spin_unlock(&tbl->slot_tbl_lock);
>> }
>> 
>> +#if defined(CONFIG_NFS_V4_1)
>> +
>> +/* Destroy the slot table */
>> +static void nfs4_destroy_slot_tables(struct nfs4_session *session)
>> +{
>> +	nfs4_shrink_slot_table(&session->fc_slot_table, 0);
>> +	nfs4_shrink_slot_table(&session->bc_slot_table, 0);
>> +}
>> +
>> /*
>>  * Initialize or reset the forechannel and backchannel tables
>>  */
>> @@ -513,4 +526,4 @@ int nfs4_init_ds_session(struct nfs_client *clp, unsigned long lease_time)
>> }
>> EXPORT_SYMBOL_GPL(nfs4_init_ds_session);
>> 
>> -
>> +#endif	/* defined(CONFIG_NFS_V4_1) */
>> diff --git a/fs/nfs/nfs4session.h b/fs/nfs/nfs4session.h
>> index 3a153d8..23d26b4 100644
>> --- a/fs/nfs/nfs4session.h
>> +++ b/fs/nfs/nfs4session.h
>> @@ -72,15 +72,24 @@ enum nfs4_session_state {
>> 	NFS4_SESSION_INITING,
>> };
>> 
>> -#if defined(CONFIG_NFS_V4_1)
>> extern struct nfs4_slot *nfs4_alloc_slot(struct nfs4_slot_table *tbl);
>> extern void nfs4_free_slot(struct nfs4_slot_table *tbl, struct nfs4_slot *slot);
>> -
>> -extern void nfs41_set_target_slotid(struct nfs4_slot_table *tbl,
>> -		u32 target_highest_slotid);
>> +extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);
>> extern void nfs41_update_target_slotid(struct nfs4_slot_table *tbl,
>> 		struct nfs4_slot *slot,
>> 		struct nfs4_sequence_res *res);
>> +bool nfs41_wake_and_assign_slot(struct nfs4_slot_table *tbl,
>> +		struct nfs4_slot *slot);
>> +void nfs41_wake_slot_table(struct nfs4_slot_table *tbl);
>> +
>> +static inline bool nfs4_slot_tbl_draining(struct nfs4_slot_table *tbl)
>> +{
>> +	return !!test_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);
>> +}
>> +
>> +#if defined(CONFIG_NFS_V4_1)
>> +extern void nfs41_set_target_slotid(struct nfs4_slot_table *tbl,
>> +		u32 target_highest_slotid);
>> 
>> extern int nfs4_setup_session_slot_tables(struct nfs4_session *ses);
>> 
>> @@ -89,17 +98,6 @@ extern void nfs4_destroy_session(struct nfs4_session *session);
>> extern int nfs4_init_session(struct nfs_client *clp);
>> extern int nfs4_init_ds_session(struct nfs_client *, unsigned long);
>> 
>> -extern void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl);
>> -
>> -static inline bool nfs4_slot_tbl_draining(struct nfs4_slot_table *tbl)
>> -{
>> -	return !!test_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);
>> -}
>> -
>> -bool nfs41_wake_and_assign_slot(struct nfs4_slot_table *tbl,
>> -		struct nfs4_slot *slot);
>> -void nfs41_wake_slot_table(struct nfs4_slot_table *tbl);
>> -
>> /*
>>  * Determine if sessions are in use.
>>  */
>> diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
>> index 6818964..8826df3 100644
>> --- a/fs/nfs/nfs4state.c
>> +++ b/fs/nfs/nfs4state.c
>> @@ -260,15 +260,6 @@ static void nfs4_end_drain_session(struct nfs_client *clp)
>> 	}
>> }
>> 
>> -/*
>> - * Signal state manager thread if session fore channel is drained
>> - */
>> -void nfs4_slot_tbl_drain_complete(struct nfs4_slot_table *tbl)
>> -{
>> -	if (nfs4_slot_tbl_draining(tbl))
>> -		complete(&tbl->complete);
>> -}
>> -
>> static int nfs4_drain_slot_tbl(struct nfs4_slot_table *tbl)
>> {
>> 	set_bit(NFS4_SLOT_TBL_DRAINING, &tbl->slot_tbl_state);
>> 
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> when compiled without CONFIG_NFS_V4_1, sparse complains:
> 
> [trondmy@leira linux-nfs]$ make O=../object.x86_64 W=1 C=2 fs/nfs/nfs4session.o
> make[2]: Nothing to be done for `all'.
> make[2]: Nothing to be done for `relocs'.
>  Using /home/trondmy/devel/kernel/linux-nfs as source for kernel
>  GEN     /home/trondmy/devel/kernel/object.x86_64/Makefile
>  CHK     include/generated/uapi/linux/version.h
>  CHK     include/generated/utsrelease.h
>  CALL    /home/trondmy/devel/kernel/linux-nfs/scripts/checksyscalls.sh
>  CHECK   /home/trondmy/devel/kernel/linux-nfs/scripts/mod/empty.c
>  CHECK   /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4session.c
> /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4session.c:304:6: warning: symbol 'nfs41_set_target_slotid' was not declared. Should it be static?
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
> /home/trondmy/devel/kernel/linux-nfs/include/linux/err.h:34:16: warning: dereference of noderef expression
>  CC [M]  fs/nfs/nfs4session.o
> /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4session.c:304:6: warning: no previous prototype for ‘nfs41_set_target_slotid’ [-Wmissing-prototypes]
> void nfs41_set_target_slotid(struct nfs4_slot_table *tbl,
>      ^
> /home/trondmy/devel/kernel/linux-nfs/fs/nfs/nfs4session.c:200:12: warning: ‘nfs4_realloc_slot_table’ defined but not used [-Wunused-function]
> static int nfs4_realloc_slot_table(struct nfs4_slot_table *tbl,

There is a trivial fix in fs/nfs/nfs4session.h.  I'm testing to see if that will cause merge conflicts with subsequent patches... or I can send a fix that applies on top of the series.  Up to you.

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: [PATCH v2 16/33] SUNRPC: Modify synopsis of rpc_client_register()

[Myklebust, Trond]

T24gVHVlLCAyMDEzLTA5LTAzIGF0IDE0OjE2IC0wNDAwLCBDaHVjayBMZXZlciB3cm90ZToNCj4g
T24gU2VwIDMsIDIwMTMsIGF0IDI6MDQgUE0sICJNeWtsZWJ1c3QsIFRyb25kIiA8VHJvbmQuTXlr
bGVidXN0QG5ldGFwcC5jb20+IHdyb3RlOg0KPiA+IEluIHBhcnRpY3VsYXIgbm90ZSB0aGF0IHRo
ZSBuZXcgY29kZSB1c2VzIHRoZSBycGNfY2xudC0+Y2xfcGFyZW50IGNoYWluDQo+ID4gaW4gb3Jk
ZXIgdG8gaW1wbGVtZW50IHNoYXJpbmcgb2YgdGhlIFJQQ1NFQ19HU1MgY2FjaGVzIGFuZCBwaXBl
IHVwY2FsbHMuDQo+IA0KPiBUaGlzIG1pZ2h0IGJlIGEgcHJvYmxlbSB3aGVuIHN3YXBwaW5nIGlu
IGEgdHJhbnNwb3J0IHRvIGEgZGVzdGluYXRpb24gc2VydmVyLCBmb3IgZXhhbXBsZS4NCg0KWW91
J3JlIGFscmVhZHkgc3dhcHBpbmcgdGhlIGNsX3hwcnQgYW5kIHRoZSBjbF9hdXRoLCB3aHkgc2hv
dWxkbid0IHlvdQ0KYWxzbyBzd2FwIHRoZSBjbF9wYXJlbnQgaW4gdGhpcyBzaXR1YXRpb24/DQoN
Cj4gSG93IHdvdWxkIHlvdSBsaWtlIHRvIHByb2NlZWQgd2l0aCBtZXJnaW5nIHRoZXNlPyAgSG9s
ZCBvZmYgdW50aWwgMy4xMywgdGFrZSB1cCB0byAxNS8zMyBub3cgYW5kIHRoZSByZXN0IGluIDMu
MTMsIG9yIHRha2UgdGhlIHdob2xlIHNlcmllcyBub3cgYW5kIGZpeCBpdCB1cCBhZnRlciB0aGUg
d2luZG93IGNsb3Nlcz8NCg0KSSBjYW4gdGFrZSB1cCB0byAxNS8zMyBub3csIGJ1dCBJJ2QgbGlr
ZSB0aG9zZSB3YXJuaW5ncyBmaXhlZC4gQ2FuIHlvdQ0KdGhyb3cgaW4gc29tZSAnZml4dXAhJyBj
b21taXRzIGZvciB0aGUgMiBpc3N1ZXMgdGhhdCB3ZXJlIHBvaW50ZWQgb3V0Pw0KDQotLSANClRy
b25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJv
bmQuTXlrbGVidXN0QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQo=

Re: [PATCH v2 16/33] SUNRPC: Modify synopsis of rpc_client_register()

[Chuck Lever]

On Sep 3, 2013, at 2:23 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:

> On Tue, 2013-09-03 at 14:16 -0400, Chuck Lever wrote:
>> On Sep 3, 2013, at 2:04 PM, "Myklebust, Trond" <Trond.Myklebust@netapp.com> wrote:
>>> In particular note that the new code uses the rpc_clnt->cl_parent chain
>>> in order to implement sharing of the RPCSEC_GSS caches and pipe upcalls.
>> 
>> This might be a problem when swapping in a transport to a destination server, for example.
> 
> You're already swapping the cl_xprt and the cl_auth, why shouldn't you
> also swap the cl_parent in this situation?

Sounds reasonable.


>> How would you like to proceed with merging these?  Hold off until 3.13, take up to 15/33 now and the rest in 3.13, or take the whole series now and fix it up after the window closes?
> 
> I can take up to 15/33 now, but I'd like those warnings fixed. Can you
> throw in some 'fixup!' commits for the 2 issues that were pointed out?

Having up to 15/33 merged will be very helpful, thanks.  I'll work up fixes that apply on 15/33, and post them later today or tomorrow (depending on how fast the little legs of my build machine can run).


-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com