All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v8 00/11] Fix OPEN/CLOSE races
@ 2017-11-06 20:28 Trond Myklebust
  2017-11-06 20:28 ` [PATCH v8 01/11] NFSv4: Fix OPEN / CLOSE race Trond Myklebust
  2017-11-06 22:46 ` [PATCH v8 00/11] Fix OPEN/CLOSE races Andrew W Elble
  0 siblings, 2 replies; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

v2:
- Fix a sleep-while-atomic issue
- Clean up.    
- Add a tracepoint to help document wait incidents.
v3:
- Fix a state update issue
v4:
- Fix a race due to setting the state->flags before waiting on another
  compound (can cause issues if the other compound has an OPEN_DOWNGRADE).
- Fix stateid seqid wraparound.
v5:
- Fix race with wakeup after clearing NFS_STATE_CHANGE_WAIT
- Fix CLOSE, DELEGRETURN and LAYOUTRETURN issues with
  NFS4ERR_OLD_STATEID, which were causing stateid leakage
- cleanups
- Fix a typo in nfs_rename...
v6:
- Fix a compile issue when CONFIG_NFS_V4_1=n
v7:
- Fix a typo in nfs4_refresh_open_stateid (thanks Andrew Elble)
- Minor cleanups in nfs4_close_done, nfs4_delegreturn_done,
  nfs4_layoutreturn_done.
v8:
- Really fix compile issue when CONFIG_NFS_V4_1=n
- nfs_inode_find_state_and_recover() should also try to match the
  open_stateid.

Trond Myklebust (11):
  NFSv4: Fix OPEN / CLOSE race
  NFSv4: Add a tracepoint to document open stateid updates
  NFSv4: Fix open create exclusive when the server reboots
  NFS: Fix a typo in nfs_rename()
  NFSv4: Retry CLOSE and DELEGRETURN on NFS4ERR_OLD_STATEID.
  NFSv4: Don't try to CLOSE if the stateid 'other' field has changed
  pNFS: Retry NFS4ERR_OLD_STATEID errors in layoutreturn-on-close
  NFSv4: Retry NFS4ERR_OLD_STATEID errors in layoutreturn
  NFSv4: cleanup nfs4_close_done
  NFSv4: Clean up nfs4_delegreturn_done
  NFSv4: Check the open stateid when searching for expired state

 fs/nfs/delegation.c |  27 +++++
 fs/nfs/delegation.h |   1 +
 fs/nfs/dir.c        |   2 +-
 fs/nfs/nfs4_fs.h    |   7 ++
 fs/nfs/nfs4proc.c   | 293 +++++++++++++++++++++++++++++++++++++---------------
 fs/nfs/nfs4state.c  |  31 +++++-
 fs/nfs/nfs4trace.h  |   2 +
 fs/nfs/pnfs.c       |  18 ++++
 fs/nfs/pnfs.h       |   6 ++
 9 files changed, 300 insertions(+), 87 deletions(-)

-- 
2.13.6

^ permalink raw reply	[flat|nested] 20+ messages in thread
* [PATCH v8 01/11] NFSv4: Fix OPEN / CLOSE race
  2017-11-06 20:28 [PATCH v8 00/11] Fix OPEN/CLOSE races Trond Myklebust
@ 2017-11-06 20:28 ` Trond Myklebust
  2017-11-06 20:28   ` [PATCH v8 02/11] NFSv4: Add a tracepoint to document open stateid updates Trond Myklebust
  2017-11-06 22:46 ` [PATCH v8 00/11] Fix OPEN/CLOSE races Andrew W Elble
  1 sibling, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

Ben Coddington has noted the following race between OPEN and CLOSE
on a single client.

Process 1		Process 2		Server
=========		=========		======

1)  OPEN file
2)			OPEN file
3)						Process OPEN (1) seqid=1
4)						Process OPEN (2) seqid=2
5)						Reply OPEN (2)
6)			Receive reply (2)
7)			new stateid, seqid=2

8)			CLOSE file, using
			stateid w/ seqid=2
9)						Reply OPEN (1)
10(						Process CLOSE (8)
11)						Reply CLOSE (8)
12)						Forget stateid
						file closed

13)			Receive reply (7)
14)			Forget stateid
			file closed.

15) Receive reply (1).
16) New stateid seqid=1
    is really the same
    stateid that was
    closed.

IOW: the reply to the first OPEN is delayed. Since "Process 2" does
not wait before closing the file, and it does not cache the closed
stateid, then when the delayed reply is finally received, it is treated
as setting up a new stateid by the client.

The fix is to ensure that the client processes the OPEN and CLOSE calls
in the same order in which the server processed them.

This commit ensures that we examine the seqid of the stateid
returned by OPEN. If it is a new stateid, we assume the seqid
must be equal to the value 1, and that each state transition
increments the seqid value by 1 (See RFC7530, Section 9.1.4.2,
and RFC5661, Section 8.2.2).

If the tracker sees that an OPEN returns with a seqid that is greater
than the cached seqid + 1, then it bumps a flag to ensure that the
caller waits for the RPCs carrying the missing seqids to complete.

Note that there can still be pathologies where the server crashes before
it can even send us the missing seqids. Since the OPEN call is still
holding a slot when it waits here, that could cause the recovery to
stall forever. To avoid that, we time out after a 5 second wait.

Reported-by: Benjamin Coddington <bcodding@redhat.com>
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4_fs.h   |   3 ++
 fs/nfs/nfs4proc.c  | 154 +++++++++++++++++++++++++++++++++++++++++------------
 fs/nfs/nfs4state.c |   1 +
 3 files changed, 123 insertions(+), 35 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index a73144b3cb8c..b07124ae5be1 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -162,6 +162,7 @@ enum {
 	NFS_STATE_POSIX_LOCKS,		/* Posix locks are supported */
 	NFS_STATE_RECOVERY_FAILED,	/* OPEN stateid state recovery failed */
 	NFS_STATE_MAY_NOTIFY_LOCK,	/* server may CB_NOTIFY_LOCK */
+	NFS_STATE_CHANGE_WAIT,		/* A state changing operation is outstanding */
 };
 
 struct nfs4_state {
@@ -185,6 +186,8 @@ struct nfs4_state {
 	unsigned int n_rdwr;		/* Number of read/write references */
 	fmode_t state;			/* State on the server (R,W, or RW) */
 	atomic_t count;
+
+	wait_queue_head_t waitq;
 };
 
 
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 96b2077e691d..c0d6b48f9e45 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1378,6 +1378,25 @@ static bool nfs_open_stateid_recover_openmode(struct nfs4_state *state)
 }
 #endif /* CONFIG_NFS_V4_1 */
 
+static void nfs_state_log_update_open_stateid(struct nfs4_state *state)
+{
+	if (test_and_clear_bit(NFS_STATE_CHANGE_WAIT, &state->flags))
+		wake_up_all(&state->waitq);
+}
+
+static void nfs_state_log_out_of_order_open_stateid(struct nfs4_state *state,
+		const nfs4_stateid *stateid)
+{
+	u32 state_seqid = be32_to_cpu(state->open_stateid.seqid);
+	u32 stateid_seqid = be32_to_cpu(stateid->seqid);
+
+	if (stateid_seqid == state_seqid + 1U ||
+	    (stateid_seqid == 1U && state_seqid == 0xffffffffU))
+		nfs_state_log_update_open_stateid(state);
+	else
+		set_bit(NFS_STATE_CHANGE_WAIT, &state->flags);
+}
+
 static void nfs_test_and_clear_all_open_stateid(struct nfs4_state *state)
 {
 	struct nfs_client *clp = state->owner->so_server->nfs_client;
@@ -1393,18 +1412,32 @@ static void nfs_test_and_clear_all_open_stateid(struct nfs4_state *state)
 		nfs4_state_mark_reclaim_nograce(clp, state);
 }
 
+/*
+ * Check for whether or not the caller may update the open stateid
+ * to the value passed in by stateid.
+ *
+ * Note: This function relies heavily on the server implementing
+ * RFC7530 Section 9.1.4.2, and RFC5661 Section 8.2.2
+ * correctly.
+ * i.e. The stateid seqids have to be initialised to 1, and
+ * are then incremented on every state transition.
+ */
 static bool nfs_need_update_open_stateid(struct nfs4_state *state,
-		const nfs4_stateid *stateid, nfs4_stateid *freeme)
+		const nfs4_stateid *stateid)
 {
-	if (test_and_set_bit(NFS_OPEN_STATE, &state->flags) == 0)
-		return true;
-	if (!nfs4_stateid_match_other(stateid, &state->open_stateid)) {
-		nfs4_stateid_copy(freeme, &state->open_stateid);
-		nfs_test_and_clear_all_open_stateid(state);
+	if (test_bit(NFS_OPEN_STATE, &state->flags) == 0 ||
+	    !nfs4_stateid_match_other(stateid, &state->open_stateid)) {
+		if (stateid->seqid == cpu_to_be32(1))
+			nfs_state_log_update_open_stateid(state);
+		else
+			set_bit(NFS_STATE_CHANGE_WAIT, &state->flags);
 		return true;
 	}
-	if (nfs4_stateid_is_newer(stateid, &state->open_stateid))
+
+	if (nfs4_stateid_is_newer(stateid, &state->open_stateid)) {
+		nfs_state_log_out_of_order_open_stateid(state, stateid);
 		return true;
+	}
 	return false;
 }
 
@@ -1443,11 +1476,13 @@ static void nfs_clear_open_stateid_locked(struct nfs4_state *state,
 	if (nfs4_stateid_match_other(stateid, &state->open_stateid) &&
 	    !nfs4_stateid_is_newer(stateid, &state->open_stateid)) {
 		nfs_resync_open_stateid_locked(state);
-		return;
+		goto out;
 	}
 	if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
 		nfs4_stateid_copy(&state->stateid, stateid);
 	nfs4_stateid_copy(&state->open_stateid, stateid);
+out:
+	nfs_state_log_update_open_stateid(state);
 }
 
 static void nfs_clear_open_stateid(struct nfs4_state *state,
@@ -1464,29 +1499,57 @@ static void nfs_clear_open_stateid(struct nfs4_state *state,
 }
 
 static void nfs_set_open_stateid_locked(struct nfs4_state *state,
-		const nfs4_stateid *stateid, fmode_t fmode,
-		nfs4_stateid *freeme)
+		const nfs4_stateid *stateid, nfs4_stateid *freeme)
 {
-	switch (fmode) {
-		case FMODE_READ:
-			set_bit(NFS_O_RDONLY_STATE, &state->flags);
+	DEFINE_WAIT(wait);
+	int status = 0;
+	for (;;) {
+
+		if (!nfs_need_update_open_stateid(state, stateid))
+			return;
+		if (!test_bit(NFS_STATE_CHANGE_WAIT, &state->flags))
 			break;
-		case FMODE_WRITE:
-			set_bit(NFS_O_WRONLY_STATE, &state->flags);
+		if (status)
 			break;
-		case FMODE_READ|FMODE_WRITE:
-			set_bit(NFS_O_RDWR_STATE, &state->flags);
+		/* Rely on seqids for serialisation with NFSv4.0 */
+		if (!nfs4_has_session(NFS_SERVER(state->inode)->nfs_client))
+			break;
+
+		prepare_to_wait(&state->waitq, &wait, TASK_KILLABLE);
+		/*
+		 * Ensure we process the state changes in the same order
+		 * in which the server processed them by delaying the
+		 * update of the stateid until we are in sequence.
+		 */
+		write_sequnlock(&state->seqlock);
+		spin_unlock(&state->owner->so_lock);
+		rcu_read_unlock();
+		if (!signal_pending(current)) {
+			if (schedule_timeout(5*HZ) == 0)
+				status = -EAGAIN;
+			else
+				status = 0;
+		} else
+			status = -EINTR;
+		finish_wait(&state->waitq, &wait);
+		rcu_read_lock();
+		spin_lock(&state->owner->so_lock);
+		write_seqlock(&state->seqlock);
 	}
-	if (!nfs_need_update_open_stateid(state, stateid, freeme))
-		return;
+
+	if (!nfs4_stateid_match_other(stateid, &state->open_stateid)) {
+		nfs4_stateid_copy(freeme, &state->open_stateid);
+		nfs_test_and_clear_all_open_stateid(state);
+	}
+
 	if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
 		nfs4_stateid_copy(&state->stateid, stateid);
 	nfs4_stateid_copy(&state->open_stateid, stateid);
+	nfs_state_log_update_open_stateid(state);
 }
 
-static void __update_open_stateid(struct nfs4_state *state,
+static void nfs_state_set_open_stateid(struct nfs4_state *state,
 		const nfs4_stateid *open_stateid,
-		const nfs4_stateid *deleg_stateid,
 		fmode_t fmode,
 		nfs4_stateid *freeme)
 {
@@ -1494,17 +1557,34 @@ static void __update_open_stateid(struct nfs4_state *state,
 	 * Protect the call to nfs4_state_set_mode_locked and
 	 * serialise the stateid update
 	 */
-	spin_lock(&state->owner->so_lock);
 	write_seqlock(&state->seqlock);
-	if (deleg_stateid != NULL) {
-		nfs4_stateid_copy(&state->stateid, deleg_stateid);
-		set_bit(NFS_DELEGATED_STATE, &state->flags);
+	nfs_set_open_stateid_locked(state, open_stateid, freeme);
+	switch (fmode) {
+	case FMODE_READ:
+		set_bit(NFS_O_RDONLY_STATE, &state->flags);
+		break;
+	case FMODE_WRITE:
+		set_bit(NFS_O_WRONLY_STATE, &state->flags);
+		break;
+	case FMODE_READ|FMODE_WRITE:
+		set_bit(NFS_O_RDWR_STATE, &state->flags);
 	}
-	if (open_stateid != NULL)
-		nfs_set_open_stateid_locked(state, open_stateid, fmode, freeme);
+	set_bit(NFS_OPEN_STATE, &state->flags);
+	write_sequnlock(&state->seqlock);
+}
+
+static void nfs_state_set_delegation(struct nfs4_state *state,
+		const nfs4_stateid *deleg_stateid,
+		fmode_t fmode)
+{
+	/*
+	 * Protect the call to nfs4_state_set_mode_locked and
+	 * serialise the stateid update
+	 */
+	write_seqlock(&state->seqlock);
+	nfs4_stateid_copy(&state->stateid, deleg_stateid);
+	set_bit(NFS_DELEGATED_STATE, &state->flags);
 	write_sequnlock(&state->seqlock);
-	update_open_stateflags(state, fmode);
-	spin_unlock(&state->owner->so_lock);
 }
 
 static int update_open_stateid(struct nfs4_state *state,
@@ -1522,6 +1602,12 @@ static int update_open_stateid(struct nfs4_state *state,
 	fmode &= (FMODE_READ|FMODE_WRITE);
 
 	rcu_read_lock();
+	spin_lock(&state->owner->so_lock);
+	if (open_stateid != NULL) {
+		nfs_state_set_open_stateid(state, open_stateid, fmode, &freeme);
+		ret = 1;
+	}
+
 	deleg_cur = rcu_dereference(nfsi->delegation);
 	if (deleg_cur == NULL)
 		goto no_delegation;
@@ -1538,18 +1624,16 @@ static int update_open_stateid(struct nfs4_state *state,
 		goto no_delegation_unlock;
 
 	nfs_mark_delegation_referenced(deleg_cur);
-	__update_open_stateid(state, open_stateid, &deleg_cur->stateid,
-			fmode, &freeme);
+	nfs_state_set_delegation(state, &deleg_cur->stateid, fmode);
 	ret = 1;
 no_delegation_unlock:
 	spin_unlock(&deleg_cur->lock);
 no_delegation:
+	if (ret)
+		update_open_stateflags(state, fmode);
+	spin_unlock(&state->owner->so_lock);
 	rcu_read_unlock();
 
-	if (!ret && open_stateid != NULL) {
-		__update_open_stateid(state, open_stateid, NULL, fmode, &freeme);
-		ret = 1;
-	}
 	if (test_bit(NFS_STATE_RECLAIM_NOGRACE, &state->flags))
 		nfs4_schedule_state_manager(clp);
 	if (freeme.type != 0)
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 0378e2257ca7..d615b7cdfa8f 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -645,6 +645,7 @@ nfs4_alloc_open_state(void)
 	INIT_LIST_HEAD(&state->lock_states);
 	spin_lock_init(&state->state_lock);
 	seqlock_init(&state->seqlock);
+	init_waitqueue_head(&state->waitq);
 	return state;
 }
 
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 02/11] NFSv4: Add a tracepoint to document open stateid updates
  2017-11-06 20:28 ` [PATCH v8 01/11] NFSv4: Fix OPEN / CLOSE race Trond Myklebust
@ 2017-11-06 20:28   ` Trond Myklebust
  2017-11-06 20:28     ` [PATCH v8 03/11] NFSv4: Fix open create exclusive when the server reboots Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c  | 3 +++
 fs/nfs/nfs4trace.h | 2 ++
 2 files changed, 5 insertions(+)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index c0d6b48f9e45..f44eef8c92fe 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1481,6 +1481,7 @@ static void nfs_clear_open_stateid_locked(struct nfs4_state *state,
 	if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
 		nfs4_stateid_copy(&state->stateid, stateid);
 	nfs4_stateid_copy(&state->open_stateid, stateid);
+	trace_nfs4_open_stateid_update(state->inode, stateid, 0);
 out:
 	nfs_state_log_update_open_stateid(state);
 }
@@ -1524,6 +1525,7 @@ static void nfs_set_open_stateid_locked(struct nfs4_state *state,
 		write_sequnlock(&state->seqlock);
 		spin_unlock(&state->owner->so_lock);
 		rcu_read_unlock();
+		trace_nfs4_open_stateid_update_wait(state->inode, stateid, 0);
 		if (!signal_pending(current)) {
 			if (schedule_timeout(5*HZ) == 0)
 				status = -EAGAIN;
@@ -1545,6 +1547,7 @@ static void nfs_set_open_stateid_locked(struct nfs4_state *state,
 	if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
 		nfs4_stateid_copy(&state->stateid, stateid);
 	nfs4_stateid_copy(&state->open_stateid, stateid);
+	trace_nfs4_open_stateid_update(state->inode, stateid, status);
 	nfs_state_log_update_open_stateid(state);
 }
 
diff --git a/fs/nfs/nfs4trace.h b/fs/nfs/nfs4trace.h
index e7c6275519b0..bf92cfd627fe 100644
--- a/fs/nfs/nfs4trace.h
+++ b/fs/nfs/nfs4trace.h
@@ -1066,6 +1066,8 @@ DECLARE_EVENT_CLASS(nfs4_inode_stateid_event,
 
 DEFINE_NFS4_INODE_STATEID_EVENT(nfs4_setattr);
 DEFINE_NFS4_INODE_STATEID_EVENT(nfs4_delegreturn);
+DEFINE_NFS4_INODE_STATEID_EVENT(nfs4_open_stateid_update);
+DEFINE_NFS4_INODE_STATEID_EVENT(nfs4_open_stateid_update_wait);
 
 DECLARE_EVENT_CLASS(nfs4_getattr_event,
 		TP_PROTO(
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 03/11] NFSv4: Fix open create exclusive when the server reboots
  2017-11-06 20:28   ` [PATCH v8 02/11] NFSv4: Add a tracepoint to document open stateid updates Trond Myklebust
@ 2017-11-06 20:28     ` Trond Myklebust
  2017-11-06 20:28       ` [PATCH v8 04/11] NFS: Fix a typo in nfs_rename() Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

If the server that does not implement NFSv4.1 persistent session
semantics reboots while we are performing an exclusive create,
then the return value of NFS4ERR_DELAY when we replay the open
during the grace period causes us to lose the verifier.
When the grace period expires, and we present a new verifier,
the server will then correctly reply NFS4ERR_EXIST.

This commit ensures that we always present the same verifier when
replaying the OPEN.

Reported-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c | 41 ++++++++++++++++++++++++++---------------
 1 file changed, 26 insertions(+), 15 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index f44eef8c92fe..3309ee35e73c 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1088,6 +1088,12 @@ struct nfs4_opendata {
 	int rpc_status;
 };
 
+struct nfs4_open_createattrs {
+	struct nfs4_label *label;
+	struct iattr *sattr;
+	const __u32 verf[2];
+};
+
 static bool nfs4_clear_cap_atomic_open_v1(struct nfs_server *server,
 		int err, struct nfs4_exception *exception)
 {
@@ -1157,8 +1163,7 @@ static void nfs4_init_opendata_res(struct nfs4_opendata *p)
 
 static struct nfs4_opendata *nfs4_opendata_alloc(struct dentry *dentry,
 		struct nfs4_state_owner *sp, fmode_t fmode, int flags,
-		const struct iattr *attrs,
-		struct nfs4_label *label,
+		const struct nfs4_open_createattrs *c,
 		enum open_claim_type4 claim,
 		gfp_t gfp_mask)
 {
@@ -1166,6 +1171,7 @@ static struct nfs4_opendata *nfs4_opendata_alloc(struct dentry *dentry,
 	struct inode *dir = d_inode(parent);
 	struct nfs_server *server = NFS_SERVER(dir);
 	struct nfs_seqid *(*alloc_seqid)(struct nfs_seqid_counter *, gfp_t);
+	struct nfs4_label *label = (c != NULL) ? c->label : NULL;
 	struct nfs4_opendata *p;
 
 	p = kzalloc(sizeof(*p), gfp_mask);
@@ -1231,15 +1237,11 @@ static struct nfs4_opendata *nfs4_opendata_alloc(struct dentry *dentry,
 	case NFS4_OPEN_CLAIM_DELEG_PREV_FH:
 		p->o_arg.fh = NFS_FH(d_inode(dentry));
 	}
-	if (attrs != NULL && attrs->ia_valid != 0) {
-		__u32 verf[2];
-
+	if (c != NULL && c->sattr != NULL && c->sattr->ia_valid != 0) {
 		p->o_arg.u.attrs = &p->attrs;
-		memcpy(&p->attrs, attrs, sizeof(p->attrs));
+		memcpy(&p->attrs, c->sattr, sizeof(p->attrs));
 
-		verf[0] = jiffies;
-		verf[1] = current->pid;
-		memcpy(p->o_arg.u.verifier.data, verf,
+		memcpy(p->o_arg.u.verifier.data, c->verf,
 				sizeof(p->o_arg.u.verifier.data));
 	}
 	p->c_arg.fh = &p->o_res.fh;
@@ -1892,7 +1894,7 @@ static struct nfs4_opendata *nfs4_open_recoverdata_alloc(struct nfs_open_context
 	struct nfs4_opendata *opendata;
 
 	opendata = nfs4_opendata_alloc(ctx->dentry, state->owner, 0, 0,
-			NULL, NULL, claim, GFP_NOFS);
+			NULL, claim, GFP_NOFS);
 	if (opendata == NULL)
 		return ERR_PTR(-ENOMEM);
 	opendata->state = state;
@@ -2823,8 +2825,7 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 static int _nfs4_do_open(struct inode *dir,
 			struct nfs_open_context *ctx,
 			int flags,
-			struct iattr *sattr,
-			struct nfs4_label *label,
+			const struct nfs4_open_createattrs *c,
 			int *opened)
 {
 	struct nfs4_state_owner  *sp;
@@ -2836,6 +2837,8 @@ static int _nfs4_do_open(struct inode *dir,
 	struct nfs4_threshold **ctx_th = &ctx->mdsthreshold;
 	fmode_t fmode = ctx->mode & (FMODE_READ|FMODE_WRITE|FMODE_EXEC);
 	enum open_claim_type4 claim = NFS4_OPEN_CLAIM_NULL;
+	struct iattr *sattr = c->sattr;
+	struct nfs4_label *label = c->label;
 	struct nfs4_label *olabel = NULL;
 	int status;
 
@@ -2854,8 +2857,8 @@ static int _nfs4_do_open(struct inode *dir,
 	status = -ENOMEM;
 	if (d_really_is_positive(dentry))
 		claim = NFS4_OPEN_CLAIM_FH;
-	opendata = nfs4_opendata_alloc(dentry, sp, fmode, flags, sattr,
-			label, claim, GFP_KERNEL);
+	opendata = nfs4_opendata_alloc(dentry, sp, fmode, flags,
+			c, claim, GFP_KERNEL);
 	if (opendata == NULL)
 		goto err_put_state_owner;
 
@@ -2936,10 +2939,18 @@ static struct nfs4_state *nfs4_do_open(struct inode *dir,
 	struct nfs_server *server = NFS_SERVER(dir);
 	struct nfs4_exception exception = { };
 	struct nfs4_state *res;
+	struct nfs4_open_createattrs c = {
+		.label = label,
+		.sattr = sattr,
+		.verf = {
+			[0] = (__u32)jiffies,
+			[1] = (__u32)current->pid,
+		},
+	};
 	int status;
 
 	do {
-		status = _nfs4_do_open(dir, ctx, flags, sattr, label, opened);
+		status = _nfs4_do_open(dir, ctx, flags, &c, opened);
 		res = ctx->state;
 		trace_nfs4_open_file(ctx, flags, status);
 		if (status == 0)
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 04/11] NFS: Fix a typo in nfs_rename()
  2017-11-06 20:28     ` [PATCH v8 03/11] NFSv4: Fix open create exclusive when the server reboots Trond Myklebust
@ 2017-11-06 20:28       ` Trond Myklebust
  2017-11-06 20:28         ` [PATCH v8 05/11] NFSv4: Retry CLOSE and DELEGRETURN on NFS4ERR_OLD_STATEID Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

On successful rename, the "old_dentry" is retained and is attached to
the "new_dir", so we need to call nfs_set_verifier() accordingly.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/dir.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 5ceaeb1f6fb6..9dda2e0ef85e 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -2064,7 +2064,7 @@ int nfs_rename(struct inode *old_dir, struct dentry *old_dentry,
 		 * should mark the directories for revalidation.
 		 */
 		d_move(old_dentry, new_dentry);
-		nfs_set_verifier(new_dentry,
+		nfs_set_verifier(old_dentry,
 					nfs_save_change_attribute(new_dir));
 	} else if (error == -ENOENT)
 		nfs_dentry_handle_enoent(old_dentry);
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 05/11] NFSv4: Retry CLOSE and DELEGRETURN on NFS4ERR_OLD_STATEID.
  2017-11-06 20:28       ` [PATCH v8 04/11] NFS: Fix a typo in nfs_rename() Trond Myklebust
@ 2017-11-06 20:28         ` Trond Myklebust
  2017-11-06 20:28           ` [PATCH v8 06/11] NFSv4: Don't try to CLOSE if the stateid 'other' field has changed Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

If we're racing with an OPEN, then retry the operation instead of
declaring it a success.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
[Andrew W Elble: Fix a typo in nfs4_refresh_open_stateid]
---
 fs/nfs/delegation.c | 27 +++++++++++++++++++++++++++
 fs/nfs/delegation.h |  1 +
 fs/nfs/nfs4_fs.h    |  2 ++
 fs/nfs/nfs4proc.c   | 21 +++++++++++++++++++--
 fs/nfs/nfs4state.c  | 16 ++++++++++++++++
 5 files changed, 65 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/delegation.c b/fs/nfs/delegation.c
index 606dd3871f66..ade44ca0c66c 100644
--- a/fs/nfs/delegation.c
+++ b/fs/nfs/delegation.c
@@ -1041,6 +1041,33 @@ int nfs_delegations_present(struct nfs_client *clp)
 }
 
 /**
+ * nfs4_refresh_delegation_stateid - Update delegation stateid seqid
+ * @dst: stateid to refresh
+ * @inode: inode to check
+ *
+ * Returns "true" and updates "dst->seqid" * if inode had a delegation
+ * that matches our delegation stateid. Otherwise "false" is returned.
+ */
+bool nfs4_refresh_delegation_stateid(nfs4_stateid *dst, struct inode *inode)
+{
+	struct nfs_delegation *delegation;
+	bool ret = false;
+	if (!inode)
+		goto out;
+
+	rcu_read_lock();
+	delegation = rcu_dereference(NFS_I(inode)->delegation);
+	if (delegation != NULL &&
+	    nfs4_stateid_match_other(dst, &delegation->stateid)) {
+		dst->seqid = delegation->stateid.seqid;
+		return ret;
+	}
+	rcu_read_unlock();
+out:
+	return ret;
+}
+
+/**
  * nfs4_copy_delegation_stateid - Copy inode's state ID information
  * @inode: inode to check
  * @flags: delegation type requirement
diff --git a/fs/nfs/delegation.h b/fs/nfs/delegation.h
index ddaf2644cf13..185a09f37a89 100644
--- a/fs/nfs/delegation.h
+++ b/fs/nfs/delegation.h
@@ -62,6 +62,7 @@ int nfs4_proc_delegreturn(struct inode *inode, struct rpc_cred *cred, const nfs4
 int nfs4_open_delegation_recall(struct nfs_open_context *ctx, struct nfs4_state *state, const nfs4_stateid *stateid, fmode_t type);
 int nfs4_lock_delegation_recall(struct file_lock *fl, struct nfs4_state *state, const nfs4_stateid *stateid);
 bool nfs4_copy_delegation_stateid(struct inode *inode, fmode_t flags, nfs4_stateid *dst, struct rpc_cred **cred);
+bool nfs4_refresh_delegation_stateid(nfs4_stateid *dst, struct inode *inode);
 
 void nfs_mark_delegation_referenced(struct nfs_delegation *delegation);
 int nfs4_have_delegation(struct inode *inode, fmode_t flags);
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index b07124ae5be1..84c3adc6bf96 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -461,6 +461,8 @@ extern int nfs4_set_lock_state(struct nfs4_state *state, struct file_lock *fl);
 extern int nfs4_select_rw_stateid(struct nfs4_state *, fmode_t,
 		const struct nfs_lock_context *, nfs4_stateid *,
 		struct rpc_cred **);
+extern bool nfs4_refresh_open_stateid(nfs4_stateid *dst,
+		struct nfs4_state *state);
 
 extern struct nfs_seqid *nfs_alloc_seqid(struct nfs_seqid_counter *counter, gfp_t gfp_mask);
 extern int nfs_wait_on_sequence(struct nfs_seqid *seqid, struct rpc_task *task);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 3309ee35e73c..8abf24907102 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3199,13 +3199,21 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 
 			}
 			break;
+		case -NFS4ERR_OLD_STATEID:
+			/* Did we race with OPEN? */
+			if (nfs4_refresh_open_stateid(&calldata->arg.stateid,
+						state)) {
+				task->tk_status = 0;
+				rpc_restart_call_prepare(task);
+			}
+			goto out_release;
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_STALE_STATEID:
 		case -NFS4ERR_EXPIRED:
 			nfs4_free_revoked_stateid(server,
 					&calldata->arg.stateid,
 					task->tk_msg.rpc_cred);
-		case -NFS4ERR_OLD_STATEID:
+			/* Fallthrough */
 		case -NFS4ERR_BAD_STATEID:
 			if (!nfs4_stateid_match(&calldata->arg.stateid,
 						&state->open_stateid)) {
@@ -3214,6 +3222,7 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 			}
 			if (calldata->arg.fmode == 0)
 				break;
+			/* Fallthrough */
 		default:
 			if (nfs4_async_handle_error(task, server, state, NULL) == -EAGAIN) {
 				rpc_restart_call_prepare(task);
@@ -5810,11 +5819,19 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 		nfs4_free_revoked_stateid(data->res.server,
 				data->args.stateid,
 				task->tk_msg.rpc_cred);
+		/* Fallthrough */
 	case -NFS4ERR_BAD_STATEID:
-	case -NFS4ERR_OLD_STATEID:
 	case -NFS4ERR_STALE_STATEID:
 		task->tk_status = 0;
 		break;
+	case -NFS4ERR_OLD_STATEID:
+		if (nfs4_refresh_delegation_stateid(&data->stateid, data->inode)) {
+			task->tk_status = 0;
+			rpc_restart_call_prepare(task);
+			return;
+		}
+		task->tk_status = 0;
+		break;
 	case -NFS4ERR_ACCESS:
 		if (data->args.bitmask) {
 			data->args.bitmask = NULL;
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index d615b7cdfa8f..ccf3b4e03997 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -986,6 +986,22 @@ static int nfs4_copy_lock_stateid(nfs4_stateid *dst,
 	return ret;
 }
 
+bool nfs4_refresh_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
+{
+	bool ret;
+	int seq;
+
+	do {
+		ret = false;
+		seq = read_seqbegin(&state->seqlock);
+		if (nfs4_state_match_open_stateid_other(state, dst)) {
+			dst->seqid = state->open_stateid.seqid;
+			ret = true;
+		}
+	} while (read_seqretry(&state->seqlock, seq));
+	return ret;
+}
+
 static void nfs4_copy_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
 {
 	const nfs4_stateid *src;
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 06/11] NFSv4: Don't try to CLOSE if the stateid 'other' field has changed
  2017-11-06 20:28         ` [PATCH v8 05/11] NFSv4: Retry CLOSE and DELEGRETURN on NFS4ERR_OLD_STATEID Trond Myklebust
@ 2017-11-06 20:28           ` Trond Myklebust
  2017-11-06 20:28             ` [PATCH v8 07/11] pNFS: Retry NFS4ERR_OLD_STATEID errors in layoutreturn-on-close Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

If the stateid is no longer recognised on the server, either due to a
restart, or due to a competing CLOSE call, then we do not have to
retry. Any open contexts that triggered a reopen of the file, will
also act as triggers for any CLOSE for the updated stateids.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4_fs.h   |  2 ++
 fs/nfs/nfs4proc.c  | 14 ++++----------
 fs/nfs/nfs4state.c |  9 +++++++--
 3 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 84c3adc6bf96..cdab6e8dbacc 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -463,6 +463,8 @@ extern int nfs4_select_rw_stateid(struct nfs4_state *, fmode_t,
 		struct rpc_cred **);
 extern bool nfs4_refresh_open_stateid(nfs4_stateid *dst,
 		struct nfs4_state *state);
+extern bool nfs4_copy_open_stateid(nfs4_stateid *dst,
+		struct nfs4_state *state);
 
 extern struct nfs_seqid *nfs_alloc_seqid(struct nfs_seqid_counter *counter, gfp_t gfp_mask);
 extern int nfs_wait_on_sequence(struct nfs_seqid *seqid, struct rpc_task *task);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 8abf24907102..25d310e33f55 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3215,14 +3215,7 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 					task->tk_msg.rpc_cred);
 			/* Fallthrough */
 		case -NFS4ERR_BAD_STATEID:
-			if (!nfs4_stateid_match(&calldata->arg.stateid,
-						&state->open_stateid)) {
-				rpc_restart_call_prepare(task);
-				goto out_release;
-			}
-			if (calldata->arg.fmode == 0)
-				break;
-			/* Fallthrough */
+			break;
 		default:
 			if (nfs4_async_handle_error(task, server, state, NULL) == -EAGAIN) {
 				rpc_restart_call_prepare(task);
@@ -3254,7 +3247,6 @@ static void nfs4_close_prepare(struct rpc_task *task, void *data)
 	is_rdwr = test_bit(NFS_O_RDWR_STATE, &state->flags);
 	is_rdonly = test_bit(NFS_O_RDONLY_STATE, &state->flags);
 	is_wronly = test_bit(NFS_O_WRONLY_STATE, &state->flags);
-	nfs4_stateid_copy(&calldata->arg.stateid, &state->open_stateid);
 	/* Calculate the change in open mode */
 	calldata->arg.fmode = 0;
 	if (state->n_rdwr == 0) {
@@ -3272,7 +3264,7 @@ static void nfs4_close_prepare(struct rpc_task *task, void *data)
 		calldata->arg.fmode |= FMODE_READ|FMODE_WRITE;
 
 	if (!nfs4_valid_open_stateid(state) ||
-	    test_bit(NFS_OPEN_STATE, &state->flags) == 0)
+	    !nfs4_refresh_open_stateid(&calldata->arg.stateid, state))
 		call_close = 0;
 	spin_unlock(&state->owner->so_lock);
 
@@ -3366,6 +3358,8 @@ int nfs4_do_close(struct nfs4_state *state, gfp_t gfp_mask, int wait)
 	calldata->inode = state->inode;
 	calldata->state = state;
 	calldata->arg.fh = NFS_FH(state->inode);
+	if (!nfs4_copy_open_stateid(&calldata->arg.stateid, state))
+		goto out_free_calldata;
 	/* Serialization for the sequence id */
 	alloc_seqid = server->nfs_client->cl_mvops->alloc_seqid;
 	calldata->arg.seqid = alloc_seqid(&state->owner->so_seqid, gfp_mask);
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index ccf3b4e03997..4a6f77880992 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1002,18 +1002,23 @@ bool nfs4_refresh_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
 	return ret;
 }
 
-static void nfs4_copy_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
+bool nfs4_copy_open_stateid(nfs4_stateid *dst, struct nfs4_state *state)
 {
+	bool ret;
 	const nfs4_stateid *src;
 	int seq;
 
 	do {
+		ret = false;
 		src = &zero_stateid;
 		seq = read_seqbegin(&state->seqlock);
-		if (test_bit(NFS_OPEN_STATE, &state->flags))
+		if (test_bit(NFS_OPEN_STATE, &state->flags)) {
 			src = &state->open_stateid;
+			ret = true;
+		}
 		nfs4_stateid_copy(dst, src);
 	} while (read_seqretry(&state->seqlock, seq));
+	return ret;
 }
 
 /*
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 07/11] pNFS: Retry NFS4ERR_OLD_STATEID errors in layoutreturn-on-close
  2017-11-06 20:28           ` [PATCH v8 06/11] NFSv4: Don't try to CLOSE if the stateid 'other' field has changed Trond Myklebust
@ 2017-11-06 20:28             ` Trond Myklebust
  2017-11-06 20:28               ` [PATCH v8 08/11] NFSv4: Retry NFS4ERR_OLD_STATEID errors in layoutreturn Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

If our layoutreturn on close operation returns an NFS4ERR_OLD_STATEID,
then try to update the stateid and retry. We know that there should
be no further LAYOUTGET requests being launched.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c | 18 ++++++++++++++++--
 fs/nfs/pnfs.c     | 18 ++++++++++++++++++
 fs/nfs/pnfs.h     |  6 ++++++
 3 files changed, 40 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 25d310e33f55..d91586fad2bd 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3166,11 +3166,18 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 			calldata->arg.lr_args = NULL;
 			calldata->res.lr_res = NULL;
 			break;
+		case -NFS4ERR_OLD_STATEID:
+			if (nfs4_refresh_layout_stateid(&calldata->arg.lr_args->stateid,
+						calldata->inode)) {
+				calldata->res.lr_ret = 0;
+				rpc_restart_call_prepare(task);
+				return;
+			}
+			/* Fallthrough */
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_DELEG_REVOKED:
 		case -NFS4ERR_EXPIRED:
 		case -NFS4ERR_BAD_STATEID:
-		case -NFS4ERR_OLD_STATEID:
 		case -NFS4ERR_UNKNOWN_LAYOUTTYPE:
 		case -NFS4ERR_WRONG_CRED:
 			calldata->arg.lr_args = NULL;
@@ -5788,11 +5795,18 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 			data->args.lr_args = NULL;
 			data->res.lr_res = NULL;
 			break;
+		case -NFS4ERR_OLD_STATEID:
+			if (nfs4_refresh_layout_stateid(&data->args.lr_args->stateid,
+						data->inode)) {
+				data->res.lr_ret = 0;
+				rpc_restart_call_prepare(task);
+				return;
+			}
+			/* Fallthrough */
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_DELEG_REVOKED:
 		case -NFS4ERR_EXPIRED:
 		case -NFS4ERR_BAD_STATEID:
-		case -NFS4ERR_OLD_STATEID:
 		case -NFS4ERR_UNKNOWN_LAYOUTTYPE:
 		case -NFS4ERR_WRONG_CRED:
 			data->args.lr_args = NULL;
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 3bcd669a3152..5e4cd6a7af66 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -355,6 +355,24 @@ pnfs_clear_lseg_state(struct pnfs_layout_segment *lseg,
 }
 
 /*
+ * Update the seqid of a layout stateid
+ */
+bool nfs4_refresh_layout_stateid(nfs4_stateid *dst, struct inode *inode)
+{
+	struct pnfs_layout_hdr *lo;
+	bool ret = false;
+
+	spin_lock(&inode->i_lock);
+	lo = NFS_I(inode)->layout;
+	if (lo && nfs4_stateid_match_other(dst, &lo->plh_stateid)) {
+		dst->seqid = lo->plh_stateid.seqid;
+		ret = true;
+	}
+	spin_unlock(&inode->i_lock);
+	return ret;
+}
+
+/*
  * Mark a pnfs_layout_hdr and all associated layout segments as invalid
  *
  * In order to continue using the pnfs_layout_hdr, a full recovery
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 87f144f14d1e..d467fc9f17a4 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -251,6 +251,7 @@ int pnfs_destroy_layouts_byfsid(struct nfs_client *clp,
 		bool is_recall);
 int pnfs_destroy_layouts_byclid(struct nfs_client *clp,
 		bool is_recall);
+bool nfs4_refresh_layout_stateid(nfs4_stateid *dst, struct inode *inode);
 void pnfs_put_layout_hdr(struct pnfs_layout_hdr *lo);
 void pnfs_set_layout_stateid(struct pnfs_layout_hdr *lo,
 			     const nfs4_stateid *new,
@@ -764,6 +765,11 @@ static inline void nfs4_pnfs_v3_ds_connect_unload(void)
 {
 }
 
+static inline bool nfs4_refresh_layout_stateid(nfs4_stateid *dst,
+		struct inode *inode)
+{
+	return false;
+}
 #endif /* CONFIG_NFS_V4_1 */
 
 #if IS_ENABLED(CONFIG_NFS_V4_2)
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 08/11] NFSv4: Retry NFS4ERR_OLD_STATEID errors in layoutreturn
  2017-11-06 20:28             ` [PATCH v8 07/11] pNFS: Retry NFS4ERR_OLD_STATEID errors in layoutreturn-on-close Trond Myklebust
@ 2017-11-06 20:28               ` Trond Myklebust
  2017-11-06 20:28                 ` [PATCH v8 09/11] NFSv4: cleanup nfs4_close_done Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

If our layoutreturn returns an NFS4ERR_OLD_STATEID, then try to
update the stateid and retry.

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index d91586fad2bd..0d6941d548f8 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -8767,18 +8767,27 @@ static void nfs4_layoutreturn_done(struct rpc_task *task, void *calldata)
 
 	server = NFS_SERVER(lrp->args.inode);
 	switch (task->tk_status) {
+	case -NFS4ERR_OLD_STATEID:
+		if (nfs4_refresh_layout_stateid(&lrp->args.stateid,
+					lrp->args.inode))
+			goto out_restart;
+		/* Fallthrough */
 	default:
 		task->tk_status = 0;
+		/* Fallthrough */
 	case 0:
 		break;
 	case -NFS4ERR_DELAY:
 		if (nfs4_async_handle_error(task, server, NULL, NULL) != -EAGAIN)
 			break;
-		nfs4_sequence_free_slot(&lrp->res.seq_res);
-		rpc_restart_call_prepare(task);
-		return;
+		goto out_restart;
 	}
 	dprintk("<-- %s\n", __func__);
+	return;
+out_restart:
+	task->tk_status = 0;
+	nfs4_sequence_free_slot(&lrp->res.seq_res);
+	rpc_restart_call_prepare(task);
 }
 
 static void nfs4_layoutreturn_release(void *calldata)
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 09/11] NFSv4: cleanup nfs4_close_done
  2017-11-06 20:28               ` [PATCH v8 08/11] NFSv4: Retry NFS4ERR_OLD_STATEID errors in layoutreturn Trond Myklebust
@ 2017-11-06 20:28                 ` Trond Myklebust
  2017-11-06 20:28                   ` [PATCH v8 10/11] NFSv4: Clean up nfs4_delegreturn_done Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c | 35 ++++++++++++++++-------------------
 1 file changed, 16 insertions(+), 19 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0d6941d548f8..1f16cd17b437 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -3168,11 +3168,8 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 			break;
 		case -NFS4ERR_OLD_STATEID:
 			if (nfs4_refresh_layout_stateid(&calldata->arg.lr_args->stateid,
-						calldata->inode)) {
-				calldata->res.lr_ret = 0;
-				rpc_restart_call_prepare(task);
-				return;
-			}
+						calldata->inode))
+				goto lr_restart;
 			/* Fallthrough */
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_DELEG_REVOKED:
@@ -3182,9 +3179,7 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 		case -NFS4ERR_WRONG_CRED:
 			calldata->arg.lr_args = NULL;
 			calldata->res.lr_res = NULL;
-			calldata->res.lr_ret = 0;
-			rpc_restart_call_prepare(task);
-			return;
+			goto lr_restart;
 		}
 	}
 
@@ -3200,19 +3195,15 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 			if (calldata->arg.bitmask != NULL) {
 				calldata->arg.bitmask = NULL;
 				calldata->res.fattr = NULL;
-				task->tk_status = 0;
-				rpc_restart_call_prepare(task);
-				goto out_release;
+				goto out_restart;
 
 			}
 			break;
 		case -NFS4ERR_OLD_STATEID:
 			/* Did we race with OPEN? */
 			if (nfs4_refresh_open_stateid(&calldata->arg.stateid,
-						state)) {
-				task->tk_status = 0;
-				rpc_restart_call_prepare(task);
-			}
+						state))
+				goto out_restart;
 			goto out_release;
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_STALE_STATEID:
@@ -3224,17 +3215,23 @@ static void nfs4_close_done(struct rpc_task *task, void *data)
 		case -NFS4ERR_BAD_STATEID:
 			break;
 		default:
-			if (nfs4_async_handle_error(task, server, state, NULL) == -EAGAIN) {
-				rpc_restart_call_prepare(task);
-				goto out_release;
-			}
+			if (nfs4_async_handle_error(task, server, state, NULL) == -EAGAIN)
+				goto out_restart;
 	}
 	nfs_clear_open_stateid(state, &calldata->arg.stateid,
 			res_stateid, calldata->arg.fmode);
 out_release:
+	task->tk_status = 0;
 	nfs_release_seqid(calldata->arg.seqid);
 	nfs_refresh_inode(calldata->inode, &calldata->fattr);
 	dprintk("%s: done, ret = %d!\n", __func__, task->tk_status);
+	return;
+lr_restart:
+	calldata->res.lr_ret = 0;
+out_restart:
+	task->tk_status = 0;
+	rpc_restart_call_prepare(task);
+	goto out_release;
 }
 
 static void nfs4_close_prepare(struct rpc_task *task, void *data)
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 10/11] NFSv4: Clean up nfs4_delegreturn_done
  2017-11-06 20:28                 ` [PATCH v8 09/11] NFSv4: cleanup nfs4_close_done Trond Myklebust
@ 2017-11-06 20:28                   ` Trond Myklebust
  2017-11-06 20:28                     ` [PATCH v8 11/11] NFSv4: Check the open stateid when searching for expired state Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4proc.c | 32 ++++++++++++++------------------
 1 file changed, 14 insertions(+), 18 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 1f16cd17b437..f773806a3442 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5794,11 +5794,8 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 			break;
 		case -NFS4ERR_OLD_STATEID:
 			if (nfs4_refresh_layout_stateid(&data->args.lr_args->stateid,
-						data->inode)) {
-				data->res.lr_ret = 0;
-				rpc_restart_call_prepare(task);
-				return;
-			}
+						data->inode))
+				goto lr_restart;
 			/* Fallthrough */
 		case -NFS4ERR_ADMIN_REVOKED:
 		case -NFS4ERR_DELEG_REVOKED:
@@ -5808,9 +5805,7 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 		case -NFS4ERR_WRONG_CRED:
 			data->args.lr_args = NULL;
 			data->res.lr_res = NULL;
-			data->res.lr_ret = 0;
-			rpc_restart_call_prepare(task);
-			return;
+			goto lr_restart;
 		}
 	}
 
@@ -5830,29 +5825,30 @@ static void nfs4_delegreturn_done(struct rpc_task *task, void *calldata)
 		task->tk_status = 0;
 		break;
 	case -NFS4ERR_OLD_STATEID:
-		if (nfs4_refresh_delegation_stateid(&data->stateid, data->inode)) {
-			task->tk_status = 0;
-			rpc_restart_call_prepare(task);
-			return;
-		}
+		if (nfs4_refresh_delegation_stateid(&data->stateid, data->inode))
+			goto out_restart;
 		task->tk_status = 0;
 		break;
 	case -NFS4ERR_ACCESS:
 		if (data->args.bitmask) {
 			data->args.bitmask = NULL;
 			data->res.fattr = NULL;
-			task->tk_status = 0;
-			rpc_restart_call_prepare(task);
-			return;
+			goto out_restart;
 		}
+		/* Fallthrough */
 	default:
 		if (nfs4_async_handle_error(task, data->res.server,
 					    NULL, NULL) == -EAGAIN) {
-			rpc_restart_call_prepare(task);
-			return;
+			goto out_restart;
 		}
 	}
 	data->rpc_status = task->tk_status;
+	return;
+lr_restart:
+	data->res.lr_ret = 0;
+out_restart:
+	task->tk_status = 0;
+	rpc_restart_call_prepare(task);
 }
 
 static void nfs4_delegreturn_release(void *calldata)
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* [PATCH v8 11/11] NFSv4: Check the open stateid when searching for expired state
  2017-11-06 20:28                   ` [PATCH v8 10/11] NFSv4: Clean up nfs4_delegreturn_done Trond Myklebust
@ 2017-11-06 20:28                     ` Trond Myklebust
  0 siblings, 0 replies; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 20:28 UTC (permalink / raw)
  To: Benjamin Coddington, Anna Schumaker; +Cc: linux-nfs

Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>
---
 fs/nfs/nfs4state.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index 4a6f77880992..4415eda2ed35 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -1431,6 +1431,11 @@ void nfs_inode_find_state_and_recover(struct inode *inode,
 			found = true;
 			continue;
 		}
+		if (nfs4_stateid_match_other(&state->open_stateid, stateid) &&
+		    nfs4_state_mark_reclaim_nograce(clp, state)) {
+			found = true;
+			continue;
+		}
 		if (nfs_state_lock_state_matches_stateid(state, stateid) &&
 		    nfs4_state_mark_reclaim_nograce(clp, state))
 			found = true;
-- 
2.13.6

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-06 20:28 [PATCH v8 00/11] Fix OPEN/CLOSE races Trond Myklebust
  2017-11-06 20:28 ` [PATCH v8 01/11] NFSv4: Fix OPEN / CLOSE race Trond Myklebust
@ 2017-11-06 22:46 ` Andrew W Elble
  2017-11-06 22:50   ` Trond Myklebust
  1 sibling, 1 reply; 20+ messages in thread
From: Andrew W Elble @ 2017-11-06 22:46 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: Benjamin Coddington, Anna Schumaker, linux-nfs

Trond Myklebust <trond.myklebust@primarydata.com> writes:

> v8:
> - Really fix compile issue when CONFIG_NFS_V4_1=n
> - nfs_inode_find_state_and_recover() should also try to match the
>   open_stateid.

Seeing a lot of TEST_STATEID's for the invalid stateid go over the
wire - this also makes the server hit this quite a bit:

pr_warn_ratelimited("NFSD: client %s testing state ID "
                        "with incorrect client ID\n", addr_str);

Thanks,

Andy

-- 
Andrew W. Elble
aweits@discipline.rit.edu
Infrastructure Engineer, Communications Technical Lead
Rochester Institute of Technology
PGP: BFAD 8461 4CCF DC95 DA2C B0EB 965B 082E 863E C912

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-06 22:46 ` [PATCH v8 00/11] Fix OPEN/CLOSE races Andrew W Elble
@ 2017-11-06 22:50   ` Trond Myklebust
  2017-11-06 23:13     ` Andrew W Elble
  2017-11-07 16:59     ` Andrew W Elble
  0 siblings, 2 replies; 20+ messages in thread
From: Trond Myklebust @ 2017-11-06 22:50 UTC (permalink / raw)
  To: aweits; +Cc: bcodding redhat, anna.schumaker, linux-nfs

T24gTW9uLCAyMDE3LTExLTA2IGF0IDE3OjQ2IC0wNTAwLCBBbmRyZXcgVyBFbGJsZSB3cm90ZToN
Cj4gVHJvbmQgTXlrbGVidXN0IDx0cm9uZC5teWtsZWJ1c3RAcHJpbWFyeWRhdGEuY29tPiB3cml0
ZXM6DQo+IA0KPiA+IHY4Og0KPiA+IC0gUmVhbGx5IGZpeCBjb21waWxlIGlzc3VlIHdoZW4gQ09O
RklHX05GU19WNF8xPW4NCj4gPiAtIG5mc19pbm9kZV9maW5kX3N0YXRlX2FuZF9yZWNvdmVyKCkg
c2hvdWxkIGFsc28gdHJ5IHRvIG1hdGNoIHRoZQ0KPiA+ICAgb3Blbl9zdGF0ZWlkLg0KPiANCj4g
U2VlaW5nIGEgbG90IG9mIFRFU1RfU1RBVEVJRCdzIGZvciB0aGUgaW52YWxpZCBzdGF0ZWlkIGdv
IG92ZXIgdGhlDQo+IHdpcmUgLSB0aGlzIGFsc28gbWFrZXMgdGhlIHNlcnZlciBoaXQgdGhpcyBx
dWl0ZSBhIGJpdDoNCj4gDQo+IHByX3dhcm5fcmF0ZWxpbWl0ZWQoIk5GU0Q6IGNsaWVudCAlcyB0
ZXN0aW5nIHN0YXRlIElEICINCj4gICAgICAgICAgICAgICAgICAgICAgICAgIndpdGggaW5jb3Jy
ZWN0IGNsaWVudCBJRFxuIiwgYWRkcl9zdHIpOw0KPiANCg0KSSdtIG5vdCBzZWVpbmcgdGhhdCBh
dCBhbGwuIENhbiB5b3UgcGxlYXNlIGVsYWJvcmF0ZSBvbiB3aGljaCBzZXJ2ZXINCmVycm9ycyBh
cmUgdHJpZ2dlcmluZyB0aGlzPyBJJ2Qgbm90IGV4cGVjdCB0byBldmVyIHNlZSBURVNUX1NUQVRF
SUQgb24NCmEgbm9ybWFsIHJ1bi4NCg0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBj
bGllbnQgbWFpbnRhaW5lciwgUHJpbWFyeURhdGENCnRyb25kLm15a2xlYnVzdEBwcmltYXJ5ZGF0
YS5jb20NCg==

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-06 22:50   ` Trond Myklebust
@ 2017-11-06 23:13     ` Andrew W Elble
  2017-11-07 16:59     ` Andrew W Elble
  1 sibling, 0 replies; 20+ messages in thread
From: Andrew W Elble @ 2017-11-06 23:13 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: bcodding redhat, anna.schumaker, linux-nfs

Trond Myklebust <trondmy@primarydata.com> writes:

>> Seeing a lot of TEST_STATEID's for the invalid stateid go over the
>> wire - this also makes the server hit this quite a bit:
>> 
>> pr_warn_ratelimited("NFSD: client %s testing state ID "
>>                         "with incorrect client ID\n", addr_str);
>> 
>
> I'm not seeing that at all. Can you please elaborate on which server
> errors are triggering this? I'd not expect to ever see TEST_STATEID on
> a normal run.

BTW, this is mounting localhost on a test laptop, running blogbench in the
mounted directory "blogbench -d ."

https://www.pureftpd.org/project/blogbench

Working to gather more information clientside.

patch-explanation (at least for the server messages)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 0d98d73bd84e..e3c57f5af85e 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -73,6 +73,7 @@
 #define ZERO_STATEID(stateid) (!memcmp((stateid), &zero_stateid, sizeof(stateid_t)))
 #define ONE_STATEID(stateid)  (!memcmp((stateid), &one_stateid, sizeof(stateid_t)))
 #define CURRENT_STATEID(stateid) (!memcmp((stateid), &currentstateid, sizeof(stateid_t)))
+#define CLOSE_STATEID(stateid)  (!memcmp((stateid), &close_stateid, sizeof(stateid_t)))
 
 /* forward declarations */
 static bool check_for_locks(struct nfs4_file *fp, struct nfs4_lockowner *lowner);
@@ -4880,7 +4881,7 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
 	struct nfs4_stid *s;
 	__be32 status = nfserr_bad_stateid;
 
-	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
+	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid) || CLOSE_STATEID(stateid))
 		return status;
 	/* Client debugging aid. */
 	if (!same_clid(&stateid->si_opaque.so_clid, &cl->cl_clientid)) {
@@ -4938,7 +4939,7 @@ static __be32 nfsd4_validate_stateid(struct nfs4_client *cl, stateid_t *stateid)
 	else if (typemask & NFS4_DELEG_STID)
 		typemask |= NFS4_REVOKED_DELEG_STID;
 
-	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
+	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid) || CLOSE_STATEID(stateid))
 		return nfserr_bad_stateid;
 	status = lookup_clientid(&stateid->si_opaque.so_clid, cstate, nn);
 	if (status == nfserr_stale_clientid) {



-- 
Andrew W. Elble
aweits@discipline.rit.edu
Infrastructure Engineer, Communications Technical Lead
Rochester Institute of Technology
PGP: BFAD 8461 4CCF DC95 DA2C B0EB 965B 082E 863E C912

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-06 22:50   ` Trond Myklebust
  2017-11-06 23:13     ` Andrew W Elble
@ 2017-11-07 16:59     ` Andrew W Elble
  2017-11-07 18:35       ` Trond Myklebust
  1 sibling, 1 reply; 20+ messages in thread
From: Andrew W Elble @ 2017-11-07 16:59 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: bcodding redhat, anna.schumaker, linux-nfs

Trond Myklebust <trondmy@primarydata.com> writes:

> On Mon, 2017-11-06 at 17:46 -0500, Andrew W Elble wrote:
>> Trond Myklebust <trond.myklebust@primarydata.com> writes:
>> 
>> > v8:
>> > - Really fix compile issue when CONFIG_NFS_V4_1=n
>> > - nfs_inode_find_state_and_recover() should also try to match the
>> >   open_stateid.
>> 
>> Seeing a lot of TEST_STATEID's for the invalid stateid go over the
>> wire - this also makes the server hit this quite a bit:
>> 
>> pr_warn_ratelimited("NFSD: client %s testing state ID "
>>                         "with incorrect client ID\n", addr_str);
>> 
>
> I'm not seeing that at all. Can you please elaborate on which server
> errors are triggering this? I'd not expect to ever see TEST_STATEID on
> a normal run.

Hopefully this helps.

Not completely sure this is correct, but this makes the client no longer
test the _invalid_ stateid:

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 6eccbd1b98cc..7dc914a6356f 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1436,6 +1436,8 @@ static void nfs_clear_open_stateid_locked(struct nfs4_state *state,
                nfs_resync_open_stateid_locked(state);
                goto out;
        }
+       if (test_bit(NFS_OPEN_STATE, &state->flags) == 0)
+               stateid->type = NFS4_INVALID_STATEID_TYPE;
        if (test_bit(NFS_DELEGATED_STATE, &state->flags) == 0)
                nfs4_stateid_copy(&state->stateid, stateid);
        nfs4_stateid_copy(&state->open_stateid, stateid);


The TEST_STATEID's are being issued from the "freeme" path of
update_open_stateid() via nfs4_opendata_to_nfs4_state(). I put a tracepoint
there:

@@ -1597,10 +1599,11 @@ static int update_open_stateid(struct nfs4_state *state,
 
 	if (test_bit(NFS_STATE_RECLAIM_NOGRACE, &state->flags))
 		nfs4_schedule_state_manager(clp);
-	if (freeme.type != 0)
+	if (freeme.type != 0) {
 		nfs4_test_and_free_stateid(server, &freeme,
 				state->owner->so_cred);
-
+		trace_nfs4_test_and_free_stateid(state->inode, &freeme, 0);
+	}
 	return ret;
 }


Even with the topmost patch in place, TEST_STATEID's still do occur with non-invalid stateids.
Here is some trace data of a stateid that had such an event:

       blogbench-2271  [001] ....    24.776203: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=1:0x65b62c64
       blogbench-2271  [001] ....    24.776233: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=1:0x65b62c64 openstateid=1:0x65b62c64
       blogbench-2198  [002] .N..    24.777914: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=2:0x65b62c64
       blogbench-2198  [002] ....    24.778016: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=2:0x65b62c64 openstateid=2:0x65b62c64
     kworker/0:3-587   [000] ....    24.778132: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=1:0x65b62c64
     kworker/1:2-113   [001] ....    24.779489: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/2:2-330   [002] ....    24.779516: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/0:3-587   [000] ....    24.779730: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/3:2-165   [003] ....    24.779763: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/2:2-330   [002] ....    24.779862: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/1:2-113   [001] ....    24.779991: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/0:3-587   [000] ....    24.780049: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
       blogbench-2254  [003] ....    24.780421: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=3:0x65b62c64
       blogbench-2254  [003] ....    24.780425: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=3:0x65b62c64 openstateid=3:0x65b62c64
       blogbench-2195  [003] ....    24.780473: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=4:0x65b62c64
       blogbench-2195  [003] ....    24.780478: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=4:0x65b62c64 openstateid=4:0x65b62c64
       blogbench-2219  [003] ....    24.780576: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=5:0x65b62c64
       blogbench-2219  [003] ....    24.780581: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=5:0x65b62c64 openstateid=5:0x65b62c64
       blogbench-2223  [003] ....    24.780608: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=6:0x65b62c64
       blogbench-2223  [003] ....    24.780612: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=6:0x65b62c64 openstateid=6:0x65b62c64
       blogbench-2256  [003] ....    24.780637: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=7:0x65b62c64
       blogbench-2256  [003] ....    24.780642: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=7:0x65b62c64 openstateid=7:0x65b62c64
       blogbench-2206  [003] ....    24.780676: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=8:0x65b62c64
       blogbench-2206  [003] ....    24.780680: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=8:0x65b62c64 openstateid=8:0x65b62c64
     kworker/0:3-587   [000] ....    24.780698: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
       blogbench-2211  [003] ....    24.780716: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=9:0x65b62c64
       blogbench-2211  [003] ....    24.780721: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=9:0x65b62c64 openstateid=9:0x65b62c64
       blogbench-2242  [003] ....    24.780753: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=10:0x65b62c64
       blogbench-2242  [003] ....    24.780758: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=10:0x65b62c64 openstateid=10:0x65b62c64
       blogbench-2253  [003] ....    24.780789: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=11:0x65b62c64
       blogbench-2253  [003] ....    24.780794: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=11:0x65b62c64 openstateid=11:0x65b62c64
       blogbench-2262  [003] ....    24.780857: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=12:0x65b62c64
       blogbench-2262  [003] ....    24.780862: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=12:0x65b62c64 openstateid=12:0x65b62c64
       blogbench-2281  [003] ....    24.780894: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=13:0x65b62c64
       blogbench-2281  [003] ....    24.780899: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=13:0x65b62c64 openstateid=13:0x65b62c64
       blogbench-2287  [003] ....    24.780930: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=14:0x65b62c64
       blogbench-2287  [003] ....    24.780935: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=14:0x65b62c64 openstateid=14:0x65b62c64
     kworker/1:2-113   [001] ....    24.780944: nfs4_close: error=-10024 (OLD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=4:0x65b62c64
       blogbench-2291  [003] ....    24.780975: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=15:0x65b62c64
       blogbench-2291  [003] ....    24.780979: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:73739313 fhandle=0x8bf9d249 name=00:2e:70677670/article-6.xml stateid=15:0x65b62c64 openstateid=15:0x65b62c64
     kworker/0:3-587   [000] ....    24.780989: nfs4_close: error=-10024 (OLD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=2:0x65b62c64
     kworker/0:3-587   [000] ....    24.781122: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=3:0x65b62c64
     kworker/2:2-330   [002] ....    24.781250: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=5:0x65b62c64
     kworker/2:2-330   [002] ....    24.781261: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=6:0x65b62c64
     kworker/2:2-330   [002] ....    24.781269: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=7:0x65b62c64
     kworker/3:2-165   [003] ....    24.781409: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=8:0x65b62c64
     kworker/3:2-165   [003] ....    24.781423: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=8:0x65b62c64
     kworker/1:2-113   [001] ....    24.781514: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=10:0x65b62c64
     kworker/1:2-113   [001] ....    24.781625: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=9:0x65b62c64
     kworker/1:2-113   [001] ....    24.781639: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=12:0x65b62c64
     kworker/3:2-165   [003] ....    24.782038: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=11:0x65b62c64
     kworker/1:2-113   [001] ....    24.782054: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782054: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=13:0x65b62c64
     kworker/3:2-165   [003] ....    24.782069: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=14:0x65b62c64
     kworker/1:2-113   [001] ....    24.782071: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782082: nfs4_close: error=-10024 (OLD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=14:0x65b62c64
     kworker/3:2-165   [003] ....    24.782114: nfs4_close: error=0 (OK) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782128: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/2:2-330   [002] ....    24.782144: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782152: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782167: nfs4_close: error=-10025 (BAD_STATEID) fmode=closed fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/2:2-330   [002] ....    24.782423: nfs4_close: error=-10025 (BAD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782469: nfs4_close: error=-10025 (BAD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782484: nfs4_close: error=-10025 (BAD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/3:2-165   [003] ....    24.782498: nfs4_close: error=-10025 (BAD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
     kworker/1:2-113   [001] ....    24.782548: nfs4_close: error=-10025 (BAD_STATEID) fmode=READ fileid=00:2e:73739313 fhandle=0x8bf9d249 openstateid=15:0x65b62c64
       blogbench-2238  [001] ....    24.782669: nfs4_test_and_free_stateid: error=0 (OK) fileid=00:2e:73739313 fhandle=0x8bf9d249 stateid=15:0x65b62c64
       blogbench-2232  [002] ....    38.135974: nfs4_open_stateid_update: error=0 (OK) fileid=00:2e:210215186 fhandle=0x33c12cc7 stateid=1:0x765b624f
       blogbench-2232  [002] ....    38.135983: nfs4_open_file: error=0 (OK) flags=32768 (0x8000) fmode=READ|0x801c fileid=00:2e:210215186 fhandle=0x33c12cc7 name=00:2e:210169499/picture-37.jpg stateid=1:0x765b624f openstateid=1:0x765b624f
     kworker/1:2-113   [001] ....    38.138993: nfs4_close: error=0 (OK) fmode=closed fileid=00:2e:210215186 fhandle=0x33c12cc7 openstateid=1:0x765b624f

Thanks,

Andy

-- 
Andrew W. Elble
aweits@discipline.rit.edu
Infrastructure Engineer, Communications Technical Lead
Rochester Institute of Technology
PGP: BFAD 8461 4CCF DC95 DA2C B0EB 965B 082E 863E C912

^ permalink raw reply related	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-07 16:59     ` Andrew W Elble
@ 2017-11-07 18:35       ` Trond Myklebust
  2017-11-07 19:00         ` Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-07 18:35 UTC (permalink / raw)
  To: aweits; +Cc: bcodding redhat, anna.schumaker, linux-nfs

T24gVHVlLCAyMDE3LTExLTA3IGF0IDExOjU5IC0wNTAwLCBBbmRyZXcgVyBFbGJsZSB3cm90ZToN
Cj4gVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkYXRhLmNvbT4gd3JpdGVzOg0KPiAN
Cj4gPiBPbiBNb24sIDIwMTctMTEtMDYgYXQgMTc6NDYgLTA1MDAsIEFuZHJldyBXIEVsYmxlIHdy
b3RlOg0KPiA+ID4gVHJvbmQgTXlrbGVidXN0IDx0cm9uZC5teWtsZWJ1c3RAcHJpbWFyeWRhdGEu
Y29tPiB3cml0ZXM6DQo+ID4gPiANCj4gPiA+ID4gdjg6DQo+ID4gPiA+IC0gUmVhbGx5IGZpeCBj
b21waWxlIGlzc3VlIHdoZW4gQ09ORklHX05GU19WNF8xPW4NCj4gPiA+ID4gLSBuZnNfaW5vZGVf
ZmluZF9zdGF0ZV9hbmRfcmVjb3ZlcigpIHNob3VsZCBhbHNvIHRyeSB0byBtYXRjaA0KPiA+ID4g
PiB0aGUNCj4gPiA+ID4gICBvcGVuX3N0YXRlaWQuDQo+ID4gPiANCj4gPiA+IFNlZWluZyBhIGxv
dCBvZiBURVNUX1NUQVRFSUQncyBmb3IgdGhlIGludmFsaWQgc3RhdGVpZCBnbyBvdmVyDQo+ID4g
PiB0aGUNCj4gPiA+IHdpcmUgLSB0aGlzIGFsc28gbWFrZXMgdGhlIHNlcnZlciBoaXQgdGhpcyBx
dWl0ZSBhIGJpdDoNCj4gPiA+IA0KPiA+ID4gcHJfd2Fybl9yYXRlbGltaXRlZCgiTkZTRDogY2xp
ZW50ICVzIHRlc3Rpbmcgc3RhdGUgSUQgIg0KPiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAg
IndpdGggaW5jb3JyZWN0IGNsaWVudCBJRFxuIiwgYWRkcl9zdHIpOw0KPiA+ID4gDQo+ID4gDQo+
ID4gSSdtIG5vdCBzZWVpbmcgdGhhdCBhdCBhbGwuIENhbiB5b3UgcGxlYXNlIGVsYWJvcmF0ZSBv
biB3aGljaA0KPiA+IHNlcnZlcg0KPiA+IGVycm9ycyBhcmUgdHJpZ2dlcmluZyB0aGlzPyBJJ2Qg
bm90IGV4cGVjdCB0byBldmVyIHNlZSBURVNUX1NUQVRFSUQNCj4gPiBvbg0KPiA+IGEgbm9ybWFs
IHJ1bi4NCj4gDQo+IEhvcGVmdWxseSB0aGlzIGhlbHBzLg0KPiANCj4gTm90IGNvbXBsZXRlbHkg
c3VyZSB0aGlzIGlzIGNvcnJlY3QsIGJ1dCB0aGlzIG1ha2VzIHRoZSBjbGllbnQgbm8NCj4gbG9u
Z2VyDQo+IHRlc3QgdGhlIF9pbnZhbGlkXyBzdGF0ZWlkOg0KPiANCj4gZGlmZiAtLWdpdCBhL2Zz
L25mcy9uZnM0cHJvYy5jIGIvZnMvbmZzL25mczRwcm9jLmMNCj4gaW5kZXggNmVjY2JkMWI5OGNj
Li43ZGM5MTRhNjM1NmYgMTAwNjQ0DQo+IC0tLSBhL2ZzL25mcy9uZnM0cHJvYy5jDQo+ICsrKyBi
L2ZzL25mcy9uZnM0cHJvYy5jDQo+IEBAIC0xNDM2LDYgKzE0MzYsOCBAQCBzdGF0aWMgdm9pZA0K
PiBuZnNfY2xlYXJfb3Blbl9zdGF0ZWlkX2xvY2tlZChzdHJ1Y3QgbmZzNF9zdGF0ZSAqc3RhdGUs
DQo+ICAgICAgICAgICAgICAgICBuZnNfcmVzeW5jX29wZW5fc3RhdGVpZF9sb2NrZWQoc3RhdGUp
Ow0KPiAgICAgICAgICAgICAgICAgZ290byBvdXQ7DQo+ICAgICAgICAgfQ0KPiArICAgICAgIGlm
ICh0ZXN0X2JpdChORlNfT1BFTl9TVEFURSwgJnN0YXRlLT5mbGFncykgPT0gMCkNCj4gKyAgICAg
ICAgICAgICAgIHN0YXRlaWQtPnR5cGUgPSBORlM0X0lOVkFMSURfU1RBVEVJRF9UWVBFOw0KPiAg
ICAgICAgIGlmICh0ZXN0X2JpdChORlNfREVMRUdBVEVEX1NUQVRFLCAmc3RhdGUtPmZsYWdzKSA9
PSAwKQ0KPiAgICAgICAgICAgICAgICAgbmZzNF9zdGF0ZWlkX2NvcHkoJnN0YXRlLT5zdGF0ZWlk
LCBzdGF0ZWlkKTsNCj4gICAgICAgICBuZnM0X3N0YXRlaWRfY29weSgmc3RhdGUtPm9wZW5fc3Rh
dGVpZCwgc3RhdGVpZCk7DQo+IA0KPiANCj4gVGhlIFRFU1RfU1RBVEVJRCdzIGFyZSBiZWluZyBp
c3N1ZWQgZnJvbSB0aGUgImZyZWVtZSIgcGF0aCBvZg0KPiB1cGRhdGVfb3Blbl9zdGF0ZWlkKCkg
dmlhIG5mczRfb3BlbmRhdGFfdG9fbmZzNF9zdGF0ZSgpLiBJIHB1dCBhDQo+IHRyYWNlcG9pbnQN
Cj4gdGhlcmU6DQoNCkFoLCBJIHNlZS4gQ2FuIHdlIGluIHRoYXQgY2FzZSBwbGVhc2UgcmF0aGVy
IGRvIHRoZSBmb2xsb3dpbmc/DQoNCjg8LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCkZyb20gZTNlZDIxYzE5NGY0MjA3Y2YzYTAwYjYyNGZl
ODZiOWE5YjE4OGMxMyBNb24gU2VwIDE3IDAwOjAwOjAwIDIwMDENCkZyb206IFRyb25kIE15a2xl
YnVzdCA8dHJvbmQubXlrbGVidXN0QHByaW1hcnlkYXRhLmNvbT4NCkRhdGU6IFR1ZSwgNyBOb3Yg
MjAxNyAxMzoxMDo0NiAtMDUwMA0KU3ViamVjdDogW1BBVENIIDEvMl0gTkZTdjQ6IG5mc19zZXRf
b3Blbl9zdGF0ZWlkIG11c3Qgbm90IHRyaWdnZXIgc3RhdGUNCiByZWNvdmVyeSBmb3IgY2xvc2Vk
IHN0YXRlDQoNCkluIG5mc19zZXRfb3Blbl9zdGF0ZWlkX2xvY2tlZCwgd2UgbXVzdCBpZ25vcmUg
c3RhdGVpZHMgZnJvbSBjbG9zZWQgc3RhdGUuDQoNClJlcG9ydGVkLWJ5OiBBbmRyZXcgVyBFbGJs
ZSA8YXdlaXRzQHJpdC5lZHU+DQpTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWtsZWJ1c3QgPHRyb25k
Lm15a2xlYnVzdEBwcmltYXJ5ZGF0YS5jb20+DQotLS0NCiBmcy9uZnMvbmZzNHByb2MuYyB8IDMg
KystDQogMSBmaWxlIGNoYW5nZWQsIDIgaW5zZXJ0aW9ucygrKSwgMSBkZWxldGlvbigtKQ0KDQpk
aWZmIC0tZ2l0IGEvZnMvbmZzL25mczRwcm9jLmMgYi9mcy9uZnMvbmZzNHByb2MuYw0KaW5kZXgg
ZDUzNmQ0ZWQ5OTBmLi4wYWE4NDYxZTczNDEgMTAwNjQ0DQotLS0gYS9mcy9uZnMvbmZzNHByb2Mu
Yw0KKysrIGIvZnMvbmZzL25mczRwcm9jLmMNCkBAIC0xNTQxLDcgKzE1NDEsOCBAQCBzdGF0aWMg
dm9pZCBuZnNfc2V0X29wZW5fc3RhdGVpZF9sb2NrZWQoc3RydWN0IG5mczRfc3RhdGUgKnN0YXRl
LA0KIAkJd3JpdGVfc2VxbG9jaygmc3RhdGUtPnNlcWxvY2spOw0KIAl9DQogDQotCWlmICghbmZz
NF9zdGF0ZWlkX21hdGNoX290aGVyKHN0YXRlaWQsICZzdGF0ZS0+b3Blbl9zdGF0ZWlkKSkgew0K
KwlpZiAodGVzdF9iaXQoTkZTX09QRU5fU1RBVEUsICZzdGF0ZS0+ZmxhZ3MpICYmDQorCSAgICAh
bmZzNF9zdGF0ZWlkX21hdGNoX290aGVyKHN0YXRlaWQsICZzdGF0ZS0+b3Blbl9zdGF0ZWlkKSkg
ew0KIAkJbmZzNF9zdGF0ZWlkX2NvcHkoZnJlZW1lLCAmc3RhdGUtPm9wZW5fc3RhdGVpZCk7DQog
CQluZnNfdGVzdF9hbmRfY2xlYXJfYWxsX29wZW5fc3RhdGVpZChzdGF0ZSk7DQogCX0NCi0tIA0K
Mi4xMy42DQoNCi0tIA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWlu
ZXIsIFByaW1hcnlEYXRhDQp0cm9uZC5teWtsZWJ1c3RAcHJpbWFyeWRhdGEuY29tDQo=

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-07 18:35       ` Trond Myklebust
@ 2017-11-07 19:00         ` Trond Myklebust
  2017-11-07 23:03           ` Andrew W Elble
  0 siblings, 1 reply; 20+ messages in thread
From: Trond Myklebust @ 2017-11-07 19:00 UTC (permalink / raw)
  To: aweits; +Cc: bcodding redhat, anna.schumaker, linux-nfs

T24gVHVlLCAyMDE3LTExLTA3IGF0IDE4OjM1ICswMDAwLCBUcm9uZCBNeWtsZWJ1c3Qgd3JvdGU6
DQo+IA0KPiBBaCwgSSBzZWUuIENhbiB3ZSBpbiB0aGF0IGNhc2UgcGxlYXNlIHJhdGhlciBkbyB0
aGUgZm9sbG93aW5nPw0KPiANCj4gDQoNCi4uLmFuZCB0aGVuIGZvbGxvdyB1cCB3aXRoOg0KDQo4
PC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tDQpGcm9tIDU0NWUyZmRkYjcyMjY4ZjVjNTlhMzk4NjE2OWFmOTZhNDBhNmFhYjAgTW9u
IFNlcCAxNyAwMDowMDowMCAyMDAxDQpGcm9tOiBUcm9uZCBNeWtsZWJ1c3QgPHRyb25kLm15a2xl
YnVzdEBwcmltYXJ5ZGF0YS5jb20+DQpEYXRlOiBUdWUsIDcgTm92IDIwMTcgMTI6Mzk6NDQgLTA1
MDANClN1YmplY3Q6IFtQQVRDSCAyLzJdIE5GU3Y0OiBSZXBsYWNlIGNsb3NlZCBzdGF0ZWlkcyB3
aXRoIHRoZSAiaW52YWxpZCBzcGVjaWFsDQogc3RhdGVpZCINCg0KV2hlbiBkZWNvZGluZyBhIENM
T1NFLCByZXBsYWNlIHRoZSBzdGF0ZWlkIHJldHVybmVkIGJ5IHRoZSBzZXJ2ZXINCndpdGggdGhl
ICJpbnZhbGlkIHNwZWNpYWwgc3RhdGVpZCIgZGVzY3JpYmVkIGluIFJGQzU2NjEsIFNlY3Rpb24g
OC4yLjMuDQoNCkluIG5mc19zZXRfb3Blbl9zdGF0ZWlkX2xvY2tlZCwgaWdub3JlIHN0YXRlaWRz
IGZyb20gY2xvc2VkIHN0YXRlLg0KDQpTaWduZWQtb2ZmLWJ5OiBUcm9uZCBNeWtsZWJ1c3QgPHRy
b25kLm15a2xlYnVzdEBwcmltYXJ5ZGF0YS5jb20+DQotLS0NCiBmcy9uZnMvbmZzNF9mcy5oICAg
fCAgMSArDQogZnMvbmZzL25mczRzdGF0ZS5jIHwgIDggKysrKysrKysNCiBmcy9uZnMvbmZzNHhk
ci5jICAgfCAxMiArKysrKysrKysrKy0NCiAzIGZpbGVzIGNoYW5nZWQsIDIwIGluc2VydGlvbnMo
KyksIDEgZGVsZXRpb24oLSkNCg0KZGlmZiAtLWdpdCBhL2ZzL25mcy9uZnM0X2ZzLmggYi9mcy9u
ZnMvbmZzNF9mcy5oDQppbmRleCBjZGFiNmU4ZGJhY2MuLjNlZGY4MGQ2ZDYzZSAxMDA2NDQNCi0t
LSBhL2ZzL25mcy9uZnM0X2ZzLmgNCisrKyBiL2ZzL25mcy9uZnM0X2ZzLmgNCkBAIC00ODIsNiAr
NDgyLDcgQEAgZXh0ZXJuIGludCBuZnM0X3NlcXVlbmNlX2RvbmUoc3RydWN0IHJwY190YXNrICp0
YXNrLA0KIGV4dGVybiB2b2lkIG5mczRfZnJlZV9sb2NrX3N0YXRlKHN0cnVjdCBuZnNfc2VydmVy
ICpzZXJ2ZXIsIHN0cnVjdCBuZnM0X2xvY2tfc3RhdGUgKmxzcCk7DQogDQogZXh0ZXJuIGNvbnN0
IG5mczRfc3RhdGVpZCB6ZXJvX3N0YXRlaWQ7DQorZXh0ZXJuIGNvbnN0IG5mczRfc3RhdGVpZCBp
bnZhbGlkX3N0YXRlaWQ7DQogDQogLyogbmZzNHN1cGVyLmMgKi8NCiBzdHJ1Y3QgbmZzX21vdW50
X2luZm87DQpkaWZmIC0tZ2l0IGEvZnMvbmZzL25mczRzdGF0ZS5jIGIvZnMvbmZzL25mczRzdGF0
ZS5jDQppbmRleCA0NDE1ZWRhMmVkMzUuLjk4MDQ2MmQ1NzdjYSAxMDA2NDQNCi0tLSBhL2ZzL25m
cy9uZnM0c3RhdGUuYw0KKysrIGIvZnMvbmZzL25mczRzdGF0ZS5jDQpAQCAtNjksNiArNjksMTQg
QEAgY29uc3QgbmZzNF9zdGF0ZWlkIHplcm9fc3RhdGVpZCA9IHsNCiAJeyAuZGF0YSA9IHsgMCB9
IH0sDQogCS50eXBlID0gTkZTNF9TUEVDSUFMX1NUQVRFSURfVFlQRSwNCiB9Ow0KK2NvbnN0IG5m
czRfc3RhdGVpZCBpbnZhbGlkX3N0YXRlaWQgPSB7DQorCXsNCisJCS5zZXFpZCA9IGNwdV90b19i
ZTMyKDB4ZmZmZmZmZmZVKSwNCisJCS5vdGhlciA9IHsgMCB9LA0KKwl9LA0KKwkudHlwZSA9IE5G
UzRfSU5WQUxJRF9TVEFURUlEX1RZUEUsDQorfTsNCisNCiBzdGF0aWMgREVGSU5FX01VVEVYKG5m
c19jbGlkX2luaXRfbXV0ZXgpOw0KIA0KIGludCBuZnM0X2luaXRfY2xpZW50aWQoc3RydWN0IG5m
c19jbGllbnQgKmNscCwgc3RydWN0IHJwY19jcmVkICpjcmVkKQ0KZGlmZiAtLWdpdCBhL2ZzL25m
cy9uZnM0eGRyLmMgYi9mcy9uZnMvbmZzNHhkci5jDQppbmRleCAxNGVkOTc5MWVjOWMuLjc3YzY3
MjllNTdmMCAxMDA2NDQNCi0tLSBhL2ZzL25mcy9uZnM0eGRyLmMNCisrKyBiL2ZzL25mcy9uZnM0
eGRyLmMNCkBAIC00Mzg1LDYgKzQzODUsMTQgQEAgc3RhdGljIGludCBkZWNvZGVfZGVsZWdhdGlv
bl9zdGF0ZWlkKHN0cnVjdCB4ZHJfc3RyZWFtICp4ZHIsIG5mczRfc3RhdGVpZCAqc3RhdGUNCiAJ
cmV0dXJuIGRlY29kZV9zdGF0ZWlkKHhkciwgc3RhdGVpZCk7DQogfQ0KIA0KK3N0YXRpYyBpbnQg
ZGVjb2RlX2ludmFsaWRfc3RhdGVpZChzdHJ1Y3QgeGRyX3N0cmVhbSAqeGRyLCBuZnM0X3N0YXRl
aWQgKnN0YXRlaWQpDQorew0KKwluZnM0X3N0YXRlaWQgZHVtbXk7DQorDQorCW5mczRfc3RhdGVp
ZF9jb3B5KHN0YXRlaWQsICZpbnZhbGlkX3N0YXRlaWQpOw0KKwlyZXR1cm4gZGVjb2RlX3N0YXRl
aWQoeGRyLCAmZHVtbXkpOw0KK30NCisNCiBzdGF0aWMgaW50IGRlY29kZV9jbG9zZShzdHJ1Y3Qg
eGRyX3N0cmVhbSAqeGRyLCBzdHJ1Y3QgbmZzX2Nsb3NlcmVzICpyZXMpDQogew0KIAlpbnQgc3Rh
dHVzOw0KQEAgLTQzOTMsNyArNDQwMSw3IEBAIHN0YXRpYyBpbnQgZGVjb2RlX2Nsb3NlKHN0cnVj
dCB4ZHJfc3RyZWFtICp4ZHIsIHN0cnVjdCBuZnNfY2xvc2VyZXMgKnJlcykNCiAJaWYgKHN0YXR1
cyAhPSAtRUlPKQ0KIAkJbmZzX2luY3JlbWVudF9vcGVuX3NlcWlkKHN0YXR1cywgcmVzLT5zZXFp
ZCk7DQogCWlmICghc3RhdHVzKQ0KLQkJc3RhdHVzID0gZGVjb2RlX29wZW5fc3RhdGVpZCh4ZHIs
ICZyZXMtPnN0YXRlaWQpOw0KKwkJc3RhdHVzID0gZGVjb2RlX2ludmFsaWRfc3RhdGVpZCh4ZHIs
ICZyZXMtPnN0YXRlaWQpOw0KIAlyZXR1cm4gc3RhdHVzOw0KIH0NCiANCkBAIC02MTA4LDYgKzYx
MTYsOCBAQCBzdGF0aWMgaW50IGRlY29kZV9sYXlvdXRyZXR1cm4oc3RydWN0IHhkcl9zdHJlYW0g
KnhkciwNCiAJcmVzLT5scnNfcHJlc2VudCA9IGJlMzJfdG9fY3B1cChwKTsNCiAJaWYgKHJlcy0+
bHJzX3ByZXNlbnQpDQogCQlzdGF0dXMgPSBkZWNvZGVfbGF5b3V0X3N0YXRlaWQoeGRyLCAmcmVz
LT5zdGF0ZWlkKTsNCisJZWxzZQ0KKwkJbmZzNF9zdGF0ZWlkX2NvcHkoJnJlcy0+c3RhdGVpZCwg
JmludmFsaWRfc3RhdGVpZCk7DQogCXJldHVybiBzdGF0dXM7DQogb3V0X292ZXJmbG93Og0KIAlw
cmludF9vdmVyZmxvd19tc2coX19mdW5jX18sIHhkcik7DQotLSANCjIuMTMuNg0KDQotLSANClRy
b25kIE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyLCBQcmltYXJ5RGF0YQ0K
dHJvbmQubXlrbGVidXN0QHByaW1hcnlkYXRhLmNvbQ0K

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-07 19:00         ` Trond Myklebust
@ 2017-11-07 23:03           ` Andrew W Elble
  2017-11-07 23:44             ` Trond Myklebust
  0 siblings, 1 reply; 20+ messages in thread
From: Andrew W Elble @ 2017-11-07 23:03 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: bcodding redhat, anna.schumaker, linux-nfs

Trond Myklebust <trondmy@primarydata.com> writes:

> On Tue, 2017-11-07 at 18:35 +0000, Trond Myklebust wrote:
>> 
>> Ah, I see. Can we in that case please rather do the following?
>> 
>> 
>
> ...and then follow up with:
>
> 8<--------------------------------------------------------------
> From 545e2fddb72268f5c59a3986169af96a40a6aab0 Mon Sep 17 00:00:00 2001
> From: Trond Myklebust <trond.myklebust@primarydata.com>
> Date: Tue, 7 Nov 2017 12:39:44 -0500
> Subject: [PATCH 2/2] NFSv4: Replace closed stateids with the "invalid special
>  stateid"
>
> When decoding a CLOSE, replace the stateid returned by the server
> with the "invalid special stateid" described in RFC5661, Section 8.2.3.
>
> In nfs_set_open_stateid_locked, ignore stateids from closed state.
>
> Signed-off-by: Trond Myklebust <trond.myklebust@primarydata.com>

Looks good! Thanks!

-- 
Andrew W. Elble
aweits@discipline.rit.edu
Infrastructure Engineer, Communications Technical Lead
Rochester Institute of Technology
PGP: BFAD 8461 4CCF DC95 DA2C B0EB 965B 082E 863E C912

^ permalink raw reply	[flat|nested] 20+ messages in thread
* Re: [PATCH v8 00/11] Fix OPEN/CLOSE races
  2017-11-07 23:03           ` Andrew W Elble
@ 2017-11-07 23:44             ` Trond Myklebust
  0 siblings, 0 replies; 20+ messages in thread
From: Trond Myklebust @ 2017-11-07 23:44 UTC (permalink / raw)
  To: aweits; +Cc: bcodding redhat, anna.schumaker, linux-nfs

T24gVHVlLCAyMDE3LTExLTA3IGF0IDE4OjAzIC0wNTAwLCBBbmRyZXcgVyBFbGJsZSB3cm90ZToN
Cj4gVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QHByaW1hcnlkYXRhLmNvbT4gd3JpdGVzOg0KPiAN
Cj4gPiBPbiBUdWUsIDIwMTctMTEtMDcgYXQgMTg6MzUgKzAwMDAsIFRyb25kIE15a2xlYnVzdCB3
cm90ZToNCj4gPiA+IA0KPiA+ID4gQWgsIEkgc2VlLiBDYW4gd2UgaW4gdGhhdCBjYXNlIHBsZWFz
ZSByYXRoZXIgZG8gdGhlIGZvbGxvd2luZz8NCj4gPiA+IA0KPiA+ID4gDQo+ID4gDQo+ID4gLi4u
YW5kIHRoZW4gZm9sbG93IHVwIHdpdGg6DQo+ID4gDQo+ID4gODwtLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0KPiA+IEZyb20gNTQ1
ZTJmZGRiNzIyNjhmNWM1OWEzOTg2MTY5YWY5NmE0MGE2YWFiMCBNb24gU2VwIDE3IDAwOjAwOjAw
DQo+ID4gMjAwMQ0KPiA+IEZyb206IFRyb25kIE15a2xlYnVzdCA8dHJvbmQubXlrbGVidXN0QHBy
aW1hcnlkYXRhLmNvbT4NCj4gPiBEYXRlOiBUdWUsIDcgTm92IDIwMTcgMTI6Mzk6NDQgLTA1MDAN
Cj4gPiBTdWJqZWN0OiBbUEFUQ0ggMi8yXSBORlN2NDogUmVwbGFjZSBjbG9zZWQgc3RhdGVpZHMg
d2l0aCB0aGUNCj4gPiAiaW52YWxpZCBzcGVjaWFsDQo+ID4gIHN0YXRlaWQiDQo+ID4gDQo+ID4g
V2hlbiBkZWNvZGluZyBhIENMT1NFLCByZXBsYWNlIHRoZSBzdGF0ZWlkIHJldHVybmVkIGJ5IHRo
ZSBzZXJ2ZXINCj4gPiB3aXRoIHRoZSAiaW52YWxpZCBzcGVjaWFsIHN0YXRlaWQiIGRlc2NyaWJl
ZCBpbiBSRkM1NjYxLCBTZWN0aW9uDQo+ID4gOC4yLjMuDQo+ID4gDQo+ID4gSW4gbmZzX3NldF9v
cGVuX3N0YXRlaWRfbG9ja2VkLCBpZ25vcmUgc3RhdGVpZHMgZnJvbSBjbG9zZWQgc3RhdGUuDQo+
ID4gDQo+ID4gU2lnbmVkLW9mZi1ieTogVHJvbmQgTXlrbGVidXN0IDx0cm9uZC5teWtsZWJ1c3RA
cHJpbWFyeWRhdGEuY29tPg0KPiANCj4gTG9va3MgZ29vZCEgVGhhbmtzIQ0KDQpUaGFuayB5b3Ug
Zm9yIHRlc3RpbmchIEl0IGhhcyBiZWVuIHZlcnkgaGVscGZ1bC4uLg0KDQotLSANClRyb25kIE15
a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyLCBQcmltYXJ5RGF0YQ0KdHJvbmQu
bXlrbGVidXN0QHByaW1hcnlkYXRhLmNvbQ0K

^ permalink raw reply	[flat|nested] 20+ messages in thread
end of thread, other threads:[~2017-11-07 23:44 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-06 20:28 [PATCH v8 00/11] Fix OPEN/CLOSE races Trond Myklebust
2017-11-06 20:28 ` [PATCH v8 01/11] NFSv4: Fix OPEN / CLOSE race Trond Myklebust
2017-11-06 20:28   ` [PATCH v8 02/11] NFSv4: Add a tracepoint to document open stateid updates Trond Myklebust
2017-11-06 20:28     ` [PATCH v8 03/11] NFSv4: Fix open create exclusive when the server reboots Trond Myklebust
2017-11-06 20:28       ` [PATCH v8 04/11] NFS: Fix a typo in nfs_rename() Trond Myklebust
2017-11-06 20:28         ` [PATCH v8 05/11] NFSv4: Retry CLOSE and DELEGRETURN on NFS4ERR_OLD_STATEID Trond Myklebust
2017-11-06 20:28           ` [PATCH v8 06/11] NFSv4: Don't try to CLOSE if the stateid 'other' field has changed Trond Myklebust
2017-11-06 20:28             ` [PATCH v8 07/11] pNFS: Retry NFS4ERR_OLD_STATEID errors in layoutreturn-on-close Trond Myklebust
2017-11-06 20:28               ` [PATCH v8 08/11] NFSv4: Retry NFS4ERR_OLD_STATEID errors in layoutreturn Trond Myklebust
2017-11-06 20:28                 ` [PATCH v8 09/11] NFSv4: cleanup nfs4_close_done Trond Myklebust
2017-11-06 20:28                   ` [PATCH v8 10/11] NFSv4: Clean up nfs4_delegreturn_done Trond Myklebust
2017-11-06 20:28                     ` [PATCH v8 11/11] NFSv4: Check the open stateid when searching for expired state Trond Myklebust
2017-11-06 22:46 ` [PATCH v8 00/11] Fix OPEN/CLOSE races Andrew W Elble
2017-11-06 22:50   ` Trond Myklebust
2017-11-06 23:13     ` Andrew W Elble
2017-11-07 16:59     ` Andrew W Elble
2017-11-07 18:35       ` Trond Myklebust
2017-11-07 19:00         ` Trond Myklebust
2017-11-07 23:03           ` Andrew W Elble
2017-11-07 23:44             ` Trond Myklebust

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.