[PATCH 00/19] Layoutget on OPEN

[PATCH 00/19] Layoutget on OPEN

[Trond Myklebust]

This patchset is something we've been sitting on for a couple of years
in order to allow it to mature. It was mainly developed by Fred Isaman
while he was working for Primary Data, but we've had a few bugfixes
come in since then.
The main reason for delaying the upstream merge was to ensure that
adding layoutget to the OPEN compound was safe (does not cause existing
layouts to be automatically revoked) and that it gives us a real
performance benefit.

Fred Isaman (14):
  pnfs: Remove redundant assignment from nfs4_proc_layoutget().
  pnfs: Store return value of decode_layoutget for later processing
  NFS4: move ctx into nfs4_run_open_task
  pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
  pnfs: refactor send_layoutget
  pnfs: move allocations out of nfs4_proc_layoutget
  pnfs: Add conditional encode/decode of LAYOUTGET within OPEN compound
  pnfs: Move nfs4_opendata into nfs4_fs.h
  pnfs: Change pnfs_alloc_init_layoutget_args call signature
  pnfs: Add LAYOUTGET to OPEN of a new file
  pnfs: Add LAYOUTGET to OPEN of an existing file
  pnfs: Stop attempting LAYOUTGET on OPEN on failure
  pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
  pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET

Trond Myklebust (5):
  NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
    uninitialised data
  NFSv4/pnfs: Don't switch off layoutget-on-open for transient errors
  pNFS: Don't send LAYOUTGET on OPEN for read, if we already have cached
    data
  pnfs: Don't call commit on failed layoutget-on-open
  pnfs: Don't release the sequence slot until we've processed layoutget
    on open

 fs/nfs/callback_proc.c                 |   2 +
 fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
 fs/nfs/nfs4_fs.h                       |  25 +++
 fs/nfs/nfs4proc.c                      |  97 +++++----
 fs/nfs/nfs4state.c                     |   8 +
 fs/nfs/nfs4xdr.c                       |  65 +++++-
 fs/nfs/pnfs.c                          | 266 +++++++++++++++++++++----
 fs/nfs/pnfs.h                          |  28 ++-
 include/linux/nfs_fs_sb.h              |   2 +
 include/linux/nfs_xdr.h                |   4 +
 10 files changed, 398 insertions(+), 100 deletions(-)

-- 
2.17.0

[PATCH 01/19] pnfs: Remove redundant assignment from nfs4_proc_layoutget().

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

nfs_init_sequence() will clear this for us.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 66664d814b3c..6287fb6dee50 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -8797,7 +8797,6 @@ nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout, gfp_t gfp_flags)
 	lgp->args.layout.pglen = max_pages * PAGE_SIZE;
 
 	lgp->res.layoutp = &lgp->args.layout;
-	lgp->res.seq_res.sr_slot = NULL;
 	nfs4_init_sequence(&lgp->args.seq_args, &lgp->res.seq_res, 0, 0);
 
 	task = rpc_run_task(&task_setup_data);
-- 
2.17.0

[PATCH 02/19] pnfs: Store return value of decode_layoutget for later processing

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

This will be needed to seperate return value of OPEN and LAYOUTGET
when they are combined into a single RPC.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4xdr.c        | 15 ++++++++++-----
 include/linux/nfs_xdr.h |  1 +
 2 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 9b7392032321..6024980dfc9e 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -6024,7 +6024,7 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 
 	status = decode_op_hdr(xdr, OP_LAYOUTGET);
 	if (status)
-		return status;
+		goto out;
 	p = xdr_inline_decode(xdr, 4);
 	if (unlikely(!p))
 		goto out_overflow;
@@ -6037,7 +6037,8 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 	if (!layout_count) {
 		dprintk("%s: server responded with empty layout array\n",
 			__func__);
-		return -EINVAL;
+		status = -EINVAL;
+		goto out;
 	}
 
 	p = xdr_inline_decode(xdr, 28);
@@ -6062,7 +6063,8 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 		dprintk("NFS: server cheating in layoutget reply: "
 				"layout len %u > recvd %u\n",
 				res->layoutp->len, recvd);
-		return -EINVAL;
+		status = -EINVAL;
+		goto out;
 	}
 
 	if (layout_count > 1) {
@@ -6075,10 +6077,13 @@ static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
 			__func__, layout_count);
 	}
 
-	return 0;
+out:
+	res->status = status;
+	return status;
 out_overflow:
 	print_overflow_msg(__func__, xdr);
-	return -EIO;
+	status = -EIO;
+	goto out;
 }
 
 static int decode_layoutreturn(struct xdr_stream *xdr,
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 52b481dfd61e..dee11a9e91e8 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -259,6 +259,7 @@ struct nfs4_layoutget_args {
 
 struct nfs4_layoutget_res {
 	struct nfs4_sequence_res seq_res;
+	int status;
 	__u32 return_on_close;
 	struct pnfs_layout_range range;
 	__u32 type;
-- 
2.17.0

[PATCH 03/19] NFS4: move ctx into nfs4_run_open_task

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Preparing to add conditional LAYOUTGET to OPEN rpc, the LAYOUTGET
will need the ctx info.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 6287fb6dee50..830569a8d00a 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -86,7 +86,6 @@
 	| ATTR_MTIME_SET)
 
 struct nfs4_opendata;
-static int _nfs4_proc_open(struct nfs4_opendata *data);
 static int _nfs4_recover_proc_open(struct nfs4_opendata *data);
 static int nfs4_do_fsinfo(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
 static void nfs_fixup_referral_attributes(struct nfs_fattr *fattr);
@@ -2328,7 +2327,8 @@ static const struct rpc_call_ops nfs4_open_ops = {
 	.rpc_release = nfs4_open_release,
 };
 
-static int nfs4_run_open_task(struct nfs4_opendata *data, int isrecover)
+static int nfs4_run_open_task(struct nfs4_opendata *data,
+			      struct nfs_open_context *ctx)
 {
 	struct inode *dir = d_inode(data->dir);
 	struct nfs_server *server = NFS_SERVER(dir);
@@ -2351,14 +2351,16 @@ static int nfs4_run_open_task(struct nfs4_opendata *data, int isrecover)
 	};
 	int status;
 
-	nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1, isrecover);
 	kref_get(&data->kref);
 	data->rpc_done = false;
 	data->rpc_status = 0;
 	data->cancelled = false;
 	data->is_recover = false;
-	if (isrecover)
+	if (!ctx) {
+		nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1, 1);
 		data->is_recover = true;
+	} else
+		nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1, 0);
 	task = rpc_run_task(&task_setup_data);
 	if (IS_ERR(task))
 		return PTR_ERR(task);
@@ -2379,7 +2381,7 @@ static int _nfs4_recover_proc_open(struct nfs4_opendata *data)
 	struct nfs_openres *o_res = &data->o_res;
 	int status;
 
-	status = nfs4_run_open_task(data, 1);
+	status = nfs4_run_open_task(data, NULL);
 	if (status != 0 || !data->rpc_done)
 		return status;
 
@@ -2440,7 +2442,8 @@ static int nfs4_opendata_access(struct rpc_cred *cred,
 /*
  * Note: On error, nfs4_proc_open will free the struct nfs4_opendata
  */
-static int _nfs4_proc_open(struct nfs4_opendata *data)
+static int _nfs4_proc_open(struct nfs4_opendata *data,
+			   struct nfs_open_context *ctx)
 {
 	struct inode *dir = d_inode(data->dir);
 	struct nfs_server *server = NFS_SERVER(dir);
@@ -2448,7 +2451,7 @@ static int _nfs4_proc_open(struct nfs4_opendata *data)
 	struct nfs_openres *o_res = &data->o_res;
 	int status;
 
-	status = nfs4_run_open_task(data, 0);
+	status = nfs4_run_open_task(data, ctx);
 	if (!data->rpc_done)
 		return status;
 	if (status != 0) {
@@ -2799,7 +2802,7 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 
 	seq = raw_seqcount_begin(&sp->so_reclaim_seqcount);
 
-	ret = _nfs4_proc_open(opendata);
+	ret = _nfs4_proc_open(opendata, ctx);
 	if (ret != 0)
 		goto out;
 
-- 
2.17.0

[PATCH 04/19] pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Driver can set flag to allow LAYOUTGET to be sent with OPEN.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/flexfilelayout/flexfilelayout.c | 1 +
 fs/nfs/pnfs.h                          | 1 +
 2 files changed, 2 insertions(+)

diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
index c75ad982bcfc..3ae038d9c292 100644
--- a/fs/nfs/flexfilelayout/flexfilelayout.c
+++ b/fs/nfs/flexfilelayout/flexfilelayout.c
@@ -2347,6 +2347,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = {
 	.id			= LAYOUT_FLEX_FILES,
 	.name			= "LAYOUT_FLEX_FILES",
 	.owner			= THIS_MODULE,
+	.flags			= PNFS_LAYOUTGET_ON_OPEN,
 	.set_layoutdriver	= ff_layout_set_layoutdriver,
 	.alloc_layout_hdr	= ff_layout_alloc_layout_hdr,
 	.free_layout_hdr	= ff_layout_free_layout_hdr,
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index daf6cbf5c15f..f71a55f11b97 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -110,6 +110,7 @@ enum layoutdriver_policy_flags {
 	PNFS_LAYOUTRET_ON_SETATTR	= 1 << 0,
 	PNFS_LAYOUTRET_ON_ERROR		= 1 << 1,
 	PNFS_READ_WHOLE_PAGE		= 1 << 2,
+	PNFS_LAYOUTGET_ON_OPEN		= 1 << 3,
 };
 
 struct nfs4_deviceid_node;
-- 
2.17.0

[PATCH 05/19] pnfs: refactor send_layoutget

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Pull out the alloc/init part for eventual reuse by OPEN.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index ee723aa153a3..2c955c389ecf 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -915,18 +915,12 @@ pnfs_layoutgets_blocked(const struct pnfs_layout_hdr *lo)
 		test_bit(NFS_LAYOUT_BULK_RECALL, &lo->plh_flags);
 }
 
-/*
- * Get layout from server.
- *    for now, assume that whole file layouts are requested.
- *    arg->offset: 0
- *    arg->length: all ones
- */
-static struct pnfs_layout_segment *
-send_layoutget(struct pnfs_layout_hdr *lo,
+static struct nfs4_layoutget *
+pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
 	   struct nfs_open_context *ctx,
 	   nfs4_stateid *stateid,
 	   const struct pnfs_layout_range *range,
-	   long *timeout, gfp_t gfp_flags)
+	   gfp_t gfp_flags)
 {
 	struct inode *ino = lo->plh_inode;
 	struct nfs_server *server = NFS_SERVER(ino);
@@ -935,14 +929,9 @@ send_layoutget(struct pnfs_layout_hdr *lo,
 
 	dprintk("--> %s\n", __func__);
 
-	/*
-	 * Synchronously retrieve layout information from server and
-	 * store in lseg. If we race with a concurrent seqid morphing
-	 * op, then re-send the LAYOUTGET.
-	 */
 	lgp = kzalloc(sizeof(*lgp), gfp_flags);
 	if (lgp == NULL)
-		return ERR_PTR(-ENOMEM);
+		return NULL;
 
 	i_size = i_size_read(ino);
 
@@ -963,8 +952,7 @@ send_layoutget(struct pnfs_layout_hdr *lo,
 	nfs4_stateid_copy(&lgp->args.stateid, stateid);
 	lgp->gfp_flags = gfp_flags;
 	lgp->cred = lo->plh_lc_cred;
-
-	return nfs4_proc_layoutget(lgp, timeout, gfp_flags);
+	return lgp;
 }
 
 static void pnfs_clear_layoutcommit(struct inode *inode,
@@ -1694,6 +1682,7 @@ pnfs_update_layout(struct inode *ino,
 	struct nfs_client *clp = server->nfs_client;
 	struct pnfs_layout_hdr *lo = NULL;
 	struct pnfs_layout_segment *lseg = NULL;
+	struct nfs4_layoutget *lgp;
 	nfs4_stateid stateid;
 	long timeout = 0;
 	unsigned long giveup = jiffies + (clp->cl_lease_time << 1);
@@ -1838,7 +1827,15 @@ pnfs_update_layout(struct inode *ino,
 	if (arg.length != NFS4_MAX_UINT64)
 		arg.length = PAGE_ALIGN(arg.length);
 
-	lseg = send_layoutget(lo, ctx, &stateid, &arg, &timeout, gfp_flags);
+	lgp = pnfs_alloc_init_layoutget_args(lo, ctx, &stateid, &arg, gfp_flags);
+	if (!lgp) {
+		trace_pnfs_update_layout(ino, pos, count, iomode, lo, NULL,
+					 PNFS_UPDATE_LAYOUT_NOMEM);
+		atomic_dec(&lo->plh_outstanding);
+		goto out_put_layout_hdr;
+	}
+
+	lseg = nfs4_proc_layoutget(lgp, &timeout, gfp_flags);
 	trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg,
 				 PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET);
 	atomic_dec(&lo->plh_outstanding);
-- 
2.17.0

[PATCH 06/19] pnfs: move allocations out of nfs4_proc_layoutget

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

They work better in the new alloc_init function.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 15 +++------------
 fs/nfs/pnfs.c     | 12 +++++++++++-
 fs/nfs/pnfs.h     |  4 +++-
 3 files changed, 17 insertions(+), 14 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 830569a8d00a..0eaab13dff04 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -8694,7 +8694,7 @@ nfs4_layoutget_handle_exception(struct rpc_task *task,
 	return status;
 }
 
-static size_t max_response_pages(struct nfs_server *server)
+size_t max_response_pages(struct nfs_server *server)
 {
 	u32 max_resp_sz = server->nfs_client->cl_session->fc_attrs.max_resp_sz;
 	return nfs_page_array_len(0, max_resp_sz);
@@ -8715,7 +8715,7 @@ static void nfs4_free_pages(struct page **pages, size_t size)
 	kfree(pages);
 }
 
-static struct page **nfs4_alloc_pages(size_t size, gfp_t gfp_flags)
+struct page **nfs4_alloc_pages(size_t size, gfp_t gfp_flags)
 {
 	struct page **pages;
 	int i;
@@ -8761,11 +8761,10 @@ static const struct rpc_call_ops nfs4_layoutget_call_ops = {
 };
 
 struct pnfs_layout_segment *
-nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout, gfp_t gfp_flags)
+nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout)
 {
 	struct inode *inode = lgp->args.inode;
 	struct nfs_server *server = NFS_SERVER(inode);
-	size_t max_pages = max_response_pages(server);
 	struct rpc_task *task;
 	struct rpc_message msg = {
 		.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_LAYOUTGET],
@@ -8792,14 +8791,6 @@ nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout, gfp_t gfp_flags)
 	/* nfs4_layoutget_release calls pnfs_put_layout_hdr */
 	pnfs_get_layout_hdr(NFS_I(inode)->layout);
 
-	lgp->args.layout.pages = nfs4_alloc_pages(max_pages, gfp_flags);
-	if (!lgp->args.layout.pages) {
-		nfs4_layoutget_release(lgp);
-		return ERR_PTR(-ENOMEM);
-	}
-	lgp->args.layout.pglen = max_pages * PAGE_SIZE;
-
-	lgp->res.layoutp = &lgp->args.layout;
 	nfs4_init_sequence(&lgp->args.seq_args, &lgp->res.seq_res, 0, 0);
 
 	task = rpc_run_task(&task_setup_data);
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 2c955c389ecf..363420d0f3a0 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -924,6 +924,7 @@ pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
 {
 	struct inode *ino = lo->plh_inode;
 	struct nfs_server *server = NFS_SERVER(ino);
+	size_t max_pages = max_response_pages(server);
 	struct nfs4_layoutget *lgp;
 	loff_t i_size;
 
@@ -933,6 +934,15 @@ pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
 	if (lgp == NULL)
 		return NULL;
 
+	lgp->args.layout.pages = nfs4_alloc_pages(max_pages, gfp_flags);
+	if (!lgp->args.layout.pages) {
+		kfree(lgp);
+		return NULL;
+	}
+	lgp->args.layout.pglen = max_pages * PAGE_SIZE;
+	lgp->res.layoutp = &lgp->args.layout;
+
+
 	i_size = i_size_read(ino);
 
 	lgp->args.minlength = PAGE_SIZE;
@@ -1835,7 +1845,7 @@ pnfs_update_layout(struct inode *ino,
 		goto out_put_layout_hdr;
 	}
 
-	lseg = nfs4_proc_layoutget(lgp, &timeout, gfp_flags);
+	lseg = nfs4_proc_layoutget(lgp, &timeout);
 	trace_pnfs_update_layout(ino, pos, count, iomode, lo, lseg,
 				 PNFS_UPDATE_LAYOUT_SEND_LAYOUTGET);
 	atomic_dec(&lo->plh_outstanding);
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index f71a55f11b97..964a7227ea97 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -224,10 +224,12 @@ extern int pnfs_register_layoutdriver(struct pnfs_layoutdriver_type *);
 extern void pnfs_unregister_layoutdriver(struct pnfs_layoutdriver_type *);
 
 /* nfs4proc.c */
+extern size_t max_response_pages(struct nfs_server *server);
+extern struct page **nfs4_alloc_pages(size_t size, gfp_t gfp_flags);
 extern int nfs4_proc_getdeviceinfo(struct nfs_server *server,
 				   struct pnfs_device *dev,
 				   struct rpc_cred *cred);
-extern struct pnfs_layout_segment* nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout, gfp_t gfp_flags);
+extern struct pnfs_layout_segment* nfs4_proc_layoutget(struct nfs4_layoutget *lgp, long *timeout);
 extern int nfs4_proc_layoutreturn(struct nfs4_layoutreturn *lrp, bool sync);
 
 /* pnfs.c */
-- 
2.17.0

[PATCH 07/19] pnfs: Add conditional encode/decode of LAYOUTGET within OPEN compound

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c       |  1 +
 fs/nfs/nfs4xdr.c        | 50 +++++++++++++++++++++++++++++++++++++----
 include/linux/nfs_xdr.h |  2 ++
 3 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0eaab13dff04..54f2987483fa 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1081,6 +1081,7 @@ struct nfs4_opendata {
 	struct nfs4_state_owner *owner;
 	struct nfs4_state *state;
 	struct iattr attrs;
+	struct nfs4_layoutget *lgp;
 	unsigned long timestamp;
 	bool rpc_done;
 	bool file_created;
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index 6024980dfc9e..738a7be019d2 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -65,7 +65,13 @@
 /* Mapping from NFS error code to "errno" error code. */
 #define errno_NFSERR_IO		EIO
 
+struct compound_hdr;
 static int nfs4_stat_to_errno(int);
+static void encode_layoutget(struct xdr_stream *xdr,
+			     const struct nfs4_layoutget_args *args,
+			     struct compound_hdr *hdr);
+static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
+			     struct nfs4_layoutget_res *res);
 
 /* NFSv4 COMPOUND tags are only wanted for debugging purposes */
 #ifdef DEBUG
@@ -424,6 +430,8 @@ static int nfs4_stat_to_errno(int);
 #define decode_sequence_maxsz	0
 #define encode_layoutreturn_maxsz 0
 #define decode_layoutreturn_maxsz 0
+#define encode_layoutget_maxsz	0
+#define decode_layoutget_maxsz	0
 #endif /* CONFIG_NFS_V4_1 */
 
 #define NFS4_enc_compound_sz	(1024)  /* XXX: large enough? */
@@ -476,14 +484,16 @@ static int nfs4_stat_to_errno(int);
 				encode_open_maxsz + \
 				encode_access_maxsz + \
 				encode_getfh_maxsz + \
-				encode_getattr_maxsz)
+				encode_getattr_maxsz + \
+				encode_layoutget_maxsz)
 #define NFS4_dec_open_sz        (compound_decode_hdr_maxsz + \
 				decode_sequence_maxsz + \
 				decode_putfh_maxsz + \
 				decode_open_maxsz + \
 				decode_access_maxsz + \
 				decode_getfh_maxsz + \
-				decode_getattr_maxsz)
+				decode_getattr_maxsz + \
+				decode_layoutget_maxsz)
 #define NFS4_enc_open_confirm_sz \
 				(compound_encode_hdr_maxsz + \
 				 encode_putfh_maxsz + \
@@ -497,13 +507,15 @@ static int nfs4_stat_to_errno(int);
 					encode_putfh_maxsz + \
 					encode_open_maxsz + \
 					encode_access_maxsz + \
-					encode_getattr_maxsz)
+					encode_getattr_maxsz + \
+					encode_layoutget_maxsz)
 #define NFS4_dec_open_noattr_sz	(compound_decode_hdr_maxsz + \
 					decode_sequence_maxsz + \
 					decode_putfh_maxsz + \
 					decode_open_maxsz + \
 					decode_access_maxsz + \
-					decode_getattr_maxsz)
+					decode_getattr_maxsz + \
+					decode_layoutget_maxsz)
 #define NFS4_enc_open_downgrade_sz \
 				(compound_encode_hdr_maxsz + \
 				 encode_sequence_maxsz + \
@@ -2070,6 +2082,13 @@ encode_layoutreturn(struct xdr_stream *xdr,
 		    struct compound_hdr *hdr)
 {
 }
+
+static void
+encode_layoutget(struct xdr_stream *xdr,
+		      const struct nfs4_layoutget_args *args,
+		      struct compound_hdr *hdr)
+{
+}
 #endif /* CONFIG_NFS_V4_1 */
 
 /*
@@ -2316,6 +2335,12 @@ static void nfs4_xdr_enc_open(struct rpc_rqst *req, struct xdr_stream *xdr,
 	if (args->access)
 		encode_access(xdr, args->access, &hdr);
 	encode_getfattr_open(xdr, args->bitmask, args->open_bitmap, &hdr);
+	if (args->lg_args) {
+		encode_layoutget(xdr, args->lg_args, &hdr);
+		xdr_inline_pages(&req->rq_rcv_buf, hdr.replen << 2,
+				 args->lg_args->layout.pages,
+				 0, args->lg_args->layout.pglen);
+	}
 	encode_nops(&hdr);
 }
 
@@ -2356,6 +2381,12 @@ static void nfs4_xdr_enc_open_noattr(struct rpc_rqst *req,
 	if (args->access)
 		encode_access(xdr, args->access, &hdr);
 	encode_getfattr_open(xdr, args->bitmask, args->open_bitmap, &hdr);
+	if (args->lg_args) {
+		encode_layoutget(xdr, args->lg_args, &hdr);
+		xdr_inline_pages(&req->rq_rcv_buf, hdr.replen << 2,
+				 args->lg_args->layout.pages,
+				 0, args->lg_args->layout.pglen);
+	}
 	encode_nops(&hdr);
 }
 
@@ -6182,6 +6213,13 @@ int decode_layoutreturn(struct xdr_stream *xdr,
 {
 	return 0;
 }
+
+static int decode_layoutget(struct xdr_stream *xdr, struct rpc_rqst *req,
+			    struct nfs4_layoutget_res *res)
+{
+	return 0;
+}
+
 #endif /* CONFIG_NFS_V4_1 */
 
 /*
@@ -6628,6 +6666,8 @@ static int nfs4_xdr_dec_open(struct rpc_rqst *rqstp, struct xdr_stream *xdr,
 	if (res->access_request)
 		decode_access(xdr, &res->access_supported, &res->access_result);
 	decode_getfattr_label(xdr, res->f_attr, res->f_label, res->server);
+	if (res->lg_res)
+		decode_layoutget(xdr, rqstp, res->lg_res);
 out:
 	return status;
 }
@@ -6680,6 +6720,8 @@ static int nfs4_xdr_dec_open_noattr(struct rpc_rqst *rqstp,
 	if (res->access_request)
 		decode_access(xdr, &res->access_supported, &res->access_result);
 	decode_getfattr(xdr, res->f_attr, res->server);
+	if (res->lg_res)
+		decode_layoutget(xdr, rqstp, res->lg_res);
 out:
 	return status;
 }
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index dee11a9e91e8..b36be7a703ea 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -436,6 +436,7 @@ struct nfs_openargs {
 	enum createmode4	createmode;
 	const struct nfs4_label *label;
 	umode_t			umask;
+	struct nfs4_layoutget_args *lg_args;
 };
 
 struct nfs_openres {
@@ -458,6 +459,7 @@ struct nfs_openres {
 	__u32			access_request;
 	__u32			access_supported;
 	__u32			access_result;
+	struct nfs4_layoutget_res *lg_res;
 };
 
 /*
-- 
2.17.0

[PATCH 08/19] pnfs: Move nfs4_opendata into nfs4_fs.h

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

It will be needed now by the pnfs code.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4_fs.h  | 25 +++++++++++++++++++++++++
 fs/nfs/nfs4proc.c | 25 -------------------------
 fs/nfs/pnfs.c     |  1 +
 3 files changed, 26 insertions(+), 25 deletions(-)

diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 06a41aa2fdb2..137e18abb7e7 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -212,6 +212,31 @@ struct nfs4_state_recovery_ops {
 		struct rpc_cred *);
 };
 
+struct nfs4_opendata {
+	struct kref kref;
+	struct nfs_openargs o_arg;
+	struct nfs_openres o_res;
+	struct nfs_open_confirmargs c_arg;
+	struct nfs_open_confirmres c_res;
+	struct nfs4_string owner_name;
+	struct nfs4_string group_name;
+	struct nfs4_label *a_label;
+	struct nfs_fattr f_attr;
+	struct nfs4_label *f_label;
+	struct dentry *dir;
+	struct dentry *dentry;
+	struct nfs4_state_owner *owner;
+	struct nfs4_state *state;
+	struct iattr attrs;
+	struct nfs4_layoutget *lgp;
+	unsigned long timestamp;
+	bool rpc_done;
+	bool file_created;
+	bool is_recover;
+	bool cancelled;
+	int rpc_status;
+};
+
 struct nfs4_add_xprt_data {
 	struct nfs_client	*clp;
 	struct rpc_cred		*cred;
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 54f2987483fa..771439971759 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1065,31 +1065,6 @@ static void update_changeattr(struct inode *dir, struct nfs4_change_info *cinfo,
 	spin_unlock(&dir->i_lock);
 }
 
-struct nfs4_opendata {
-	struct kref kref;
-	struct nfs_openargs o_arg;
-	struct nfs_openres o_res;
-	struct nfs_open_confirmargs c_arg;
-	struct nfs_open_confirmres c_res;
-	struct nfs4_string owner_name;
-	struct nfs4_string group_name;
-	struct nfs4_label *a_label;
-	struct nfs_fattr f_attr;
-	struct nfs4_label *f_label;
-	struct dentry *dir;
-	struct dentry *dentry;
-	struct nfs4_state_owner *owner;
-	struct nfs4_state *state;
-	struct iattr attrs;
-	struct nfs4_layoutget *lgp;
-	unsigned long timestamp;
-	bool rpc_done;
-	bool file_created;
-	bool is_recover;
-	bool cancelled;
-	int rpc_status;
-};
-
 struct nfs4_open_createattrs {
 	struct nfs4_label *label;
 	struct iattr *sattr;
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 363420d0f3a0..11148414b9b6 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -37,6 +37,7 @@
 #include "nfs4trace.h"
 #include "delegation.h"
 #include "nfs42.h"
+#include "nfs4_fs.h"
 
 #define NFSDBG_FACILITY		NFSDBG_PNFS
 #define PNFS_LAYOUTGET_RETRY_TIMEOUT (120*HZ)
-- 
2.17.0

[PATCH 09/19] pnfs: Change pnfs_alloc_init_layoutget_args call signature

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Don't send in a layout, instead use the (possibly NULL) inode.

This is needed for LAYOUTGET attached to an OPEN where the inode is not
yet set.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 40 ++++++++++++++++++++++++++++------------
 1 file changed, 28 insertions(+), 12 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 11148414b9b6..07f1bbd9100c 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -916,18 +916,31 @@ pnfs_layoutgets_blocked(const struct pnfs_layout_hdr *lo)
 		test_bit(NFS_LAYOUT_BULK_RECALL, &lo->plh_flags);
 }
 
+static struct nfs_server *
+pnfs_find_server(struct inode *inode, struct nfs_open_context *ctx)
+{
+	struct nfs_server *server;
+
+	if (inode)
+		server = NFS_SERVER(inode);
+	else {
+		struct dentry *parent_dir = dget_parent(ctx->dentry);
+		server = NFS_SERVER(parent_dir->d_inode);
+		dput(parent_dir);
+	}
+	return server;
+}
+
 static struct nfs4_layoutget *
-pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
+pnfs_alloc_init_layoutget_args(struct inode *ino,
 	   struct nfs_open_context *ctx,
 	   nfs4_stateid *stateid,
 	   const struct pnfs_layout_range *range,
 	   gfp_t gfp_flags)
 {
-	struct inode *ino = lo->plh_inode;
-	struct nfs_server *server = NFS_SERVER(ino);
+	struct nfs_server *server = pnfs_find_server(ino, ctx);
 	size_t max_pages = max_response_pages(server);
 	struct nfs4_layoutget *lgp;
-	loff_t i_size;
 
 	dprintk("--> %s\n", __func__);
 
@@ -944,16 +957,19 @@ pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
 	lgp->res.layoutp = &lgp->args.layout;
 
 
-	i_size = i_size_read(ino);
 
 	lgp->args.minlength = PAGE_SIZE;
 	if (lgp->args.minlength > range->length)
 		lgp->args.minlength = range->length;
-	if (range->iomode == IOMODE_READ) {
-		if (range->offset >= i_size)
-			lgp->args.minlength = 0;
-		else if (i_size - range->offset < lgp->args.minlength)
-			lgp->args.minlength = i_size - range->offset;
+	if (ino) {
+		loff_t i_size = i_size_read(ino);
+
+		if (range->iomode == IOMODE_READ) {
+			if (range->offset >= i_size)
+				lgp->args.minlength = 0;
+			else if (i_size - range->offset < lgp->args.minlength)
+				lgp->args.minlength = i_size - range->offset;
+		}
 	}
 	lgp->args.maxcount = PNFS_LAYOUT_MAXSIZE;
 	pnfs_copy_range(&lgp->args.range, range);
@@ -962,7 +978,7 @@ pnfs_alloc_init_layoutget_args(struct pnfs_layout_hdr *lo,
 	lgp->args.ctx = get_nfs_open_context(ctx);
 	nfs4_stateid_copy(&lgp->args.stateid, stateid);
 	lgp->gfp_flags = gfp_flags;
-	lgp->cred = lo->plh_lc_cred;
+	lgp->cred = ctx->cred;
 	return lgp;
 }
 
@@ -1838,7 +1854,7 @@ pnfs_update_layout(struct inode *ino,
 	if (arg.length != NFS4_MAX_UINT64)
 		arg.length = PAGE_ALIGN(arg.length);
 
-	lgp = pnfs_alloc_init_layoutget_args(lo, ctx, &stateid, &arg, gfp_flags);
+	lgp = pnfs_alloc_init_layoutget_args(ino, ctx, &stateid, &arg, gfp_flags);
 	if (!lgp) {
 		trace_pnfs_update_layout(ino, pos, count, iomode, lo, NULL,
 					 PNFS_UPDATE_LAYOUT_NOMEM);
-- 
2.17.0

[PATCH 10/19] pnfs: Add LAYOUTGET to OPEN of a new file

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

This triggers when have no pre-existing inode to attach to.
The preexisting case is saved for later.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c  | 20 ++++++++++--
 fs/nfs/nfs4state.c |  8 +++++
 fs/nfs/pnfs.c      | 81 ++++++++++++++++++++++++++++++++++++++++++++--
 fs/nfs/pnfs.h      | 17 ++++++++++
 4 files changed, 121 insertions(+), 5 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 771439971759..a73c83f17a86 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -86,6 +86,7 @@
 	| ATTR_MTIME_SET)
 
 struct nfs4_opendata;
+static void nfs4_layoutget_release(void *calldata);
 static int _nfs4_recover_proc_open(struct nfs4_opendata *data);
 static int nfs4_do_fsinfo(struct nfs_server *, struct nfs_fh *, struct nfs_fsinfo *);
 static void nfs_fixup_referral_attributes(struct nfs_fattr *fattr);
@@ -878,6 +879,10 @@ nfs4_sequence_process_interrupted(struct nfs_client *client,
 
 #else	/* !CONFIG_NFS_V4_1 */
 
+static void nfs4_layoutget_release(void *calldata)
+{
+}
+
 static int nfs4_sequence_process(struct rpc_task *task, struct nfs4_sequence_res *res)
 {
 	return nfs40_sequence_done(task, res);
@@ -1245,6 +1250,8 @@ static void nfs4_opendata_free(struct kref *kref)
 			struct nfs4_opendata, kref);
 	struct super_block *sb = p->dentry->d_sb;
 
+	if (p->lgp)
+		nfs4_layoutget_release(p->lgp);
 	nfs_free_seqid(p->o_arg.seqid);
 	nfs4_sequence_free_slot(&p->o_res.seq_res);
 	if (p->state != NULL)
@@ -2335,8 +2342,10 @@ static int nfs4_run_open_task(struct nfs4_opendata *data,
 	if (!ctx) {
 		nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1, 1);
 		data->is_recover = true;
-	} else
+	} else {
 		nfs4_init_sequence(&o_arg->seq_args, &o_res->seq_res, 1, 0);
+		pnfs_lgopen_prepare(data, ctx);
+	}
 	task = rpc_run_task(&task_setup_data);
 	if (IS_ERR(task))
 		return PTR_ERR(task);
@@ -2816,7 +2825,10 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 		nfs_inode_attach_open_context(ctx);
 		if (read_seqcount_retry(&sp->so_reclaim_seqcount, seq))
 			nfs4_schedule_stateid_recovery(server, state);
+		else
+			pnfs_parse_lgopen(state->inode, opendata->lgp, ctx);
 	}
+
 out:
 	return ret;
 }
@@ -8718,13 +8730,15 @@ static void nfs4_layoutget_release(void *calldata)
 {
 	struct nfs4_layoutget *lgp = calldata;
 	struct inode *inode = lgp->args.inode;
-	struct nfs_server *server = NFS_SERVER(inode);
-	size_t max_pages = max_response_pages(server);
+	size_t max_pages = lgp->args.layout.pglen / PAGE_SIZE;
 
 	dprintk("--> %s\n", __func__);
 	nfs4_sequence_free_slot(&lgp->res.seq_res);
 	nfs4_free_pages(lgp->args.layout.pages, max_pages);
 	pnfs_put_layout_hdr(NFS_I(inode)->layout);
+	if (inode)
+		pnfs_put_layout_hdr(NFS_I(inode)->layout);
+	put_rpccred(lgp->cred);
 	put_nfs_open_context(lgp->args.ctx);
 	kfree(calldata);
 	dprintk("<-- %s\n", __func__);
diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index c10a422efe6f..2bf2eaa08ca7 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -77,6 +77,14 @@ const nfs4_stateid invalid_stateid = {
 	.type = NFS4_INVALID_STATEID_TYPE,
 };
 
+const nfs4_stateid current_stateid = {
+	{
+		/* Funky initialiser keeps older gcc versions happy */
+		.data = { 0x0, 0x0, 0x0, 0x1, 0 },
+	},
+	.type = NFS4_SPECIAL_STATEID_TYPE,
+};
+
 static DEFINE_MUTEX(nfs_clid_init_mutex);
 
 int nfs4_init_clientid(struct nfs_client *clp, struct rpc_cred *cred)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 07f1bbd9100c..a0a2484c3aed 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -934,7 +934,7 @@ pnfs_find_server(struct inode *inode, struct nfs_open_context *ctx)
 static struct nfs4_layoutget *
 pnfs_alloc_init_layoutget_args(struct inode *ino,
 	   struct nfs_open_context *ctx,
-	   nfs4_stateid *stateid,
+	   const nfs4_stateid *stateid,
 	   const struct pnfs_layout_range *range,
 	   gfp_t gfp_flags)
 {
@@ -978,7 +978,7 @@ pnfs_alloc_init_layoutget_args(struct inode *ino,
 	lgp->args.ctx = get_nfs_open_context(ctx);
 	nfs4_stateid_copy(&lgp->args.stateid, stateid);
 	lgp->gfp_flags = gfp_flags;
-	lgp->cred = ctx->cred;
+	lgp->cred = get_rpccred(ctx->cred);
 	return lgp;
 }
 
@@ -1943,6 +1943,83 @@ pnfs_sanity_check_layout_range(struct pnfs_layout_range *range)
 	return true;
 }
 
+extern const nfs4_stateid current_stateid;
+
+static void _lgopen_prepare_attached(struct nfs4_opendata *data,
+				     struct nfs_open_context *ctx)
+{
+	/* STUB */
+}
+
+static void _lgopen_prepare_floating(struct nfs4_opendata *data,
+				     struct nfs_open_context *ctx)
+{
+	struct pnfs_layout_range rng = {
+		.iomode = (data->o_arg.fmode & FMODE_WRITE) ?
+			  IOMODE_RW: IOMODE_READ,
+		.offset = 0,
+		.length = NFS4_MAX_UINT64,
+	};
+	struct nfs4_layoutget *lgp;
+
+	lgp = pnfs_alloc_init_layoutget_args(NULL, ctx, &current_stateid,
+					     &rng, GFP_KERNEL);
+	if (!lgp)
+		return;
+	data->lgp = lgp;
+	data->o_arg.lg_args = &lgp->args;
+	data->o_res.lg_res = &lgp->res;
+}
+
+void pnfs_lgopen_prepare(struct nfs4_opendata *data,
+			 struct nfs_open_context *ctx)
+{
+	struct nfs_server *server = NFS_SERVER(data->dir->d_inode);
+
+	if (!(pnfs_enabled_sb(server) &&
+	      server->pnfs_curr_ld->flags & PNFS_LAYOUTGET_ON_OPEN))
+		return;
+	/* Could check on max_ops, but currently hardcoded high enough */
+	if (data->state)
+		_lgopen_prepare_attached(data, ctx);
+	else
+		_lgopen_prepare_floating(data, ctx);
+}
+
+void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
+		       struct nfs_open_context *ctx)
+{
+	struct pnfs_layout_hdr *lo;
+	struct pnfs_layout_segment *lseg;
+	u32 iomode;
+
+	if (!lgp || lgp->res.layoutp->len == 0)
+		return;
+	if (!lgp->args.inode) {
+		/* Need to grab lo */
+		spin_lock(&ino->i_lock);
+		lo = pnfs_find_alloc_layout(ino, ctx, GFP_KERNEL);
+		atomic_inc(&lo->plh_outstanding);
+		spin_unlock(&ino->i_lock);
+		lgp->args.inode = ino;
+	} else
+		lo = NFS_I(lgp->args.inode)->layout;
+	pnfs_get_layout_hdr(lo);
+
+	lseg = pnfs_layout_process(lgp);
+	atomic_dec(&lo->plh_outstanding);
+	if (IS_ERR(lseg)) {
+		/* ignore lseg, but would like to mark not to try lgopen */
+		/* clear some lo flags - first and fail ???? */
+	} else {
+		iomode = lgp->args.range.iomode;
+		pnfs_layout_clear_fail_bit(lo, pnfs_iomode_to_fail_bit(iomode));
+		pnfs_put_lseg(lseg);
+	}
+	pnfs_clear_first_layoutget(lo);
+	pnfs_put_layout_hdr(lo);
+}
+
 struct pnfs_layout_segment *
 pnfs_layout_process(struct nfs4_layoutget *lgp)
 {
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 964a7227ea97..b110c09ea89c 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -35,6 +35,8 @@
 #include <linux/nfs_page.h>
 #include <linux/workqueue.h>
 
+struct nfs4_opendata;
+
 enum {
 	NFS_LSEG_VALID = 0,	/* cleared when lseg is recalled/returned */
 	NFS_LSEG_ROC,		/* roc bit received from server */
@@ -378,6 +380,10 @@ void pnfs_layout_mark_request_commit(struct nfs_page *req,
 				     struct pnfs_layout_segment *lseg,
 				     struct nfs_commit_info *cinfo,
 				     u32 ds_commit_idx);
+void pnfs_lgopen_prepare(struct nfs4_opendata *data,
+			 struct nfs_open_context *ctx);
+void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
+		       struct nfs_open_context *ctx);
 
 static inline bool nfs_have_layout(struct inode *inode)
 {
@@ -778,6 +784,17 @@ static inline bool nfs4_refresh_layout_stateid(nfs4_stateid *dst,
 {
 	return false;
 }
+
+static inline void pnfs_lgopen_prepare(struct nfs4_opendata *data,
+		struct nfs_open_context *ctx)
+{
+}
+
+static inline void pnfs_parse_lgopen(struct inode *ino,
+		struct nfs4_layoutget *lgp,
+		struct nfs_open_context *ctx)
+{
+}
 #endif /* CONFIG_NFS_V4_1 */
 
 #if IS_ENABLED(CONFIG_NFS_V4_2)
-- 
2.17.0

[PATCH 11/19] pnfs: Add LAYOUTGET to OPEN of an existing file

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 90 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 73 insertions(+), 17 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index a0a2484c3aed..258b68110277 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -921,9 +921,9 @@ pnfs_find_server(struct inode *inode, struct nfs_open_context *ctx)
 {
 	struct nfs_server *server;
 
-	if (inode)
+	if (inode) {
 		server = NFS_SERVER(inode);
-	else {
+	} else {
 		struct dentry *parent_dir = dget_parent(ctx->dentry);
 		server = NFS_SERVER(parent_dir->d_inode);
 		dput(parent_dir);
@@ -1686,6 +1686,22 @@ static void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo)
 	wake_up_bit(bitlock, NFS_LAYOUT_FIRST_LAYOUTGET);
 }
 
+static void _add_to_server_list(struct pnfs_layout_hdr *lo,
+				struct nfs_server *server)
+{
+	if (list_empty(&lo->plh_layouts)) {
+		struct nfs_client *clp = server->nfs_client;
+
+		/* The lo must be on the clp list if there is any
+		 * chance of a CB_LAYOUTRECALL(FILE) coming in.
+		 */
+		spin_lock(&clp->cl_lock);
+		if (list_empty(&lo->plh_layouts))
+			list_add_tail(&lo->plh_layouts, &server->layouts);
+		spin_unlock(&clp->cl_lock);
+	}
+}
+
 /*
  * Layout segment is retreived from the server if not cached.
  * The appropriate layout segment is referenced and returned to the caller.
@@ -1836,15 +1852,7 @@ pnfs_update_layout(struct inode *ino,
 	atomic_inc(&lo->plh_outstanding);
 	spin_unlock(&ino->i_lock);
 
-	if (list_empty(&lo->plh_layouts)) {
-		/* The lo must be on the clp list if there is any
-		 * chance of a CB_LAYOUTRECALL(FILE) coming in.
-		 */
-		spin_lock(&clp->cl_lock);
-		if (list_empty(&lo->plh_layouts))
-			list_add_tail(&lo->plh_layouts, &server->layouts);
-		spin_unlock(&clp->cl_lock);
-	}
+	_add_to_server_list(lo, server);
 
 	pg_offset = arg.offset & ~PAGE_MASK;
 	if (pg_offset) {
@@ -1943,12 +1951,62 @@ pnfs_sanity_check_layout_range(struct pnfs_layout_range *range)
 	return true;
 }
 
+static struct pnfs_layout_hdr *
+_pnfs_grab_empty_layout(struct inode *ino, struct nfs_open_context *ctx)
+{
+	struct pnfs_layout_hdr *lo;
+
+	spin_lock(&ino->i_lock);
+	lo = pnfs_find_alloc_layout(ino, ctx, GFP_KERNEL);
+	if (!lo)
+		goto out_unlock;
+	if (!test_bit(NFS_LAYOUT_INVALID_STID, &lo->plh_flags))
+		goto out_unlock;
+	if (test_bit(NFS_LAYOUT_RETURN, &lo->plh_flags))
+		goto out_unlock;
+	if (pnfs_layoutgets_blocked(lo))
+		goto out_unlock;
+	if (test_and_set_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags))
+		goto out_unlock;
+	atomic_inc(&lo->plh_outstanding);
+	spin_unlock(&ino->i_lock);
+	_add_to_server_list(lo, NFS_SERVER(ino));
+	return lo;
+
+out_unlock:
+	spin_unlock(&ino->i_lock);
+	pnfs_put_layout_hdr(lo);
+	return NULL;
+}
+
 extern const nfs4_stateid current_stateid;
 
 static void _lgopen_prepare_attached(struct nfs4_opendata *data,
 				     struct nfs_open_context *ctx)
 {
-	/* STUB */
+	struct inode *ino = data->dentry->d_inode;
+	struct pnfs_layout_range rng = {
+		.iomode = (data->o_arg.fmode & FMODE_WRITE) ?
+			  IOMODE_RW: IOMODE_READ,
+		.offset = 0,
+		.length = NFS4_MAX_UINT64,
+	};
+	struct nfs4_layoutget *lgp;
+	struct pnfs_layout_hdr *lo;
+
+	lo = _pnfs_grab_empty_layout(ino, ctx);
+	if (!lo)
+		return;
+	lgp = pnfs_alloc_init_layoutget_args(ino, ctx, &current_stateid,
+					     &rng, GFP_KERNEL);
+	if (!lgp) {
+		pnfs_clear_first_layoutget(lo);
+		pnfs_put_layout_hdr(lo);
+		return;
+	}
+	data->lgp = lgp;
+	data->o_arg.lg_args = &lgp->args;
+	data->o_res.lg_res = &lgp->res;
 }
 
 static void _lgopen_prepare_floating(struct nfs4_opendata *data,
@@ -1996,11 +2054,9 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 	if (!lgp || lgp->res.layoutp->len == 0)
 		return;
 	if (!lgp->args.inode) {
-		/* Need to grab lo */
-		spin_lock(&ino->i_lock);
-		lo = pnfs_find_alloc_layout(ino, ctx, GFP_KERNEL);
-		atomic_inc(&lo->plh_outstanding);
-		spin_unlock(&ino->i_lock);
+		lo = _pnfs_grab_empty_layout(ino, ctx);
+		if (!lo)
+			return;
 		lgp->args.inode = ino;
 	} else
 		lo = NFS_I(lgp->args.inode)->layout;
-- 
2.17.0

[PATCH 12/19] pnfs: Stop attempting LAYOUTGET on OPEN on failure

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c         |  4 +++-
 fs/nfs/pnfs.c             | 19 ++++++++++++++++++-
 include/linux/nfs_fs_sb.h |  1 +
 3 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index a73c83f17a86..659ba6f0ca9b 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -9507,7 +9507,8 @@ static const struct nfs4_minor_version_ops nfs_v4_1_minor_ops = {
 		| NFS_CAP_ATOMIC_OPEN
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
-		| NFS_CAP_ATOMIC_OPEN_V1,
+		| NFS_CAP_ATOMIC_OPEN_V1
+		| NFS_CAP_LGOPEN,
 	.init_client = nfs41_init_client,
 	.shutdown_client = nfs41_shutdown_client,
 	.match_stateid = nfs41_match_stateid,
@@ -9532,6 +9533,7 @@ static const struct nfs4_minor_version_ops nfs_v4_2_minor_ops = {
 		| NFS_CAP_POSIX_LOCK
 		| NFS_CAP_STATEID_NFSV41
 		| NFS_CAP_ATOMIC_OPEN_V1
+		| NFS_CAP_LGOPEN
 		| NFS_CAP_ALLOCATE
 		| NFS_CAP_COPY
 		| NFS_CAP_DEALLOCATE
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 258b68110277..b2158e5bf7f7 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -2038,6 +2038,8 @@ void pnfs_lgopen_prepare(struct nfs4_opendata *data,
 	      server->pnfs_curr_ld->flags & PNFS_LAYOUTGET_ON_OPEN))
 		return;
 	/* Could check on max_ops, but currently hardcoded high enough */
+	if (!nfs_server_capable(data->dir->d_inode, NFS_CAP_LGOPEN))
+		return;
 	if (data->state)
 		_lgopen_prepare_attached(data, ctx);
 	else
@@ -2051,8 +2053,23 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 	struct pnfs_layout_segment *lseg;
 	u32 iomode;
 
-	if (!lgp || lgp->res.layoutp->len == 0)
+	if (!lgp)
 		return;
+	dprintk("%s: entered with status %i\n", __func__, lgp->res.status);
+	if (lgp->res.status) {
+		switch (lgp->res.status) {
+		case -NFS4ERR_DELAY:
+		case -NFS4ERR_GRACE:
+		case -NFS4ERR_LAYOUTTRYLATER:
+			break;
+		default:
+			/* FIXME - Any error not listed above permanently
+			 * halts lgopen attempts.
+			 */
+			NFS_SERVER(ino)->caps &= ~NFS_CAP_LGOPEN;
+		}
+		return;
+	}
 	if (!lgp->args.inode) {
 		lo = _pnfs_grab_empty_layout(ino, ctx);
 		if (!lo)
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 4e735be53e70..2c18d618604e 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -235,6 +235,7 @@ struct nfs_server {
 #define NFS_CAP_ACLS		(1U << 3)
 #define NFS_CAP_ATOMIC_OPEN	(1U << 4)
 /* #define NFS_CAP_CHANGE_ATTR	(1U << 5) */
+#define NFS_CAP_LGOPEN		(1U << 5)
 #define NFS_CAP_FILEID		(1U << 6)
 #define NFS_CAP_MODE		(1U << 7)
 #define NFS_CAP_NLINK		(1U << 8)
-- 
2.17.0

[PATCH 13/19] pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

Since the LAYOUTGET on OPEN can be sent without prior inode information,
existing methods to prevent LAYOUTGET from being sent while processing
CB_LAYOUTRECALL don't work.  Track if a recall occurred while LAYOUTGET
was being sent, and if so ignore the results.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/callback_proc.c    | 2 ++
 fs/nfs/pnfs.c             | 8 +++++++-
 include/linux/nfs_fs_sb.h | 1 +
 include/linux/nfs_xdr.h   | 1 +
 4 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index a50d7813e3ea..d561161b7c3e 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -322,6 +322,8 @@ static u32 initiate_bulk_draining(struct nfs_client *clp,
 static u32 do_callback_layoutrecall(struct nfs_client *clp,
 				    struct cb_layoutrecallargs *args)
 {
+	write_seqcount_begin(&clp->cl_callback_count);
+	write_seqcount_end(&clp->cl_callback_count);
 	if (args->cbl_recall_type == RETURN_FILE)
 		return initiate_file_draining(clp, args);
 	return initiate_bulk_draining(clp, args);
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index b2158e5bf7f7..5621b85a5c24 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -979,6 +979,7 @@ pnfs_alloc_init_layoutget_args(struct inode *ino,
 	nfs4_stateid_copy(&lgp->args.stateid, stateid);
 	lgp->gfp_flags = gfp_flags;
 	lgp->cred = get_rpccred(ctx->cred);
+	lgp->callback_count = raw_seqcount_begin(&server->nfs_client->cl_callback_count);
 	return lgp;
 }
 
@@ -2051,6 +2052,7 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 {
 	struct pnfs_layout_hdr *lo;
 	struct pnfs_layout_segment *lseg;
+	struct nfs_server *srv = NFS_SERVER(ino);
 	u32 iomode;
 
 	if (!lgp)
@@ -2066,7 +2068,7 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 			/* FIXME - Any error not listed above permanently
 			 * halts lgopen attempts.
 			 */
-			NFS_SERVER(ino)->caps &= ~NFS_CAP_LGOPEN;
+			srv->caps &= ~NFS_CAP_LGOPEN;
 		}
 		return;
 	}
@@ -2079,6 +2081,9 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 		lo = NFS_I(lgp->args.inode)->layout;
 	pnfs_get_layout_hdr(lo);
 
+	if (read_seqcount_retry(&srv->nfs_client->cl_callback_count,
+				lgp->callback_count))
+		goto out;
 	lseg = pnfs_layout_process(lgp);
 	atomic_dec(&lo->plh_outstanding);
 	if (IS_ERR(lseg)) {
@@ -2089,6 +2094,7 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 		pnfs_layout_clear_fail_bit(lo, pnfs_iomode_to_fail_bit(iomode));
 		pnfs_put_lseg(lseg);
 	}
+out:
 	pnfs_clear_first_layoutget(lo);
 	pnfs_put_layout_hdr(lo);
 }
diff --git a/include/linux/nfs_fs_sb.h b/include/linux/nfs_fs_sb.h
index 2c18d618604e..74ae3e1d19a0 100644
--- a/include/linux/nfs_fs_sb.h
+++ b/include/linux/nfs_fs_sb.h
@@ -28,6 +28,7 @@ struct nfs41_impl_id;
 struct nfs_client {
 	refcount_t		cl_count;
 	atomic_t		cl_mds_count;
+	seqcount_t		cl_callback_count;
 	int			cl_cons_state;	/* current construction state (-ve: init error) */
 #define NFS_CS_READY		0		/* ready to be used */
 #define NFS_CS_INITING		1		/* busy initialising */
diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index b36be7a703ea..b777eb7da15e 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -271,6 +271,7 @@ struct nfs4_layoutget {
 	struct nfs4_layoutget_args args;
 	struct nfs4_layoutget_res res;
 	struct rpc_cred *cred;
+	unsigned callback_count;
 	gfp_t gfp_flags;
 };
 
-- 
2.17.0

[PATCH 14/19] pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET

[Trond Myklebust]

From: Fred Isaman <fred.isaman@gmail.com>

The flag was not always being cleared after LAYOUTGET on OPEN.

Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 15 ++++++++++++++-
 fs/nfs/pnfs.c     |  9 ++-------
 fs/nfs/pnfs.h     |  6 ++++++
 3 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 659ba6f0ca9b..53f93c3993c8 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -1244,6 +1244,19 @@ static struct nfs4_opendata *nfs4_opendata_alloc(struct dentry *dentry,
 	return NULL;
 }
 
+static void nfs4_lgopen_release(void *calldata)
+{
+	struct nfs4_layoutget *lgp = calldata;
+	struct inode *inode = lgp->args.inode;
+
+	if (inode) {
+		struct pnfs_layout_hdr *lo = NFS_I(inode)->layout;
+		atomic_dec(&lo->plh_outstanding);
+		pnfs_clear_first_layoutget(lo);
+	}
+	nfs4_layoutget_release(lgp);
+}
+
 static void nfs4_opendata_free(struct kref *kref)
 {
 	struct nfs4_opendata *p = container_of(kref,
@@ -1251,7 +1264,7 @@ static void nfs4_opendata_free(struct kref *kref)
 	struct super_block *sb = p->dentry->d_sb;
 
 	if (p->lgp)
-		nfs4_layoutget_release(p->lgp);
+		nfs4_lgopen_release(p->lgp);
 	nfs_free_seqid(p->o_arg.seqid);
 	nfs4_sequence_free_slot(&p->o_res.seq_res);
 	if (p->state != NULL)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 5621b85a5c24..037e6b9b047b 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1678,7 +1678,7 @@ static bool pnfs_prepare_to_retry_layoutget(struct pnfs_layout_hdr *lo)
 				   TASK_UNINTERRUPTIBLE);
 }
 
-static void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo)
+void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo)
 {
 	unsigned long *bitlock = &lo->plh_flags;
 
@@ -2079,13 +2079,11 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 		lgp->args.inode = ino;
 	} else
 		lo = NFS_I(lgp->args.inode)->layout;
-	pnfs_get_layout_hdr(lo);
 
 	if (read_seqcount_retry(&srv->nfs_client->cl_callback_count,
 				lgp->callback_count))
-		goto out;
+		return;
 	lseg = pnfs_layout_process(lgp);
-	atomic_dec(&lo->plh_outstanding);
 	if (IS_ERR(lseg)) {
 		/* ignore lseg, but would like to mark not to try lgopen */
 		/* clear some lo flags - first and fail ???? */
@@ -2094,9 +2092,6 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 		pnfs_layout_clear_fail_bit(lo, pnfs_iomode_to_fail_bit(iomode));
 		pnfs_put_lseg(lseg);
 	}
-out:
-	pnfs_clear_first_layoutget(lo);
-	pnfs_put_layout_hdr(lo);
 }
 
 struct pnfs_layout_segment *
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index b110c09ea89c..fc714caff52e 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -380,6 +380,7 @@ void pnfs_layout_mark_request_commit(struct nfs_page *req,
 				     struct pnfs_layout_segment *lseg,
 				     struct nfs_commit_info *cinfo,
 				     u32 ds_commit_idx);
+void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo);
 void pnfs_lgopen_prepare(struct nfs4_opendata *data,
 			 struct nfs_open_context *ctx);
 void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
@@ -795,6 +796,11 @@ static inline void pnfs_parse_lgopen(struct inode *ino,
 		struct nfs_open_context *ctx)
 {
 }
+
+static inline void pnfs_clear_first_layoutget(struct pnfs_layout_hdr *lo)
+{
+}
+
 #endif /* CONFIG_NFS_V4_1 */
 
 #if IS_ENABLED(CONFIG_NFS_V4_2)
-- 
2.17.0

[PATCH 15/19] NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse uninitialised data

[Trond Myklebust]

From: Trond Myklebust <trond.myklebust@primarydata.com>

We need to ensure that pnfs_parse_lgopen() doesn't try to parse a
struct nfs4_layoutget_res that was not filled by a successful call
to decode_layoutget(). This can happen if we performed a cached open,
or if either the OP_ACCESS or OP_GETATTR operations preceding the
OP_LAYOUTGET in the compound returned an error.

By initialising the 'status' field to NFS4ERR_DELAY, we ensure that
pnfs_parse_lgopen() won't try to interpret the structure.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 037e6b9b047b..3da4f404c2fa 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -956,7 +956,8 @@ pnfs_alloc_init_layoutget_args(struct inode *ino,
 	lgp->args.layout.pglen = max_pages * PAGE_SIZE;
 	lgp->res.layoutp = &lgp->args.layout;
 
-
+	/* Don't confuse uninitialised result and success */
+	lgp->res.status = -NFS4ERR_DELAY;
 
 	lgp->args.minlength = PAGE_SIZE;
 	if (lgp->args.minlength > range->length)
-- 
2.17.0

[PATCH 16/19] NFSv4/pnfs: Don't switch off layoutget-on-open for transient errors

[Trond Myklebust]

From: Trond Myklebust <trond.myklebust@primarydata.com>

Ensure that we only switch off the LAYOUTGET operation in the OPEN
compound when the server is truly broken, and/or it is complaining
that the compound is too large.
Currently, we end up turning off the functionality permanently,
even for transient errors such as EACCES or ENOSPC.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 3da4f404c2fa..8c6ce63f5ed0 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -2061,14 +2061,22 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 	dprintk("%s: entered with status %i\n", __func__, lgp->res.status);
 	if (lgp->res.status) {
 		switch (lgp->res.status) {
-		case -NFS4ERR_DELAY:
-		case -NFS4ERR_GRACE:
-		case -NFS4ERR_LAYOUTTRYLATER:
-			break;
 		default:
-			/* FIXME - Any error not listed above permanently
-			 * halts lgopen attempts.
-			 */
+			break;
+		/*
+		 * Halt lgopen attempts if the server doesn't recognise
+		 * the "current stateid" value, the layout type, or the
+		 * layoutget operation as being valid.
+		 * Also if it complains about too many ops in the compound
+		 * or of the request/reply being too big.
+		 */
+		case -NFS4ERR_BAD_STATEID:
+		case -NFS4ERR_NOTSUPP:
+		case -NFS4ERR_REP_TOO_BIG:
+		case -NFS4ERR_REP_TOO_BIG_TO_CACHE:
+		case -NFS4ERR_REQ_TOO_BIG:
+		case -NFS4ERR_TOO_MANY_OPS:
+		case -NFS4ERR_UNKNOWN_LAYOUTTYPE:
 			srv->caps &= ~NFS_CAP_LGOPEN;
 		}
 		return;
-- 
2.17.0

[PATCH 17/19] pNFS: Don't send LAYOUTGET on OPEN for read, if we already have cached data

[Trond Myklebust]

From: Trond Myklebust <trond.myklebust@primarydata.com>

If we're only opening the file for reading, and the file is empty and/or
we already have cached data, then heuristically optimise away the
LAYOUTGET.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 8c6ce63f5ed0..5fc4d84d9326 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1996,6 +1996,11 @@ static void _lgopen_prepare_attached(struct nfs4_opendata *data,
 	struct nfs4_layoutget *lgp;
 	struct pnfs_layout_hdr *lo;
 
+	/* Heuristic: don't send layoutget if we have cached data */
+	if (rng.iomode == IOMODE_READ &&
+	   (i_size_read(ino) == 0 || ino->i_mapping->nrpages != 0))
+		return;
+
 	lo = _pnfs_grab_empty_layout(ino, ctx);
 	if (!lo)
 		return;
-- 
2.17.0

[PATCH 18/19] pnfs: Don't call commit on failed layoutget-on-open

[Trond Myklebust]

If the layoutget on open call failed, we can't really commit the inode,
so don't bother calling it.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/pnfs.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 5fc4d84d9326..545e9e2c6201 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -2098,10 +2098,7 @@ void pnfs_parse_lgopen(struct inode *ino, struct nfs4_layoutget *lgp,
 				lgp->callback_count))
 		return;
 	lseg = pnfs_layout_process(lgp);
-	if (IS_ERR(lseg)) {
-		/* ignore lseg, but would like to mark not to try lgopen */
-		/* clear some lo flags - first and fail ???? */
-	} else {
+	if (!IS_ERR(lseg)) {
 		iomode = lgp->args.range.iomode;
 		pnfs_layout_clear_fail_bit(lo, pnfs_iomode_to_fail_bit(iomode));
 		pnfs_put_lseg(lseg);
@@ -2173,8 +2170,6 @@ pnfs_layout_process(struct nfs4_layoutget *lgp)
 	spin_unlock(&ino->i_lock);
 	lseg->pls_layout = lo;
 	NFS_SERVER(ino)->pnfs_curr_ld->free_lseg(lseg);
-	if (!pnfs_layout_is_valid(lo))
-		nfs_commit_inode(ino, 0);
 	return ERR_PTR(-EAGAIN);
 }
 
-- 
2.17.0

[PATCH 19/19] pnfs: Don't release the sequence slot until we've processed layoutget on open

[Trond Myklebust]

If the server recalls the layout that was just handed out, we risk hitting
a race as described in RFC5661 Section 2.10.6.3 unless we ensure that we
release the sequence slot after processing the LAYOUTGET operation that
was sent as part of the OPEN compound.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/nfs4proc.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 53f93c3993c8..e664e0c82700 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -2804,7 +2804,7 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 	if (ret != 0)
 		goto out;
 
-	state = nfs4_opendata_to_nfs4_state(opendata);
+	state = _nfs4_opendata_to_nfs4_state(opendata);
 	ret = PTR_ERR(state);
 	if (IS_ERR(state))
 		goto out;
@@ -2843,6 +2843,7 @@ static int _nfs4_open_and_get_state(struct nfs4_opendata *opendata,
 	}
 
 out:
+	nfs4_sequence_free_slot(&opendata->o_res.seq_res);
 	return ret;
 }
 
-- 
2.17.0

Re: [PATCH 04/19] pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN

[Olga Kornievskaia]

Hi Trond,

This feature is enabled for Flexfiles layout type. Is there a reason
that it shouldn't be generic for all pnfs?

On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com> wrote:
> From: Fred Isaman <fred.isaman@gmail.com>
>
> Driver can set flag to allow LAYOUTGET to be sent with OPEN.
>
> Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
> Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> ---
>  fs/nfs/flexfilelayout/flexfilelayout.c | 1 +
>  fs/nfs/pnfs.h                          | 1 +
>  2 files changed, 2 insertions(+)
>
> diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c b/fs/nfs/flexfilelayout/flexfilelayout.c
> index c75ad982bcfc..3ae038d9c292 100644
> --- a/fs/nfs/flexfilelayout/flexfilelayout.c
> +++ b/fs/nfs/flexfilelayout/flexfilelayout.c
> @@ -2347,6 +2347,7 @@ static struct pnfs_layoutdriver_type flexfilelayout_type = {
>         .id                     = LAYOUT_FLEX_FILES,
>         .name                   = "LAYOUT_FLEX_FILES",
>         .owner                  = THIS_MODULE,
> +       .flags                  = PNFS_LAYOUTGET_ON_OPEN,
>         .set_layoutdriver       = ff_layout_set_layoutdriver,
>         .alloc_layout_hdr       = ff_layout_alloc_layout_hdr,
>         .free_layout_hdr        = ff_layout_free_layout_hdr,
> diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
> index daf6cbf5c15f..f71a55f11b97 100644
> --- a/fs/nfs/pnfs.h
> +++ b/fs/nfs/pnfs.h
> @@ -110,6 +110,7 @@ enum layoutdriver_policy_flags {
>         PNFS_LAYOUTRET_ON_SETATTR       = 1 << 0,
>         PNFS_LAYOUTRET_ON_ERROR         = 1 << 1,
>         PNFS_READ_WHOLE_PAGE            = 1 << 2,
> +       PNFS_LAYOUTGET_ON_OPEN          = 1 << 3,
>  };
>
>  struct nfs4_deviceid_node;
> --
> 2.17.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 04/19] pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN

[Trond Myklebust]

T24gV2VkLCAyMDE4LTA1LTMwIGF0IDE2OjEwIC0wNDAwLCBPbGdhIEtvcm5pZXZza2FpYSB3cm90
ZToNCj4gSGkgVHJvbmQsDQo+IA0KPiBUaGlzIGZlYXR1cmUgaXMgZW5hYmxlZCBmb3IgRmxleGZp
bGVzIGxheW91dCB0eXBlLiBJcyB0aGVyZSBhIHJlYXNvbg0KPiB0aGF0IGl0IHNob3VsZG4ndCBi
ZSBnZW5lcmljIGZvciBhbGwgcG5mcz8NCg0KWWVzIGFuZCBuby4gSXQgcmVsaWVzIG9uIHVzZSBv
ZiB0aGUgImN1cnJlbnQgc3RhdGVpZCIgZmVhdHVyZSwgd2hpY2gNCmhhcyBub3QgcHJldmlvdXNs
eSBzZWVuIGhlYXZ5IHVzZSwgYW5kIHRoZSBhYmlsaXR5IG9mIHRoZSBwTkZTIHNlcnZlcg0KdG8g
aGFuZGxlIGxheW91dGdldCByYWNlcyBjb3JyZWN0bHkgKHdoZW4gdGhlIGNsaWVudCBzZW5kcyAy
IGxheW91dGdldA0KcmVxdWVzdHMgaW4gcGFyYWxsZWwsIGJvdGggdXNpbmcgdGhlIG9wZW4gc3Rh
dGVpZCkuIElmIHRoZSBzZXJ2ZXIgaXMNCmludmFsaWRhdGluZyBvbmUgb2YgdGhlIHJlc3VsdGlu
ZyBsYXlvdXQgc3RhdGVpZHMgYXMgaXQgc2hvdWxkLCB0aGVuIGl0DQpoYWQgYmV0dGVyIGVpdGhl
ciBoYXZlIGEgZ29vZCBmZW5jaW5nIG1lY2hhbmlzbSwgb3IgaXQgbXVzdCByZWNhbGwgdGhhdA0K
bGF5b3V0IHN0YXRlaWQgYmVmb3JlIGhhbmRpbmcgb3V0IHRoZSBzZWNvbmQgbGF5b3V0Lg0KDQpG
b3IgdGhhdCByZWFzb24sIEknZCBsaWtlIHRvIGVuc3VyZSB0aGF0IHdlIGF0IGxlYXN0IHRlc3Qg
dGhlIGV4aXN0aW5nDQpwTkZTIGltcGxlbWVudGF0aW9ucyB0byBlbnN1cmUgd2UgZG9uJ3Qgc2Vl
IHJlZ3Jlc3Npb25zIGJlZm9yZSB3ZQ0KZW5hYmxlIHRoZSBmZWF0dXJlLg0KDQoNCj4gT24gV2Vk
LCBNYXkgMzAsIDIwMTggYXQgMjowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0cm9uZG15QGdtYWls
LmNvbT4NCj4gd3JvdGU6DQo+ID4gRnJvbTogRnJlZCBJc2FtYW4gPGZyZWQuaXNhbWFuQGdtYWls
LmNvbT4NCj4gPiANCj4gPiBEcml2ZXIgY2FuIHNldCBmbGFnIHRvIGFsbG93IExBWU9VVEdFVCB0
byBiZSBzZW50IHdpdGggT1BFTi4NCj4gPiANCj4gPiBTaWduZWQtb2ZmLWJ5OiBGcmVkIElzYW1h
biA8ZnJlZC5pc2FtYW5AZ21haWwuY29tPg0KPiA+IFNpZ25lZC1vZmYtYnk6IFRyb25kIE15a2xl
YnVzdCA8dHJvbmQubXlrbGVidXN0QGhhbW1lcnNwYWNlLmNvbT4NCj4gPiAtLS0NCj4gPiAgZnMv
bmZzL2ZsZXhmaWxlbGF5b3V0L2ZsZXhmaWxlbGF5b3V0LmMgfCAxICsNCj4gPiAgZnMvbmZzL3Bu
ZnMuaCAgICAgICAgICAgICAgICAgICAgICAgICAgfCAxICsNCj4gPiAgMiBmaWxlcyBjaGFuZ2Vk
LCAyIGluc2VydGlvbnMoKykNCj4gPiANCj4gPiBkaWZmIC0tZ2l0IGEvZnMvbmZzL2ZsZXhmaWxl
bGF5b3V0L2ZsZXhmaWxlbGF5b3V0LmMNCj4gPiBiL2ZzL25mcy9mbGV4ZmlsZWxheW91dC9mbGV4
ZmlsZWxheW91dC5jDQo+ID4gaW5kZXggYzc1YWQ5ODJiY2ZjLi4zYWUwMzhkOWMyOTIgMTAwNjQ0
DQo+ID4gLS0tIGEvZnMvbmZzL2ZsZXhmaWxlbGF5b3V0L2ZsZXhmaWxlbGF5b3V0LmMNCj4gPiAr
KysgYi9mcy9uZnMvZmxleGZpbGVsYXlvdXQvZmxleGZpbGVsYXlvdXQuYw0KPiA+IEBAIC0yMzQ3
LDYgKzIzNDcsNyBAQCBzdGF0aWMgc3RydWN0IHBuZnNfbGF5b3V0ZHJpdmVyX3R5cGUNCj4gPiBm
bGV4ZmlsZWxheW91dF90eXBlID0gew0KPiA+ICAgICAgICAgLmlkICAgICAgICAgICAgICAgICAg
ICAgPSBMQVlPVVRfRkxFWF9GSUxFUywNCj4gPiAgICAgICAgIC5uYW1lICAgICAgICAgICAgICAg
ICAgID0gIkxBWU9VVF9GTEVYX0ZJTEVTIiwNCj4gPiAgICAgICAgIC5vd25lciAgICAgICAgICAg
ICAgICAgID0gVEhJU19NT0RVTEUsDQo+ID4gKyAgICAgICAuZmxhZ3MgICAgICAgICAgICAgICAg
ICA9IFBORlNfTEFZT1VUR0VUX09OX09QRU4sDQo+ID4gICAgICAgICAuc2V0X2xheW91dGRyaXZl
ciAgICAgICA9IGZmX2xheW91dF9zZXRfbGF5b3V0ZHJpdmVyLA0KPiA+ICAgICAgICAgLmFsbG9j
X2xheW91dF9oZHIgICAgICAgPSBmZl9sYXlvdXRfYWxsb2NfbGF5b3V0X2hkciwNCj4gPiAgICAg
ICAgIC5mcmVlX2xheW91dF9oZHIgICAgICAgID0gZmZfbGF5b3V0X2ZyZWVfbGF5b3V0X2hkciwN
Cj4gPiBkaWZmIC0tZ2l0IGEvZnMvbmZzL3BuZnMuaCBiL2ZzL25mcy9wbmZzLmgNCj4gPiBpbmRl
eCBkYWY2Y2JmNWMxNWYuLmY3MWE1NWYxMWI5NyAxMDA2NDQNCj4gPiAtLS0gYS9mcy9uZnMvcG5m
cy5oDQo+ID4gKysrIGIvZnMvbmZzL3BuZnMuaA0KPiA+IEBAIC0xMTAsNiArMTEwLDcgQEAgZW51
bSBsYXlvdXRkcml2ZXJfcG9saWN5X2ZsYWdzIHsNCj4gPiAgICAgICAgIFBORlNfTEFZT1VUUkVU
X09OX1NFVEFUVFIgICAgICAgPSAxIDw8IDAsDQo+ID4gICAgICAgICBQTkZTX0xBWU9VVFJFVF9P
Tl9FUlJPUiAgICAgICAgID0gMSA8PCAxLA0KPiA+ICAgICAgICAgUE5GU19SRUFEX1dIT0xFX1BB
R0UgICAgICAgICAgICA9IDEgPDwgMiwNCj4gPiArICAgICAgIFBORlNfTEFZT1VUR0VUX09OX09Q
RU4gICAgICAgICAgPSAxIDw8IDMsDQo+ID4gIH07DQo+ID4gDQo+ID4gIHN0cnVjdCBuZnM0X2Rl
dmljZWlkX25vZGU7DQo+ID4gLS0NCj4gPiAyLjE3LjANCj4gPiANCj4gPiAtLQ0KPiA+IFRvIHVu
c3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51
eC0NCj4gPiBuZnMiIGluDQo+ID4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2
Z2VyLmtlcm5lbC5vcmcNCj4gPiBNb3JlIG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5r
ZXJuZWwub3JnL21ham9yZG9tby1pbmZvLmh0bWwNCj4gDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJl
IGZyb20gdGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiDQo+
IGluDQo+IHRoZSBib2R5IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3Jn
DQo+IE1vcmUgbWFqb3Jkb21vIGluZm8gYXQgIGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jk
b21vLWluZm8uaHRtbA0KLS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFp
bnRhaW5lciwgSGFtbWVyc3BhY2UNCnRyb25kLm15a2xlYnVzdEBoYW1tZXJzcGFjZS5jb20NCg0K

Re: [PATCH 04/19] pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN

[Olga Kornievskaia]

On Thu, May 31, 2018 at 8:41 AM Trond Myklebust <trondmy@hammerspace.com> wrote:
>
> On Wed, 2018-05-30 at 16:10 -0400, Olga Kornievskaia wrote:
> > Hi Trond,
> >
> > This feature is enabled for Flexfiles layout type. Is there a reason
> > that it shouldn't be generic for all pnfs?
>
> Yes and no. It relies on use of the "current stateid" feature, which
> has not previously seen heavy use, and the ability of the pNFS server
> to handle layoutget races correctly (when the client sends 2 layoutget
> requests in parallel, both using the open stateid). If the server is
> invalidating one of the resulting layout stateids as it should, then it
> had better either have a good fencing mechanism, or it must recall that
> layout stateid before handing out the second layout.
>
> For that reason, I'd like to ensure that we at least test the existing
> pNFS implementations to ensure we don't see regressions before we
> enable the feature.

Hi Trond,

I'm getting back to trying to add LAYOUTGET to the OPEN compound for
the file layout type. I'm looking at your reply and trying to figure
out what kind of testing I should try to do. You mention a race where
a client sends 2 layoutget requests in parallel. You say "using the
open stateid", but when the LAYOUTGET is added to the OPEN, it'll be
using as you said current stateid. So is the problem sending 2
concurrent OPENs for the same file (same owner)? Then server should
issue the reply to the 1st, then a layout recall and reply to the 2nd?

But for the testing is your suggestion to instrument sending 2
concurrent opens with the new code and see that happens?

Thank you.

> > On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com>
> > wrote:
> > > From: Fred Isaman <fred.isaman@gmail.com>
> > >
> > > Driver can set flag to allow LAYOUTGET to be sent with OPEN.
> > >
> > > Signed-off-by: Fred Isaman <fred.isaman@gmail.com>
> > > Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
> > > ---
> > >  fs/nfs/flexfilelayout/flexfilelayout.c | 1 +
> > >  fs/nfs/pnfs.h                          | 1 +
> > >  2 files changed, 2 insertions(+)
> > >
> > > diff --git a/fs/nfs/flexfilelayout/flexfilelayout.c
> > > b/fs/nfs/flexfilelayout/flexfilelayout.c
> > > index c75ad982bcfc..3ae038d9c292 100644
> > > --- a/fs/nfs/flexfilelayout/flexfilelayout.c
> > > +++ b/fs/nfs/flexfilelayout/flexfilelayout.c
> > > @@ -2347,6 +2347,7 @@ static struct pnfs_layoutdriver_type
> > > flexfilelayout_type = {
> > >         .id                     = LAYOUT_FLEX_FILES,
> > >         .name                   = "LAYOUT_FLEX_FILES",
> > >         .owner                  = THIS_MODULE,
> > > +       .flags                  = PNFS_LAYOUTGET_ON_OPEN,
> > >         .set_layoutdriver       = ff_layout_set_layoutdriver,
> > >         .alloc_layout_hdr       = ff_layout_alloc_layout_hdr,
> > >         .free_layout_hdr        = ff_layout_free_layout_hdr,
> > > diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
> > > index daf6cbf5c15f..f71a55f11b97 100644
> > > --- a/fs/nfs/pnfs.h
> > > +++ b/fs/nfs/pnfs.h
> > > @@ -110,6 +110,7 @@ enum layoutdriver_policy_flags {
> > >         PNFS_LAYOUTRET_ON_SETATTR       = 1 << 0,
> > >         PNFS_LAYOUTRET_ON_ERROR         = 1 << 1,
> > >         PNFS_READ_WHOLE_PAGE            = 1 << 2,
> > > +       PNFS_LAYOUTGET_ON_OPEN          = 1 << 3,
> > >  };
> > >
> > >  struct nfs4_deviceid_node;
> > > --
> > > 2.17.0
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-
> > > nfs" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> > in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
>

Re: [PATCH 00/19] Layoutget on OPEN

[Olga Kornievskaia]

Hi Trond,

Do you by any chance have this patch set somewhere in your git
branches? I'd like to test this against NetApp.

Thank you.

On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com> wrote:
> This patchset is something we've been sitting on for a couple of years
> in order to allow it to mature. It was mainly developed by Fred Isaman
> while he was working for Primary Data, but we've had a few bugfixes
> come in since then.
> The main reason for delaying the upstream merge was to ensure that
> adding layoutget to the OPEN compound was safe (does not cause existing
> layouts to be automatically revoked) and that it gives us a real
> performance benefit.
>
> Fred Isaman (14):
>   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>   pnfs: Store return value of decode_layoutget for later processing
>   NFS4: move ctx into nfs4_run_open_task
>   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>   pnfs: refactor send_layoutget
>   pnfs: move allocations out of nfs4_proc_layoutget
>   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN compound
>   pnfs: Move nfs4_opendata into nfs4_fs.h
>   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>   pnfs: Add LAYOUTGET to OPEN of a new file
>   pnfs: Add LAYOUTGET to OPEN of an existing file
>   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
>
> Trond Myklebust (5):
>   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>     uninitialised data
>   NFSv4/pnfs: Don't switch off layoutget-on-open for transient errors
>   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have cached
>     data
>   pnfs: Don't call commit on failed layoutget-on-open
>   pnfs: Don't release the sequence slot until we've processed layoutget
>     on open
>
>  fs/nfs/callback_proc.c                 |   2 +
>  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>  fs/nfs/nfs4_fs.h                       |  25 +++
>  fs/nfs/nfs4proc.c                      |  97 +++++----
>  fs/nfs/nfs4state.c                     |   8 +
>  fs/nfs/nfs4xdr.c                       |  65 +++++-
>  fs/nfs/pnfs.c                          | 266 +++++++++++++++++++++----
>  fs/nfs/pnfs.h                          |  28 ++-
>  include/linux/nfs_fs_sb.h              |   2 +
>  include/linux/nfs_xdr.h                |   4 +
>  10 files changed, 398 insertions(+), 100 deletions(-)
>
> --
> 2.17.0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/19] Layoutget on OPEN

[Trond Myklebust]

T24gV2VkLCAyMDE4LTA1LTMwIGF0IDE0OjI1IC0wNDAwLCBPbGdhIEtvcm5pZXZza2FpYSB3cm90
ZToNCj4gSGkgVHJvbmQsDQo+IA0KPiBEbyB5b3UgYnkgYW55IGNoYW5jZSBoYXZlIHRoaXMgcGF0
Y2ggc2V0IHNvbWV3aGVyZSBpbiB5b3VyIGdpdA0KPiBicmFuY2hlcz8gSSdkIGxpa2UgdG8gdGVz
dCB0aGlzIGFnYWluc3QgTmV0QXBwLg0KPiANCj4gVGhhbmsgeW91Lg0KPiANCg0KSSd2ZSBqdXN0
IHB1c2hlZCBpdCBvdXQgdG8gbXkgdGVzdGluZyBicmFuY2ggb24gZ2l0LmxpbnV4LW5mcy5vcmcu
DQoNCj4gT24gV2VkLCBNYXkgMzAsIDIwMTggYXQgMjowNSBQTSwgVHJvbmQgTXlrbGVidXN0IDx0
cm9uZG15QGdtYWlsLmNvbT4NCj4gd3JvdGU6DQo+ID4gVGhpcyBwYXRjaHNldCBpcyBzb21ldGhp
bmcgd2UndmUgYmVlbiBzaXR0aW5nIG9uIGZvciBhIGNvdXBsZSBvZg0KPiA+IHllYXJzDQo+ID4g
aW4gb3JkZXIgdG8gYWxsb3cgaXQgdG8gbWF0dXJlLiBJdCB3YXMgbWFpbmx5IGRldmVsb3BlZCBi
eSBGcmVkDQo+ID4gSXNhbWFuDQo+ID4gd2hpbGUgaGUgd2FzIHdvcmtpbmcgZm9yIFByaW1hcnkg
RGF0YSwgYnV0IHdlJ3ZlIGhhZCBhIGZldyBidWdmaXhlcw0KPiA+IGNvbWUgaW4gc2luY2UgdGhl
bi4NCj4gPiBUaGUgbWFpbiByZWFzb24gZm9yIGRlbGF5aW5nIHRoZSB1cHN0cmVhbSBtZXJnZSB3
YXMgdG8gZW5zdXJlIHRoYXQNCj4gPiBhZGRpbmcgbGF5b3V0Z2V0IHRvIHRoZSBPUEVOIGNvbXBv
dW5kIHdhcyBzYWZlIChkb2VzIG5vdCBjYXVzZQ0KPiA+IGV4aXN0aW5nDQo+ID4gbGF5b3V0cyB0
byBiZSBhdXRvbWF0aWNhbGx5IHJldm9rZWQpIGFuZCB0aGF0IGl0IGdpdmVzIHVzIGEgcmVhbA0K
PiA+IHBlcmZvcm1hbmNlIGJlbmVmaXQuDQo+ID4gDQo+ID4gRnJlZCBJc2FtYW4gKDE0KToNCj4g
PiAgIHBuZnM6IFJlbW92ZSByZWR1bmRhbnQgYXNzaWdubWVudCBmcm9tIG5mczRfcHJvY19sYXlv
dXRnZXQoKS4NCj4gPiAgIHBuZnM6IFN0b3JlIHJldHVybiB2YWx1ZSBvZiBkZWNvZGVfbGF5b3V0
Z2V0IGZvciBsYXRlciBwcm9jZXNzaW5nDQo+ID4gICBORlM0OiBtb3ZlIGN0eCBpbnRvIG5mczRf
cnVuX29wZW5fdGFzaw0KPiA+ICAgcG5mczogQWRkIGxheW91dCBkcml2ZXIgZmxhZyBQTkZTX0xB
WU9VVEdFVF9PTl9PUEVODQo+ID4gICBwbmZzOiByZWZhY3RvciBzZW5kX2xheW91dGdldA0KPiA+
ICAgcG5mczogbW92ZSBhbGxvY2F0aW9ucyBvdXQgb2YgbmZzNF9wcm9jX2xheW91dGdldA0KPiA+
ICAgcG5mczogQWRkIGNvbmRpdGlvbmFsIGVuY29kZS9kZWNvZGUgb2YgTEFZT1VUR0VUIHdpdGhp
biBPUEVODQo+ID4gY29tcG91bmQNCj4gPiAgIHBuZnM6IE1vdmUgbmZzNF9vcGVuZGF0YSBpbnRv
IG5mczRfZnMuaA0KPiA+ICAgcG5mczogQ2hhbmdlIHBuZnNfYWxsb2NfaW5pdF9sYXlvdXRnZXRf
YXJncyBjYWxsIHNpZ25hdHVyZQ0KPiA+ICAgcG5mczogQWRkIExBWU9VVEdFVCB0byBPUEVOIG9m
IGEgbmV3IGZpbGUNCj4gPiAgIHBuZnM6IEFkZCBMQVlPVVRHRVQgdG8gT1BFTiBvZiBhbiBleGlz
dGluZyBmaWxlDQo+ID4gICBwbmZzOiBTdG9wIGF0dGVtcHRpbmcgTEFZT1VUR0VUIG9uIE9QRU4g
b24gZmFpbHVyZQ0KPiA+ICAgcG5mczogQWRkIGJhcnJpZXIgdG8gcHJldmVudCBsZ29wZW4gdXNp
bmcgTEFZT1VUR0VUIGR1cmluZyByZWNhbGwNCj4gPiAgIHBuZnM6IEZpeCBtYW5pcHVsYXRpb24g
b2YgTkZTX0xBWU9VVF9GSVJTVF9MQVlPVVRHRVQNCj4gPiANCj4gPiBUcm9uZCBNeWtsZWJ1c3Qg
KDUpOg0KPiA+ICAgTkZTdjQvcG5mczogRW5zdXJlIHBuZnNfcGFyc2VfbGdvcGVuKCkgd29uJ3Qg
dHJ5IHRvIHBhcnNlDQo+ID4gICAgIHVuaW5pdGlhbGlzZWQgZGF0YQ0KPiA+ICAgTkZTdjQvcG5m
czogRG9uJ3Qgc3dpdGNoIG9mZiBsYXlvdXRnZXQtb24tb3BlbiBmb3IgdHJhbnNpZW50DQo+ID4g
ZXJyb3JzDQo+ID4gICBwTkZTOiBEb24ndCBzZW5kIExBWU9VVEdFVCBvbiBPUEVOIGZvciByZWFk
LCBpZiB3ZSBhbHJlYWR5IGhhdmUNCj4gPiBjYWNoZWQNCj4gPiAgICAgZGF0YQ0KPiA+ICAgcG5m
czogRG9uJ3QgY2FsbCBjb21taXQgb24gZmFpbGVkIGxheW91dGdldC1vbi1vcGVuDQo+ID4gICBw
bmZzOiBEb24ndCByZWxlYXNlIHRoZSBzZXF1ZW5jZSBzbG90IHVudGlsIHdlJ3ZlIHByb2Nlc3Nl
ZA0KPiA+IGxheW91dGdldA0KPiA+ICAgICBvbiBvcGVuDQo+ID4gDQo+ID4gIGZzL25mcy9jYWxs
YmFja19wcm9jLmMgICAgICAgICAgICAgICAgIHwgICAyICsNCj4gPiAgZnMvbmZzL2ZsZXhmaWxl
bGF5b3V0L2ZsZXhmaWxlbGF5b3V0LmMgfCAgIDEgKw0KPiA+ICBmcy9uZnMvbmZzNF9mcy5oICAg
ICAgICAgICAgICAgICAgICAgICB8ICAyNSArKysNCj4gPiAgZnMvbmZzL25mczRwcm9jLmMgICAg
ICAgICAgICAgICAgICAgICAgfCAgOTcgKysrKystLS0tDQo+ID4gIGZzL25mcy9uZnM0c3RhdGUu
YyAgICAgICAgICAgICAgICAgICAgIHwgICA4ICsNCj4gPiAgZnMvbmZzL25mczR4ZHIuYyAgICAg
ICAgICAgICAgICAgICAgICAgfCAgNjUgKysrKystDQo+ID4gIGZzL25mcy9wbmZzLmMgICAgICAg
ICAgICAgICAgICAgICAgICAgIHwgMjY2DQo+ID4gKysrKysrKysrKysrKysrKysrKysrLS0tLQ0K
PiA+ICBmcy9uZnMvcG5mcy5oICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAyOCArKy0NCj4g
PiAgaW5jbHVkZS9saW51eC9uZnNfZnNfc2IuaCAgICAgICAgICAgICAgfCAgIDIgKw0KPiA+ICBp
bmNsdWRlL2xpbnV4L25mc194ZHIuaCAgICAgICAgICAgICAgICB8ICAgNCArDQo+ID4gIDEwIGZp
bGVzIGNoYW5nZWQsIDM5OCBpbnNlcnRpb25zKCspLCAxMDAgZGVsZXRpb25zKC0pDQo+ID4gDQo+
ID4gLS0NCj4gPiAyLjE3LjANCj4gPiANCj4gPiAtLQ0KPiA+IFRvIHVuc3Vic2NyaWJlIGZyb20g
dGhpcyBsaXN0OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC0NCj4gPiBuZnMiIGlu
DQo+ID4gdGhlIGJvZHkgb2YgYSBtZXNzYWdlIHRvIG1ham9yZG9tb0B2Z2VyLmtlcm5lbC5vcmcN
Cj4gPiBNb3JlIG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3JnL21ham9y
ZG9tby1pbmZvLmh0bWwNCj4gDQo+IC0tDQo+IFRvIHVuc3Vic2NyaWJlIGZyb20gdGhpcyBsaXN0
OiBzZW5kIHRoZSBsaW5lICJ1bnN1YnNjcmliZSBsaW51eC1uZnMiDQo+IGluDQo+IHRoZSBib2R5
IG9mIGEgbWVzc2FnZSB0byBtYWpvcmRvbW9Admdlci5rZXJuZWwub3JnDQo+IE1vcmUgbWFqb3Jk
b21vIGluZm8gYXQgIGh0dHA6Ly92Z2VyLmtlcm5lbC5vcmcvbWFqb3Jkb21vLWluZm8uaHRtbA0K
LS0gDQpUcm9uZCBNeWtsZWJ1c3QNCkxpbnV4IE5GUyBjbGllbnQgbWFpbnRhaW5lciwgSGFtbWVy
c3BhY2UNCnRyb25kLm15a2xlYnVzdEBoYW1tZXJzcGFjZS5jb20NCg0K

Re: [PATCH 00/19] Layoutget on OPEN

[Olga Kornievskaia]

Awesome thanks.

On Wed, May 30, 2018 at 2:36 PM, Trond Myklebust
<trondmy@hammerspace.com> wrote:
> On Wed, 2018-05-30 at 14:25 -0400, Olga Kornievskaia wrote:
>> Hi Trond,
>>
>> Do you by any chance have this patch set somewhere in your git
>> branches? I'd like to test this against NetApp.
>>
>> Thank you.
>>
>
> I've just pushed it out to my testing branch on git.linux-nfs.org.
>
>> On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com>
>> wrote:
>> > This patchset is something we've been sitting on for a couple of
>> > years
>> > in order to allow it to mature. It was mainly developed by Fred
>> > Isaman
>> > while he was working for Primary Data, but we've had a few bugfixes
>> > come in since then.
>> > The main reason for delaying the upstream merge was to ensure that
>> > adding layoutget to the OPEN compound was safe (does not cause
>> > existing
>> > layouts to be automatically revoked) and that it gives us a real
>> > performance benefit.
>> >
>> > Fred Isaman (14):
>> >   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>> >   pnfs: Store return value of decode_layoutget for later processing
>> >   NFS4: move ctx into nfs4_run_open_task
>> >   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>> >   pnfs: refactor send_layoutget
>> >   pnfs: move allocations out of nfs4_proc_layoutget
>> >   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN
>> > compound
>> >   pnfs: Move nfs4_opendata into nfs4_fs.h
>> >   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>> >   pnfs: Add LAYOUTGET to OPEN of a new file
>> >   pnfs: Add LAYOUTGET to OPEN of an existing file
>> >   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>> >   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>> >   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
>> >
>> > Trond Myklebust (5):
>> >   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>> >     uninitialised data
>> >   NFSv4/pnfs: Don't switch off layoutget-on-open for transient
>> > errors
>> >   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have
>> > cached
>> >     data
>> >   pnfs: Don't call commit on failed layoutget-on-open
>> >   pnfs: Don't release the sequence slot until we've processed
>> > layoutget
>> >     on open
>> >
>> >  fs/nfs/callback_proc.c                 |   2 +
>> >  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>> >  fs/nfs/nfs4_fs.h                       |  25 +++
>> >  fs/nfs/nfs4proc.c                      |  97 +++++----
>> >  fs/nfs/nfs4state.c                     |   8 +
>> >  fs/nfs/nfs4xdr.c                       |  65 +++++-
>> >  fs/nfs/pnfs.c                          | 266
>> > +++++++++++++++++++++----
>> >  fs/nfs/pnfs.h                          |  28 ++-
>> >  include/linux/nfs_fs_sb.h              |   2 +
>> >  include/linux/nfs_xdr.h                |   4 +
>> >  10 files changed, 398 insertions(+), 100 deletions(-)
>> >
>> > --
>> > 2.17.0
>> >
>> > --
>> > To unsubscribe from this list: send the line "unsubscribe linux-
>> > nfs" in
>> > the body of a message to majordomo@vger.kernel.org
>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>> in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> Trond Myklebust
> Linux NFS client maintainer, Hammerspace
> trond.myklebust@hammerspace.com
>

Re: [PATCH 00/19] Layoutget on OPEN

[Olga Kornievskaia]

It doesn't compile. Am I missing something?

ERROR: "nfs4_inode_make_writeable" [fs/nfs/nfs.ko] undefined!
ERROR: "nfs4_inode_return_delegation" [fs/nfs/nfs.ko] undefined!

On Wed, May 30, 2018 at 2:37 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
> Awesome thanks.
>
> On Wed, May 30, 2018 at 2:36 PM, Trond Myklebust
> <trondmy@hammerspace.com> wrote:
>> On Wed, 2018-05-30 at 14:25 -0400, Olga Kornievskaia wrote:
>>> Hi Trond,
>>>
>>> Do you by any chance have this patch set somewhere in your git
>>> branches? I'd like to test this against NetApp.
>>>
>>> Thank you.
>>>
>>
>> I've just pushed it out to my testing branch on git.linux-nfs.org.
>>
>>> On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com>
>>> wrote:
>>> > This patchset is something we've been sitting on for a couple of
>>> > years
>>> > in order to allow it to mature. It was mainly developed by Fred
>>> > Isaman
>>> > while he was working for Primary Data, but we've had a few bugfixes
>>> > come in since then.
>>> > The main reason for delaying the upstream merge was to ensure that
>>> > adding layoutget to the OPEN compound was safe (does not cause
>>> > existing
>>> > layouts to be automatically revoked) and that it gives us a real
>>> > performance benefit.
>>> >
>>> > Fred Isaman (14):
>>> >   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>>> >   pnfs: Store return value of decode_layoutget for later processing
>>> >   NFS4: move ctx into nfs4_run_open_task
>>> >   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>>> >   pnfs: refactor send_layoutget
>>> >   pnfs: move allocations out of nfs4_proc_layoutget
>>> >   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN
>>> > compound
>>> >   pnfs: Move nfs4_opendata into nfs4_fs.h
>>> >   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>>> >   pnfs: Add LAYOUTGET to OPEN of a new file
>>> >   pnfs: Add LAYOUTGET to OPEN of an existing file
>>> >   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>>> >   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>>> >   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
>>> >
>>> > Trond Myklebust (5):
>>> >   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>>> >     uninitialised data
>>> >   NFSv4/pnfs: Don't switch off layoutget-on-open for transient
>>> > errors
>>> >   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have
>>> > cached
>>> >     data
>>> >   pnfs: Don't call commit on failed layoutget-on-open
>>> >   pnfs: Don't release the sequence slot until we've processed
>>> > layoutget
>>> >     on open
>>> >
>>> >  fs/nfs/callback_proc.c                 |   2 +
>>> >  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>>> >  fs/nfs/nfs4_fs.h                       |  25 +++
>>> >  fs/nfs/nfs4proc.c                      |  97 +++++----
>>> >  fs/nfs/nfs4state.c                     |   8 +
>>> >  fs/nfs/nfs4xdr.c                       |  65 +++++-
>>> >  fs/nfs/pnfs.c                          | 266
>>> > +++++++++++++++++++++----
>>> >  fs/nfs/pnfs.h                          |  28 ++-
>>> >  include/linux/nfs_fs_sb.h              |   2 +
>>> >  include/linux/nfs_xdr.h                |   4 +
>>> >  10 files changed, 398 insertions(+), 100 deletions(-)
>>> >
>>> > --
>>> > 2.17.0
>>> >
>>> > --
>>> > To unsubscribe from this list: send the line "unsubscribe linux-
>>> > nfs" in
>>> > the body of a message to majordomo@vger.kernel.org
>>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>> in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> --
>> Trond Myklebust
>> Linux NFS client maintainer, Hammerspace
>> trond.myklebust@hammerspace.com
>>

Re: [PATCH 00/19] Layoutget on OPEN

[Olga Kornievskaia]

I added EXPORT_SYMBOL_GPL for those function to get passed it.

On Wed, May 30, 2018 at 3:27 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
> It doesn't compile. Am I missing something?
>
> ERROR: "nfs4_inode_make_writeable" [fs/nfs/nfs.ko] undefined!
> ERROR: "nfs4_inode_return_delegation" [fs/nfs/nfs.ko] undefined!
>
> On Wed, May 30, 2018 at 2:37 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
>> Awesome thanks.
>>
>> On Wed, May 30, 2018 at 2:36 PM, Trond Myklebust
>> <trondmy@hammerspace.com> wrote:
>>> On Wed, 2018-05-30 at 14:25 -0400, Olga Kornievskaia wrote:
>>>> Hi Trond,
>>>>
>>>> Do you by any chance have this patch set somewhere in your git
>>>> branches? I'd like to test this against NetApp.
>>>>
>>>> Thank you.
>>>>
>>>
>>> I've just pushed it out to my testing branch on git.linux-nfs.org.
>>>
>>>> On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com>
>>>> wrote:
>>>> > This patchset is something we've been sitting on for a couple of
>>>> > years
>>>> > in order to allow it to mature. It was mainly developed by Fred
>>>> > Isaman
>>>> > while he was working for Primary Data, but we've had a few bugfixes
>>>> > come in since then.
>>>> > The main reason for delaying the upstream merge was to ensure that
>>>> > adding layoutget to the OPEN compound was safe (does not cause
>>>> > existing
>>>> > layouts to be automatically revoked) and that it gives us a real
>>>> > performance benefit.
>>>> >
>>>> > Fred Isaman (14):
>>>> >   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>>>> >   pnfs: Store return value of decode_layoutget for later processing
>>>> >   NFS4: move ctx into nfs4_run_open_task
>>>> >   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>>>> >   pnfs: refactor send_layoutget
>>>> >   pnfs: move allocations out of nfs4_proc_layoutget
>>>> >   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN
>>>> > compound
>>>> >   pnfs: Move nfs4_opendata into nfs4_fs.h
>>>> >   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>>>> >   pnfs: Add LAYOUTGET to OPEN of a new file
>>>> >   pnfs: Add LAYOUTGET to OPEN of an existing file
>>>> >   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>>>> >   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>>>> >   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
>>>> >
>>>> > Trond Myklebust (5):
>>>> >   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>>>> >     uninitialised data
>>>> >   NFSv4/pnfs: Don't switch off layoutget-on-open for transient
>>>> > errors
>>>> >   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have
>>>> > cached
>>>> >     data
>>>> >   pnfs: Don't call commit on failed layoutget-on-open
>>>> >   pnfs: Don't release the sequence slot until we've processed
>>>> > layoutget
>>>> >     on open
>>>> >
>>>> >  fs/nfs/callback_proc.c                 |   2 +
>>>> >  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>>>> >  fs/nfs/nfs4_fs.h                       |  25 +++
>>>> >  fs/nfs/nfs4proc.c                      |  97 +++++----
>>>> >  fs/nfs/nfs4state.c                     |   8 +
>>>> >  fs/nfs/nfs4xdr.c                       |  65 +++++-
>>>> >  fs/nfs/pnfs.c                          | 266
>>>> > +++++++++++++++++++++----
>>>> >  fs/nfs/pnfs.h                          |  28 ++-
>>>> >  include/linux/nfs_fs_sb.h              |   2 +
>>>> >  include/linux/nfs_xdr.h                |   4 +
>>>> >  10 files changed, 398 insertions(+), 100 deletions(-)
>>>> >
>>>> > --
>>>> > 2.17.0
>>>> >
>>>> > --
>>>> > To unsubscribe from this list: send the line "unsubscribe linux-
>>>> > nfs" in
>>>> > the body of a message to majordomo@vger.kernel.org
>>>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>> in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> --
>>> Trond Myklebust
>>> Linux NFS client maintainer, Hammerspace
>>> trond.myklebust@hammerspace.com
>>>

Re: [PATCH 00/19] Layoutget on OPEN

[Olga Kornievskaia]

I get a kernel oops

[  101.881058] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000088
[  101.884302] PGD 800000002e0dc067 P4D 800000002e0dc067 PUD 2e0dd067 PMD 0
[  101.886773] Oops: 0000 [#1] SMP PTI
[  101.888026] Modules linked in: nfs_layout_nfsv41_files
rpcsec_gss_krb5 nfsv4 dns_resolver nfs ib_iser libiscsi
scsi_transport_iscsi ib_srp scsi_transport_srp ib_ipoib rdma_ucm
ib_ucm ib_uverbs ib_umad rpcrdma rdma_cm iw_cm ib_cm ib_core rfcomm
fuse ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT
nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute bridge stp llc
ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6
ip6table_mangle ip6table_security ip6table_raw iptable_nat
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
iptable_mangle iptable_security iptable_raw ebtable_filter ebtables
ip6table_filter ip6_tables iptable_filter bnep snd_seq_midi
snd_seq_midi_event coretemp crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel pcbc btusb btrtl uvcvideo btbcm snd_ens1371
btintel
[  101.913196]  snd_ac97_codec bluetooth ac97_bus aesni_intel snd_seq
videobuf2_vmalloc crypto_simd videobuf2_memops videobuf2_v4l2 cryptd
glue_helper videodev vmw_balloon snd_pcm pcspkr ext4 snd_timer mbcache
jbd2 snd_rawmidi videobuf2_common snd_seq_device snd rfkill
ecdh_generic vmw_vmci shpchp soundcore i2c_piix4 nfsd nfs_acl lockd
auth_rpcgss grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom
ata_generic vmwgfx pata_acpi drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm sd_mod drm crc32c_intel ahci ata_piix
libahci serio_raw libata mptspi scsi_transport_spi e1000 i2c_core
mptscsih mptbase dm_mirror dm_region_hash dm_log dm_mod
[  101.934175] CPU: 0 PID: 2485 Comm: bash Tainted: G        W
4.17.0-rc7+ #111
[  101.938064] Hardware name: VMware, Inc. VMware Virtual
Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
[  101.943395] RIP: 0010:fl_pnfs_update_layout.constprop.22+0x51/0x150
[nfs_layout_nfsv41_files]
[  101.947586] RSP: 0018:ffffb00443027ac0 EFLAGS: 00010246
[  101.949513] RAX: 0000000000000000 RBX: ffff9739bada46c0 RCX: 00000001802a0028
[  101.952094] RDX: 00000001802a0029 RSI: 0000000000000000 RDI: 0000000040000000
[  101.954862] RBP: ffff97396cd70608 R08: ffff9739b3b2d200 R09: 00000001802a0028
[  101.957999] R10: 00000000b3b2d101 R11: ffff9739b3b2d200 R12: 0000000001400040
[  101.960552] R13: 0000000000000000 R14: ffffb00443027d90 R15: ffff9739b9126000
[  101.963083] FS:  00007f31d31b0740(0000) GS:ffff9739bb600000(0000)
knlGS:0000000000000000
[  101.966021] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  101.968468] CR2: 0000000000000088 CR3: 000000002e0f0006 CR4: 00000000001606f0
[  101.971240] Call Trace:
[  101.972178]  filelayout_pg_init_write+0x223/0x270 [nfs_layout_nfsv41_files]
[  101.974741]  __nfs_pageio_add_request+0x117/0x3c0 [nfs]
[  101.976637]  ? radix_tree_tag_clear+0x9c/0xd0
[  101.978230]  nfs_pageio_add_request+0x17f/0x310 [nfs]
[  101.980123]  nfs_do_writepage+0x18e/0x390 [nfs]
[  101.981774]  nfs_writepages_callback+0xf/0x20 [nfs]
[  101.983526]  write_cache_pages+0x21f/0x460
[  101.985077]  ? nfs_do_writepage+0x390/0x390 [nfs]
[  101.986853]  ? wb_wakeup_delayed+0x52/0x60
[  101.988315]  ? __set_page_dirty_nobuffers+0x119/0x180
[  101.990125]  ? _cond_resched+0x15/0x30
[  101.991481]  ? kmem_cache_alloc_trace+0xae/0x1c0
[  101.993151]  nfs_writepages+0xb2/0x120 [nfs]
[  101.994705]  do_writepages+0x17/0x70
[  101.996012]  __filemap_fdatawrite_range+0xc5/0x100
[  101.997727]  filemap_write_and_wait_range+0x30/0x80
[  101.999453]  nfs_file_fsync+0x39/0x1f0 [nfs]
[  102.000991]  filp_close+0x2a/0x70
[  102.002176]  do_dup2+0x79/0xd0
[  102.003293]  __x64_sys_dup2+0x1a/0x60
[  102.004639]  do_syscall_64+0x55/0x180
[  102.005951]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  102.007771] RIP: 0033:0x7f31d289ff17
[  102.009093] RSP: 002b:00007fff269668b8 EFLAGS: 00000246 ORIG_RAX:
0000000000000021
[  102.011772] RAX: ffffffffffffffda RBX: 0000000002644790 RCX: 00007f31d289ff17
[  102.014323] RDX: 0000000000000001 RSI: 0000000000000001 RDI: 000000000000000a
[  102.016852] RBP: 00007fff26966940 R08: 000000000000000a R09: 000000000263a250
[  102.019380] R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000007
[  102.021928] R13: 0000000000000009 R14: 0000000000000001 R15: 0000000000000000
[  102.024454] Code: b6 fc f8 ff 48 85 c0 48 89 c3 74 13 48 3d 00 f0
ff ff 77 0b 48 83 7b 78 00 48 8b 45 e8 74 0c 48 83 c4 08 48 89 d8 5b
5d 41 5c c3 <48> 8b 90 88 00 00 00 48 8b 80 90 00 00 00 48 8d 73 68 44
89 e1
[  102.031048] RIP: fl_pnfs_update_layout.constprop.22+0x51/0x150
[nfs_layout_nfsv41_files] RSP: ffffb00443027ac0
[  102.034692] CR2: 0000000000000088
[  102.036225] ---[ end trace ae6b78ef3992a438 ]---
[  102.037969] Kernel panic - not syncing: Fatal exception
[  102.039901] Kernel Offset: 0x36600000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[  102.043717] ---[ end Kernel panic - not syncing: Fatal exception ]---

On Wed, May 30, 2018 at 3:29 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
> I added EXPORT_SYMBOL_GPL for those function to get passed it.
>
> On Wed, May 30, 2018 at 3:27 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
>> It doesn't compile. Am I missing something?
>>
>> ERROR: "nfs4_inode_make_writeable" [fs/nfs/nfs.ko] undefined!
>> ERROR: "nfs4_inode_return_delegation" [fs/nfs/nfs.ko] undefined!
>>
>> On Wed, May 30, 2018 at 2:37 PM, Olga Kornievskaia <aglo@umich.edu> wrote:
>>> Awesome thanks.
>>>
>>> On Wed, May 30, 2018 at 2:36 PM, Trond Myklebust
>>> <trondmy@hammerspace.com> wrote:
>>>> On Wed, 2018-05-30 at 14:25 -0400, Olga Kornievskaia wrote:
>>>>> Hi Trond,
>>>>>
>>>>> Do you by any chance have this patch set somewhere in your git
>>>>> branches? I'd like to test this against NetApp.
>>>>>
>>>>> Thank you.
>>>>>
>>>>
>>>> I've just pushed it out to my testing branch on git.linux-nfs.org.
>>>>
>>>>> On Wed, May 30, 2018 at 2:05 PM, Trond Myklebust <trondmy@gmail.com>
>>>>> wrote:
>>>>> > This patchset is something we've been sitting on for a couple of
>>>>> > years
>>>>> > in order to allow it to mature. It was mainly developed by Fred
>>>>> > Isaman
>>>>> > while he was working for Primary Data, but we've had a few bugfixes
>>>>> > come in since then.
>>>>> > The main reason for delaying the upstream merge was to ensure that
>>>>> > adding layoutget to the OPEN compound was safe (does not cause
>>>>> > existing
>>>>> > layouts to be automatically revoked) and that it gives us a real
>>>>> > performance benefit.
>>>>> >
>>>>> > Fred Isaman (14):
>>>>> >   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>>>>> >   pnfs: Store return value of decode_layoutget for later processing
>>>>> >   NFS4: move ctx into nfs4_run_open_task
>>>>> >   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>>>>> >   pnfs: refactor send_layoutget
>>>>> >   pnfs: move allocations out of nfs4_proc_layoutget
>>>>> >   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN
>>>>> > compound
>>>>> >   pnfs: Move nfs4_opendata into nfs4_fs.h
>>>>> >   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>>>>> >   pnfs: Add LAYOUTGET to OPEN of a new file
>>>>> >   pnfs: Add LAYOUTGET to OPEN of an existing file
>>>>> >   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>>>>> >   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>>>>> >   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
>>>>> >
>>>>> > Trond Myklebust (5):
>>>>> >   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>>>>> >     uninitialised data
>>>>> >   NFSv4/pnfs: Don't switch off layoutget-on-open for transient
>>>>> > errors
>>>>> >   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have
>>>>> > cached
>>>>> >     data
>>>>> >   pnfs: Don't call commit on failed layoutget-on-open
>>>>> >   pnfs: Don't release the sequence slot until we've processed
>>>>> > layoutget
>>>>> >     on open
>>>>> >
>>>>> >  fs/nfs/callback_proc.c                 |   2 +
>>>>> >  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>>>>> >  fs/nfs/nfs4_fs.h                       |  25 +++
>>>>> >  fs/nfs/nfs4proc.c                      |  97 +++++----
>>>>> >  fs/nfs/nfs4state.c                     |   8 +
>>>>> >  fs/nfs/nfs4xdr.c                       |  65 +++++-
>>>>> >  fs/nfs/pnfs.c                          | 266
>>>>> > +++++++++++++++++++++----
>>>>> >  fs/nfs/pnfs.h                          |  28 ++-
>>>>> >  include/linux/nfs_fs_sb.h              |   2 +
>>>>> >  include/linux/nfs_xdr.h                |   4 +
>>>>> >  10 files changed, 398 insertions(+), 100 deletions(-)
>>>>> >
>>>>> > --
>>>>> > 2.17.0
>>>>> >
>>>>> > --
>>>>> > To unsubscribe from this list: send the line "unsubscribe linux-
>>>>> > nfs" in
>>>>> > the body of a message to majordomo@vger.kernel.org
>>>>> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>>> in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>> --
>>>> Trond Myklebust
>>>> Linux NFS client maintainer, Hammerspace
>>>> trond.myklebust@hammerspace.com
>>>>

Re: [PATCH 00/19] Layoutget on OPEN

[Trond Myklebust]

T24gV2VkLCAyMDE4LTA1LTMwIGF0IDE1OjQ3IC0wNDAwLCBPbGdhIEtvcm5pZXZza2FpYSB3cm90
ZToNCj4gSSBnZXQgYSBrZXJuZWwgb29wcw0KPiANCj4gWyAgMTAxLjg4MTA1OF0gQlVHOiB1bmFi
bGUgdG8gaGFuZGxlIGtlcm5lbCBOVUxMIHBvaW50ZXIgZGVyZWZlcmVuY2UNCj4gYXQgMDAwMDAw
MDAwMDAwMDA4OA0KPiBbICAxMDEuODg0MzAyXSBQR0QgODAwMDAwMDAyZTBkYzA2NyBQNEQgODAw
MDAwMDAyZTBkYzA2NyBQVUQgMmUwZGQwNjcNCj4gUE1EIDANCj4gWyAgMTAxLjg4Njc3M10gT29w
czogMDAwMCBbIzFdIFNNUCBQVEkNCj4gWyAgMTAxLjg4ODAyNl0gTW9kdWxlcyBsaW5rZWQgaW46
IG5mc19sYXlvdXRfbmZzdjQxX2ZpbGVzDQo+IHJwY3NlY19nc3Nfa3JiNSBuZnN2NCBkbnNfcmVz
b2x2ZXIgbmZzIGliX2lzZXIgbGliaXNjc2kNCj4gc2NzaV90cmFuc3BvcnRfaXNjc2kgaWJfc3Jw
IHNjc2lfdHJhbnNwb3J0X3NycCBpYl9pcG9pYiByZG1hX3VjbQ0KPiBpYl91Y20gaWJfdXZlcmJz
IGliX3VtYWQgcnBjcmRtYSByZG1hX2NtIGl3X2NtIGliX2NtIGliX2NvcmUgcmZjb21tDQo+IGZ1
c2UgaXA2dF9ycGZpbHRlciBpcHRfUkVKRUNUIG5mX3JlamVjdF9pcHY0IGlwNnRfUkVKRUNUDQo+
IG5mX3JlamVjdF9pcHY2IHh0X2Nvbm50cmFjayBlYnRhYmxlX25hdCBlYnRhYmxlX2Jyb3V0ZSBi
cmlkZ2Ugc3RwIGxsYw0KPiBpcDZ0YWJsZV9uYXQgbmZfY29ubnRyYWNrX2lwdjYgbmZfZGVmcmFn
X2lwdjYgbmZfbmF0X2lwdjYNCj4gaXA2dGFibGVfbWFuZ2xlIGlwNnRhYmxlX3NlY3VyaXR5IGlw
NnRhYmxlX3JhdyBpcHRhYmxlX25hdA0KPiBuZl9jb25udHJhY2tfaXB2NCBuZl9kZWZyYWdfaXB2
NCBuZl9uYXRfaXB2NCBuZl9uYXQgbmZfY29ubnRyYWNrDQo+IGlwdGFibGVfbWFuZ2xlIGlwdGFi
bGVfc2VjdXJpdHkgaXB0YWJsZV9yYXcgZWJ0YWJsZV9maWx0ZXIgZWJ0YWJsZXMNCj4gaXA2dGFi
bGVfZmlsdGVyIGlwNl90YWJsZXMgaXB0YWJsZV9maWx0ZXIgYm5lcCBzbmRfc2VxX21pZGkNCj4g
c25kX3NlcV9taWRpX2V2ZW50IGNvcmV0ZW1wIGNyY3QxMGRpZl9wY2xtdWwgY3JjMzJfcGNsbXVs
DQo+IGdoYXNoX2NsbXVsbmlfaW50ZWwgcGNiYyBidHVzYiBidHJ0bCB1dmN2aWRlbyBidGJjbSBz
bmRfZW5zMTM3MQ0KPiBidGludGVsDQo+IFsgIDEwMS45MTMxOTZdICBzbmRfYWM5N19jb2RlYyBi
bHVldG9vdGggYWM5N19idXMgYWVzbmlfaW50ZWwgc25kX3NlcQ0KPiB2aWRlb2J1ZjJfdm1hbGxv
YyBjcnlwdG9fc2ltZCB2aWRlb2J1ZjJfbWVtb3BzIHZpZGVvYnVmMl92NGwyIGNyeXB0ZA0KPiBn
bHVlX2hlbHBlciB2aWRlb2RldiB2bXdfYmFsbG9vbiBzbmRfcGNtIHBjc3BrciBleHQ0IHNuZF90
aW1lcg0KPiBtYmNhY2hlDQo+IGpiZDIgc25kX3Jhd21pZGkgdmlkZW9idWYyX2NvbW1vbiBzbmRf
c2VxX2RldmljZSBzbmQgcmZraWxsDQo+IGVjZGhfZ2VuZXJpYyB2bXdfdm1jaSBzaHBjaHAgc291
bmRjb3JlIGkyY19waWl4NCBuZnNkIG5mc19hY2wgbG9ja2QNCj4gYXV0aF9ycGNnc3MgZ3JhY2Ug
c3VucnBjIGlwX3RhYmxlcyB4ZnMgbGliY3JjMzJjIHNyX21vZCBjZHJvbQ0KPiBhdGFfZ2VuZXJp
YyB2bXdnZnggcGF0YV9hY3BpIGRybV9rbXNfaGVscGVyIHN5c2NvcHlhcmVhIHN5c2ZpbGxyZWN0
DQo+IHN5c2ltZ2JsdCBmYl9zeXNfZm9wcyB0dG0gc2RfbW9kIGRybSBjcmMzMmNfaW50ZWwgYWhj
aSBhdGFfcGlpeA0KPiBsaWJhaGNpIHNlcmlvX3JhdyBsaWJhdGEgbXB0c3BpIHNjc2lfdHJhbnNw
b3J0X3NwaSBlMTAwMCBpMmNfY29yZQ0KPiBtcHRzY3NpaCBtcHRiYXNlIGRtX21pcnJvciBkbV9y
ZWdpb25faGFzaCBkbV9sb2cgZG1fbW9kDQo+IFsgIDEwMS45MzQxNzVdIENQVTogMCBQSUQ6IDI0
ODUgQ29tbTogYmFzaCBUYWludGVkOiBHICAgICAgICBXDQo+IDQuMTcuMC1yYzcrICMxMTENCj4g
WyAgMTAxLjkzODA2NF0gSGFyZHdhcmUgbmFtZTogVk13YXJlLCBJbmMuIFZNd2FyZSBWaXJ0dWFs
DQo+IFBsYXRmb3JtLzQ0MEJYIERlc2t0b3AgUmVmZXJlbmNlIFBsYXRmb3JtLCBCSU9TIDYuMDAg
MDcvMDIvMjAxNQ0KPiBbICAxMDEuOTQzMzk1XSBSSVA6DQo+IDAwMTA6ZmxfcG5mc191cGRhdGVf
bGF5b3V0LmNvbnN0cHJvcC4yMisweDUxLzB4MTUwDQo+IFtuZnNfbGF5b3V0X25mc3Y0MV9maWxl
c10NCj4gWyAgMTAxLjk0NzU4Nl0gUlNQOiAwMDE4OmZmZmZiMDA0NDMwMjdhYzAgRUZMQUdTOiAw
MDAxMDI0Ng0KPiBbICAxMDEuOTQ5NTEzXSBSQVg6IDAwMDAwMDAwMDAwMDAwMDAgUkJYOiBmZmZm
OTczOWJhZGE0NmMwIFJDWDoNCj4gMDAwMDAwMDE4MDJhMDAyOA0KPiBbICAxMDEuOTUyMDk0XSBS
RFg6IDAwMDAwMDAxODAyYTAwMjkgUlNJOiAwMDAwMDAwMDAwMDAwMDAwIFJESToNCj4gMDAwMDAw
MDA0MDAwMDAwMA0KPiBbICAxMDEuOTU0ODYyXSBSQlA6IGZmZmY5NzM5NmNkNzA2MDggUjA4OiBm
ZmZmOTczOWIzYjJkMjAwIFIwOToNCj4gMDAwMDAwMDE4MDJhMDAyOA0KPiBbICAxMDEuOTU3OTk5
XSBSMTA6IDAwMDAwMDAwYjNiMmQxMDEgUjExOiBmZmZmOTczOWIzYjJkMjAwIFIxMjoNCj4gMDAw
MDAwMDAwMTQwMDA0MA0KPiBbICAxMDEuOTYwNTUyXSBSMTM6IDAwMDAwMDAwMDAwMDAwMDAgUjE0
OiBmZmZmYjAwNDQzMDI3ZDkwIFIxNToNCj4gZmZmZjk3MzliOTEyNjAwMA0KPiBbICAxMDEuOTYz
MDgzXSBGUzogIDAwMDA3ZjMxZDMxYjA3NDAoMDAwMCkgR1M6ZmZmZjk3MzliYjYwMDAwMCgwMDAw
KQ0KPiBrbmxHUzowMDAwMDAwMDAwMDAwMDAwDQo+IFsgIDEwMS45NjYwMjFdIENTOiAgMDAxMCBE
UzogMDAwMCBFUzogMDAwMCBDUjA6IDAwMDAwMDAwODAwNTAwMzMNCj4gWyAgMTAxLjk2ODQ2OF0g
Q1IyOiAwMDAwMDAwMDAwMDAwMDg4IENSMzogMDAwMDAwMDAyZTBmMDAwNiBDUjQ6DQo+IDAwMDAw
MDAwMDAxNjA2ZjANCj4gWyAgMTAxLjk3MTI0MF0gQ2FsbCBUcmFjZToNCj4gWyAgMTAxLjk3MjE3
OF0gIGZpbGVsYXlvdXRfcGdfaW5pdF93cml0ZSsweDIyMy8weDI3MA0KPiBbbmZzX2xheW91dF9u
ZnN2NDFfZmlsZXNdDQo+IFsgIDEwMS45NzQ3NDFdICBfX25mc19wYWdlaW9fYWRkX3JlcXVlc3Qr
MHgxMTcvMHgzYzAgW25mc10NCj4gWyAgMTAxLjk3NjYzN10gID8gcmFkaXhfdHJlZV90YWdfY2xl
YXIrMHg5Yy8weGQwDQo+IFsgIDEwMS45NzgyMzBdICBuZnNfcGFnZWlvX2FkZF9yZXF1ZXN0KzB4
MTdmLzB4MzEwIFtuZnNdDQo+IFsgIDEwMS45ODAxMjNdICBuZnNfZG9fd3JpdGVwYWdlKzB4MThl
LzB4MzkwIFtuZnNdDQo+IFsgIDEwMS45ODE3NzRdICBuZnNfd3JpdGVwYWdlc19jYWxsYmFjaysw
eGYvMHgyMCBbbmZzXQ0KPiBbICAxMDEuOTgzNTI2XSAgd3JpdGVfY2FjaGVfcGFnZXMrMHgyMWYv
MHg0NjANCj4gWyAgMTAxLjk4NTA3N10gID8gbmZzX2RvX3dyaXRlcGFnZSsweDM5MC8weDM5MCBb
bmZzXQ0KPiBbICAxMDEuOTg2ODUzXSAgPyB3Yl93YWtldXBfZGVsYXllZCsweDUyLzB4NjANCj4g
WyAgMTAxLjk4ODMxNV0gID8gX19zZXRfcGFnZV9kaXJ0eV9ub2J1ZmZlcnMrMHgxMTkvMHgxODAN
Cj4gWyAgMTAxLjk5MDEyNV0gID8gX2NvbmRfcmVzY2hlZCsweDE1LzB4MzANCj4gWyAgMTAxLjk5
MTQ4MV0gID8ga21lbV9jYWNoZV9hbGxvY190cmFjZSsweGFlLzB4MWMwDQo+IFsgIDEwMS45OTMx
NTFdICBuZnNfd3JpdGVwYWdlcysweGIyLzB4MTIwIFtuZnNdDQo+IFsgIDEwMS45OTQ3MDVdICBk
b193cml0ZXBhZ2VzKzB4MTcvMHg3MA0KPiBbICAxMDEuOTk2MDEyXSAgX19maWxlbWFwX2ZkYXRh
d3JpdGVfcmFuZ2UrMHhjNS8weDEwMA0KPiBbICAxMDEuOTk3NzI3XSAgZmlsZW1hcF93cml0ZV9h
bmRfd2FpdF9yYW5nZSsweDMwLzB4ODANCj4gWyAgMTAxLjk5OTQ1M10gIG5mc19maWxlX2ZzeW5j
KzB4MzkvMHgxZjAgW25mc10NCj4gWyAgMTAyLjAwMDk5MV0gIGZpbHBfY2xvc2UrMHgyYS8weDcw
DQo+IFsgIDEwMi4wMDIxNzZdICBkb19kdXAyKzB4NzkvMHhkMA0KPiBbICAxMDIuMDAzMjkzXSAg
X194NjRfc3lzX2R1cDIrMHgxYS8weDYwDQo+IFsgIDEwMi4wMDQ2MzldICBkb19zeXNjYWxsXzY0
KzB4NTUvMHgxODANCj4gWyAgMTAyLjAwNTk1MV0gIGVudHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdm
cmFtZSsweDQ0LzB4YTkNCj4gWyAgMTAyLjAwNzc3MV0gUklQOiAwMDMzOjB4N2YzMWQyODlmZjE3
DQo+IFsgIDEwMi4wMDkwOTNdIFJTUDogMDAyYjowMDAwN2ZmZjI2OTY2OGI4IEVGTEFHUzogMDAw
MDAyNDYgT1JJR19SQVg6DQo+IDAwMDAwMDAwMDAwMDAwMjENCj4gWyAgMTAyLjAxMTc3Ml0gUkFY
OiBmZmZmZmZmZmZmZmZmZmRhIFJCWDogMDAwMDAwMDAwMjY0NDc5MCBSQ1g6DQo+IDAwMDA3ZjMx
ZDI4OWZmMTcNCj4gWyAgMTAyLjAxNDMyM10gUkRYOiAwMDAwMDAwMDAwMDAwMDAxIFJTSTogMDAw
MDAwMDAwMDAwMDAwMSBSREk6DQo+IDAwMDAwMDAwMDAwMDAwMGENCj4gWyAgMTAyLjAxNjg1Ml0g
UkJQOiAwMDAwN2ZmZjI2OTY2OTQwIFIwODogMDAwMDAwMDAwMDAwMDAwYSBSMDk6DQo+IDAwMDAw
MDAwMDI2M2EyNTANCj4gWyAgMTAyLjAxOTM4MF0gUjEwOiAwMDAwMDAwMDAwMDAwMDAzIFIxMTog
MDAwMDAwMDAwMDAwMDI0NiBSMTI6DQo+IDAwMDAwMDAwMDAwMDAwMDcNCj4gWyAgMTAyLjAyMTky
OF0gUjEzOiAwMDAwMDAwMDAwMDAwMDA5IFIxNDogMDAwMDAwMDAwMDAwMDAwMSBSMTU6DQo+IDAw
MDAwMDAwMDAwMDAwMDANCj4gWyAgMTAyLjAyNDQ1NF0gQ29kZTogYjYgZmMgZjggZmYgNDggODUg
YzAgNDggODkgYzMgNzQgMTMgNDggM2QgMDAgZjANCj4gZmYgZmYgNzcgMGIgNDggODMgN2IgNzgg
MDAgNDggOGIgNDUgZTggNzQgMGMgNDggODMgYzQgMDggNDggODkgZDggNWINCj4gNWQgNDEgNWMg
YzMgPDQ4PiA4YiA5MCA4OCAwMCAwMCAwMCA0OCA4YiA4MCA5MCAwMCAwMCAwMCA0OCA4ZCA3MyA2
OA0KPiA0NA0KPiA4OSBlMQ0KPiBbICAxMDIuMDMxMDQ4XSBSSVA6IGZsX3BuZnNfdXBkYXRlX2xh
eW91dC5jb25zdHByb3AuMjIrMHg1MS8weDE1MA0KPiBbbmZzX2xheW91dF9uZnN2NDFfZmlsZXNd
IFJTUDogZmZmZmIwMDQ0MzAyN2FjMA0KPiBbICAxMDIuMDM0NjkyXSBDUjI6IDAwMDAwMDAwMDAw
MDAwODgNCj4gWyAgMTAyLjAzNjIyNV0gLS0tWyBlbmQgdHJhY2UgYWU2Yjc4ZWYzOTkyYTQzOCBd
LS0tDQo+IFsgIDEwMi4wMzc5NjldIEtlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBGYXRhbCBl
eGNlcHRpb24NCj4gWyAgMTAyLjAzOTkwMV0gS2VybmVsIE9mZnNldDogMHgzNjYwMDAwMCBmcm9t
IDB4ZmZmZmZmZmY4MTAwMDAwMA0KPiAocmVsb2NhdGlvbiByYW5nZTogMHhmZmZmZmZmZjgwMDAw
MDAwLTB4ZmZmZmZmZmZiZmZmZmZmZikNCj4gWyAgMTAyLjA0MzcxN10gLS0tWyBlbmQgS2VybmVs
IHBhbmljIC0gbm90IHN5bmNpbmc6IEZhdGFsIGV4Y2VwdGlvbg0KPiBdLS0tDQoNClVyZ2guLiBU
aGF0IHdhcyBhIHBvcnRpbmcgZXJyb3I6IE1lYSBjdWxwYS4uLg0KDQpTaG91bGQgYmUgZml4ZWQg
aW4gdGhlIHRlc3RpbmcgYnJhbmNoIG5vdy4NCg0KQ2hlZXJzDQogIFRyb25kDQotLSANClRyb25k
IE15a2xlYnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyLCBIYW1tZXJzcGFjZQ0KdHJv
bmQubXlrbGVidXN0QGhhbW1lcnNwYWNlLmNvbQ0KDQo=

Re: [PATCH 00/19] Layoutget on OPEN

[J. Bruce Fields]

On Wed, May 30, 2018 at 02:05:34PM -0400, Trond Myklebust wrote:
> This patchset is something we've been sitting on for a couple of years
> in order to allow it to mature. It was mainly developed by Fred Isaman
> while he was working for Primary Data, but we've had a few bugfixes
> come in since then.
> The main reason for delaying the upstream merge was to ensure that
> adding layoutget to the OPEN compound was safe (does not cause existing
> layouts to be automatically revoked) and that it gives us a real
> performance benefit.

Do you have any of the performance data?

Or just a rough summary explaining when it helps most and by how much?

--b.

> 
> Fred Isaman (14):
>   pnfs: Remove redundant assignment from nfs4_proc_layoutget().
>   pnfs: Store return value of decode_layoutget for later processing
>   NFS4: move ctx into nfs4_run_open_task
>   pnfs: Add layout driver flag PNFS_LAYOUTGET_ON_OPEN
>   pnfs: refactor send_layoutget
>   pnfs: move allocations out of nfs4_proc_layoutget
>   pnfs: Add conditional encode/decode of LAYOUTGET within OPEN compound
>   pnfs: Move nfs4_opendata into nfs4_fs.h
>   pnfs: Change pnfs_alloc_init_layoutget_args call signature
>   pnfs: Add LAYOUTGET to OPEN of a new file
>   pnfs: Add LAYOUTGET to OPEN of an existing file
>   pnfs: Stop attempting LAYOUTGET on OPEN on failure
>   pnfs: Add barrier to prevent lgopen using LAYOUTGET during recall
>   pnfs: Fix manipulation of NFS_LAYOUT_FIRST_LAYOUTGET
> 
> Trond Myklebust (5):
>   NFSv4/pnfs: Ensure pnfs_parse_lgopen() won't try to parse
>     uninitialised data
>   NFSv4/pnfs: Don't switch off layoutget-on-open for transient errors
>   pNFS: Don't send LAYOUTGET on OPEN for read, if we already have cached
>     data
>   pnfs: Don't call commit on failed layoutget-on-open
>   pnfs: Don't release the sequence slot until we've processed layoutget
>     on open
> 
>  fs/nfs/callback_proc.c                 |   2 +
>  fs/nfs/flexfilelayout/flexfilelayout.c |   1 +
>  fs/nfs/nfs4_fs.h                       |  25 +++
>  fs/nfs/nfs4proc.c                      |  97 +++++----
>  fs/nfs/nfs4state.c                     |   8 +
>  fs/nfs/nfs4xdr.c                       |  65 +++++-
>  fs/nfs/pnfs.c                          | 266 +++++++++++++++++++++----
>  fs/nfs/pnfs.h                          |  28 ++-
>  include/linux/nfs_fs_sb.h              |   2 +
>  include/linux/nfs_xdr.h                |   4 +
>  10 files changed, 398 insertions(+), 100 deletions(-)
> 
> -- 
> 2.17.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/19] Layoutget on OPEN

[Trond Myklebust]

T24gVGh1LCAyMDE4LTA1LTMxIGF0IDE0OjU5IC0wNDAwLCBKLiBCcnVjZSBGaWVsZHMgd3JvdGU6
DQo+IE9uIFdlZCwgTWF5IDMwLCAyMDE4IGF0IDAyOjA1OjM0UE0gLTA0MDAsIFRyb25kIE15a2xl
YnVzdCB3cm90ZToNCj4gPiBUaGlzIHBhdGNoc2V0IGlzIHNvbWV0aGluZyB3ZSd2ZSBiZWVuIHNp
dHRpbmcgb24gZm9yIGEgY291cGxlIG9mDQo+ID4geWVhcnMNCj4gPiBpbiBvcmRlciB0byBhbGxv
dyBpdCB0byBtYXR1cmUuIEl0IHdhcyBtYWlubHkgZGV2ZWxvcGVkIGJ5IEZyZWQNCj4gPiBJc2Ft
YW4NCj4gPiB3aGlsZSBoZSB3YXMgd29ya2luZyBmb3IgUHJpbWFyeSBEYXRhLCBidXQgd2UndmUg
aGFkIGEgZmV3IGJ1Z2ZpeGVzDQo+ID4gY29tZSBpbiBzaW5jZSB0aGVuLg0KPiA+IFRoZSBtYWlu
IHJlYXNvbiBmb3IgZGVsYXlpbmcgdGhlIHVwc3RyZWFtIG1lcmdlIHdhcyB0byBlbnN1cmUgdGhh
dA0KPiA+IGFkZGluZyBsYXlvdXRnZXQgdG8gdGhlIE9QRU4gY29tcG91bmQgd2FzIHNhZmUgKGRv
ZXMgbm90IGNhdXNlDQo+ID4gZXhpc3RpbmcNCj4gPiBsYXlvdXRzIHRvIGJlIGF1dG9tYXRpY2Fs
bHkgcmV2b2tlZCkgYW5kIHRoYXQgaXQgZ2l2ZXMgdXMgYSByZWFsDQo+ID4gcGVyZm9ybWFuY2Ug
YmVuZWZpdC4NCj4gDQo+IERvIHlvdSBoYXZlIGFueSBvZiB0aGUgcGVyZm9ybWFuY2UgZGF0YT8N
Cj4gDQo+IE9yIGp1c3QgYSByb3VnaCBzdW1tYXJ5IGV4cGxhaW5pbmcgd2hlbiBpdCBoZWxwcyBt
b3N0IGFuZCBieSBob3cNCj4gbXVjaD8NCj4gDQoNCkl0J3MgdmVyeSB3b3JrbG9hZCBkZXBlbmRl
bnQ6DQoNCklmIHlvdSBoYXZlIGxhcmdlIGZpbGVzIHRoYXQgYXJlIGJlaW5nIHdyaXR0ZW4gZnJv
bSBzY3JhdGNoLCB0aGVuIHRoZQ0KbGF5b3V0Z2V0IG9uIE9QRU4gbWFrZXMgbm8gZGlmZmVyZW5j
ZSwgc2luY2UgdGhlIG92ZXJoZWFkIG9mIGEgc2luZ2xlDQpsYXlvdXRnZXQgdG8gdGhlIG1ldGFk
YXRhIHNlcnZlciBpcyBxdWlja2x5IGFtb3J0aXplZC4NCg0KVGhlIG1haW4gZGlmZmVyZW5jZSBp
cyB3aXRoIHdvcmtsb2FkcyBpbnZvbHZpbmcgbG90cyBvZiBzbWFsbCBmaWxlcywNCndoZXJlIHlv
dSB3YW50IHRvIG9wZW4sIHRoZW4gc2VuZCBvbmUgb3IgdHdvIHdyaXRlcywgdGhlbiBjbG9zZTsg
aWYgeW91DQpjYW4gZWxpbWluYXRlIG9uZSByb3VuZCB0cmlwIHRvIHRoZSBNRFMsIHRoZW4gdGhh
dCBjYW4gaW1wcm92ZQ0KcGVyZm9ybWFuY2UgYnkgYSBiaXQgKG5vdCBhIGh1Z2UgYW1vdW50KS4g
RGl0dG8gZm9yIHdvcmtsb2FkcyB3aGVyZSB5b3UNCm9wZW4sIHRoZW4gc2VuZCBhIHJlYWQsIHRo
ZW4gY2xvc2UsIGhvd2V2ZXIgcmVhZCBjYWNoaW5nIGNhbiBxdWlja2x5DQplbGltaW5hdGUgYW55
IGFkdmFudGFnZSBvZiBzZW5kaW5nIGEgcHJlLWVtcHRpdmUgbGF5b3V0Z2V0IGhlcmUuDQoNCi0t
IA0KVHJvbmQgTXlrbGVidXN0DQpMaW51eCBORlMgY2xpZW50IG1haW50YWluZXIsIEhhbW1lcnNw
YWNlDQp0cm9uZC5teWtsZWJ1c3RAaGFtbWVyc3BhY2UuY29tDQoNCg==