[PATCH 00/10] orangefs changes for 4.12

[PATCH 00/10] orangefs changes for 4.12

[Martin Brandenburg]

Mike,

Here are the OrangeFS changes I intend for the 4.12 merge window.

The biggest change here is my readdir work.  The old readdir code was
fragile in a number of ways relating to small buffers passed to
getdents.  The new code fixes all of that by separating making server
requests for more data from responding to the user.  OrangeFS cannot
list directories from an arbitrary offset, so this saves the directory
as it is received from the server.  Then it can continue from the point
that the user buffer got too full.  Before if all of the entries did not
fit, it would do a new server request on the next call and skip those
that did not fit.  This also handles directory seeking, which was
completely ignored in the old code.

I increased the number of directory entries to request in a server
request from 96 to 512, which is the OrangeFS client maximum.  The limit
of 96 appears to have been chosen because it is small enough that most
directories will fit in the buffer passed to getdents.

I've included some patches for statx.  Depending on how much testing we
can get done over the next couple weeks, we may want to hold these for
4.13.  It is very helpful to OrangeFS's performance to skip fetching
size if it is not needed.  I can't help but wonder if I've missed
something.  It passes xfstests, but I don't think it's been through all
our internal tests.

Then there are some miscellaneous fixes.

Martin Brandenburg (10):
  orangefs: remove unused get_fsid_from_ino
  orangefs: fix bounds check for listxattr
  orangefs: clean up oversize xattr validation
  orangefs: do not set getattr_time on orangefs_lookup
  orangefs: rewrite readdir to fix several bugs
  orangefs: support llseek on directories
  orangefs: support very large directories
  orangefs: remove ORANGEFS_READDIR macros
  orangefs: implement statx
  orangefs: do not check possibly stale size on truncate

 fs/orangefs/dir.c                | 598 ++++++++++++++++++---------------------
 fs/orangefs/downcall.h           |  21 +-
 fs/orangefs/file.c               |   6 +-
 fs/orangefs/inode.c              |  18 +-
 fs/orangefs/namei.c              |   5 +-
 fs/orangefs/orangefs-dev-proto.h |   7 +-
 fs/orangefs/orangefs-kernel.h    |   9 +-
 fs/orangefs/orangefs-utils.c     |  63 +++--
 fs/orangefs/protocol.h           |   9 +-
 fs/orangefs/xattr.c              |  26 +-
 10 files changed, 363 insertions(+), 399 deletions(-)

-- 
2.1.4

[PATCH 01/10] orangefs: remove unused get_fsid_from_ino

[Martin Brandenburg]

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/orangefs-kernel.h | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/fs/orangefs/orangefs-kernel.h b/fs/orangefs/orangefs-kernel.h
index 5e48a0b..24b0648 100644
--- a/fs/orangefs/orangefs-kernel.h
+++ b/fs/orangefs/orangefs-kernel.h
@@ -339,11 +339,6 @@ static inline struct orangefs_khandle *get_khandle_from_ino(struct inode *inode)
 	return &(ORANGEFS_I(inode)->refn.khandle);
 }
 
-static inline __s32 get_fsid_from_ino(struct inode *inode)
-{
-	return ORANGEFS_I(inode)->refn.fs_id;
-}
-
 static inline ino_t get_ino_from_khandle(struct inode *inode)
 {
 	struct orangefs_khandle *khandle;
-- 
2.1.4

[PATCH 02/10] orangefs: fix bounds check for listxattr

[Martin Brandenburg]

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/xattr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/orangefs/xattr.c b/fs/orangefs/xattr.c
index 74a81b1..fba4db7 100644
--- a/fs/orangefs/xattr.c
+++ b/fs/orangefs/xattr.c
@@ -358,7 +358,7 @@ ssize_t orangefs_listxattr(struct dentry *dentry, char *buffer, size_t size)
 
 	returned_count = new_op->downcall.resp.listxattr.returned_count;
 	if (returned_count < 0 ||
-	    returned_count >= ORANGEFS_MAX_XATTR_LISTLEN) {
+	    returned_count > ORANGEFS_MAX_XATTR_LISTLEN) {
 		gossip_err("%s: impossible value for returned_count:%d:\n",
 		__func__,
 		returned_count);
-- 
2.1.4

[PATCH 03/10] orangefs: clean up oversize xattr validation

[Martin Brandenburg]

Also don't check flags as this has been validated by the VFS already.

Fix an off-by-one error in the max size checking.

Stop logging just because userspace wants to write attributes which do
not fit.

This and the previous commit fix xfstests generic/020.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/xattr.c | 24 +++++++-----------------
 1 file changed, 7 insertions(+), 17 deletions(-)

diff --git a/fs/orangefs/xattr.c b/fs/orangefs/xattr.c
index fba4db7..237c9c0 100644
--- a/fs/orangefs/xattr.c
+++ b/fs/orangefs/xattr.c
@@ -76,11 +76,8 @@ ssize_t orangefs_inode_getxattr(struct inode *inode, const char *name,
 	if (S_ISLNK(inode->i_mode))
 		return -EOPNOTSUPP;
 
-	if (strlen(name) >= ORANGEFS_MAX_XATTR_NAMELEN) {
-		gossip_err("Invalid key length (%d)\n",
-			   (int)strlen(name));
+	if (strlen(name) > ORANGEFS_MAX_XATTR_NAMELEN)
 		return -EINVAL;
-	}
 
 	fsuid = from_kuid(&init_user_ns, current_fsuid());
 	fsgid = from_kgid(&init_user_ns, current_fsgid());
@@ -172,6 +169,9 @@ static int orangefs_inode_removexattr(struct inode *inode, const char *name,
 	struct orangefs_kernel_op_s *new_op = NULL;
 	int ret = -ENOMEM;
 
+	if (strlen(name) > ORANGEFS_MAX_XATTR_NAMELEN)
+		return -EINVAL;
+
 	down_write(&orangefs_inode->xattr_sem);
 	new_op = op_alloc(ORANGEFS_VFS_OP_REMOVEXATTR);
 	if (!new_op)
@@ -231,23 +231,13 @@ int orangefs_inode_setxattr(struct inode *inode, const char *name,
 		     "%s: name %s, buffer_size %zd\n",
 		     __func__, name, size);
 
-	if (size >= ORANGEFS_MAX_XATTR_VALUELEN ||
-	    flags < 0) {
-		gossip_err("orangefs_inode_setxattr: bogus values of size(%d), flags(%d)\n",
-			   (int)size,
-			   flags);
+	if (size > ORANGEFS_MAX_XATTR_VALUELEN)
+		return -EINVAL;
+	if (strlen(name) > ORANGEFS_MAX_XATTR_NAMELEN)
 		return -EINVAL;
-	}
 
 	internal_flag = convert_to_internal_xattr_flags(flags);
 
-	if (strlen(name) >= ORANGEFS_MAX_XATTR_NAMELEN) {
-		gossip_err
-		    ("orangefs_inode_setxattr: bogus key size (%d)\n",
-		     (int)(strlen(name)));
-		return -EINVAL;
-	}
-
 	/* This is equivalent to a removexattr */
 	if (size == 0 && value == NULL) {
 		gossip_debug(GOSSIP_XATTR_DEBUG,
-- 
2.1.4

[PATCH 04/10] orangefs: do not set getattr_time on orangefs_lookup

[Martin Brandenburg]

Since orangefs_lookup calls orangefs_iget which calls
orangefs_inode_getattr, getattr_time will get set.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/namei.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/fs/orangefs/namei.c b/fs/orangefs/namei.c
index a290ff6..7c31593 100644
--- a/fs/orangefs/namei.c
+++ b/fs/orangefs/namei.c
@@ -193,8 +193,6 @@ static struct dentry *orangefs_lookup(struct inode *dir, struct dentry *dentry,
 		goto out;
 	}
 
-	ORANGEFS_I(inode)->getattr_time = jiffies - 1;
-
 	gossip_debug(GOSSIP_NAME_DEBUG,
 		     "%s:%s:%d "
 		     "Found good inode [%lu] with count [%d]\n",
-- 
2.1.4

[PATCH 05/10] orangefs: rewrite readdir to fix several bugs

[Martin Brandenburg]

In the past, readdir assumed that the user buffer will be large enough
that all entries from the server will fit.  If this was not true,
entries would be skipped.

Since it works now, request 512 entries rather than 96 per server
operation.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/dir.c                | 514 ++++++++++++++-------------------------
 fs/orangefs/downcall.h           |  21 +-
 fs/orangefs/orangefs-dev-proto.h |   7 +-
 3 files changed, 188 insertions(+), 354 deletions(-)

diff --git a/fs/orangefs/dir.c b/fs/orangefs/dir.c
index 284373a..3f82a73 100644
--- a/fs/orangefs/dir.c
+++ b/fs/orangefs/dir.c
@@ -1,7 +1,5 @@
 /*
- * (C) 2001 Clemson University and The University of Chicago
- *
- * See COPYING in top-level directory.
+ * Copyright 2017 Omnibond Systems, L.L.C.
  */
 
 #include "protocol.h"
@@ -9,388 +7,230 @@
 #include "orangefs-bufmap.h"
 
 /*
- * decode routine used by kmod to deal with the blob sent from
- * userspace for readdirs. The blob contains zero or more of these
- * sub-blobs:
- *   __u32 - represents length of the character string that follows.
- *   string - between 1 and ORANGEFS_NAME_MAX bytes long.
- *   padding - (if needed) to cause the __u32 plus the string to be
- *             eight byte aligned.
- *   khandle - sizeof(khandle) bytes.
+ * There can be up to 512 directory entries.  Each entry contains a four byte
+ * string size, a string, and a 16 byte khandle.  The string can be up to 256
+ * bytes and is encoded with a trailing zero and four byte padding for the
+ * khandle.
  */
-static long decode_dirents(char *ptr, size_t size,
-                           struct orangefs_readdir_response_s *readdir)
-{
-	int i;
-	struct orangefs_readdir_response_s *rd =
-		(struct orangefs_readdir_response_s *) ptr;
-	char *buf = ptr;
-	int khandle_size = sizeof(struct orangefs_khandle);
-	size_t offset = offsetof(struct orangefs_readdir_response_s,
-				dirent_array);
-	/* 8 reflects eight byte alignment */
-	int smallest_blob = khandle_size + 8;
-	__u32 len;
-	int aligned_len;
-	int sizeof_u32 = sizeof(__u32);
-	long ret;
-
-	gossip_debug(GOSSIP_DIR_DEBUG, "%s: size:%zu:\n", __func__, size);
-
-	/* size is = offset on empty dirs, > offset on non-empty dirs... */
-	if (size < offset) {
-		gossip_err("%s: size:%zu: offset:%zu:\n",
-			   __func__,
-			   size,
-			   offset);
-		ret = -EINVAL;
-		goto out;
-	}
-
-        if ((size == offset) && (readdir->orangefs_dirent_outcount != 0)) {
-		gossip_err("%s: size:%zu: dirent_outcount:%d:\n",
-			   __func__,
-			   size,
-			   readdir->orangefs_dirent_outcount);
-		ret = -EINVAL;
-		goto out;
-	}
-
-	readdir->token = rd->token;
-	readdir->orangefs_dirent_outcount = rd->orangefs_dirent_outcount;
-	readdir->dirent_array = kcalloc(readdir->orangefs_dirent_outcount,
-					sizeof(*readdir->dirent_array),
-					GFP_KERNEL);
-	if (readdir->dirent_array == NULL) {
-		gossip_err("%s: kcalloc failed.\n", __func__);
-		ret = -ENOMEM;
-		goto out;
-	}
-
-	buf += offset;
-	size -= offset;
-
-	for (i = 0; i < readdir->orangefs_dirent_outcount; i++) {
-		if (size < smallest_blob) {
-			gossip_err("%s: size:%zu: smallest_blob:%d:\n",
-				   __func__,
-				   size,
-				   smallest_blob);
-			ret = -EINVAL;
-			goto free;
-		}
-
-		len = *(__u32 *)buf;
-		if ((len < 1) || (len > ORANGEFS_NAME_MAX)) {
-			gossip_err("%s: len:%d:\n", __func__, len);
-			ret = -EINVAL;
-			goto free;
-		}
-
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "%s: size:%zu: len:%d:\n",
-			     __func__,
-			     size,
-			     len);
-
-		readdir->dirent_array[i].d_name = buf + sizeof_u32;
-		readdir->dirent_array[i].d_length = len;
-
-		/*
-		 * Calculate "aligned" length of this string and its
-		 * associated __u32 descriptor.
-		 */
-		aligned_len = ((sizeof_u32 + len + 1) + 7) & ~7;
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "%s: aligned_len:%d:\n",
-			     __func__,
-			     aligned_len);
-
-		/*
-		 * The end of the blob should coincide with the end
-		 * of the last sub-blob.
-		 */
-		if (size < aligned_len + khandle_size) {
-			gossip_err("%s: ran off the end of the blob.\n",
-				   __func__);
-			ret = -EINVAL;
-			goto free;
-		}
-		size -= aligned_len + khandle_size;
+#define MAX_DIRECTORY ((4 + 257 + 3 + 16)*512)
 
-		buf += aligned_len;
-
-		readdir->dirent_array[i].khandle =
-			*(struct orangefs_khandle *) buf;
-		buf += khandle_size;
-	}
-	ret = buf - ptr;
-	gossip_debug(GOSSIP_DIR_DEBUG, "%s: returning:%ld:\n", __func__, ret);
-	goto out;
-
-free:
-	kfree(readdir->dirent_array);
-	readdir->dirent_array = NULL;
-
-out:
-	return ret;
-}
+struct orangefs_dir {
+	__u64 token;
+	void *directory;
+	size_t i, len;
+	int error;
+};
 
 /*
- * Read directory entries from an instance of an open directory.
+ * The userspace component sends several directory entries of the following
+ * format.  The first four bytes are the string length not including a trailing
+ * zero byte.  This is followed by the string and a trailing zero padded to the
+ * next four byte boundry.  This is followed by the sixteen byte khandle padded
+ * to the next eight byte boundry.
+ *
+ * The trailer_buf starts with a struct orangefs_readdir_response_s which must
+ * be skipped to get to the directory data.
  */
-static int orangefs_readdir(struct file *file, struct dir_context *ctx)
-{
-	int ret = 0;
-	int buffer_index;
-	/*
-	 * ptoken supports Orangefs' distributed directory logic, added
-	 * in 2.9.2.
-	 */
-	__u64 *ptoken = file->private_data;
-	__u64 pos = 0;
-	ino_t ino = 0;
-	struct dentry *dentry = file->f_path.dentry;
-	struct orangefs_kernel_op_s *new_op = NULL;
-	struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(dentry->d_inode);
-	struct orangefs_readdir_response_s readdir_response;
-	void *dents_buf;
-	int i = 0;
-	int len = 0;
-	ino_t current_ino = 0;
-	char *current_entry = NULL;
-	long bytes_decoded;
-
-	gossip_debug(GOSSIP_DIR_DEBUG,
-		     "%s: ctx->pos:%lld, ptoken = %llu\n",
-		     __func__,
-		     lld(ctx->pos),
-		     llu(*ptoken));
-
-	pos = (__u64) ctx->pos;
-
-	/* are we done? */
-	if (pos == ORANGEFS_READDIR_END) {
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "Skipping to termination path\n");
-		return 0;
-	}
 
-	gossip_debug(GOSSIP_DIR_DEBUG,
-		     "orangefs_readdir called on %pd (pos=%llu)\n",
-		     dentry, llu(pos));
-
-	memset(&readdir_response, 0, sizeof(readdir_response));
-
-	new_op = op_alloc(ORANGEFS_VFS_OP_READDIR);
-	if (!new_op)
+static int orangefs_dir_more(struct orangefs_inode_s *oi,
+    struct orangefs_dir *od, struct dentry *dentry)
+{
+	const size_t offset = sizeof(struct orangefs_readdir_response_s);
+	struct orangefs_readdir_response_s *resp;
+	struct orangefs_kernel_op_s *op;
+	int bufi, r;
+
+	op = op_alloc(ORANGEFS_VFS_OP_READDIR);
+	if (!op) {
+		od->error = -ENOMEM;
 		return -ENOMEM;
+	}
 
 	/*
-	 * Only the indices are shared. No memory is actually shared, but the
-	 * mechanism is used.
+	 * Despite the badly named field, readdir does not use shared memory.
+	 * However, there are a limited number of readdir slots, which must be
+	 * allocated here.  This flag simply tells the op scheduler to return
+	 * the op here for retry.
 	 */
-	new_op->uses_shared_memory = 1;
-	new_op->upcall.req.readdir.refn = orangefs_inode->refn;
-	new_op->upcall.req.readdir.max_dirent_count =
+	op->uses_shared_memory = 1;
+	op->upcall.req.readdir.refn = oi->refn;
+	op->upcall.req.readdir.token = od->token;
+	op->upcall.req.readdir.max_dirent_count =
 	    ORANGEFS_MAX_DIRENT_COUNT_READDIR;
 
-	gossip_debug(GOSSIP_DIR_DEBUG,
-		     "%s: upcall.req.readdir.refn.khandle: %pU\n",
-		     __func__,
-		     &new_op->upcall.req.readdir.refn.khandle);
-
-	new_op->upcall.req.readdir.token = *ptoken;
-
-get_new_buffer_index:
-	buffer_index = orangefs_readdir_index_get();
-	if (buffer_index < 0) {
-		ret = buffer_index;
-		gossip_lerr("orangefs_readdir: orangefs_readdir_index_get() failure (%d)\n",
-			    ret);
-		goto out_free_op;
+again:
+	bufi = orangefs_readdir_index_get();
+	if (bufi < 0) {
+		op_release(op);
+		od->error = bufi;
+		return bufi;
 	}
-	new_op->upcall.req.readdir.buf_index = buffer_index;
 
-	ret = service_operation(new_op,
-				"orangefs_readdir",
-				get_interruptible_flag(dentry->d_inode));
+	op->upcall.req.readdir.buf_index = bufi;
 
-	gossip_debug(GOSSIP_DIR_DEBUG,
-		     "Readdir downcall status is %d.  ret:%d\n",
-		     new_op->downcall.status,
-		     ret);
+	r = service_operation(op, "orangefs_readdir",
+	    get_interruptible_flag(dentry->d_inode));
 
-	orangefs_readdir_index_put(buffer_index);
+	orangefs_readdir_index_put(bufi);
 
-	if (ret == -EAGAIN && op_state_purged(new_op)) {
-		/* Client-core indices are invalid after it restarted. */
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			"%s: Getting new buffer_index for retry of readdir..\n",
-			 __func__);
-		goto get_new_buffer_index;
-	}
-
-	if (ret == -EIO && op_state_purged(new_op)) {
-		gossip_err("%s: Client is down. Aborting readdir call.\n",
-			__func__);
-		goto out_free_op;
+	if (op_state_purged(op)) {
+		if (r == -EAGAIN) {
+			vfree(op->downcall.trailer_buf);
+			goto again;
+		} else if (r == -EIO) {
+			vfree(op->downcall.trailer_buf);
+			op_release(op);
+			od->error = r;
+			return r;
+		}
 	}
 
-	if (ret < 0 || new_op->downcall.status != 0) {
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "Readdir request failed.  Status:%d\n",
-			     new_op->downcall.status);
-		if (ret >= 0)
-			ret = new_op->downcall.status;
-		goto out_free_op;
-	}
+	if (r < 0) {
+		vfree(op->downcall.trailer_buf);
+		op_release(op);
+		od->error = r;
+		return r;
+	} else if (op->downcall.status) {
+		vfree(op->downcall.trailer_buf);
+		op_release(op);
+		od->error = op->downcall.status;
+		return op->downcall.status;
+	}
+
+	resp = (struct orangefs_readdir_response_s *)op->downcall.trailer_buf;
+	od->token = resp->token;
+
+	if (od->len + op->downcall.trailer_size - offset <= MAX_DIRECTORY) {
+		memcpy(od->directory + od->len,
+		    op->downcall.trailer_buf + offset,
+		    op->downcall.trailer_size - offset);
+		od->len += op->downcall.trailer_size - offset;
+	} else {
+		/* This limit was chosen based on protocol limits. */
+		gossip_err("orangefs_dir_more: userspace sent too much data\n");
+		vfree(op->downcall.trailer_buf);
+		op_release(op);
+		od->error = -EIO;
+		return -EIO;
+	}
+
+	vfree(op->downcall.trailer_buf);
+	op_release(op);
+	return 0;
+}
 
-	dents_buf = new_op->downcall.trailer_buf;
-	if (dents_buf == NULL) {
-		gossip_err("Invalid NULL buffer in readdir response\n");
-		ret = -ENOMEM;
-		goto out_free_op;
-	}
+static int orangefs_dir_fill(struct orangefs_inode_s *oi,
+    struct orangefs_dir *od, struct dentry *dentry, struct dir_context *ctx)
+{
+	struct orangefs_khandle *khandle;
+	__u32 *len, padlen;
+	char *s;
+	while (od->i < od->len) {
+		if (od->len < od->i + sizeof *len)
+			goto eio;
+		len = od->directory + od->i;
+		/* len does not include trailing zero but padlen does. */
+		padlen = (*len + 1) + (4 - (*len + 1)%4)%4;
+		if (od->len < od->i + sizeof *len + padlen + sizeof *khandle)
+			goto eio;
+		s = od->directory + od->i + sizeof *len;
+		if (s[*len] != 0)
+			goto eio;
+		khandle = od->directory + od->i + sizeof *len + padlen;
+
+		if (!dir_emit(ctx, s, *len, orangefs_khandle_to_ino(khandle),
+		    DT_UNKNOWN))
+			return 0;
+		od->i += sizeof *len + padlen + sizeof *khandle;
+		od->i = od->i + (8 - od->i%8)%8;
+		ctx->pos = 2 + od->i;
+	}
+	BUG_ON(od->i > od->len);
+	return 0;
+eio:
+	gossip_err("orangefs_dir_fill: userspace returns corrupt data\n");
+	od->error = -EIO;
+	return -EIO;
+}
 
-	bytes_decoded = decode_dirents(dents_buf, new_op->downcall.trailer_size,
-					&readdir_response);
-	if (bytes_decoded < 0) {
-		ret = bytes_decoded;
-		gossip_err("Could not decode readdir from buffer %d\n", ret);
-		goto out_vfree;
-	}
+static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
+{
+	struct orangefs_inode_s *oi;
+	struct orangefs_dir *od;
+	struct dentry *dentry;
+	int r;
 
-	if (bytes_decoded != new_op->downcall.trailer_size) {
-		gossip_err("orangefs_readdir: # bytes decoded (%ld) "
-			   "!= trailer size (%ld)\n",
-			   bytes_decoded,
-			   (long)new_op->downcall.trailer_size);
-		ret = -EINVAL;
-		goto out_destroy_handle;
-	}
+	dentry = file->f_path.dentry;
+	oi = ORANGEFS_I(dentry->d_inode);
+	od = file->private_data;
 
-	/*
-	 *  orangefs doesn't actually store dot and dot-dot, but
-	 *  we need to have them represented.
-	 */
-	if (pos == 0) {
-		ino = get_ino_from_khandle(dentry->d_inode);
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "%s: calling dir_emit of \".\" with pos = %llu\n",
-			     __func__,
-			     llu(pos));
-		ret = dir_emit(ctx, ".", 1, ino, DT_DIR);
-		pos += 1;
-	}
+	if (od->error)
+		return od->error;
 
-	if (pos == 1) {
-		ino = get_parent_ino_from_dentry(dentry);
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "%s: calling dir_emit of \"..\" with pos = %llu\n",
-			     __func__,
-			     llu(pos));
-		ret = dir_emit(ctx, "..", 2, ino, DT_DIR);
-		pos += 1;
+	if (ctx->pos == 0) {
+		if (!dir_emit_dot(file, ctx))
+			return 0;
+		ctx->pos++;
 	}
-
-	/*
-	 * we stored ORANGEFS_ITERATE_NEXT in ctx->pos last time around
-	 * to prevent "finding" dot and dot-dot on any iteration
-	 * other than the first.
-	 */
-	if (ctx->pos == ORANGEFS_ITERATE_NEXT)
-		ctx->pos = 0;
-
-	gossip_debug(GOSSIP_DIR_DEBUG,
-		     "%s: dirent_outcount:%d:\n",
-		     __func__,
-		     readdir_response.orangefs_dirent_outcount);
-	for (i = ctx->pos;
-	     i < readdir_response.orangefs_dirent_outcount;
-	     i++) {
-		len = readdir_response.dirent_array[i].d_length;
-		current_entry = readdir_response.dirent_array[i].d_name;
-		current_ino = orangefs_khandle_to_ino(
-			&readdir_response.dirent_array[i].khandle);
-
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			     "calling dir_emit for %s with len %d"
-			     ", ctx->pos %ld\n",
-			     current_entry,
-			     len,
-			     (unsigned long)ctx->pos);
-		/*
-		 * type is unknown. We don't return object type
-		 * in the dirent_array. This leaves getdents
-		 * clueless about type.
-		 */
-		ret =
-		    dir_emit(ctx, current_entry, len, current_ino, DT_UNKNOWN);
-		if (!ret)
-			break;
+	if (ctx->pos == 1) {
+		if (!dir_emit_dotdot(file, ctx))
+			return 0;
 		ctx->pos++;
-		gossip_debug(GOSSIP_DIR_DEBUG,
-			      "%s: ctx->pos:%lld\n",
-			      __func__,
-			      lld(ctx->pos));
-
 	}
 
-	/*
-	 * we ran all the way through the last batch, set up for
-	 * getting another batch...
-	 */
-	if (ret) {
-		*ptoken = readdir_response.token;
-		ctx->pos = ORANGEFS_ITERATE_NEXT;
+	r = 0;
+
+	if (od->i < od->len) {
+		r = orangefs_dir_fill(oi, od, dentry, ctx);
+		if (r)
+			return r;
 	}
 
-	/*
-	 * Did we hit the end of the directory?
-	 */
-	if (readdir_response.token == ORANGEFS_READDIR_END) {
-		gossip_debug(GOSSIP_DIR_DEBUG,
-		"End of dir detected; setting ctx->pos to ORANGEFS_READDIR_END.\n");
-		ctx->pos = ORANGEFS_READDIR_END;
+	if (od->token != ORANGEFS_READDIR_END) {
+		r = orangefs_dir_more(oi, od, dentry);
+		if (r)
+			return r;
+		r = orangefs_dir_fill(oi, od, dentry, ctx);
 	}
 
-out_destroy_handle:
-	/* kfree(NULL) is safe */
-	kfree(readdir_response.dirent_array);
-out_vfree:
-	gossip_debug(GOSSIP_DIR_DEBUG, "vfree %p\n", dents_buf);
-	vfree(dents_buf);
-out_free_op:
-	op_release(new_op);
-	gossip_debug(GOSSIP_DIR_DEBUG, "orangefs_readdir returning %d\n", ret);
-	return ret;
+	return r;
 }
 
 static int orangefs_dir_open(struct inode *inode, struct file *file)
 {
-	__u64 *ptoken;
-
-	file->private_data = kmalloc(sizeof(__u64), GFP_KERNEL);
+	struct orangefs_dir *od;
+	file->private_data = kmalloc(sizeof(struct orangefs_dir), GFP_KERNEL);
 	if (!file->private_data)
 		return -ENOMEM;
-
-	ptoken = file->private_data;
-	*ptoken = ORANGEFS_READDIR_START;
+	od = file->private_data;
+	od->token = ORANGEFS_READDIR_START;
+	/*
+	 * XXX: It seems wasteful to allocate such a large buffer for
+	 * each request.  Most will be much smaller.
+	 */
+	od->directory = alloc_pages_exact(MAX_DIRECTORY, GFP_KERNEL);
+	if (!od->directory) {
+		kfree(file->private_data);
+		return -ENOMEM;
+	}
+	od->i = 0;
+	od->len = 0;
+	od->error = 0;
 	return 0;
 }
 
 static int orangefs_dir_release(struct inode *inode, struct file *file)
 {
+	struct orangefs_dir *od = file->private_data;
 	orangefs_flush_inode(inode);
-	kfree(file->private_data);
+	free_pages_exact(od->directory, MAX_DIRECTORY);
+	kfree(od);
 	return 0;
 }
 
-/** ORANGEFS implementation of VFS directory operations */
 const struct file_operations orangefs_dir_operations = {
 	.read = generic_read_dir,
-	.iterate = orangefs_readdir,
+	.iterate = orangefs_dir_iterate,
 	.open = orangefs_dir_open,
-	.release = orangefs_dir_release,
+	.release = orangefs_dir_release
 };
diff --git a/fs/orangefs/downcall.h b/fs/orangefs/downcall.h
index 3b8923f..e27bd23 100644
--- a/fs/orangefs/downcall.h
+++ b/fs/orangefs/downcall.h
@@ -40,16 +40,6 @@ struct orangefs_mkdir_response {
 	struct orangefs_object_kref refn;
 };
 
-/*
- * duplication of some system interface structures so that I don't have
- * to allocate extra memory
- */
-struct orangefs_dirent {
-	char *d_name;
-	int d_length;
-	struct orangefs_khandle khandle;
-};
-
 struct orangefs_statfs_response {
 	__s64 block_size;
 	__s64 blocks_total;
@@ -136,7 +126,16 @@ struct orangefs_readdir_response_s {
 	__u64 directory_version;
 	__u32 __pad2;
 	__u32 orangefs_dirent_outcount;
-	struct orangefs_dirent *dirent_array;
+	/* the response is followed by a blob
+ * decode routine used by kmod to deal with the blob sent from
+ * userspace for readdirs. The blob contains zero or more of these
+ * sub-blobs:
+ *   __u32 - represents length of the character string that follows.
+ *   string - between 1 and ORANGEFS_NAME_MAX bytes long.
+ *   padding - (if needed) to cause the __u32 plus the string to be
+ *             eight byte aligned.
+ *   khandle - sizeof(khandle) bytes.
+	 */
 };
 
 #endif /* __DOWNCALL_H */
diff --git a/fs/orangefs/orangefs-dev-proto.h b/fs/orangefs/orangefs-dev-proto.h
index f380f9ed..efe08c7 100644
--- a/fs/orangefs/orangefs-dev-proto.h
+++ b/fs/orangefs/orangefs-dev-proto.h
@@ -52,12 +52,7 @@
  */
 #define ORANGEFS_MAX_DEBUG_STRING_LEN	0x00000800
 
-/*
- * The maximum number of directory entries in a single request is 96.
- * XXX: Why can this not be higher. The client-side code can handle up to 512.
- * XXX: What happens if we expect more than the client can return?
- */
-#define ORANGEFS_MAX_DIRENT_COUNT_READDIR 96
+#define ORANGEFS_MAX_DIRENT_COUNT_READDIR 512
 
 #include "upcall.h"
 #include "downcall.h"
-- 
2.1.4

[PATCH 06/10] orangefs: support llseek on directories

[Martin Brandenburg]

This and the previous commit fix xfstests generic/257.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/dir.c | 48 ++++++++++++++++++++++++++++++++++--------------
 1 file changed, 34 insertions(+), 14 deletions(-)

diff --git a/fs/orangefs/dir.c b/fs/orangefs/dir.c
index 3f82a73..c48220f 100644
--- a/fs/orangefs/dir.c
+++ b/fs/orangefs/dir.c
@@ -17,7 +17,7 @@
 struct orangefs_dir {
 	__u64 token;
 	void *directory;
-	size_t i, len;
+	size_t len;
 	int error;
 };
 
@@ -124,31 +124,36 @@ static int orangefs_dir_fill(struct orangefs_inode_s *oi,
 {
 	struct orangefs_khandle *khandle;
 	__u32 *len, padlen;
+	loff_t i;
 	char *s;
-	while (od->i < od->len) {
-		if (od->len < od->i + sizeof *len)
+	i = ctx->pos - 2;
+	while (i < od->len) {
+		if (od->len < i + sizeof *len)
 			goto eio;
-		len = od->directory + od->i;
+		len = od->directory + i;
 		/* len does not include trailing zero but padlen does. */
 		padlen = (*len + 1) + (4 - (*len + 1)%4)%4;
-		if (od->len < od->i + sizeof *len + padlen + sizeof *khandle)
+		if (od->len < i + sizeof *len + padlen + sizeof *khandle)
 			goto eio;
-		s = od->directory + od->i + sizeof *len;
+		s = od->directory + i + sizeof *len;
 		if (s[*len] != 0)
 			goto eio;
-		khandle = od->directory + od->i + sizeof *len + padlen;
+		khandle = od->directory + i + sizeof *len + padlen;
 
 		if (!dir_emit(ctx, s, *len, orangefs_khandle_to_ino(khandle),
 		    DT_UNKNOWN))
 			return 0;
-		od->i += sizeof *len + padlen + sizeof *khandle;
-		od->i = od->i + (8 - od->i%8)%8;
-		ctx->pos = 2 + od->i;
+		i += sizeof *len + padlen + sizeof *khandle;
+		i = i + (8 - i%8)%8;
+		ctx->pos = i + 2;
 	}
-	BUG_ON(od->i > od->len);
+	BUG_ON(i > od->len);
 	return 0;
 eio:
-	gossip_err("orangefs_dir_fill: userspace returns corrupt data\n");
+	/*
+	 * Here either data from userspace is corrupt or the application has
+	 * sought to an invalid location.
+	 */
 	od->error = -EIO;
 	return -EIO;
 }
@@ -180,12 +185,27 @@ static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
 
 	r = 0;
 
-	if (od->i < od->len) {
+	/*
+	 * Must read more if the user has sought past what has been read so far.
+	 * Stop a user who has sought past the end.
+	 */
+	while (od->token != ORANGEFS_READDIR_END && ctx->pos - 2 > od->len) {
+		r = orangefs_dir_more(oi, od, dentry);
+		if (r)
+			return r;
+	}
+	if (od->token == ORANGEFS_READDIR_END && ctx->pos - 2 > od->len) {
+		return -EIO;
+	}
+
+	/* Then try to fill if there's any left in the buffer. */
+	if (ctx->pos - 2 < od->len) {
 		r = orangefs_dir_fill(oi, od, dentry, ctx);
 		if (r)
 			return r;
 	}
 
+	/* Finally get some more and try to fill. */
 	if (od->token != ORANGEFS_READDIR_END) {
 		r = orangefs_dir_more(oi, od, dentry);
 		if (r)
@@ -213,7 +233,6 @@ static int orangefs_dir_open(struct inode *inode, struct file *file)
 		kfree(file->private_data);
 		return -ENOMEM;
 	}
-	od->i = 0;
 	od->len = 0;
 	od->error = 0;
 	return 0;
@@ -229,6 +248,7 @@ static int orangefs_dir_release(struct inode *inode, struct file *file)
 }
 
 const struct file_operations orangefs_dir_operations = {
+	.llseek = default_llseek,
 	.read = generic_read_dir,
 	.iterate = orangefs_dir_iterate,
 	.open = orangefs_dir_open,
-- 
2.1.4

[PATCH 07/10] orangefs: support very large directories

[Martin Brandenburg]

This works by maintaining a linked list of pages which the directory
has been read into rather than one giant fixed-size buffer.

This replaces code which limits the total directory size to the total
amount that could be returned in one server request.  Since filenames
are usually considerably shorter than the maximum, the old code could
usually handle several server requests before running out of space.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/dir.c | 246 ++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 172 insertions(+), 74 deletions(-)

diff --git a/fs/orangefs/dir.c b/fs/orangefs/dir.c
index c48220f..7afde1b 100644
--- a/fs/orangefs/dir.c
+++ b/fs/orangefs/dir.c
@@ -6,18 +6,15 @@
 #include "orangefs-kernel.h"
 #include "orangefs-bufmap.h"
 
-/*
- * There can be up to 512 directory entries.  Each entry contains a four byte
- * string size, a string, and a 16 byte khandle.  The string can be up to 256
- * bytes and is encoded with a trailing zero and four byte padding for the
- * khandle.
- */
-#define MAX_DIRECTORY ((4 + 257 + 3 + 16)*512)
+struct orangefs_dir_page {
+	struct orangefs_dir_page *next;
+	size_t len;
+};
 
 struct orangefs_dir {
 	__u64 token;
-	void *directory;
-	size_t len;
+	struct orangefs_dir_page *page;
+	loff_t end;
 	int error;
 };
 
@@ -32,20 +29,12 @@ struct orangefs_dir {
  * be skipped to get to the directory data.
  */
 
-static int orangefs_dir_more(struct orangefs_inode_s *oi,
-    struct orangefs_dir *od, struct dentry *dentry)
+static int do_readdir(struct orangefs_inode_s *oi, struct orangefs_dir *od,
+    struct dentry *dentry, struct orangefs_kernel_op_s *op)
 {
-	const size_t offset = sizeof(struct orangefs_readdir_response_s);
 	struct orangefs_readdir_response_s *resp;
-	struct orangefs_kernel_op_s *op;
 	int bufi, r;
 
-	op = op_alloc(ORANGEFS_VFS_OP_READDIR);
-	if (!op) {
-		od->error = -ENOMEM;
-		return -ENOMEM;
-	}
-
 	/*
 	 * Despite the badly named field, readdir does not use shared memory.
 	 * However, there are a limited number of readdir slots, which must be
@@ -61,7 +50,6 @@ static int orangefs_dir_more(struct orangefs_inode_s *oi,
 again:
 	bufi = orangefs_readdir_index_get();
 	if (bufi < 0) {
-		op_release(op);
 		od->error = bufi;
 		return bufi;
 	}
@@ -79,7 +67,6 @@ static int orangefs_dir_more(struct orangefs_inode_s *oi,
 			goto again;
 		} else if (r == -EIO) {
 			vfree(op->downcall.trailer_buf);
-			op_release(op);
 			od->error = r;
 			return r;
 		}
@@ -87,75 +74,188 @@ static int orangefs_dir_more(struct orangefs_inode_s *oi,
 
 	if (r < 0) {
 		vfree(op->downcall.trailer_buf);
-		op_release(op);
 		od->error = r;
 		return r;
 	} else if (op->downcall.status) {
 		vfree(op->downcall.trailer_buf);
-		op_release(op);
 		od->error = op->downcall.status;
 		return op->downcall.status;
 	}
 
 	resp = (struct orangefs_readdir_response_s *)op->downcall.trailer_buf;
 	od->token = resp->token;
+	return 0;
+}
 
-	if (od->len + op->downcall.trailer_size - offset <= MAX_DIRECTORY) {
-		memcpy(od->directory + od->len,
-		    op->downcall.trailer_buf + offset,
-		    op->downcall.trailer_size - offset);
-		od->len += op->downcall.trailer_size - offset;
-	} else {
-		/* This limit was chosen based on protocol limits. */
-		gossip_err("orangefs_dir_more: userspace sent too much data\n");
-		vfree(op->downcall.trailer_buf);
-		op_release(op);
-		od->error = -EIO;
-		return -EIO;
+static int parse_readdir(struct orangefs_dir *od,
+    struct orangefs_kernel_op_s *op)
+{
+	const size_t offset = sizeof(struct orangefs_readdir_response_s);
+	size_t last, i = offset, count = 1;
+	struct orangefs_dir_page *page;
+
+	page = od->page;
+	while (page->next) {
+		page = page->next;
+		count++;
+	}
+
+	while (i < op->downcall.trailer_size) {
+		__u32 *len, padlen;
+		/* Calculate how much will fit into the current page. */
+		last = i;
+		while (i < op->downcall.trailer_size) {
+			len = (void *)op->downcall.trailer_buf + i;
+			padlen = (4 + *len + 1 + 16) +
+			    (8 - (4 + *len + 1 + 16)%8)%8;
+			if (i + padlen > op->downcall.trailer_size) {
+				gossip_err(
+				    "orangefs_dir_more: userspace corrupt");
+				return -EIO;
+			}
+			if (page->len + i + padlen - last >
+			    PAGE_SIZE - sizeof *page)
+				break;
+			i += padlen;
+		}
+
+		memcpy((void *)page + sizeof *page + page->len,
+		    op->downcall.trailer_buf + last, i - last);
+		page->len += i - last;
+		/*
+		 * Must add 2.  Once from the current page (count++ below has
+		 * not been done here) and once to move past the final page.
+		 * The iteration code moves to the beginning of the next page
+		 * when it runs out, so the end pointer must point to the
+		 * beginning of the page after the last page.
+		 */
+		od->end = (count + 2) << PAGE_SHIFT;
+		BUG_ON(page->len > PAGE_SIZE - sizeof *page);
+
+		/* New page for next bit. */
+
+		/*
+		 * Should this wait if there's more room in the page?  Sounds
+		 * hard and error-prone to attempt to measure and at best saves
+		 * only a few kilobytes.
+		 */
+
+		page->next = (void *)__get_free_page(GFP_KERNEL);
+		if (!page->next)
+			return -ENOMEM;
+		page = page->next;
+		count++;
+		page->next = NULL;
+		page->len = 0;
+
+		/* Must make progress. */
+		BUG_ON(last == i);
+	}
+
+	return 0;
+}
+
+static int orangefs_dir_more(struct orangefs_inode_s *oi,
+    struct orangefs_dir *od, struct dentry *dentry)
+{
+	struct orangefs_kernel_op_s *op;
+	int r;
+
+	op = op_alloc(ORANGEFS_VFS_OP_READDIR);
+	if (!op) {
+		od->error = -ENOMEM;
+		return -ENOMEM;
+	}
+	r = do_readdir(oi, od, dentry, op);
+	if (r) {
+		od->error = r;
+		goto out;
+	}
+	r = parse_readdir(od, op);
+	if (r) {
+		od->error = r;
+		goto out;
 	}
 
+	od->error = 0;
 	vfree(op->downcall.trailer_buf);
+out:
 	op_release(op);
-	return 0;
+	return od->error;
 }
 
-static int orangefs_dir_fill(struct orangefs_inode_s *oi,
-    struct orangefs_dir *od, struct dentry *dentry, struct dir_context *ctx)
+static int fill_from_page(struct orangefs_dir_page *page,
+    struct dir_context *ctx)
 {
 	struct orangefs_khandle *khandle;
 	__u32 *len, padlen;
 	loff_t i;
 	char *s;
-	i = ctx->pos - 2;
-	while (i < od->len) {
-		if (od->len < i + sizeof *len)
-			goto eio;
-		len = od->directory + i;
+	i = ctx->pos & ~PAGE_MASK;
+
+	/* The file offset from userspace is too large. */
+	if (i > page->len)
+		return -EIO;
+
+	while (i < page->len) {
+		if (page->len < i + sizeof *len)
+			return -EIO;
+		len = (void *)page + sizeof *page + i;
 		/* len does not include trailing zero but padlen does. */
 		padlen = (*len + 1) + (4 - (*len + 1)%4)%4;
-		if (od->len < i + sizeof *len + padlen + sizeof *khandle)
-			goto eio;
-		s = od->directory + i + sizeof *len;
+		if (page->len < i + sizeof *len + padlen + sizeof *khandle)
+			return -EIO;
+		s = (void *)page + sizeof *page + i + sizeof *len;
 		if (s[*len] != 0)
-			goto eio;
-		khandle = od->directory + i + sizeof *len + padlen;
-
+			return -EIO;
+		khandle = (void *)page + sizeof *page +
+		    i + sizeof *len + padlen;
 		if (!dir_emit(ctx, s, *len, orangefs_khandle_to_ino(khandle),
 		    DT_UNKNOWN))
 			return 0;
 		i += sizeof *len + padlen + sizeof *khandle;
 		i = i + (8 - i%8)%8;
-		ctx->pos = i + 2;
+		BUG_ON(i > page->len);
+		ctx->pos = (ctx->pos & PAGE_MASK) | i;
+	}
+	return 1;
+}
+
+static int orangefs_dir_fill(struct orangefs_inode_s *oi,
+    struct orangefs_dir *od, struct dentry *dentry, struct dir_context *ctx)
+{
+	struct orangefs_dir_page *page;
+	size_t count;
+
+	count = ((ctx->pos & PAGE_MASK) >> PAGE_SHIFT) - 1;
+
+	page = od->page;
+	while (page->next && count) {
+		count--;
+		page = page->next;
+	}
+	/* This means the userspace file offset is invalid. */
+	if (count) {
+		od->error = -EIO;
+		return -EIO;
+	}
+
+	while (page && page->len) {
+		int r;
+		r = fill_from_page(page, ctx);
+		if (r < 0) {
+			od->error = r;
+			return r;
+		} else if (r == 0) {
+			/* Userspace buffer is full. */
+			break;
+		} else {
+			/* The page ran out of data.  Move to the next page. */
+			ctx->pos = (ctx->pos & PAGE_MASK) + (1 << PAGE_SHIFT);
+			page = page->next;
+		}
 	}
-	BUG_ON(i > od->len);
 	return 0;
-eio:
-	/*
-	 * Here either data from userspace is corrupt or the application has
-	 * sought to an invalid location.
-	 */
-	od->error = -EIO;
-	return -EIO;
 }
 
 static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
@@ -180,7 +280,7 @@ static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
 	if (ctx->pos == 1) {
 		if (!dir_emit_dotdot(file, ctx))
 			return 0;
-		ctx->pos++;
+		ctx->pos = 1 << PAGE_SHIFT;
 	}
 
 	r = 0;
@@ -189,17 +289,16 @@ static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
 	 * Must read more if the user has sought past what has been read so far.
 	 * Stop a user who has sought past the end.
 	 */
-	while (od->token != ORANGEFS_READDIR_END && ctx->pos - 2 > od->len) {
+	while (od->token != ORANGEFS_READDIR_END && ctx->pos > od->end) {
 		r = orangefs_dir_more(oi, od, dentry);
 		if (r)
 			return r;
 	}
-	if (od->token == ORANGEFS_READDIR_END && ctx->pos - 2 > od->len) {
+	if (od->token == ORANGEFS_READDIR_END && ctx->pos > od->end)
 		return -EIO;
-	}
 
 	/* Then try to fill if there's any left in the buffer. */
-	if (ctx->pos - 2 < od->len) {
+	if (ctx->pos < od->end) {
 		r = orangefs_dir_fill(oi, od, dentry, ctx);
 		if (r)
 			return r;
@@ -224,16 +323,10 @@ static int orangefs_dir_open(struct inode *inode, struct file *file)
 		return -ENOMEM;
 	od = file->private_data;
 	od->token = ORANGEFS_READDIR_START;
-	/*
-	 * XXX: It seems wasteful to allocate such a large buffer for
-	 * each request.  Most will be much smaller.
-	 */
-	od->directory = alloc_pages_exact(MAX_DIRECTORY, GFP_KERNEL);
-	if (!od->directory) {
-		kfree(file->private_data);
-		return -ENOMEM;
-	}
-	od->len = 0;
+	od->page = (void *)__get_free_page(GFP_KERNEL);
+	od->page->next = NULL;
+	od->page->len = 0;
+	od->end = 1 << PAGE_SHIFT;
 	od->error = 0;
 	return 0;
 }
@@ -241,8 +334,13 @@ static int orangefs_dir_open(struct inode *inode, struct file *file)
 static int orangefs_dir_release(struct inode *inode, struct file *file)
 {
 	struct orangefs_dir *od = file->private_data;
+	struct orangefs_dir_page *page = od->page;
 	orangefs_flush_inode(inode);
-	free_pages_exact(od->directory, MAX_DIRECTORY);
+	do {
+		struct orangefs_dir_page *next = page->next;
+		free_page((unsigned long)page);
+		page = next;
+	} while (page);
 	kfree(od);
 	return 0;
 }
-- 
2.1.4

[PATCH 08/10] orangefs: remove ORANGEFS_READDIR macros

[Martin Brandenburg]

They are clones of the ORANGEFS_ITERATE macros in use elsewhere.  Delete
ORANGEFS_ITERATE_NEXT which is a hack previously used by readdir.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/dir.c      | 8 ++++----
 fs/orangefs/protocol.h | 9 ++-------
 2 files changed, 6 insertions(+), 11 deletions(-)

diff --git a/fs/orangefs/dir.c b/fs/orangefs/dir.c
index 7afde1b..9cc755b 100644
--- a/fs/orangefs/dir.c
+++ b/fs/orangefs/dir.c
@@ -289,12 +289,12 @@ static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
 	 * Must read more if the user has sought past what has been read so far.
 	 * Stop a user who has sought past the end.
 	 */
-	while (od->token != ORANGEFS_READDIR_END && ctx->pos > od->end) {
+	while (od->token != ORANGEFS_ITERATE_END && ctx->pos > od->end) {
 		r = orangefs_dir_more(oi, od, dentry);
 		if (r)
 			return r;
 	}
-	if (od->token == ORANGEFS_READDIR_END && ctx->pos > od->end)
+	if (od->token == ORANGEFS_ITERATE_END && ctx->pos > od->end)
 		return -EIO;
 
 	/* Then try to fill if there's any left in the buffer. */
@@ -305,7 +305,7 @@ static int orangefs_dir_iterate(struct file *file, struct dir_context *ctx)
 	}
 
 	/* Finally get some more and try to fill. */
-	if (od->token != ORANGEFS_READDIR_END) {
+	if (od->token != ORANGEFS_ITERATE_END) {
 		r = orangefs_dir_more(oi, od, dentry);
 		if (r)
 			return r;
@@ -322,7 +322,7 @@ static int orangefs_dir_open(struct inode *inode, struct file *file)
 	if (!file->private_data)
 		return -ENOMEM;
 	od = file->private_data;
-	od->token = ORANGEFS_READDIR_START;
+	od->token = ORANGEFS_ITERATE_START;
 	od->page = (void *)__get_free_page(GFP_KERNEL);
 	od->page->next = NULL;
 	od->page->len = 0;
diff --git a/fs/orangefs/protocol.h b/fs/orangefs/protocol.h
index 971307a..48bcc1b 100644
--- a/fs/orangefs/protocol.h
+++ b/fs/orangefs/protocol.h
@@ -138,13 +138,8 @@ typedef __s64 ORANGEFS_offset;
 #define ORANGEFS_G_SGID    (1 << 10)
 #define ORANGEFS_U_SUID    (1 << 11)
 
-/* definition taken from stdint.h */
-#define INT32_MAX (2147483647)
-#define ORANGEFS_ITERATE_START    (INT32_MAX - 1)
-#define ORANGEFS_ITERATE_END      (INT32_MAX - 2)
-#define ORANGEFS_ITERATE_NEXT     (INT32_MAX - 3)
-#define ORANGEFS_READDIR_START ORANGEFS_ITERATE_START
-#define ORANGEFS_READDIR_END   ORANGEFS_ITERATE_END
+#define ORANGEFS_ITERATE_START    2147483646
+#define ORANGEFS_ITERATE_END      2147483645
 #define ORANGEFS_IMMUTABLE_FL FS_IMMUTABLE_FL
 #define ORANGEFS_APPEND_FL    FS_APPEND_FL
 #define ORANGEFS_NOATIME_FL   FS_NOATIME_FL
-- 
2.1.4

[PATCH 09/10] orangefs: implement statx

[Martin Brandenburg]

Fortunately OrangeFS has had a getattr request mask for a long time.

The server basically has two difficulty levels for attributes.  Fetching
any attribute except size requires communicating with the metadata
server for that handle.  Since all the attributes are right there, it
makes sense to return them all.  Fetching the size requires
communicating with every I/O server (that the file is distributed
across).  Therefore if asked for anything except size, get everything
except size, and if asked for size, get everything.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/file.c            |  6 +++--
 fs/orangefs/inode.c           | 15 +++++++----
 fs/orangefs/namei.c           |  3 +++
 fs/orangefs/orangefs-kernel.h |  4 ++-
 fs/orangefs/orangefs-utils.c  | 63 +++++++++++++++++++++++++++++--------------
 5 files changed, 63 insertions(+), 28 deletions(-)

diff --git a/fs/orangefs/file.c b/fs/orangefs/file.c
index e6bbc80..b421df1 100644
--- a/fs/orangefs/file.c
+++ b/fs/orangefs/file.c
@@ -475,7 +475,8 @@ static ssize_t orangefs_file_write_iter(struct kiocb *iocb, struct iov_iter *ite
 
 	/* Make sure generic_write_checks sees an up to date inode size. */
 	if (file->f_flags & O_APPEND) {
-		rc = orangefs_inode_getattr(file->f_mapping->host, 0, 1);
+		rc = orangefs_inode_getattr(file->f_mapping->host, 0, 1,
+		    STATX_SIZE);
 		if (rc == -ESTALE)
 			rc = -EIO;
 		if (rc) {
@@ -693,7 +694,8 @@ static loff_t orangefs_file_llseek(struct file *file, loff_t offset, int origin)
 		 * NOTE: We are only interested in file size here,
 		 * so we set mask accordingly.
 		 */
-		ret = orangefs_inode_getattr(file->f_mapping->host, 0, 1);
+		ret = orangefs_inode_getattr(file->f_mapping->host, 0, 1,
+		    STATX_SIZE);
 		if (ret == -ESTALE)
 			ret = -EIO;
 		if (ret) {
diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c
index a304bf3..ee28364 100644
--- a/fs/orangefs/inode.c
+++ b/fs/orangefs/inode.c
@@ -161,7 +161,7 @@ static int orangefs_setattr_size(struct inode *inode, struct iattr *iattr)
 		     iattr->ia_size);
 
 	/* Ensure that we have a up to date size, so we know if it changed. */
-	ret = orangefs_inode_getattr(inode, 0, 1);
+	ret = orangefs_inode_getattr(inode, 0, 1, STATX_SIZE);
 	if (ret == -ESTALE)
 		ret = -EIO;
 	if (ret) {
@@ -256,13 +256,18 @@ int orangefs_getattr(const struct path *path, struct kstat *stat,
 		     "orangefs_getattr: called on %pd\n",
 		     path->dentry);
 
-	ret = orangefs_inode_getattr(inode, 0, 0);
+	ret = orangefs_inode_getattr(inode, 0, 0, request_mask);
 	if (ret == 0) {
 		generic_fillattr(inode, stat);
 
 		/* override block size reported to stat */
 		orangefs_inode = ORANGEFS_I(inode);
 		stat->blksize = orangefs_inode->blksize;
+
+		if (request_mask & STATX_SIZE)
+			stat->result_mask = STATX_BASIC_STATS;
+		else
+			stat->result_mask = STATX_BASIC_STATS & ~STATX_SIZE;
 	}
 	return ret;
 }
@@ -277,7 +282,7 @@ int orangefs_permission(struct inode *inode, int mask)
 	gossip_debug(GOSSIP_INODE_DEBUG, "%s: refreshing\n", __func__);
 
 	/* Make sure the permission (and other common attrs) are up to date. */
-	ret = orangefs_inode_getattr(inode, 0, 0);
+	ret = orangefs_inode_getattr(inode, 0, 0, STATX_MODE);
 	if (ret < 0)
 		return ret;
 
@@ -375,7 +380,7 @@ struct inode *orangefs_iget(struct super_block *sb, struct orangefs_object_kref
 	if (!inode || !(inode->i_state & I_NEW))
 		return inode;
 
-	error = orangefs_inode_getattr(inode, 1, 1);
+	error = orangefs_inode_getattr(inode, 1, 1, STATX_ALL);
 	if (error) {
 		iget_failed(inode);
 		return ERR_PTR(error);
@@ -420,7 +425,7 @@ struct inode *orangefs_new_inode(struct super_block *sb, struct inode *dir,
 	orangefs_set_inode(inode, ref);
 	inode->i_ino = hash;	/* needed for stat etc */
 
-	error = orangefs_inode_getattr(inode, 1, 1);
+	error = orangefs_inode_getattr(inode, 1, 1, STATX_ALL);
 	if (error)
 		goto out_iput;
 
diff --git a/fs/orangefs/namei.c b/fs/orangefs/namei.c
index 7c31593..478e88b 100644
--- a/fs/orangefs/namei.c
+++ b/fs/orangefs/namei.c
@@ -74,6 +74,7 @@ static int orangefs_create(struct inode *dir,
 	unlock_new_inode(inode);
 	orangefs_set_timeout(dentry);
 	ORANGEFS_I(inode)->getattr_time = jiffies - 1;
+	ORANGEFS_I(inode)->getattr_mask = STATX_BASIC_STATS;
 
 	gossip_debug(GOSSIP_NAME_DEBUG,
 		     "%s: dentry instantiated for %pd\n",
@@ -322,6 +323,7 @@ static int orangefs_symlink(struct inode *dir,
 	unlock_new_inode(inode);
 	orangefs_set_timeout(dentry);
 	ORANGEFS_I(inode)->getattr_time = jiffies - 1;
+	ORANGEFS_I(inode)->getattr_mask = STATX_BASIC_STATS;
 
 	gossip_debug(GOSSIP_NAME_DEBUG,
 		     "Inode (Symlink) %pU -> %pd\n",
@@ -386,6 +388,7 @@ static int orangefs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode
 	unlock_new_inode(inode);
 	orangefs_set_timeout(dentry);
 	ORANGEFS_I(inode)->getattr_time = jiffies - 1;
+	ORANGEFS_I(inode)->getattr_mask = STATX_BASIC_STATS;
 
 	gossip_debug(GOSSIP_NAME_DEBUG,
 		     "Inode (Directory) %pU -> %pd\n",
diff --git a/fs/orangefs/orangefs-kernel.h b/fs/orangefs/orangefs-kernel.h
index 24b0648..087bb51 100644
--- a/fs/orangefs/orangefs-kernel.h
+++ b/fs/orangefs/orangefs-kernel.h
@@ -215,6 +215,7 @@ struct orangefs_inode_s {
 	unsigned long pinode_flags;
 
 	unsigned long getattr_time;
+	u32 getattr_mask;
 };
 
 #define P_ATIME_FLAG 0
@@ -494,7 +495,8 @@ int orangefs_inode_setxattr(struct inode *inode,
 			 size_t size,
 			 int flags);
 
-int orangefs_inode_getattr(struct inode *inode, int new, int bypass);
+int orangefs_inode_getattr(struct inode *inode, int new, int bypass,
+    u32 request_mask);
 
 int orangefs_inode_check_changed(struct inode *inode);
 
diff --git a/fs/orangefs/orangefs-utils.c b/fs/orangefs/orangefs-utils.c
index 9b96b99..fcbf4e5 100644
--- a/fs/orangefs/orangefs-utils.c
+++ b/fs/orangefs/orangefs-utils.c
@@ -251,7 +251,8 @@ static int orangefs_inode_is_stale(struct inode *inode, int new,
 	return 0;
 }
 
-int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
+int orangefs_inode_getattr(struct inode *inode, int new, int bypass,
+    u32 request_mask)
 {
 	struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode);
 	struct orangefs_kernel_op_s *new_op;
@@ -262,7 +263,13 @@ int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
 	    get_khandle_from_ino(inode));
 
 	if (!new && !bypass) {
-		if (time_before(jiffies, orangefs_inode->getattr_time))
+		/*
+		 * Must have all the attributes in the mask and be within cache
+		 * time.
+		 */
+		if ((request_mask & orangefs_inode->getattr_mask) ==
+		    request_mask &&
+		    time_before(jiffies, orangefs_inode->getattr_time))
 			return 0;
 	}
 
@@ -270,7 +277,15 @@ int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
 	if (!new_op)
 		return -ENOMEM;
 	new_op->upcall.req.getattr.refn = orangefs_inode->refn;
-	new_op->upcall.req.getattr.mask = ORANGEFS_ATTR_SYS_ALL_NOHINT;
+	/*
+	 * Size is the hardest attribute to get.  The incremental cost of any
+	 * other attribute is essentially zero.
+	 */
+	if (request_mask & STATX_SIZE || new)
+		new_op->upcall.req.getattr.mask = ORANGEFS_ATTR_SYS_ALL_NOHINT;
+	else
+		new_op->upcall.req.getattr.mask =
+		    ORANGEFS_ATTR_SYS_ALL_NOHINT & ~ORANGEFS_ATTR_SYS_SIZE;
 
 	ret = service_operation(new_op, __func__,
 	    get_interruptible_flag(inode));
@@ -291,25 +306,29 @@ int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
 	case S_IFREG:
 		inode->i_flags = orangefs_inode_flags(&new_op->
 		    downcall.resp.getattr.attributes);
-		inode_size = (loff_t)new_op->
-		    downcall.resp.getattr.attributes.size;
-		rounded_up_size =
-		    (inode_size + (4096 - (inode_size % 4096)));
-		inode->i_size = inode_size;
-		orangefs_inode->blksize =
-		    new_op->downcall.resp.getattr.attributes.blksize;
-		spin_lock(&inode->i_lock);
-		inode->i_bytes = inode_size;
-		inode->i_blocks =
-		    (unsigned long)(rounded_up_size / 512);
-		spin_unlock(&inode->i_lock);
+		if (request_mask & STATX_SIZE || new) {
+			inode_size = (loff_t)new_op->
+			    downcall.resp.getattr.attributes.size;
+			rounded_up_size =
+			    (inode_size + (4096 - (inode_size % 4096)));
+			inode->i_size = inode_size;
+			orangefs_inode->blksize =
+			    new_op->downcall.resp.getattr.attributes.blksize;
+			spin_lock(&inode->i_lock);
+			inode->i_bytes = inode_size;
+			inode->i_blocks =
+			    (unsigned long)(rounded_up_size / 512);
+			spin_unlock(&inode->i_lock);
+		}
 		break;
 	case S_IFDIR:
-		inode->i_size = PAGE_SIZE;
-		orangefs_inode->blksize = i_blocksize(inode);
-		spin_lock(&inode->i_lock);
-		inode_set_bytes(inode, inode->i_size);
-		spin_unlock(&inode->i_lock);
+		if (request_mask & STATX_SIZE || new) {
+			inode->i_size = PAGE_SIZE;
+			orangefs_inode->blksize = i_blocksize(inode);
+			spin_lock(&inode->i_lock);
+			inode_set_bytes(inode, inode->i_size);
+			spin_unlock(&inode->i_lock);
+		}
 		set_nlink(inode, 1);
 		break;
 	case S_IFLNK:
@@ -349,6 +368,10 @@ int orangefs_inode_getattr(struct inode *inode, int new, int bypass)
 
 	orangefs_inode->getattr_time = jiffies +
 	    orangefs_getattr_timeout_msecs*HZ/1000;
+	if (request_mask & STATX_SIZE || new)
+		orangefs_inode->getattr_mask = STATX_BASIC_STATS;
+	else
+		orangefs_inode->getattr_mask = STATX_BASIC_STATS & ~STATX_SIZE;
 	ret = 0;
 out:
 	op_release(new_op);
-- 
2.1.4

[PATCH 10/10] orangefs: do not check possibly stale size on truncate

[Martin Brandenburg]

Let the server figure this out because our size might be out of date or
not present.

The bug was that

	xfs_io -f -t -c "pread -v 0 100" /mnt/foo
	echo "Test" > /mnt/foo
	xfs_io -f -t -c "pread -v 0 100" /mnt/foo

fail because the second truncate did not happen if nothing had requested
the size after the write in echo.  Thus i_size was zero (not present)
and the orangefs_setattr though i_size was zero and there was nothing to
do.

Signed-off-by: Martin Brandenburg <martin@omnibond.com>
---
 fs/orangefs/inode.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c
index ee28364..7f82e56 100644
--- a/fs/orangefs/inode.c
+++ b/fs/orangefs/inode.c
@@ -218,8 +218,7 @@ int orangefs_setattr(struct dentry *dentry, struct iattr *iattr)
 	if (ret)
 		goto out;
 
-	if ((iattr->ia_valid & ATTR_SIZE) &&
-	    iattr->ia_size != i_size_read(inode)) {
+	if (iattr->ia_valid & ATTR_SIZE) {
 		ret = orangefs_setattr_size(inode, iattr);
 		if (ret)
 			goto out;
-- 
2.1.4