[PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

Miklos,

This version provides a solution for some interesting non-samefs cases:
- All the ext* family
- Many other fs with default encode_fh
- xfs that is not huge with overlay 'xino' mount option
- tmpfs that is not on a machine with jurassic uptime with 'xino'

I tested this with Chandan's upstream overlay/041 xfstest for
consistent d_ino in non-samefs setup. Results are:
- Test passes for ext4
- Test fails for xfs
- Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
  but this is because of a test bug
- With the test bug fix available at [2] test passes
  with xfs and OVERLAY_MOUNT_OPTIONS=-oxino

All the exportfs tests also pass with these changes and
either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.

Changes since v7:
- Drop patches for building impure cache for non-samefs subdirs
- Dropped patch "update cache version of impure parent on rename"
  because it is not relevant to this series
- Remap lower inode numbers for 32bit inode file systems
- Add mount option 'xino' for opting-in to use high inode bits

Thanks,
Amir.

[1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
[2] https://github.com/amir73il/xfstests/commits/overlayfs-devel

Amir Goldstein (8):
  ovl: move include of ovl_entry.h into overlayfs.h
  ovl: allocate anonymous devs for lowerdirs
  ovl: factor out ovl_map_dev_ino() helper
  ovl: return anonymous st_dev for lower inodes
  ovl: relax same fs constraint for constant st_ino
  ovl: constant st_ino for non-samefs with xino
  ovl: consistent d_ino for non-samefs with xino
  ovl: add support for 'xino' mount option

Chandan Rajendra (1):
  ovl: re-structure overlay lower layers in-memory

 fs/overlayfs/copy_up.c   |   1 -
 fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
 fs/overlayfs/namei.c     |  42 +++++++--------
 fs/overlayfs/overlayfs.h |   9 ++--
 fs/overlayfs/ovl_entry.h |  19 ++++++-
 fs/overlayfs/readdir.c   |  49 +++++++++++++++---
 fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
 fs/overlayfs/util.c      |  39 ++++++++++++--
 8 files changed, 277 insertions(+), 102 deletions(-)

-- 
2.7.4

[PATCH v8 1/9] ovl: move include of ovl_entry.h into overlayfs.h

[Amir Goldstein]

Most overlayfs c files already explicitly include ovl_entry.h
to use overlay entry struct definitions and upcoming changes
are going to require even more c files to include this header.

All overlayfs c files include overlayfs.h and overlayfs.h itself
refers to some structs defined in ovl_entry.h, so it seems more
logic to include ovl_entry.h from overlayfs.h than from c files.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/copy_up.c   | 1 -
 fs/overlayfs/inode.c     | 1 -
 fs/overlayfs/namei.c     | 1 -
 fs/overlayfs/overlayfs.h | 1 +
 fs/overlayfs/super.c     | 1 -
 fs/overlayfs/util.c      | 1 -
 6 files changed, 1 insertion(+), 5 deletions(-)

diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
index c441f9387a1b..9311d183b532 100644
--- a/fs/overlayfs/copy_up.c
+++ b/fs/overlayfs/copy_up.c
@@ -22,7 +22,6 @@
 #include <linux/ratelimit.h>
 #include <linux/exportfs.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 #define OVL_COPY_UP_CHUNK_SIZE (1 << 20)
 
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 2949448d0c7d..b772e5a2a730 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -14,7 +14,6 @@
 #include <linux/posix_acl.h>
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 {
diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index a12dc10bf726..505a4b8902fc 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -15,7 +15,6 @@
 #include <linux/mount.h>
 #include <linux/exportfs.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 struct ovl_lookup_data {
 	struct qstr name;
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index d53157ccf0d7..1cf3bdd193a4 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -9,6 +9,7 @@
 
 #include <linux/kernel.h>
 #include <linux/uuid.h>
+#include "ovl_entry.h"
 
 enum ovl_path_type {
 	__OVL_PATH_UPPER	= (1 << 0),
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index f5738e96a052..8702803ba328 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -18,7 +18,6 @@
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 MODULE_AUTHOR("Miklos Szeredi <miklos@szeredi.hu>");
 MODULE_DESCRIPTION("Overlay filesystem");
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 51ca8bd16009..9158d17bb320 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -17,7 +17,6 @@
 #include <linux/namei.h>
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
-#include "ovl_entry.h"
 
 int ovl_want_write(struct dentry *dentry)
 {
-- 
2.7.4

[PATCH v8 2/9] ovl: re-structure overlay lower layers in-memory

[Amir Goldstein]

From: Chandan Rajendra <chandan@linux.vnet.ibm.com>

Define new structures to represent overlay instance lower layers and
overlay merge dir lower layers to make room for storing more per layer
information in-memory.

Instead of keeping the fs instance lower layers in an array of struct
vfsmount, keep them in an array of new struct ovl_layer, that has a
pointer to struct vfsmount.

Instead of keeping the dentry lower layers in an array of struct path,
keep them in an array of new struct ovl_path, that has a pointer to
struct dentry and to struct ovl_layer.

Store fs root layer index in struct layer to get from non-root dir layer
object to root dir layer layer object.

[amir: split re-structure from anonymous bdev patch
       store layer index in layer struct]

Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/namei.c     | 41 ++++++++++++++-----------------
 fs/overlayfs/overlayfs.h |  5 ++--
 fs/overlayfs/ovl_entry.h | 15 ++++++++++--
 fs/overlayfs/readdir.c   |  4 +--
 fs/overlayfs/super.c     | 63 +++++++++++++++++++++++++++---------------------
 fs/overlayfs/util.c      | 14 ++++++++++-
 6 files changed, 84 insertions(+), 58 deletions(-)

diff --git a/fs/overlayfs/namei.c b/fs/overlayfs/namei.c
index 505a4b8902fc..7b89f5944e04 100644
--- a/fs/overlayfs/namei.c
+++ b/fs/overlayfs/namei.c
@@ -285,16 +285,15 @@ static int ovl_lookup_layer(struct dentry *base, struct ovl_lookup_data *d,
 
 
 static int ovl_check_origin(struct dentry *upperdentry,
-			    struct path *lowerstack, unsigned int numlower,
-			    struct path **stackp, unsigned int *ctrp)
+			    struct ovl_path *lower, unsigned int numlower,
+			    struct ovl_path **stackp, unsigned int *ctrp)
 {
 	struct vfsmount *mnt;
 	struct dentry *origin = NULL;
 	int i;
 
-
 	for (i = 0; i < numlower; i++) {
-		mnt = lowerstack[i].mnt;
+		mnt = lower[i].layer->mnt;
 		origin = ovl_get_origin(upperdentry, mnt);
 		if (IS_ERR(origin))
 			return PTR_ERR(origin);
@@ -308,12 +307,12 @@ static int ovl_check_origin(struct dentry *upperdentry,
 
 	BUG_ON(*ctrp);
 	if (!*stackp)
-		*stackp = kmalloc(sizeof(struct path), GFP_KERNEL);
+		*stackp = kmalloc(sizeof(struct ovl_path), GFP_KERNEL);
 	if (!*stackp) {
 		dput(origin);
 		return -ENOMEM;
 	}
-	**stackp = (struct path) { .dentry = origin, .mnt = mnt };
+	**stackp = (struct ovl_path){.dentry = origin, .layer = lower[i].layer};
 	*ctrp = 1;
 
 	return 0;
@@ -383,13 +382,13 @@ int ovl_verify_origin(struct dentry *dentry, struct vfsmount *mnt,
  * OVL_XATTR_ORIGIN and that origin file handle can be decoded to lower path.
  * Return 0 on match, -ESTALE on mismatch or stale origin, < 0 on error.
  */
-int ovl_verify_index(struct dentry *index, struct path *lowerstack,
+int ovl_verify_index(struct dentry *index, struct ovl_path *lower,
 		     unsigned int numlower)
 {
 	struct ovl_fh *fh = NULL;
 	size_t len;
-	struct path origin = { };
-	struct path *stack = &origin;
+	struct ovl_path origin = { };
+	struct ovl_path *stack = &origin;
 	unsigned int ctr = 0;
 	int err;
 
@@ -428,7 +427,7 @@ int ovl_verify_index(struct dentry *index, struct path *lowerstack,
 	if (err)
 		goto fail;
 
-	err = ovl_check_origin(index, lowerstack, numlower, &stack, &ctr);
+	err = ovl_check_origin(index, lower, numlower, &stack, &ctr);
 	if (!err && !ctr)
 		err = -ESTALE;
 	if (err)
@@ -567,7 +566,8 @@ int ovl_path_next(int idx, struct dentry *dentry, struct path *path)
 		idx++;
 	}
 	BUG_ON(idx > oe->numlower);
-	*path = oe->lowerstack[idx - 1];
+	path->dentry = oe->lowerstack[idx - 1].dentry;
+	path->mnt = oe->lowerstack[idx - 1].layer->mnt;
 
 	return (idx < oe->numlower) ? idx + 1 : -1;
 }
@@ -580,7 +580,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 	struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
 	struct ovl_entry *poe = dentry->d_parent->d_fsdata;
 	struct ovl_entry *roe = dentry->d_sb->s_root->d_fsdata;
-	struct path *stack = NULL;
+	struct ovl_path *stack = NULL;
 	struct dentry *upperdir, *upperdentry = NULL;
 	struct dentry *index = NULL;
 	unsigned int ctr = 0;
@@ -645,17 +645,17 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 
 	if (!d.stop && poe->numlower) {
 		err = -ENOMEM;
-		stack = kcalloc(ofs->numlower, sizeof(struct path),
+		stack = kcalloc(ofs->numlower, sizeof(struct ovl_path),
 				GFP_KERNEL);
 		if (!stack)
 			goto out_put_upper;
 	}
 
 	for (i = 0; !d.stop && i < poe->numlower; i++) {
-		struct path lowerpath = poe->lowerstack[i];
+		struct ovl_path lower = poe->lowerstack[i];
 
 		d.last = i == poe->numlower - 1;
-		err = ovl_lookup_layer(lowerpath.dentry, &d, &this);
+		err = ovl_lookup_layer(lower.dentry, &d, &this);
 		if (err)
 			goto out_put;
 
@@ -663,7 +663,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 			continue;
 
 		stack[ctr].dentry = this;
-		stack[ctr].mnt = lowerpath.mnt;
+		stack[ctr].layer = lower.layer;
 		ctr++;
 
 		if (d.stop)
@@ -671,13 +671,8 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 
 		if (d.redirect && d.redirect[0] == '/' && poe != roe) {
 			poe = roe;
-
 			/* Find the current layer on the root dentry */
-			for (i = 0; i < poe->numlower; i++)
-				if (poe->lowerstack[i].mnt == lowerpath.mnt)
-					break;
-			if (WARN_ON(i == poe->numlower))
-				break;
+			i = lower.layer->idx - 1;
 		}
 	}
 
@@ -699,7 +694,7 @@ struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
 		goto out_put;
 
 	oe->opaque = upperopaque;
-	memcpy(oe->lowerstack, stack, sizeof(struct path) * ctr);
+	memcpy(oe->lowerstack, stack, sizeof(struct ovl_path) * ctr);
 	dentry->d_fsdata = oe;
 
 	if (upperdentry)
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 1cf3bdd193a4..6d43e0f3d6f5 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -203,6 +203,7 @@ void ovl_path_lower(struct dentry *dentry, struct path *path);
 enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path);
 struct dentry *ovl_dentry_upper(struct dentry *dentry);
 struct dentry *ovl_dentry_lower(struct dentry *dentry);
+struct ovl_layer *ovl_layer_lower(struct dentry *dentry);
 struct dentry *ovl_dentry_real(struct dentry *dentry);
 struct dentry *ovl_i_dentry_upper(struct inode *inode);
 struct inode *ovl_inode_upper(struct inode *inode);
@@ -251,7 +252,7 @@ static inline bool ovl_is_impuredir(struct dentry *dentry)
 /* namei.c */
 int ovl_verify_origin(struct dentry *dentry, struct vfsmount *mnt,
 		      struct dentry *origin, bool is_upper, bool set);
-int ovl_verify_index(struct dentry *index, struct path *lowerstack,
+int ovl_verify_index(struct dentry *index, struct ovl_path *lower,
 		     unsigned int numlower);
 int ovl_get_index_name(struct dentry *origin, struct qstr *name);
 int ovl_path_next(int idx, struct dentry *dentry, struct path *path);
@@ -268,7 +269,7 @@ int ovl_check_d_type_supported(struct path *realpath);
 void ovl_workdir_cleanup(struct inode *dir, struct vfsmount *mnt,
 			 struct dentry *dentry, int level);
 int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
-			 struct path *lowerstack, unsigned int numlower);
+			 struct ovl_path *lower, unsigned int numlower);
 
 /* inode.c */
 int ovl_set_nlink_upper(struct dentry *dentry);
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 25d9b5adcd42..7573f98ddec1 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -17,11 +17,22 @@ struct ovl_config {
 	bool index;
 };
 
+struct ovl_layer {
+	struct vfsmount *mnt;
+	/* Index of this layer in mount (upper == 0) */
+	int idx;
+};
+
+struct ovl_path {
+	struct ovl_layer *layer;
+	struct dentry *dentry;
+};
+
 /* private information held for overlayfs's superblock */
 struct ovl_fs {
 	struct vfsmount *upper_mnt;
 	unsigned numlower;
-	struct vfsmount **lower_mnt;
+	struct ovl_layer *lower_layers;
 	/* workbasedir is the path at workdir= mount option */
 	struct dentry *workbasedir;
 	/* workdir is the 'work' directory under workbasedir */
@@ -52,7 +63,7 @@ struct ovl_entry {
 		struct rcu_head rcu;
 	};
 	unsigned numlower;
-	struct path lowerstack[];
+	struct ovl_path lowerstack[];
 };
 
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
index 333de85d6de4..4f2479b6ef91 100644
--- a/fs/overlayfs/readdir.c
+++ b/fs/overlayfs/readdir.c
@@ -1019,7 +1019,7 @@ void ovl_workdir_cleanup(struct inode *dir, struct vfsmount *mnt,
 }
 
 int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
-			 struct path *lowerstack, unsigned int numlower)
+			 struct ovl_path *lower, unsigned int numlower)
 {
 	int err;
 	struct dentry *index = NULL;
@@ -1054,7 +1054,7 @@ int ovl_indexdir_cleanup(struct dentry *dentry, struct vfsmount *mnt,
 			index = NULL;
 			break;
 		}
-		err = ovl_verify_index(index, lowerstack, numlower);
+		err = ovl_verify_index(index, lower, numlower);
 		/* Cleanup stale and orphan index entries */
 		if (err && (err == -ESTALE || err == -ENOENT))
 			err = ovl_cleanup(dir, index);
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 8702803ba328..ceca3a9ac27f 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -220,8 +220,8 @@ static void ovl_put_super(struct super_block *sb)
 		ovl_inuse_unlock(ufs->upper_mnt->mnt_root);
 	mntput(ufs->upper_mnt);
 	for (i = 0; i < ufs->numlower; i++)
-		mntput(ufs->lower_mnt[i]);
-	kfree(ufs->lower_mnt);
+		mntput(ufs->lower_layers[i].mnt);
+	kfree(ufs->lower_layers);
 
 	kfree(ufs->config.lowerdir);
 	kfree(ufs->config.upperdir);
@@ -1026,24 +1026,27 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	}
 
 	err = -ENOMEM;
-	ufs->lower_mnt = kcalloc(numlower, sizeof(struct vfsmount *), GFP_KERNEL);
-	if (ufs->lower_mnt == NULL)
+	ufs->lower_layers = kcalloc(numlower, sizeof(struct ovl_layer),
+				    GFP_KERNEL);
+	if (ufs->lower_layers == NULL)
 		goto out_put_workdir;
 	for (i = 0; i < numlower; i++) {
-		struct vfsmount *mnt = clone_private_mount(&stack[i]);
+		struct vfsmount *mnt;
 
+		mnt = clone_private_mount(&stack[i]);
 		err = PTR_ERR(mnt);
 		if (IS_ERR(mnt)) {
 			pr_err("overlayfs: failed to clone lowerpath\n");
-			goto out_put_lower_mnt;
+			goto out_put_lower_layers;
 		}
 		/*
-		 * Make lower_mnt R/O.  That way fchmod/fchown on lower file
+		 * Make lower layers R/O.  That way fchmod/fchown on lower file
 		 * will fail instead of modifying lower fs.
 		 */
 		mnt->mnt_flags |= MNT_READONLY | MNT_NOATIME;
 
-		ufs->lower_mnt[ufs->numlower] = mnt;
+		ufs->lower_layers[ufs->numlower].mnt = mnt;
+		ufs->lower_layers[ufs->numlower].idx = i + 1;
 		ufs->numlower++;
 
 		/* Check if all lower layers are on same sb */
@@ -1059,13 +1062,25 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb != ufs->same_sb)
 		ufs->same_sb = NULL;
 
+	err = -ENOMEM;
+	oe = ovl_alloc_entry(numlower);
+	if (!oe)
+		goto out_put_lower_layers;
+
+	for (i = 0; i < numlower; i++) {
+		oe->lowerstack[i].dentry = stack[i].dentry;
+		oe->lowerstack[i].layer = &(ufs->lower_layers[i]);
+	}
+
 	if (!(ovl_force_readonly(ufs)) && ufs->config.index) {
 		/* Verify lower root is upper root origin */
-		err = ovl_verify_origin(upperpath.dentry, ufs->lower_mnt[0],
-					stack[0].dentry, false, true);
+		err = ovl_verify_origin(upperpath.dentry,
+					oe->lowerstack[0].layer->mnt,
+					oe->lowerstack[0].dentry,
+					false, true);
 		if (err) {
 			pr_err("overlayfs: failed to verify upper root origin\n");
-			goto out_put_lower_mnt;
+			goto out_free_oe;
 		}
 
 		ufs->indexdir = ovl_workdir_create(sb, ufs, workpath.dentry,
@@ -1081,7 +1096,8 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 			if (!err)
 				err = ovl_indexdir_cleanup(ufs->indexdir,
 							   ufs->upper_mnt,
-							   stack, numlower);
+							   oe->lowerstack,
+							   numlower);
 		}
 		if (err || !ufs->indexdir)
 			pr_warn("overlayfs: try deleting index dir or mounting with '-o index=off' to disable inodes index.\n");
@@ -1106,11 +1122,6 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	/* Never override disk quota limits or use reserved space */
 	cap_lower(cred->cap_effective, CAP_SYS_RESOURCE);
 
-	err = -ENOMEM;
-	oe = ovl_alloc_entry(numlower);
-	if (!oe)
-		goto out_put_cred;
-
 	sb->s_magic = OVERLAYFS_SUPER_MAGIC;
 	sb->s_op = &ovl_super_operations;
 	sb->s_xattr = ovl_xattr_handlers;
@@ -1119,11 +1130,12 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 
 	root_dentry = d_make_root(ovl_new_inode(sb, S_IFDIR, 0));
 	if (!root_dentry)
-		goto out_free_oe;
+		goto out_put_cred;
 
 	mntput(upperpath.mnt);
 	for (i = 0; i < numlower; i++)
 		mntput(stack[i].mnt);
+	kfree(stack);
 	mntput(workpath.mnt);
 	kfree(lowertmp);
 
@@ -1132,11 +1144,6 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 		if (ovl_is_impuredir(upperpath.dentry))
 			ovl_set_flag(OVL_IMPURE, d_inode(root_dentry));
 	}
-	for (i = 0; i < numlower; i++) {
-		oe->lowerstack[i].dentry = stack[i].dentry;
-		oe->lowerstack[i].mnt = ufs->lower_mnt[i];
-	}
-	kfree(stack);
 
 	root_dentry->d_fsdata = oe;
 
@@ -1147,16 +1154,16 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 
 	return 0;
 
-out_free_oe:
-	kfree(oe);
 out_put_cred:
 	put_cred(ufs->creator_cred);
 out_put_indexdir:
 	dput(ufs->indexdir);
-out_put_lower_mnt:
+out_free_oe:
+	kfree(oe);
+out_put_lower_layers:
 	for (i = 0; i < ufs->numlower; i++)
-		mntput(ufs->lower_mnt[i]);
-	kfree(ufs->lower_mnt);
+		mntput(ufs->lower_layers[i].mnt);
+	kfree(ufs->lower_layers);
 out_put_workdir:
 	dput(ufs->workdir);
 	mntput(ufs->upper_mnt);
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 9158d17bb320..3c3b026c189b 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -124,7 +124,12 @@ void ovl_path_lower(struct dentry *dentry, struct path *path)
 {
 	struct ovl_entry *oe = dentry->d_fsdata;
 
-	*path = oe->numlower ? oe->lowerstack[0] : (struct path) { };
+	if (oe->numlower) {
+		path->mnt = oe->lowerstack[0].layer->mnt;
+		path->dentry = oe->lowerstack[0].dentry;
+	} else {
+		*path = (struct path) { };
+	}
 }
 
 enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path)
@@ -151,6 +156,13 @@ struct dentry *ovl_dentry_lower(struct dentry *dentry)
 	return oe->numlower ? oe->lowerstack[0].dentry : NULL;
 }
 
+struct ovl_layer *ovl_layer_lower(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	return oe->numlower ? oe->lowerstack[0].layer : NULL;
+}
+
 struct dentry *ovl_dentry_real(struct dentry *dentry)
 {
 	return ovl_dentry_upper(dentry) ?: ovl_dentry_lower(dentry);
-- 
2.7.4

[PATCH v8 3/9] ovl: allocate anonymous devs for lowerdirs

[Amir Goldstein]

Generate unique values of st_dev per lower layer for non-samefs
overlay mount. The unique values are obtained by allocating anonymous
bdevs for each of the lowerdirs in the overlayfs instance.

The anonymous bdev is going to be returned by stat(2) for lowerdir
non-dir entries in non-samefs case.

[amir: split from ovl_getattr() and re-structure patches]

Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/super.c     | 18 ++++++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 7573f98ddec1..63ef5ae9275b 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -21,6 +21,7 @@ struct ovl_layer {
 	struct vfsmount *mnt;
 	/* Index of this layer in mount (upper == 0) */
 	int idx;
+	dev_t pseudo_dev;
 };
 
 struct ovl_path {
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index ceca3a9ac27f..0b8fe8dca187 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -219,8 +219,10 @@ static void ovl_put_super(struct super_block *sb)
 	if (ufs->upper_mnt && ufs->upperdir_locked)
 		ovl_inuse_unlock(ufs->upper_mnt->mnt_root);
 	mntput(ufs->upper_mnt);
-	for (i = 0; i < ufs->numlower; i++)
+	for (i = 0; i < ufs->numlower; i++) {
 		mntput(ufs->lower_layers[i].mnt);
+		free_anon_bdev(ufs->lower_layers[i].pseudo_dev);
+	}
 	kfree(ufs->lower_layers);
 
 	kfree(ufs->config.lowerdir);
@@ -1032,11 +1034,19 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 		goto out_put_workdir;
 	for (i = 0; i < numlower; i++) {
 		struct vfsmount *mnt;
+		dev_t dev;
+
+		err = get_anon_bdev(&dev);
+		if (err) {
+			pr_err("overlayfs: failed to get anonymous bdev for lowerpath\n");
+			goto out_put_lower_layers;
+		}
 
 		mnt = clone_private_mount(&stack[i]);
 		err = PTR_ERR(mnt);
 		if (IS_ERR(mnt)) {
 			pr_err("overlayfs: failed to clone lowerpath\n");
+			free_anon_bdev(dev);
 			goto out_put_lower_layers;
 		}
 		/*
@@ -1047,6 +1057,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 
 		ufs->lower_layers[ufs->numlower].mnt = mnt;
 		ufs->lower_layers[ufs->numlower].idx = i + 1;
+		ufs->lower_layers[ufs->numlower].pseudo_dev = dev;
 		ufs->numlower++;
 
 		/* Check if all lower layers are on same sb */
@@ -1161,8 +1172,11 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 out_free_oe:
 	kfree(oe);
 out_put_lower_layers:
-	for (i = 0; i < ufs->numlower; i++)
+	for (i = 0; i < ufs->numlower; i++) {
+		if (ufs->lower_layers[i].mnt)
+			free_anon_bdev(ufs->lower_layers[i].pseudo_dev);
 		mntput(ufs->lower_layers[i].mnt);
+	}
 	kfree(ufs->lower_layers);
 out_put_workdir:
 	dput(ufs->workdir);
-- 
2.7.4

[PATCH v8 4/9] ovl: factor out ovl_map_dev_ino() helper

[Amir Goldstein]

A helper for ovl_getattr() to map the values of st_dev and st_ino
according to constant st_ino rules.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c | 52 ++++++++++++++++++++++++++++++++++------------------
 1 file changed, 34 insertions(+), 18 deletions(-)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index b772e5a2a730..6c54ecff56ee 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -58,6 +58,34 @@ int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 	return err;
 }
 
+static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
+{
+	bool samefs = ovl_same_sb(dentry->d_sb);
+
+	if (samefs) {
+		/*
+		 * When all layers are on the same fs, all real inode
+		 * number are unique, so we use the overlay st_dev,
+		 * which is friendly to du -x.
+		 */
+		stat->dev = dentry->d_sb->s_dev;
+	} else if (S_ISDIR(dentry->d_inode->i_mode)) {
+		/*
+		 * Always use the overlay st_dev for directories, so 'find
+		 * -xdev' will scan the entire overlay mount and won't cross the
+		 * overlay mount boundaries.
+		 *
+		 * If not all layers are on the same fs the pair {real st_ino;
+		 * overlay st_dev} is not unique, so use the non persistent
+		 * overlay st_ino for directories.
+		 */
+		stat->dev = dentry->d_sb->s_dev;
+		stat->ino = dentry->d_inode->i_ino;
+	}
+
+	return 0;
+}
+
 int ovl_getattr(const struct path *path, struct kstat *stat,
 		u32 request_mask, unsigned int flags)
 {
@@ -75,13 +103,10 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 		goto out;
 
 	/*
-	 * When all layers are on the same fs, all real inode number are
-	 * unique, so we use the overlay st_dev, which is friendly to du -x.
-	 *
-	 * We also use st_ino of the copy up origin, if we know it.
+	 * When all layers are on the same fs, we use st_ino of copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
 	 *
-	 * If filesystem supports NFS export ops, this also guaranties
+	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
 	if (ovl_same_sb(dentry->d_sb)) {
@@ -108,21 +133,12 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 			    ovl_test_flag(OVL_INDEX, d_inode(dentry)))
 				stat->ino = lowerstat.ino;
 		}
-		stat->dev = dentry->d_sb->s_dev;
-	} else if (is_dir) {
-		/*
-		 * If not all layers are on the same fs the pair {real st_ino;
-		 * overlay st_dev} is not unique, so use the non persistent
-		 * overlay st_ino.
-		 *
-		 * Always use the overlay st_dev for directories, so 'find
-		 * -xdev' will scan the entire overlay mount and won't cross the
-		 * overlay mount boundaries.
-		 */
-		stat->dev = dentry->d_sb->s_dev;
-		stat->ino = dentry->d_inode->i_ino;
 	}
 
+	err = ovl_map_dev_ino(dentry, stat);
+	if (err)
+		goto out;
+
 	/*
 	 * It's probably not worth it to count subdirs to get the
 	 * correct link count.  nlink=1 seems to pacify 'find' and
-- 
2.7.4

[PATCH v8 5/9] ovl: return anonymous st_dev for lower inodes

[Amir Goldstein]

For non-samefs setup, to make sure that st_dev/st_ino pair is unique
across the system, we return a unique anonymous st_dev for stat(2)
of lower layer inode.

A following patch is going to fix constant st_dev/st_ino across copy up
by returning origin st_dev/st_ino for copied up objects.

If the st_dev/st_ino for copied up object would have been the same as
that of the real underlying lower file, running diff on underlying lower
file and overlay copied up file would result in diff reporting that the
2 files are equal when in fact, they may have different content.

[amir: split from allocate anonymous bdev patch
       simplify get pseudo dev code]

Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 6c54ecff56ee..29b0adc77ebe 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -15,6 +15,7 @@
 #include <linux/ratelimit.h>
 #include "overlayfs.h"
 
+
 int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 {
 	int err;
@@ -60,6 +61,7 @@ int ovl_setattr(struct dentry *dentry, struct iattr *attr)
 
 static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 {
+	struct ovl_layer *lower_layer = ovl_layer_lower(dentry);
 	bool samefs = ovl_same_sb(dentry->d_sb);
 
 	if (samefs) {
@@ -81,6 +83,14 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 		 */
 		stat->dev = dentry->d_sb->s_dev;
 		stat->ino = dentry->d_inode->i_ino;
+	} else if (lower_layer) {
+		/*
+		 * For non-samefs setup, if we cannot map all layers st_ino
+		 * to a unified address space, we need to make sure that st_dev
+		 * is unique per layer. Upper layer uses real st_dev and lower
+		 * layers use the unique anonymous bdev.
+		 */
+		stat->dev = lower_layer->pseudo_dev;
 	}
 
 	return 0;
-- 
2.7.4

[PATCH v8 6/9] ovl: relax same fs constraint for constant st_ino

[Amir Goldstein]

For the case of all layers not on the same fs, return the copy up origin
inode st_dev/st_ino for non-dir from stat(2).

This guaranties constant st_dev/st_ino for non-dir across copy up.
Like the same fs case, st_ino of non-dir is also persistent.

If the st_dev/st_ino for copied up object would have been the same as
that of the real underlying lower file, running diff on underlying lower
file and overlay copied up file would result in diff reporting that the
two files are equal when in fact, they may have different content.

Therefore, unlike the same fs case, st_dev is not uniform across all
overlay object, which is less friednly to du -x.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 29b0adc77ebe..80a4b61a1149 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -104,6 +104,7 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 	struct path realpath;
 	const struct cred *old_cred;
 	bool is_dir = S_ISDIR(dentry->d_inode->i_mode);
+	bool samefs = ovl_same_sb(dentry->d_sb);
 	int err;
 
 	type = ovl_path_real(dentry, &realpath);
@@ -113,13 +114,13 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 		goto out;
 
 	/*
-	 * When all layers are on the same fs, we use st_ino of copy up origin.
+	 * For non-dir or same fs, we use st_ino of the copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
 	 *
 	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
-	if (ovl_same_sb(dentry->d_sb)) {
+	if (!is_dir || samefs) {
 		if (OVL_TYPE_ORIGIN(type)) {
 			struct kstat lowerstat;
 			u32 lowermask = STATX_INO | (!is_dir ? STATX_NLINK : 0);
@@ -130,7 +131,9 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 			if (err)
 				goto out;
 
-			WARN_ON_ONCE(stat->dev != lowerstat.dev);
+			if (samefs)
+				WARN_ON_ONCE(stat->dev != lowerstat.dev);
+
 			/*
 			 * Lower hardlinks may be broken on copy up to different
 			 * upper files, so we cannot use the lower origin st_ino
-- 
2.7.4

[PATCH v8 7/9] ovl: constant st_ino for non-samefs with xino

[Amir Goldstein]

When overlay layers are not all on the same fs, but all inode numbers
of underlying fs are not using 'xino' high bits, use the high bits to
partition the overlay st_ino address space.  The high bits hold the
layer index (upper is 0).  This way overlay inode numbers are unique
and all inodes use overlay st_dev.  Inode numbers are also persistent
for a given layer configuration.

Currently, our only indication for available high ino bits is from a
filesystem that supports file handles and uses the default encode_fh()
operation, which encodes a 32bit inode number.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/inode.c     | 23 ++++++++++++++++++++++-
 fs/overlayfs/overlayfs.h |  3 ++-
 fs/overlayfs/ovl_entry.h |  2 ++
 fs/overlayfs/super.c     | 30 +++++++++++++++++++++++-------
 fs/overlayfs/util.c      | 24 +++++++++++++++++++++---
 5 files changed, 70 insertions(+), 12 deletions(-)

diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 80a4b61a1149..521c4d21eb7b 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -63,6 +63,7 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 {
 	struct ovl_layer *lower_layer = ovl_layer_lower(dentry);
 	bool samefs = ovl_same_sb(dentry->d_sb);
+	int xinobits = ovl_xino_bits(dentry->d_sb);
 
 	if (samefs) {
 		/*
@@ -71,6 +72,24 @@ static int ovl_map_dev_ino(struct dentry *dentry, struct kstat *stat)
 		 * which is friendly to du -x.
 		 */
 		stat->dev = dentry->d_sb->s_dev;
+	} else if (xinobits) {
+		/*
+		 * All inode numbers of underlying fs should not be using the
+		 * high xinobits, so we use high xinobits to partition the
+		 * overlay st_ino address space. The high bits holds the layer
+		 * index (upper is 0). This way overlay inode numbers are unique
+		 * and all inodes use overlay st_dev. Inode numbers are also
+		 * persistent for a given layer configuration.
+		 */
+		if (stat->ino >> (64 - xinobits)) {
+			pr_warn_ratelimited("overlayfs: inode number too big (%pd2, ino=%llu, xinobits=%d)\n",
+					    dentry, stat->ino, xinobits);
+			return -EOVERFLOW;
+		}
+
+		if (lower_layer)
+			stat->ino |= ((u64)lower_layer->idx) << (64 - xinobits);
+		stat->dev = dentry->d_sb->s_dev;
 	} else if (S_ISDIR(dentry->d_inode->i_mode)) {
 		/*
 		 * Always use the overlay st_dev for directories, so 'find
@@ -116,11 +135,13 @@ int ovl_getattr(const struct path *path, struct kstat *stat,
 	/*
 	 * For non-dir or same fs, we use st_ino of the copy up origin.
 	 * This guaranties constant st_dev/st_ino across copy up.
+	 * With xino feature and non-samefs, we use st_ino of the copy up
+	 * origin masked with high bits that represent the layer id.
 	 *
 	 * If lower filesystem supports NFS file handles, this also guaranties
 	 * persistent st_ino across mount cycle.
 	 */
-	if (!is_dir || samefs) {
+	if (!is_dir || samefs || ovl_xino_bits(dentry->d_sb)) {
 		if (OVL_TYPE_ORIGIN(type)) {
 			struct kstat lowerstat;
 			u32 lowermask = STATX_INO | (!is_dir ? STATX_NLINK : 0);
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
index 6d43e0f3d6f5..29569bf1cc6e 100644
--- a/fs/overlayfs/overlayfs.h
+++ b/fs/overlayfs/overlayfs.h
@@ -192,7 +192,8 @@ void ovl_drop_write(struct dentry *dentry);
 struct dentry *ovl_workdir(struct dentry *dentry);
 const struct cred *ovl_override_creds(struct super_block *sb);
 struct super_block *ovl_same_sb(struct super_block *sb);
-bool ovl_can_decode_fh(struct super_block *sb);
+int ovl_xino_bits(struct super_block *sb);
+int ovl_can_decode_fh(struct super_block *sb);
 struct dentry *ovl_indexdir(struct super_block *sb);
 struct ovl_entry *ovl_alloc_entry(unsigned int numlower);
 bool ovl_dentry_remote(struct dentry *dentry);
diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 63ef5ae9275b..226cf94838a3 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -52,6 +52,8 @@ struct ovl_fs {
 	/* Did we take the inuse lock? */
 	bool upperdir_locked;
 	bool workdir_locked;
+	/* Inode numbers in all layers do not use the high xino_bits */
+	int xino_bits;
 };
 
 /* private information held for every overlayfs dentry */
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 0b8fe8dca187..ddedfca9305c 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -17,6 +17,7 @@
 #include <linux/statfs.h>
 #include <linux/seq_file.h>
 #include <linux/posix_acl_xattr.h>
+#include <linux/exportfs.h>
 #include "overlayfs.h"
 
 MODULE_AUTHOR("Miklos Szeredi <miklos@szeredi.hu>");
@@ -629,6 +630,7 @@ static int ovl_check_namelen(struct path *path, struct ovl_fs *ofs,
 static int ovl_lower_dir(const char *name, struct path *path,
 			 struct ovl_fs *ofs, int *stack_depth, bool *remote)
 {
+	int fh_type;
 	int err;
 
 	err = ovl_mount_dir_noesc(name, path);
@@ -648,10 +650,14 @@ static int ovl_lower_dir(const char *name, struct path *path,
 	 * The inodes index feature needs to encode and decode file
 	 * handles, so it requires that all layers support them.
 	 */
-	if (ofs->config.index && !ovl_can_decode_fh(path->dentry->d_sb)) {
+	fh_type = ovl_can_decode_fh(path->dentry->d_sb);
+	if (ofs->config.index && !fh_type) {
 		ofs->config.index = false;
 		pr_warn("overlayfs: fs on '%s' does not support file handles, falling back to index=off.\n", name);
 	}
+	/* Check if lower fs has 32bit inode numbers */
+	if (fh_type != FILEID_INO32_GEN)
+		ofs->xino_bits = 0;
 
 	return 0;
 
@@ -842,6 +848,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	unsigned int i;
 	bool remote = false;
 	struct cred *cred;
+	int fh_type;
 	int err;
 
 	err = -ENOMEM;
@@ -961,6 +968,8 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 		goto out_put_lowerpath;
 	}
 
+	/* Assume underlaying fs uses 32bit inodes unless proven otherwise */
+	ufs->xino_bits = 32;
 	if (ufs->config.upperdir) {
 		ufs->upper_mnt = clone_private_mount(&upperpath);
 		err = PTR_ERR(ufs->upper_mnt);
@@ -1017,14 +1026,17 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 			} else {
 				vfs_removexattr(ufs->workdir, OVL_XATTR_OPAQUE);
 			}
+		}
 
-			/* Check if upper/work fs supports file handles */
-			if (ufs->config.index &&
-			    !ovl_can_decode_fh(ufs->workdir->d_sb)) {
-				ufs->config.index = false;
-				pr_warn("overlayfs: upper fs does not support file handles, falling back to index=off.\n");
-			}
+		/* Check if upper fs supports file handles */
+		fh_type = ovl_can_decode_fh(ufs->upper_mnt->mnt_sb);
+		if (ufs->config.index && !fh_type) {
+			ufs->config.index = false;
+			pr_warn("overlayfs: upper fs does not support file handles, falling back to index=off.\n");
 		}
+		/* Check if upper fs has 32bit inode numbers */
+		if (fh_type != FILEID_INO32_GEN)
+			ufs->xino_bits = 0;
 	}
 
 	err = -ENOMEM;
@@ -1073,6 +1085,10 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb != ufs->same_sb)
 		ufs->same_sb = NULL;
 
+	/* When all layers on same fs, overlay can use real inode numbers */
+	if (ufs->same_sb)
+		ufs->xino_bits = 0;
+
 	err = -ENOMEM;
 	oe = ovl_alloc_entry(numlower);
 	if (!oe)
diff --git a/fs/overlayfs/util.c b/fs/overlayfs/util.c
index 3c3b026c189b..255ae337fc46 100644
--- a/fs/overlayfs/util.c
+++ b/fs/overlayfs/util.c
@@ -50,10 +50,28 @@ struct super_block *ovl_same_sb(struct super_block *sb)
 	return ofs->same_sb;
 }
 
-bool ovl_can_decode_fh(struct super_block *sb)
+int ovl_xino_bits(struct super_block *sb)
 {
-	return (sb->s_export_op && sb->s_export_op->fh_to_dentry &&
-		!uuid_is_null(&sb->s_uuid));
+	struct ovl_fs *ofs = sb->s_fs_info;
+
+	return ofs->xino_bits;
+}
+
+/*
+ * Check if underlying fs supports file handles and try to determine encoding
+ * type, in order to deduce maximum inode number used by fs.
+ *
+ * Return 0 if file handles are not supported.
+ * Return 1 (FILEID_INO32_GEN) if fs uses the default 32bit inode encoding.
+ * Return -1 if fs uses a non default encoding with unknown inode size.
+ */
+int ovl_can_decode_fh(struct super_block *sb)
+{
+	if (!sb->s_export_op || !sb->s_export_op->fh_to_dentry ||
+	    uuid_is_null(&sb->s_uuid))
+		return 0;
+
+	return sb->s_export_op->encode_fh ? -1 : FILEID_INO32_GEN;
 }
 
 struct dentry *ovl_indexdir(struct super_block *sb)
-- 
2.7.4

[PATCH v8 8/9] ovl: consistent d_ino for non-samefs with xino

[Amir Goldstein]

When overlay layers are not all on the same fs, but all inode numbers
of underlying fs do not use the high 'xino' bits, overlay st_ino values
are constant and persistent.

In that case, relax non-samefs constraint for consistent d_ino and always
iterate non-merge dir using ovl_fill_real() actor so we can remap lower
inode numbers to unique layer range.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/readdir.c | 45 +++++++++++++++++++++++++++++++++++++++------
 1 file changed, 39 insertions(+), 6 deletions(-)

diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
index 4f2479b6ef91..92a6dcdbdaf7 100644
--- a/fs/overlayfs/readdir.c
+++ b/fs/overlayfs/readdir.c
@@ -120,6 +120,10 @@ static bool ovl_calc_d_ino(struct ovl_readdir_data *rdd,
 	if (!rdd->dentry)
 		return false;
 
+	/* Always recalc d_ino when remapping lower inode numbers */
+	if (ovl_xino_bits(rdd->dentry->d_sb))
+		return true;
+
 	/* Always recalc d_ino for parent */
 	if (strcmp(p->name, "..") == 0)
 		return true;
@@ -440,6 +444,19 @@ static struct ovl_dir_cache *ovl_cache_get(struct dentry *dentry)
 	return cache;
 }
 
+/* Map inode number to lower layer unique range */
+static u64 ovl_remap_lower_ino(u64 ino, int xinobits, int idx,
+			       const char *name, int namelen)
+{
+	if (ino >> (64 - xinobits)) {
+		pr_warn_ratelimited("overlayfs: d_ino too big (%.*s, ino=%llu, xinobits=%d)\n",
+				    namelen, name, ino, xinobits);
+		return ino;
+	}
+
+	return ino | ((u64)idx) << (64 - xinobits);
+}
+
 /*
  * Set d_ino for upper entries. Non-upper entries should always report
  * the uppermost real inode ino and should not call this function.
@@ -457,9 +474,10 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p)
 	struct dentry *this = NULL;
 	enum ovl_path_type type;
 	u64 ino = p->real_ino;
+	int xinobits = ovl_xino_bits(dir->d_sb);
 	int err = 0;
 
-	if (!ovl_same_sb(dir->d_sb))
+	if (!ovl_same_sb(dir->d_sb) && !xinobits)
 		goto out;
 
 	if (p->name[0] == '.') {
@@ -496,6 +514,10 @@ static int ovl_cache_update_ino(struct path *path, struct ovl_cache_entry *p)
 
 		WARN_ON_ONCE(dir->d_sb->s_dev != stat.dev);
 		ino = stat.ino;
+	} else if (xinobits && !OVL_TYPE_UPPER(type)) {
+		ino = ovl_remap_lower_ino(ino, xinobits,
+					  ovl_layer_lower(this)->idx,
+					  p->name, p->len);
 	}
 
 out:
@@ -616,6 +638,8 @@ struct ovl_readdir_translate {
 	struct ovl_dir_cache *cache;
 	struct dir_context ctx;
 	u64 parent_ino;
+	int layer_idx;
+	int xinobits;
 };
 
 static int ovl_fill_real(struct dir_context *ctx, const char *name,
@@ -626,14 +650,17 @@ static int ovl_fill_real(struct dir_context *ctx, const char *name,
 		container_of(ctx, struct ovl_readdir_translate, ctx);
 	struct dir_context *orig_ctx = rdt->orig_ctx;
 
-	if (rdt->parent_ino && strcmp(name, "..") == 0)
+	if (rdt->parent_ino && strcmp(name, "..") == 0) {
 		ino = rdt->parent_ino;
-	else if (rdt->cache) {
+	} else if (rdt->cache) {
 		struct ovl_cache_entry *p;
 
 		p = ovl_cache_entry_find(&rdt->cache->root, name, namelen);
 		if (p)
 			ino = p->ino;
+	} else if (rdt->xinobits) {
+		ino = ovl_remap_lower_ino(ino, rdt->xinobits, rdt->layer_idx,
+					  name, namelen);
 	}
 
 	return orig_ctx->actor(orig_ctx, name, namelen, offset, ino, d_type);
@@ -644,11 +671,16 @@ static int ovl_iterate_real(struct file *file, struct dir_context *ctx)
 	int err;
 	struct ovl_dir_file *od = file->private_data;
 	struct dentry *dir = file->f_path.dentry;
+	struct ovl_layer *lower_layer = ovl_layer_lower(dir);
 	struct ovl_readdir_translate rdt = {
 		.ctx.actor = ovl_fill_real,
 		.orig_ctx = ctx,
+		.xinobits = ovl_xino_bits(dir->d_sb),
 	};
 
+	if (rdt.xinobits && lower_layer)
+		rdt.layer_idx = lower_layer->idx;
+
 	if (OVL_TYPE_MERGE(ovl_path_type(dir->d_parent))) {
 		struct kstat stat;
 		struct path statpath = file->f_path;
@@ -688,9 +720,10 @@ static int ovl_iterate(struct file *file, struct dir_context *ctx)
 		 * dir is impure then need to adjust d_ino for copied up
 		 * entries.
 		 */
-		if (ovl_same_sb(dentry->d_sb) &&
-		    (ovl_test_flag(OVL_IMPURE, d_inode(dentry)) ||
-		     OVL_TYPE_MERGE(ovl_path_type(dentry->d_parent)))) {
+		if (ovl_xino_bits(dentry->d_sb) ||
+		    (ovl_same_sb(dentry->d_sb) &&
+		     (ovl_test_flag(OVL_IMPURE, d_inode(dentry)) ||
+		      OVL_TYPE_MERGE(ovl_path_type(dentry->d_parent))))) {
 			return ovl_iterate_real(file, ctx);
 		}
 		return iterate_dir(od->realfile, ctx);
-- 
2.7.4

[PATCH v8 9/9] ovl: add support for 'xino' mount option

[Amir Goldstein]

With mount option 'xino', mounter declares that there are enough
free high bits in underlying fs to hold the layer id.
If overlayfs does encounter underlying inodes using the high xino
bits reserved for layer id, stat(2) will return -EOVERLFLOW.

The mount option name 'xino' goes after a similar meaning mount option
of aufs, but in overlayfs case, the mapping is stateless.

An example for a use case of 'xino' is when upper/lower is on an xfs
filesystem mounted with mount option 'inode32' or on an xfs filesystem
that is not huge.  If xfs filesystem AG count is limited by device size
to under 2^(31-x), then it is safe to use x high blocks for the overlay
layer id.

Signed-off-by: Amir Goldstein <amir73il@gmail.com>
---
 fs/overlayfs/ovl_entry.h |  1 +
 fs/overlayfs/super.c     | 19 ++++++++++++++++++-
 2 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/fs/overlayfs/ovl_entry.h b/fs/overlayfs/ovl_entry.h
index 226cf94838a3..13140028f2bb 100644
--- a/fs/overlayfs/ovl_entry.h
+++ b/fs/overlayfs/ovl_entry.h
@@ -15,6 +15,7 @@ struct ovl_config {
 	bool default_permissions;
 	bool redirect_dir;
 	bool index;
+	bool xino;
 };
 
 struct ovl_layer {
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index ddedfca9305c..3f41ddf6bd0d 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -308,6 +308,8 @@ static int ovl_show_options(struct seq_file *m, struct dentry *dentry)
 	if (ufs->config.index != ovl_index_def)
 		seq_printf(m, ",index=%s",
 			   ufs->config.index ? "on" : "off");
+	if (ufs->config.xino)
+		seq_puts(m, ",xino");
 	return 0;
 }
 
@@ -341,6 +343,7 @@ enum {
 	OPT_REDIRECT_DIR_OFF,
 	OPT_INDEX_ON,
 	OPT_INDEX_OFF,
+	OPT_XINO,
 	OPT_ERR,
 };
 
@@ -353,6 +356,7 @@ static const match_table_t ovl_tokens = {
 	{OPT_REDIRECT_DIR_OFF,		"redirect_dir=off"},
 	{OPT_INDEX_ON,			"index=on"},
 	{OPT_INDEX_OFF,			"index=off"},
+	{OPT_XINO,			"xino"},
 	{OPT_ERR,			NULL}
 };
 
@@ -433,6 +437,10 @@ static int ovl_parse_opt(char *opt, struct ovl_config *config)
 			config->index = false;
 			break;
 
+		case OPT_XINO:
+			config->xino = true;
+			break;
+
 		default:
 			pr_err("overlayfs: unrecognized mount option \"%s\" or missing value\n", p);
 			return -EINVAL;
@@ -1085,9 +1093,18 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	else if (ufs->upper_mnt->mnt_sb != ufs->same_sb)
 		ufs->same_sb = NULL;
 
-	/* When all layers on same fs, overlay can use real inode numbers */
+	/*
+	 * When all layers on same fs, overlay can use real inode numbers.
+	 * With mount option 'xino', mounter declares that there are enough
+	 * free high bits in underlying fs to hold the layer id.
+	 * If overlayfs does encounter underlying inodes using the high xino
+	 * bits reserved for layer id, stat(2) will return -EOVERLFLOW.
+	 */
+	BUILD_BUG_ON(ilog2(OVL_MAX_STACK) > 31);
 	if (ufs->same_sb)
 		ufs->xino_bits = 0;
+	else if (ufs->config.xino && !ufs->xino_bits)
+		ufs->xino_bits = ilog2(ufs->numlower) + 1;
 
 	err = -ENOMEM;
 	oe = ovl_alloc_entry(numlower);
-- 
2.7.4

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Vivek Goyal]

Hi Miklos,

These patches apply on top of which tree/branch. I am trying linus's
latest tree and these fail to apply.

Vivek

On Tue, Nov 07, 2017 at 06:58:00PM +0200, Amir Goldstein wrote:
> Miklos,
> 
> This version provides a solution for some interesting non-samefs cases:
> - All the ext* family
> - Many other fs with default encode_fh
> - xfs that is not huge with overlay 'xino' mount option
> - tmpfs that is not on a machine with jurassic uptime with 'xino'
> 
> I tested this with Chandan's upstream overlay/041 xfstest for
> consistent d_ino in non-samefs setup. Results are:
> - Test passes for ext4
> - Test fails for xfs
> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>   but this is because of a test bug
> - With the test bug fix available at [2] test passes
>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
> 
> All the exportfs tests also pass with these changes and
> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
> 
> Changes since v7:
> - Drop patches for building impure cache for non-samefs subdirs
> - Dropped patch "update cache version of impure parent on rename"
>   because it is not relevant to this series
> - Remap lower inode numbers for 32bit inode file systems
> - Add mount option 'xino' for opting-in to use high inode bits
> 
> Thanks,
> Amir.
> 
> [1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
> [2] https://github.com/amir73il/xfstests/commits/overlayfs-devel
> 
> Amir Goldstein (8):
>   ovl: move include of ovl_entry.h into overlayfs.h
>   ovl: allocate anonymous devs for lowerdirs
>   ovl: factor out ovl_map_dev_ino() helper
>   ovl: return anonymous st_dev for lower inodes
>   ovl: relax same fs constraint for constant st_ino
>   ovl: constant st_ino for non-samefs with xino
>   ovl: consistent d_ino for non-samefs with xino
>   ovl: add support for 'xino' mount option
> 
> Chandan Rajendra (1):
>   ovl: re-structure overlay lower layers in-memory
> 
>  fs/overlayfs/copy_up.c   |   1 -
>  fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
>  fs/overlayfs/namei.c     |  42 +++++++--------
>  fs/overlayfs/overlayfs.h |   9 ++--
>  fs/overlayfs/ovl_entry.h |  19 ++++++-
>  fs/overlayfs/readdir.c   |  49 +++++++++++++++---
>  fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
>  fs/overlayfs/util.c      |  39 ++++++++++++--
>  8 files changed, 277 insertions(+), 102 deletions(-)
> 
> -- 
> 2.7.4

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Tue, Nov 7, 2017 at 9:39 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> Hi Miklos,
>
> These patches apply on top of which tree/branch. I am trying linus's
> latest tree and these fail to apply.
>

Hi Vivek,

Sorry, I forgot to mention, those patches are based on top of
the series "Overlayfs fixes for exposed whiteouts" posted on Nov 1
and one lockdep fix patch posted on Oct 27.

You can fetch them all from my tree. I write the branch below [1]
just forgot to refer to it in the text...

FYI, there are 3 more patches on my overlayfs-devel branch [3],
which I intend to post for v4.15. They are complementary patches
to the "fixes for exposed whiteouts" series.

Amir.

>
> On Tue, Nov 07, 2017 at 06:58:00PM +0200, Amir Goldstein wrote:
>> Miklos,
>>
>> This version provides a solution for some interesting non-samefs cases:
>> - All the ext* family
>> - Many other fs with default encode_fh
>> - xfs that is not huge with overlay 'xino' mount option
>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>
>> I tested this with Chandan's upstream overlay/041 xfstest for
>> consistent d_ino in non-samefs setup. Results are:
>> - Test passes for ext4
>> - Test fails for xfs
>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>   but this is because of a test bug
>> - With the test bug fix available at [2] test passes
>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>
>> All the exportfs tests also pass with these changes and
>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>
>> Changes since v7:
>> - Drop patches for building impure cache for non-samefs subdirs
>> - Dropped patch "update cache version of impure parent on rename"
>>   because it is not relevant to this series
>> - Remap lower inode numbers for 32bit inode file systems
>> - Add mount option 'xino' for opting-in to use high inode bits
>>
>> Thanks,
>> Amir.
>>
>> [1] https://github.com/amir73il/linux/commits/ovl-nonsamefs-v8
>> [2] https://github.com/amir73il/xfstests/commits/overlayfs-devel

[3] https://github.com/amir73il/linux/commits/overlayfs-devel

>>
>> Amir Goldstein (8):
>>   ovl: move include of ovl_entry.h into overlayfs.h
>>   ovl: allocate anonymous devs for lowerdirs
>>   ovl: factor out ovl_map_dev_ino() helper
>>   ovl: return anonymous st_dev for lower inodes
>>   ovl: relax same fs constraint for constant st_ino
>>   ovl: constant st_ino for non-samefs with xino
>>   ovl: consistent d_ino for non-samefs with xino
>>   ovl: add support for 'xino' mount option
>>
>> Chandan Rajendra (1):
>>   ovl: re-structure overlay lower layers in-memory
>>
>>  fs/overlayfs/copy_up.c   |   1 -
>>  fs/overlayfs/inode.c     |  91 +++++++++++++++++++++++++--------
>>  fs/overlayfs/namei.c     |  42 +++++++--------
>>  fs/overlayfs/overlayfs.h |   9 ++--
>>  fs/overlayfs/ovl_entry.h |  19 ++++++-
>>  fs/overlayfs/readdir.c   |  49 +++++++++++++++---
>>  fs/overlayfs/super.c     | 129 +++++++++++++++++++++++++++++++++--------------
>>  fs/overlayfs/util.c      |  39 ++++++++++++--
>>  8 files changed, 277 insertions(+), 102 deletions(-)
>>
>> --
>> 2.7.4

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Miklos Szeredi]

On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> Miklos,
>
> This version provides a solution for some interesting non-samefs cases:
> - All the ext* family
> - Many other fs with default encode_fh
> - xfs that is not huge with overlay 'xino' mount option
> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>
> I tested this with Chandan's upstream overlay/041 xfstest for
> consistent d_ino in non-samefs setup. Results are:
> - Test passes for ext4
> - Test fails for xfs
> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>   but this is because of a test bug
> - With the test bug fix available at [2] test passes
>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>
> All the exportfs tests also pass with these changes and
> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>
> Changes since v7:
> - Drop patches for building impure cache for non-samefs subdirs
> - Dropped patch "update cache version of impure parent on rename"
>   because it is not relevant to this series
> - Remap lower inode numbers for 32bit inode file systems
> - Add mount option 'xino' for opting-in to use high inode bits

I the meantime I went and committed v7 (with the noted changes) and
based my cleanup for ovl_fill_super() on top of that.  So that's now
pushed out to overlayfs-next, hopefully not invalidating too much of
your work.

The -oxino patches are interesting, but maybe we should leave them
brewing for another cycle.  Do you agree?

I'll try and look at the other series posted by you and Vivek to see
what else would fit in to 4.15.

Thanks,
Miklos

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> Miklos,
>>
>> This version provides a solution for some interesting non-samefs cases:
>> - All the ext* family
>> - Many other fs with default encode_fh
>> - xfs that is not huge with overlay 'xino' mount option
>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>
>> I tested this with Chandan's upstream overlay/041 xfstest for
>> consistent d_ino in non-samefs setup. Results are:
>> - Test passes for ext4
>> - Test fails for xfs
>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>   but this is because of a test bug
>> - With the test bug fix available at [2] test passes
>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>
>> All the exportfs tests also pass with these changes and
>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>
>> Changes since v7:
>> - Drop patches for building impure cache for non-samefs subdirs
>> - Dropped patch "update cache version of impure parent on rename"
>>   because it is not relevant to this series
>> - Remap lower inode numbers for 32bit inode file systems
>> - Add mount option 'xino' for opting-in to use high inode bits
>
> I the meantime I went and committed v7 (with the noted changes) and
> based my cleanup for ovl_fill_super() on top of that.  So that's now

Nice cleanup!
I don't like setting of s_d_op inside ovl_get_lowerstack().
IMO should add ufs->remote boolean because it is interesting anyway
and set s_d_op in ovl_fill_super() itself.

> pushed out to overlayfs-next, hopefully not invalidating too much of
> your work.

Not a problem. most of the work on v8 was on the last patches.
I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.

>
> The -oxino patches are interesting, but maybe we should leave them
> brewing for another cycle.  Do you agree?
>

Sure. your call.

> I'll try and look at the other series posted by you and Vivek to see
> what else would fit in to 4.15.

OK. I need to address a small comment by Zhangyi on restoring origin fh.
Will push a fix today.

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> Miklos,
>>>
>>> This version provides a solution for some interesting non-samefs cases:
>>> - All the ext* family
>>> - Many other fs with default encode_fh
>>> - xfs that is not huge with overlay 'xino' mount option
>>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>>
>>> I tested this with Chandan's upstream overlay/041 xfstest for
>>> consistent d_ino in non-samefs setup. Results are:
>>> - Test passes for ext4
>>> - Test fails for xfs
>>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>>   but this is because of a test bug
>>> - With the test bug fix available at [2] test passes
>>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>>
>>> All the exportfs tests also pass with these changes and
>>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>>
>>> Changes since v7:
>>> - Drop patches for building impure cache for non-samefs subdirs
>>> - Dropped patch "update cache version of impure parent on rename"
>>>   because it is not relevant to this series
>>> - Remap lower inode numbers for 32bit inode file systems
>>> - Add mount option 'xino' for opting-in to use high inode bits
>>
>> I the meantime I went and committed v7 (with the noted changes) and
>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>
> Nice cleanup!
> I don't like setting of s_d_op inside ovl_get_lowerstack().
> IMO should add ufs->remote boolean because it is interesting anyway
> and set s_d_op in ovl_fill_super() itself.
>

Mikos,

ovl_fill_super() cleanup caused 3 regressions with xfstest
check -overlay -g overlay/quick
overlay/011 overlay/035 - mount failures instead of r/o mount
overlay/022 - dentries in use after mount failure:

BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
[unmount of overlay overlay]
BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
[unmount of xfs vdf]
BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)
[unmount of xfs vdf]

Another regression caused by tweaking OVL_WHITEOUTS patch:
overlay/038 - A fix to the readdir regression is pushed to:
https://github.com/amir73il/linux/commits/ovl-fixes

>> pushed out to overlayfs-next, hopefully not invalidating too much of
>> your work.
>
> Not a problem. most of the work on v8 was on the last patches.
> I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.
>
>>
>> The -oxino patches are interesting, but maybe we should leave them
>> brewing for another cycle.  Do you agree?
>>
>
> Sure. your call.
>
>> I'll try and look at the other series posted by you and Vivek to see
>> what else would fit in to 4.15.
>
> OK. I need to address a small comment by Zhangyi on restoring origin fh.
> Will push a fix today.
>

Pushed rebased and fixes version to:
https://github.com/amir73il/linux/commits/ovl-whiteouts

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Wed, Nov 8, 2017 at 3:40 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>> Miklos,
>>>>
>>>> This version provides a solution for some interesting non-samefs cases:
>>>> - All the ext* family
>>>> - Many other fs with default encode_fh
>>>> - xfs that is not huge with overlay 'xino' mount option
>>>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>>>
>>>> I tested this with Chandan's upstream overlay/041 xfstest for
>>>> consistent d_ino in non-samefs setup. Results are:
>>>> - Test passes for ext4
>>>> - Test fails for xfs
>>>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>>>   but this is because of a test bug
>>>> - With the test bug fix available at [2] test passes
>>>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>>>
>>>> All the exportfs tests also pass with these changes and
>>>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>>>
>>>> Changes since v7:
>>>> - Drop patches for building impure cache for non-samefs subdirs
>>>> - Dropped patch "update cache version of impure parent on rename"
>>>>   because it is not relevant to this series
>>>> - Remap lower inode numbers for 32bit inode file systems
>>>> - Add mount option 'xino' for opting-in to use high inode bits
>>>
>>> I the meantime I went and committed v7 (with the noted changes) and
>>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>>
>> Nice cleanup!
>> I don't like setting of s_d_op inside ovl_get_lowerstack().
>> IMO should add ufs->remote boolean because it is interesting anyway
>> and set s_d_op in ovl_fill_super() itself.
>>
>
> Mikos,
>
> ovl_fill_super() cleanup caused 3 regressions with xfstest
> check -overlay -g overlay/quick
> overlay/011 overlay/035 - mount failures instead of r/o mount

Fixed by "fix free of ERR_PTR oe"

> overlay/022 - dentries in use after mount failure:
>
> BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
> [unmount of overlay overlay]
> BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
> [unmount of xfs vdf]
> BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)
> [unmount of xfs vdf]
>

Fixed by "fix double path_put() on error"

> Another regression caused by tweaking OVL_WHITEOUTS patch:
> overlay/038 - A fix to the readdir regression is pushed to:
> https://github.com/amir73il/linux/commits/ovl-fixes

Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Miklos Szeredi]

On Wed, Nov 8, 2017 at 5:50 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Nov 8, 2017 at 3:40 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>> Miklos,
>>>>>
>>>>> This version provides a solution for some interesting non-samefs cases:
>>>>> - All the ext* family
>>>>> - Many other fs with default encode_fh
>>>>> - xfs that is not huge with overlay 'xino' mount option
>>>>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>>>>
>>>>> I tested this with Chandan's upstream overlay/041 xfstest for
>>>>> consistent d_ino in non-samefs setup. Results are:
>>>>> - Test passes for ext4
>>>>> - Test fails for xfs
>>>>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>>>>   but this is because of a test bug
>>>>> - With the test bug fix available at [2] test passes
>>>>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>>>>
>>>>> All the exportfs tests also pass with these changes and
>>>>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>>>>
>>>>> Changes since v7:
>>>>> - Drop patches for building impure cache for non-samefs subdirs
>>>>> - Dropped patch "update cache version of impure parent on rename"
>>>>>   because it is not relevant to this series
>>>>> - Remap lower inode numbers for 32bit inode file systems
>>>>> - Add mount option 'xino' for opting-in to use high inode bits
>>>>
>>>> I the meantime I went and committed v7 (with the noted changes) and
>>>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>>>
>>> Nice cleanup!
>>> I don't like setting of s_d_op inside ovl_get_lowerstack().
>>> IMO should add ufs->remote boolean because it is interesting anyway
>>> and set s_d_op in ovl_fill_super() itself.
>>>
>>
>> Mikos,
>>
>> ovl_fill_super() cleanup caused 3 regressions with xfstest
>> check -overlay -g overlay/quick
>> overlay/011 overlay/035 - mount failures instead of r/o mount
>
> Fixed by "fix free of ERR_PTR oe"
>
>> overlay/022 - dentries in use after mount failure:
>>
>> BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
>> [unmount of overlay overlay]
>> BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
>> [unmount of xfs vdf]
>> BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)
>> [unmount of xfs vdf]
>>
>
> Fixed by "fix double path_put() on error"
>
>> Another regression caused by tweaking OVL_WHITEOUTS patch:
>> overlay/038 - A fix to the readdir regression is pushed to:
>> https://github.com/amir73il/linux/commits/ovl-fixes
>
> Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.

Thanks for testing and looking at the issues.  I've now set up
xfstests  as well in my test environment...

Force pushed branch with fixes to overlayfs-next.

Thanks,
Miklos

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Thu, Nov 9, 2017 at 11:33 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Wed, Nov 8, 2017 at 5:50 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Nov 8, 2017 at 3:40 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>>> Miklos,
>>>>>>
>>>>>> This version provides a solution for some interesting non-samefs cases:
>>>>>> - All the ext* family
>>>>>> - Many other fs with default encode_fh
>>>>>> - xfs that is not huge with overlay 'xino' mount option
>>>>>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>>>>>
>>>>>> I tested this with Chandan's upstream overlay/041 xfstest for
>>>>>> consistent d_ino in non-samefs setup. Results are:
>>>>>> - Test passes for ext4
>>>>>> - Test fails for xfs
>>>>>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>>>>>   but this is because of a test bug
>>>>>> - With the test bug fix available at [2] test passes
>>>>>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>>>>>
>>>>>> All the exportfs tests also pass with these changes and
>>>>>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>>>>>
>>>>>> Changes since v7:
>>>>>> - Drop patches for building impure cache for non-samefs subdirs
>>>>>> - Dropped patch "update cache version of impure parent on rename"
>>>>>>   because it is not relevant to this series
>>>>>> - Remap lower inode numbers for 32bit inode file systems
>>>>>> - Add mount option 'xino' for opting-in to use high inode bits
>>>>>
>>>>> I the meantime I went and committed v7 (with the noted changes) and
>>>>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>>>>
>>>> Nice cleanup!
>>>> I don't like setting of s_d_op inside ovl_get_lowerstack().
>>>> IMO should add ufs->remote boolean because it is interesting anyway
>>>> and set s_d_op in ovl_fill_super() itself.
>>>>
>>>
>>> Mikos,
>>>
>>> ovl_fill_super() cleanup caused 3 regressions with xfstest
>>> check -overlay -g overlay/quick
>>> overlay/011 overlay/035 - mount failures instead of r/o mount
>>
>> Fixed by "fix free of ERR_PTR oe"
>>
>>> overlay/022 - dentries in use after mount failure:
>>>
>>> BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
>>> [unmount of overlay overlay]
>>> BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
>>> [unmount of xfs vdf]
>>> BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)
>>> [unmount of xfs vdf]
>>>
>>
>> Fixed by "fix double path_put() on error"
>>
>>> Another regression caused by tweaking OVL_WHITEOUTS patch:
>>> overlay/038 - A fix to the readdir regression is pushed to:
>>> https://github.com/amir73il/linux/commits/ovl-fixes
>>
>> Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.
>
> Thanks for testing and looking at the issues.  I've now set up
> xfstests  as well in my test environment...
>

You may want to try out Ted's kvm-xfstests if you haven't already.
I've set it up to also run unionmount testsuite tests from within xfstests:

https://github.com/amir73il/xfstests-bld/commits/overlayfs-devel

> Force pushed branch with fixes to overlayfs-next.
>

Looks good.
Will test.

Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Thu, Nov 9, 2017 at 1:25 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Thu, Nov 9, 2017 at 11:33 AM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Wed, Nov 8, 2017 at 5:50 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> On Wed, Nov 8, 2017 at 3:40 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>> On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>>>>>> Miklos,
>>>>>>>
>>>>>>> This version provides a solution for some interesting non-samefs cases:
>>>>>>> - All the ext* family
>>>>>>> - Many other fs with default encode_fh
>>>>>>> - xfs that is not huge with overlay 'xino' mount option
>>>>>>> - tmpfs that is not on a machine with jurassic uptime with 'xino'
>>>>>>>
>>>>>>> I tested this with Chandan's upstream overlay/041 xfstest for
>>>>>>> consistent d_ino in non-samefs setup. Results are:
>>>>>>> - Test passes for ext4
>>>>>>> - Test fails for xfs
>>>>>>> - Test fails for xfs with OVERLAY_MOUNT_OPTIONS=-oxino,
>>>>>>>   but this is because of a test bug
>>>>>>> - With the test bug fix available at [2] test passes
>>>>>>>   with xfs and OVERLAY_MOUNT_OPTIONS=-oxino
>>>>>>>
>>>>>>> All the exportfs tests also pass with these changes and
>>>>>>> either ext4 or xfs with OVERLAY_MOUNT_OPTIONS=-oxino.
>>>>>>>
>>>>>>> Changes since v7:
>>>>>>> - Drop patches for building impure cache for non-samefs subdirs
>>>>>>> - Dropped patch "update cache version of impure parent on rename"
>>>>>>>   because it is not relevant to this series
>>>>>>> - Remap lower inode numbers for 32bit inode file systems
>>>>>>> - Add mount option 'xino' for opting-in to use high inode bits
>>>>>>
>>>>>> I the meantime I went and committed v7 (with the noted changes) and
>>>>>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>>>>>
>>>>> Nice cleanup!
>>>>> I don't like setting of s_d_op inside ovl_get_lowerstack().
>>>>> IMO should add ufs->remote boolean because it is interesting anyway
>>>>> and set s_d_op in ovl_fill_super() itself.
>>>>>
>>>>
>>>> Mikos,
>>>>
>>>> ovl_fill_super() cleanup caused 3 regressions with xfstest
>>>> check -overlay -g overlay/quick
>>>> overlay/011 overlay/035 - mount failures instead of r/o mount
>>>
>>> Fixed by "fix free of ERR_PTR oe"
>>>
>>>> overlay/022 - dentries in use after mount failure:
>>>>
>>>> BUG: Dentry ffff880079fee2f8{i=42ff,n=upper}  still in use (-1)
>>>> [unmount of overlay overlay]
>>>> BUG: Dentry ffff880079fe0338{i=2008d81,n=upper}  still in use (1)
>>>> [unmount of xfs vdf]
>>>> BUG: Dentry ffff880079fda700{i=3000100,n=ovl-lower}  still in use (1)
>>>> [unmount of xfs vdf]
>>>>
>>>
>>> Fixed by "fix double path_put() on error"
>>>
>>>> Another regression caused by tweaking OVL_WHITEOUTS patch:
>>>> overlay/038 - A fix to the readdir regression is pushed to:
>>>> https://github.com/amir73il/linux/commits/ovl-fixes
>>>
>>> Pushed those 2 fixes and 2 more review bug fixes to ovl-fixes branch.
>>
>> Thanks for testing and looking at the issues.  I've now set up
>> xfstests  as well in my test environment...
>>
>
> You may want to try out Ted's kvm-xfstests if you haven't already.
> I've set it up to also run unionmount testsuite tests from within xfstests:
>
> https://github.com/amir73il/xfstests-bld/commits/overlayfs-devel
>
>> Force pushed branch with fixes to overlayfs-next.
>>
>
> Looks good.
> Will test.
>

Tested. All good.

Did you get a chance to look at the 2 cleanup patches on overlayfs-devel:

3c964f63efcf ovl: factor out ovl_map_dev_ino() helper
8e97d9da4f2f ovl: store layer index in ovl_layer

Do you want me to post them proper?
A variant of them was posted within the v8 series.

BTW, there is a locking order issue for nested overlay I am working on
which I intend to post later today. I hope it can make it to v4.15 as well.

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Wed, Nov 8, 2017 at 3:40 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Nov 8, 2017 at 2:01 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
[...]
>>> I the meantime I went and committed v7 (with the noted changes) and
>>> based my cleanup for ovl_fill_super() on top of that.  So that's now
>>> pushed out to overlayfs-next, hopefully not invalidating too much of
>>> your work.
>>
>> Not a problem. most of the work on v8 was on the last patches.
>> I'll send 2 cleanup patches that make get_attr() code a bit cleaner to my taste.
>>

And finally, those are the cleanup patches from v8:
42448ca8681a ovl: factor out ovl_map_dev_ino() helper
52a325cd6b41 ovl: store layer index in ovl_layer

Can be found at the tip of:
https://github.com/amir73il/linux/commits/overlayfs-devel

The xino patches are also rebased and tested on:
https://github.com/amir73il/linux/commits/ovl-xino

In case you change your mind about this merge cycle.

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
[...]

> The -oxino patches are interesting, but maybe we should leave them
> brewing for another cycle.  Do you agree?
>

Miklos,

FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
of a few fixes in branch ovl-fixes:
* 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
* 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
* 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
absolute path redirects

Vivek has already written an xfstest for the middle layer opaque issue.
Eddie has already tested the NFSv3 DT_UNKNOWN issue as
well as the ovl-xino branch that fixes NFSv3 DT_UNKNOWN for the non samefs case.

You may want to pick at least the first and third patches from
ovl-fixes to v4.16.

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Miklos Szeredi]

On Sat, Mar 17, 2018 at 9:29 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> [...]
>
>> The -oxino patches are interesting, but maybe we should leave them
>> brewing for another cycle.  Do you agree?
>>
>
> Miklos,
>
> FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
> of a few fixes in branch ovl-fixes:
> * 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
> * 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
> * 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
> absolute path redirects

I'm a bit confused about this last one.  It's in Vivek's lookup fixes
series as well but in a slightly different form.  Which one should I
be looking at?

Thanks,
Miklois

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Amir Goldstein]

On Thu, Mar 22, 2018 at 4:19 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> On Sat, Mar 17, 2018 at 9:29 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> [...]
>>
>>> The -oxino patches are interesting, but maybe we should leave them
>>> brewing for another cycle.  Do you agree?
>>>
>>
>> Miklos,
>>
>> FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
>> of a few fixes in branch ovl-fixes:
>> * 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
>> * 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
>> * 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
>> absolute path redirects
>
> I'm a bit confused about this last one.  It's in Vivek's lookup fixes
> series as well but in a slightly different form.  Which one should I
> be looking at?
>

Vivek's patch is the later version (so v2).
There are 2 differences between v1 and v2:
1. Vivek's commit message is more elaborate, so should take it.
2. Vivek's patch sets only d->stop and not d->opaque
This difference is purely semantic, because d->opaque is
ignored in ovl_lookup() for anything but the upper layer.

I am fine with the semantic change, but wasn't sure if you
had other meaning in mind w.r.t d->opaque and metadata
going forward.

Thanks,
Amir.

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Miklos Szeredi]

On Thu, Mar 22, 2018 at 4:07 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> On Thu, Mar 22, 2018 at 4:19 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> On Sat, Mar 17, 2018 at 9:29 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>>> [...]
>>>
>>>> The -oxino patches are interesting, but maybe we should leave them
>>>> brewing for another cycle.  Do you agree?
>>>>
>>>
>>> Miklos,
>>>
>>> FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
>>> of a few fixes in branch ovl-fixes:
>>> * 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
>>> * 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
>>> * 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
>>> absolute path redirects
>>
>> I'm a bit confused about this last one.  It's in Vivek's lookup fixes
>> series as well but in a slightly different form.  Which one should I
>> be looking at?
>>
>
> Vivek's patch is the later version (so v2).
> There are 2 differences between v1 and v2:
> 1. Vivek's commit message is more elaborate, so should take it.
> 2. Vivek's patch sets only d->stop and not d->opaque
> This difference is purely semantic, because d->opaque is
> ignored in ovl_lookup() for anything but the upper layer.
>
> I am fine with the semantic change, but wasn't sure if you
> had other meaning in mind w.r.t d->opaque and metadata
> going forward.

I think clearing opaque it is the correct thing to do even if it
doesn't have any effect.

Thanks,
Miklos

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Vivek Goyal]

On Thu, Mar 22, 2018 at 04:36:25PM +0100, Miklos Szeredi wrote:
> On Thu, Mar 22, 2018 at 4:07 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> > On Thu, Mar 22, 2018 at 4:19 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> >> On Sat, Mar 17, 2018 at 9:29 AM, Amir Goldstein <amir73il@gmail.com> wrote:
> >>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> >>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
> >>> [...]
> >>>
> >>>> The -oxino patches are interesting, but maybe we should leave them
> >>>> brewing for another cycle.  Do you agree?
> >>>>
> >>>
> >>> Miklos,
> >>>
> >>> FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
> >>> of a few fixes in branch ovl-fixes:
> >>> * 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
> >>> * 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
> >>> * 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
> >>> absolute path redirects
> >>
> >> I'm a bit confused about this last one.  It's in Vivek's lookup fixes
> >> series as well but in a slightly different form.  Which one should I
> >> be looking at?
> >>
> >
> > Vivek's patch is the later version (so v2).
> > There are 2 differences between v1 and v2:
> > 1. Vivek's commit message is more elaborate, so should take it.
> > 2. Vivek's patch sets only d->stop and not d->opaque
> > This difference is purely semantic, because d->opaque is
> > ignored in ovl_lookup() for anything but the upper layer.
> >
> > I am fine with the semantic change, but wasn't sure if you
> > had other meaning in mind w.r.t d->opaque and metadata
> > going forward.
> 
> I think clearing opaque it is the correct thing to do even if it
> doesn't have any effect.

Hi Miklos,

I am fine with anything. I was just trying to make it semantically
more clear. That is when ovl_lookup_layer() is called, what do fields
in ovl_data{} reflect. I felt that "->opaque" should reflect the
property of last element of the path. And if we stick to it, then we
should not clear d->opauque when absolute redirect is found.

But I am not particular about it and it should not affect existing
metacopy patches I am working on. So I am fine with whatever you think
is better thing to do.

Vivek

Re: [PATCH v8 0/9] Overlayfs: constant st_ino/d_ino for non-samefs

[Miklos Szeredi]

On Fri, Mar 23, 2018 at 3:00 PM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Thu, Mar 22, 2018 at 04:36:25PM +0100, Miklos Szeredi wrote:
>> On Thu, Mar 22, 2018 at 4:07 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> > On Thu, Mar 22, 2018 at 4:19 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> >> On Sat, Mar 17, 2018 at 9:29 AM, Amir Goldstein <amir73il@gmail.com> wrote:
>> >>> On Wed, Nov 8, 2017 at 12:53 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
>> >>>> On Tue, Nov 7, 2017 at 5:58 PM, Amir Goldstein <amir73il@gmail.com> wrote:
>> >>> [...]
>> >>>
>> >>>> The -oxino patches are interesting, but maybe we should leave them
>> >>>> brewing for another cycle.  Do you agree?
>> >>>>
>> >>>
>> >>> Miklos,
>> >>>
>> >>> FYI, I pushed ovl-xino branch that is rebased on v4.16-rc5 and on top
>> >>> of a few fixes in branch ovl-fixes:
>> >>> * 5668064a61f6 - ovl: set i_ino to the value of st_ino for NFS export
>> >>> * 579515ad5c75 - ovl: opaque xattr should overrule redirect xattr
>> >>> * 0161362aeab7 - ovl: fix lookup with middle layer opaque dir and
>> >>> absolute path redirects
>> >>
>> >> I'm a bit confused about this last one.  It's in Vivek's lookup fixes
>> >> series as well but in a slightly different form.  Which one should I
>> >> be looking at?
>> >>
>> >
>> > Vivek's patch is the later version (so v2).
>> > There are 2 differences between v1 and v2:
>> > 1. Vivek's commit message is more elaborate, so should take it.
>> > 2. Vivek's patch sets only d->stop and not d->opaque
>> > This difference is purely semantic, because d->opaque is
>> > ignored in ovl_lookup() for anything but the upper layer.
>> >
>> > I am fine with the semantic change, but wasn't sure if you
>> > had other meaning in mind w.r.t d->opaque and metadata
>> > going forward.
>>
>> I think clearing opaque it is the correct thing to do even if it
>> doesn't have any effect.
>
> Hi Miklos,
>
> I am fine with anything. I was just trying to make it semantically
> more clear. That is when ovl_lookup_layer() is called, what do fields
> in ovl_data{} reflect. I felt that "->opaque" should reflect the
> property of last element of the path. And if we stick to it, then we
> should not clear d->opauque when absolute redirect is found.

Right.  In the light of the rest of your patchset it makes sense.

Thanks,
Miklos