[PATCH 0/7] overlay filesystem v9

[PATCH 0/7] overlay filesystem v9

[Miklos Szeredi]

Here's an updated version of the overlay filesystem.

Git tree is here:

  git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9


Changes from v8 to v9

- support xattr on tmpfs

- fix build after split-up

- fix remove after rename (reported by Jordi Pujol)

- fix rename failure case


To be done:

- redo i_op->open, waiting for status of atomic_open() from Al

- waiting for test results from Jordi

Thanks,
Miklos

---
Andy Whitcroft (1):
      overlayfs: add statfs support

Eric Paris (1):
      tmpfs: implement generic xattr support

Miklos Szeredi (4):
      vfs: add i_op->open()
      vfs: export do_splice_direct() to modules
      vfs: introduce clone_private_mount()
      overlay filesystem

Neil Brown (1):
      overlay: overlay filesystem documentation

---
 Documentation/filesystems/overlayfs.txt |  167 +++++++++
 MAINTAINERS                             |    7 +
 fs/Kconfig                              |   19 +-
 fs/Makefile                             |    1 +
 fs/namespace.c                          |   17 +
 fs/open.c                               |   76 +++--
 fs/overlayfs/Kconfig                    |    4 +
 fs/overlayfs/Makefile                   |    7 +
 fs/overlayfs/copy_up.c                  |  383 ++++++++++++++++++++
 fs/overlayfs/dir.c                      |  594 ++++++++++++++++++++++++++++++
 fs/overlayfs/inode.c                    |  374 +++++++++++++++++++
 fs/overlayfs/overlayfs.h                |   62 ++++
 fs/overlayfs/readdir.c                  |  560 +++++++++++++++++++++++++++++
 fs/overlayfs/super.c                    |  596 +++++++++++++++++++++++++++++++
 fs/splice.c                             |    1 +
 include/linux/fs.h                      |    2 +
 include/linux/mount.h                   |    3 +
 include/linux/shmem_fs.h                |    8 +-
 mm/shmem.c                              |  320 ++++++++++++++---
 19 files changed, 3119 insertions(+), 82 deletions(-)
 create mode 100644 Documentation/filesystems/overlayfs.txt
 create mode 100644 fs/overlayfs/Kconfig
 create mode 100644 fs/overlayfs/Makefile
 create mode 100644 fs/overlayfs/copy_up.c
 create mode 100644 fs/overlayfs/dir.c
 create mode 100644 fs/overlayfs/inode.c
 create mode 100644 fs/overlayfs/overlayfs.h
 create mode 100644 fs/overlayfs/readdir.c
 create mode 100644 fs/overlayfs/super.c
------------------------------------------------------------------------------
Changes from v7 to v8:

- split overlayfs.c into smaller files

- fix locking for copy up (reported by Al Viro)

- locking analysis of copy up vs. directory rename added as a comment

- tested with lockdep, fixed one lock annotation

- other bug fixes

------------------------------------------------------------------------------
Changes from v6 to v7

- added patches from Felix Fietkau to fix deadlocks on jffs2

- optimized directory removal

- properly clean up after copy-up and other failures

------------------------------------------------------------------------------
Changes from v5 to v6

- optimize directory merging

  o use rbtree for weeding out duplicates

  o use a cursor for current position within the stream

- instead of f_op->open_other(), implement i_op->open()

- don't share inodes for non-directory dentries - for now.  I hope
  this can come back once RCU lookup code has settled.

- misc bug fixes

------------------------------------------------------------------------------
Changes from v4 to v5

- fix copying up if fs doesn't support xattrs (Andy Whitcroft)

- clone mounts to be used internally to access the underlying
  filesystems

------------------------------------------------------------------------------
Changes from v3 to v4

- export security_inode_permission to allow overlayfs to be modular
  (Andy Whitcroft)

- add statfs support (Andy Whitcroft)

- change BUG_ON to WARN_ON

- Revert "vfs: add flag to allow rename to same inode", instead
  introduce s_op->is_same_inode()

- overlayfs: fix rename to self

- fix whiteout after rename

------------------------------------------------------------------------------
Changes from v2 to v3

 - Minimal remount support.  As overlayfs reflects the 'readonly'
   mount status in write-access to the upper filesystem, we must
   handle remount and either drop or take write access when the ro
   status changes. (NeilBrown)

 - Use correct seek function for directories.  It is incorrect to call
   generic_llseek_file on a file from a different filesystem.  For
   that we must use the seek function that the filesystem defines,
   which is called by vfs_llseek.  Also, we only want to seek the
   realfile when is_real is true.  Otherwise we just want to update
   our own f_pos pointer, so use generic_llseek_file for
   that. (NeilBrown)

 - Initialise is_real before use.  The previous patch can use
   od->is_real before it is properly initialised is llseek is called
   before readdir.  So factor out the initialisation of is_real and
   call it from both readdir and llseek when f_pos is 0. (NeilBrown)

 - Rename ovl_fill_cache to ovl_dir_read (NeilBrown)

 - Tiny optimisation in open_other handling (NeilBrown)

 - Assorted updates to Documentation/filesystems/overlayfs.txt (NeilBrown)

 - Make copy-up work for >=4G files, make it killable during copy-up.
   Need to fix recovery after a failed/interrupted copy-up.

 - Store and reference upper/lower dentries in overlay dentries.
   Store and reference upper/lower vfsmounts in overlay superblock.

 - Add necessary barriers for setting upper dentry in copyup and for
   retrieving upper dentry locklessly.

 - Make sure the right file is used for directory fsync() after
   copy-up.

 - Add locking to ovl_dir_llseek() to prevent concurrent call of
   ovl_dir_reset() with ovl_dir_read().

 - Get rid of ovl_dentry_iput().  The VFS doesn't provide enough
   locking for this function that the contents of ->d_fsdata could be
   safely updated.

 - After copying up a non-directory unhash the dentry.  This way the
   lower dentry ref, which is no longer necessary, can go away.  This
   revealed a use-after-free bug in truncate handling in
   fs/namei.c:finish_open().

 - Fix if a copy-up happens between the follow_linka the put_link
   calls.

 - Replace some WARN_ONs with BUG_ON.  Some things just _really_
   shouldn't happen.

 - Extract common code from ovl_unlink and ovl_rmdir to a helper
   function.

 - After unlink and rmdir unhash the dentry.  This will get rid of the
   lower and upper dentry references after there are no more users of
   the deleted dentry.  This is a safe replacement for the removed
   ->d_iput() functionality.

 - Added checks to unlink, rmdir and rename to verify that the
   parent-child relationship in the upper filesystem matches that of
   the overlay.  This is necessary to prevent crash and/or corruption
   if the upper filesystem topology is being modified while part of
   the overlay.

 - Optimize checking whiteout and opaque attributes.

 - Optimize copy-up on truncate: don't copy up whole file before
   truncating

 - Misc bug fixes

------------------------------------------------------------------------------
Changes from v1 to v2

 - rename "hybrid union filesystem" to "overlay filesystem" or overlayfs

 - added documentation written by Neil

 - correct st_dev for directories (reported by Neil)

 - use getattr() to get attributes from the underlying filesystems,
   this means that now an overlay filesystem itself can be the lower,
   read-only layer of another overlay

 - listxattr filters out private extended attributes

 - get write ref on the upper layer on mount unless the overlay
   itself is mounted read-only

 - raise capabilities for copy up, dealing with whiteouts and opaque
   directories.  Now the overlay works for non-root users as well

 - "rm -rf" didn't work correctly in all cases if the directory was
   copied up between opendir and the first readdir, this is now fixed
   (and the directory operations consolidated)

 - simplified copy up, this broke optimization for truncate and
   open(O_TRUNC) (now file is copied up to be immediately truncated,
   will fix)

 - st_nlink for merged directories set to 1, this is an "illegal"
   value that normal filesystems never have but some use it to
   indicate that the number of subdirectories is unknown.  Utilities
   (find, ...) seem to tolerate this well.

 - misc fixes I forgot about

[PATCH 1/7] tmpfs: implement generic xattr support

[Miklos Szeredi]

From: Eric Paris <eparis@redhat.com>

Implement generic xattrs for tmpfs filesystems.  The Feodra project, while
trying to replace suid apps with file capabilities, realized that tmpfs,
which is used on the build systems, does not support file capabilities and
thus cannot be used to build packages which use file capabilities.  Xattrs
are also needed for overlayfs.

The xattr interface is a bit odd.  If a filesystem does not implement any
{get,set,list}xattr functions the VFS will call into some random LSM hooks
and the running LSM can then implement some method for handling xattrs.
SELinux for example provides a method to support security.selinux but no
other security.* xattrs.

As it stands today when one enables CONFIG_TMPFS_POSIX_ACL tmpfs will have
xattr handler routines specifically to handle acls.  Because of this tmpfs
would loose the VFS/LSM helpers to support the running LSM.  To make up
for that tmpfs had stub functions that did nothing but call into the LSM
hooks which implement the helpers.

This new patch does not use the LSM fallback functions and instead just
implements a native get/set/list xattr feature for the full security.* and
trusted.* namespace like a normal filesystem.  This means that tmpfs can
now support both security.selinux and security.capability, which was not
previously possible.

The basic implementation is that I attach a:

struct shmem_xattr {
	struct list_head list; /* anchored by shmem_inode_info->xattr_list */
	char *name;
	size_t size;
	char value[0];
};

Into the struct shmem_inode_info for each xattr that is set.  This
implementation could easily support the user.* namespace as well, except
some care needs to be taken to prevent large amounts of unswappable memory
being allocated for unprivileged users.

[mszeredi@suse.cz: new config option, suport trusted.*, support symlinks]
Signed-off-by: Eric Paris <eparis@redhat.com>
Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
Acked-by: Serge Hallyn <serge.hallyn@ubuntu.com>
Cc: Kyle McMartin <kyle@mcmartin.ca>
Acked-by: Hugh Dickins <hughd@google.com>
Tested-by: Jordi Pujol <jordipujolp@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
 fs/Kconfig               |   18 +++-
 include/linux/shmem_fs.h |    8 +-
 mm/shmem.c               |  320 ++++++++++++++++++++++++++++++++++++++--------
 3 files changed, 290 insertions(+), 56 deletions(-)

diff --git a/fs/Kconfig b/fs/Kconfig
index f3aa9b0..979992d 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -121,9 +121,25 @@ config TMPFS
 
 	  See <file:Documentation/filesystems/tmpfs.txt> for details.
 
+config TMPFS_XATTR
+	bool "Tmpfs extended attributes"
+	depends on TMPFS
+	default n
+	help
+	  Extended attributes are name:value pairs associated with inodes by
+	  the kernel or by users (see the attr(5) manual page, or visit
+	  <http://acl.bestbits.at/> for details).
+
+	  Currently this enables support for the trusted.* and
+	  security.* namespaces.
+
+	  If unsure, say N.
+
+	  You need this for POSIX ACL support on tmpfs.
+
 config TMPFS_POSIX_ACL
 	bool "Tmpfs POSIX Access Control Lists"
-	depends on TMPFS
+	depends on TMPFS_XATTR
 	select GENERIC_ACL
 	help
 	  POSIX Access Control Lists (ACLs) support permissions for users and
diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h
index 399be5a..2b7fec8 100644
--- a/include/linux/shmem_fs.h
+++ b/include/linux/shmem_fs.h
@@ -9,6 +9,8 @@
 
 #define SHMEM_NR_DIRECT 16
 
+#define SHMEM_SYMLINK_INLINE_LEN (SHMEM_NR_DIRECT * sizeof(swp_entry_t))
+
 struct shmem_inode_info {
 	spinlock_t		lock;
 	unsigned long		flags;
@@ -17,8 +19,12 @@ struct shmem_inode_info {
 	unsigned long		next_index;	/* highest alloced index + 1 */
 	struct shared_policy	policy;		/* NUMA memory alloc policy */
 	struct page		*i_indirect;	/* top indirect blocks page */
-	swp_entry_t		i_direct[SHMEM_NR_DIRECT]; /* first blocks */
+	union {
+		swp_entry_t	i_direct[SHMEM_NR_DIRECT]; /* first blocks */
+		char		inline_symlink[SHMEM_SYMLINK_INLINE_LEN];
+	};
 	struct list_head	swaplist;	/* chain of maybes on swap */
+	struct list_head	xattr_list;	/* list of shmem_xattr */
 	struct inode		vfs_inode;
 };
 
diff --git a/mm/shmem.c b/mm/shmem.c
index 8fa27e4..eab5593c 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -99,6 +99,13 @@ static struct vfsmount *shm_mnt;
 /* Pretend that each entry is of this size in directory's i_size */
 #define BOGO_DIRENT_SIZE 20
 
+struct shmem_xattr {
+	struct list_head list;	/* anchored by shmem_inode_info->xattr_list */
+	char *name;		/* xattr name */
+	size_t size;
+	char value[0];
+};
+
 /* Flag allocation requirements to shmem_getpage and shmem_swp_alloc */
 enum sgp_type {
 	SGP_READ,	/* don't exceed i_size, don't allocate page */
@@ -822,6 +829,7 @@ static int shmem_notify_change(struct dentry *dentry, struct iattr *attr)
 static void shmem_evict_inode(struct inode *inode)
 {
 	struct shmem_inode_info *info = SHMEM_I(inode);
+	struct shmem_xattr *xattr, *nxattr;
 
 	if (inode->i_mapping->a_ops == &shmem_aops) {
 		truncate_inode_pages(inode->i_mapping, 0);
@@ -834,6 +842,11 @@ static void shmem_evict_inode(struct inode *inode)
 			mutex_unlock(&shmem_swaplist_mutex);
 		}
 	}
+
+	list_for_each_entry_safe(xattr, nxattr, &info->xattr_list, list) {
+		kfree(xattr->name);
+		kfree(xattr);
+	}
 	BUG_ON(inode->i_blocks);
 	shmem_free_inode(inode->i_sb);
 	end_writeback(inode);
@@ -1597,6 +1610,7 @@ static struct inode *shmem_get_inode(struct super_block *sb, const struct inode
 		spin_lock_init(&info->lock);
 		info->flags = flags & VM_NORESERVE;
 		INIT_LIST_HEAD(&info->swaplist);
+		INIT_LIST_HEAD(&info->xattr_list);
 		cache_no_acl(inode);
 
 		switch (mode & S_IFMT) {
@@ -1996,9 +2010,9 @@ static int shmem_symlink(struct inode *dir, struct dentry *dentry, const char *s
 
 	info = SHMEM_I(inode);
 	inode->i_size = len-1;
-	if (len <= (char *)inode - (char *)info) {
+	if (len <= SHMEM_SYMLINK_INLINE_LEN) {
 		/* do it inline */
-		memcpy(info, symname, len);
+		memcpy(info->inline_symlink, symname, len);
 		inode->i_op = &shmem_symlink_inline_operations;
 	} else {
 		error = shmem_getpage(inode, 0, &page, SGP_WRITE, NULL);
@@ -2024,7 +2038,7 @@ static int shmem_symlink(struct inode *dir, struct dentry *dentry, const char *s
 
 static void *shmem_follow_link_inline(struct dentry *dentry, struct nameidata *nd)
 {
-	nd_set_link(nd, (char *)SHMEM_I(dentry->d_inode));
+	nd_set_link(nd, SHMEM_I(dentry->d_inode)->inline_symlink);
 	return NULL;
 }
 
@@ -2048,63 +2062,253 @@ static void shmem_put_link(struct dentry *dentry, struct nameidata *nd, void *co
 	}
 }
 
-static const struct inode_operations shmem_symlink_inline_operations = {
-	.readlink	= generic_readlink,
-	.follow_link	= shmem_follow_link_inline,
-};
-
-static const struct inode_operations shmem_symlink_inode_operations = {
-	.readlink	= generic_readlink,
-	.follow_link	= shmem_follow_link,
-	.put_link	= shmem_put_link,
-};
-
-#ifdef CONFIG_TMPFS_POSIX_ACL
+#ifdef CONFIG_TMPFS_XATTR
 /*
- * Superblocks without xattr inode operations will get security.* xattr
- * support from the VFS "for free". As soon as we have any other xattrs
+ * Superblocks without xattr inode operations may get some security.* xattr
+ * support from the LSM "for free". As soon as we have any other xattrs
  * like ACLs, we also need to implement the security.* handlers at
  * filesystem level, though.
  */
 
-static size_t shmem_xattr_security_list(struct dentry *dentry, char *list,
-					size_t list_len, const char *name,
-					size_t name_len, int handler_flags)
+static int shmem_xattr_get(struct dentry *dentry, const char *name,
+			   void *buffer, size_t size)
 {
-	return security_inode_listsecurity(dentry->d_inode, list, list_len);
-}
+	struct shmem_inode_info *info;
+	struct shmem_xattr *xattr;
+	int ret = -ENODATA;
 
-static int shmem_xattr_security_get(struct dentry *dentry, const char *name,
-		void *buffer, size_t size, int handler_flags)
-{
-	if (strcmp(name, "") == 0)
-		return -EINVAL;
-	return xattr_getsecurity(dentry->d_inode, name, buffer, size);
+	info = SHMEM_I(dentry->d_inode);
+
+	spin_lock(&info->lock);
+	list_for_each_entry(xattr, &info->xattr_list, list) {
+		if (strcmp(name, xattr->name))
+			continue;
+
+		ret = xattr->size;
+		if (buffer) {
+			if (size < xattr->size)
+				ret = -ERANGE;
+			else
+				memcpy(buffer, xattr->value, xattr->size);
+		}
+		break;
+	}
+	spin_unlock(&info->lock);
+	return ret;
 }
 
-static int shmem_xattr_security_set(struct dentry *dentry, const char *name,
-		const void *value, size_t size, int flags, int handler_flags)
+static int shmem_xattr_set(struct dentry *dentry, const char *name,
+			   const void *value, size_t size, int flags)
 {
-	if (strcmp(name, "") == 0)
-		return -EINVAL;
-	return security_inode_setsecurity(dentry->d_inode, name, value,
-					  size, flags);
+	struct inode *inode = dentry->d_inode;
+	struct shmem_inode_info *info = SHMEM_I(inode);
+	struct shmem_xattr *xattr;
+	struct shmem_xattr *new_xattr = NULL;
+	size_t len;
+	int err = 0;
+
+	/* value == NULL means remove */
+	if (value) {
+		/* wrap around? */
+		len = sizeof(*new_xattr) + size;
+		if (len <= sizeof(*new_xattr))
+			return -ENOMEM;
+
+		new_xattr = kmalloc(len, GFP_KERNEL);
+		if (!new_xattr)
+			return -ENOMEM;
+
+		new_xattr->name = kstrdup(name, GFP_KERNEL);
+		if (!new_xattr->name) {
+			kfree(new_xattr);
+			return -ENOMEM;
+		}
+
+		new_xattr->size = size;
+		memcpy(new_xattr->value, value, size);
+	}
+
+	spin_lock(&info->lock);
+	list_for_each_entry(xattr, &info->xattr_list, list) {
+		if (!strcmp(name, xattr->name)) {
+			if (flags & XATTR_CREATE) {
+				xattr = new_xattr;
+				err = -EEXIST;
+			} else if (new_xattr) {
+				list_replace(&xattr->list, &new_xattr->list);
+			} else {
+				list_del(&xattr->list);
+			}
+			goto out;
+		}
+	}
+	if (flags & XATTR_REPLACE) {
+		xattr = new_xattr;
+		err = -ENODATA;
+	} else {
+		list_add(&new_xattr->list, &info->xattr_list);
+		xattr = NULL;
+	}
+out:
+	spin_unlock(&info->lock);
+	if (xattr)
+		kfree(xattr->name);
+	kfree(xattr);
+	return err;
 }
 
-static const struct xattr_handler shmem_xattr_security_handler = {
-	.prefix = XATTR_SECURITY_PREFIX,
-	.list   = shmem_xattr_security_list,
-	.get    = shmem_xattr_security_get,
-	.set    = shmem_xattr_security_set,
-};
 
 static const struct xattr_handler *shmem_xattr_handlers[] = {
+#ifdef CONFIG_TMPFS_POSIX_ACL
 	&generic_acl_access_handler,
 	&generic_acl_default_handler,
-	&shmem_xattr_security_handler,
+#endif
 	NULL
 };
+
+static int shmem_xattr_validate(const char *name)
+{
+	struct { const char *prefix; size_t len; } arr[] = {
+		{ XATTR_SECURITY_PREFIX, XATTR_SECURITY_PREFIX_LEN },
+		{ XATTR_TRUSTED_PREFIX, XATTR_TRUSTED_PREFIX_LEN }
+	};
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(arr); i++) {
+		size_t preflen = arr[i].len;
+		if (strncmp(name, arr[i].prefix, preflen) == 0) {
+			if (!name[preflen])
+				return -EINVAL;
+			return 0;
+		}
+	}
+	return -EOPNOTSUPP;
+}
+
+static ssize_t shmem_getxattr(struct dentry *dentry, const char *name,
+			      void *buffer, size_t size)
+{
+	int err;
+
+	/*
+	 * If this is a request for a synthetic attribute in the system.*
+	 * namespace use the generic infrastructure to resolve a handler
+	 * for it via sb->s_xattr.
+	 */
+	if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+		return generic_getxattr(dentry, name, buffer, size);
+
+	err = shmem_xattr_validate(name);
+	if (err)
+		return err;
+
+	return shmem_xattr_get(dentry, name, buffer, size);
+}
+
+static int shmem_setxattr(struct dentry *dentry, const char *name,
+			  const void *value, size_t size, int flags)
+{
+	int err;
+
+	/*
+	 * If this is a request for a synthetic attribute in the system.*
+	 * namespace use the generic infrastructure to resolve a handler
+	 * for it via sb->s_xattr.
+	 */
+	if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+		return generic_setxattr(dentry, name, value, size, flags);
+
+	err = shmem_xattr_validate(name);
+	if (err)
+		return err;
+
+	if (size == 0)
+		value = "";  /* empty EA, do not remove */
+
+	return shmem_xattr_set(dentry, name, value, size, flags);
+
+}
+
+static int shmem_removexattr(struct dentry *dentry, const char *name)
+{
+	int err;
+
+	/*
+	 * If this is a request for a synthetic attribute in the system.*
+	 * namespace use the generic infrastructure to resolve a handler
+	 * for it via sb->s_xattr.
+	 */
+	if (!strncmp(name, XATTR_SYSTEM_PREFIX, XATTR_SYSTEM_PREFIX_LEN))
+		return generic_removexattr(dentry, name);
+
+	err = shmem_xattr_validate(name);
+	if (err)
+		return err;
+
+	return shmem_xattr_set(dentry, name, NULL, 0, XATTR_REPLACE);
+}
+
+static bool xattr_is_trusted(const char *name)
+{
+	return !strncmp(name, XATTR_TRUSTED_PREFIX, XATTR_TRUSTED_PREFIX_LEN);
+}
+
+static ssize_t shmem_listxattr(struct dentry *dentry, char *buffer, size_t size)
+{
+	bool trusted = capable(CAP_SYS_ADMIN);
+	struct shmem_xattr *xattr;
+	struct shmem_inode_info *info;
+	size_t used = 0;
+
+	info = SHMEM_I(dentry->d_inode);
+
+	spin_lock(&info->lock);
+	list_for_each_entry(xattr, &info->xattr_list, list) {
+		size_t len;
+
+		/* skip "trusted." attributes for unprivileged callers */
+		if (!trusted && xattr_is_trusted(xattr->name))
+			continue;
+
+		len = strlen(xattr->name) + 1;
+		used += len;
+		if (buffer) {
+			if (size < used) {
+				used = -ERANGE;
+				break;
+			}
+			memcpy(buffer, xattr->name, len);
+			buffer += len;
+		}
+	}
+	spin_unlock(&info->lock);
+
+	return used;
+}
+#endif /* CONFIG_TMPFS_XATTR */
+
+static const struct inode_operations shmem_symlink_inline_operations = {
+	.readlink	= generic_readlink,
+	.follow_link	= shmem_follow_link_inline,
+#ifdef CONFIG_TMPFS_XATTR
+	.setxattr	= shmem_setxattr,
+	.getxattr	= shmem_getxattr,
+	.listxattr	= shmem_listxattr,
+	.removexattr	= shmem_removexattr,
 #endif
+};
+
+static const struct inode_operations shmem_symlink_inode_operations = {
+	.readlink	= generic_readlink,
+	.follow_link	= shmem_follow_link,
+	.put_link	= shmem_put_link,
+#ifdef CONFIG_TMPFS_XATTR
+	.setxattr	= shmem_setxattr,
+	.getxattr	= shmem_getxattr,
+	.listxattr	= shmem_listxattr,
+	.removexattr	= shmem_removexattr,
+#endif
+};
 
 static struct dentry *shmem_get_parent(struct dentry *child)
 {
@@ -2384,8 +2588,10 @@ int shmem_fill_super(struct super_block *sb, void *data, int silent)
 	sb->s_magic = TMPFS_MAGIC;
 	sb->s_op = &shmem_ops;
 	sb->s_time_gran = 1;
-#ifdef CONFIG_TMPFS_POSIX_ACL
+#ifdef CONFIG_TMPFS_XATTR
 	sb->s_xattr = shmem_xattr_handlers;
+#endif
+#ifdef CONFIG_TMPFS_POSIX_ACL
 	sb->s_flags |= MS_POSIXACL;
 #endif
 
@@ -2483,11 +2689,13 @@ static const struct file_operations shmem_file_operations = {
 static const struct inode_operations shmem_inode_operations = {
 	.setattr	= shmem_notify_change,
 	.truncate_range	= shmem_truncate_range,
+#ifdef CONFIG_TMPFS_XATTR
+	.setxattr	= shmem_setxattr,
+	.getxattr	= shmem_getxattr,
+	.listxattr	= shmem_listxattr,
+	.removexattr	= shmem_removexattr,
+#endif
 #ifdef CONFIG_TMPFS_POSIX_ACL
-	.setxattr	= generic_setxattr,
-	.getxattr	= generic_getxattr,
-	.listxattr	= generic_listxattr,
-	.removexattr	= generic_removexattr,
 	.check_acl	= generic_check_acl,
 #endif
 
@@ -2505,23 +2713,27 @@ static const struct inode_operations shmem_dir_inode_operations = {
 	.mknod		= shmem_mknod,
 	.rename		= shmem_rename,
 #endif
+#ifdef CONFIG_TMPFS_XATTR
+	.setxattr	= shmem_setxattr,
+	.getxattr	= shmem_getxattr,
+	.listxattr	= shmem_listxattr,
+	.removexattr	= shmem_removexattr,
+#endif
 #ifdef CONFIG_TMPFS_POSIX_ACL
 	.setattr	= shmem_notify_change,
-	.setxattr	= generic_setxattr,
-	.getxattr	= generic_getxattr,
-	.listxattr	= generic_listxattr,
-	.removexattr	= generic_removexattr,
 	.check_acl	= generic_check_acl,
 #endif
 };
 
 static const struct inode_operations shmem_special_inode_operations = {
+#ifdef CONFIG_TMPFS_XATTR
+	.setxattr	= shmem_setxattr,
+	.getxattr	= shmem_getxattr,
+	.listxattr	= shmem_listxattr,
+	.removexattr	= shmem_removexattr,
+#endif
 #ifdef CONFIG_TMPFS_POSIX_ACL
 	.setattr	= shmem_notify_change,
-	.setxattr	= generic_setxattr,
-	.getxattr	= generic_getxattr,
-	.listxattr	= generic_listxattr,
-	.removexattr	= generic_removexattr,
 	.check_acl	= generic_check_acl,
 #endif
 };
-- 
1.7.3.4

[PATCH 2/7] vfs: add i_op->open()

[Miklos Szeredi]

From: Miklos Szeredi <mszeredi@suse.cz>

Add a new inode operation i_op->open().  This is for stacked
filesystems that want to return a struct file from a different
filesystem.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 fs/open.c          |   76 ++++++++++++++++++++++++++++++++++------------------
 include/linux/fs.h |    2 +
 2 files changed, 52 insertions(+), 26 deletions(-)

diff --git a/fs/open.c b/fs/open.c
index b52cf01..84fa16a 100644
--- a/fs/open.c
+++ b/fs/open.c
@@ -666,8 +666,7 @@ static inline int __get_file_write_access(struct inode *inode,
 	return error;
 }
 
-static struct file *__dentry_open(struct dentry *dentry, struct vfsmount *mnt,
-					struct file *f,
+static struct file *__dentry_open(struct path *path, struct file *f,
 					int (*open)(struct inode *, struct file *),
 					const struct cred *cred)
 {
@@ -675,15 +674,16 @@ static struct file *__dentry_open(struct dentry *dentry, struct vfsmount *mnt,
 	struct inode *inode;
 	int error;
 
+	path_get(path);
 	f->f_mode = OPEN_FMODE(f->f_flags) | FMODE_LSEEK |
 				FMODE_PREAD | FMODE_PWRITE;
 
 	if (unlikely(f->f_flags & O_PATH))
 		f->f_mode = FMODE_PATH;
 
-	inode = dentry->d_inode;
+	inode = path->dentry->d_inode;
 	if (f->f_mode & FMODE_WRITE) {
-		error = __get_file_write_access(inode, mnt);
+		error = __get_file_write_access(inode, path->mnt);
 		if (error)
 			goto cleanup_file;
 		if (!special_file(inode->i_mode))
@@ -691,8 +691,7 @@ static struct file *__dentry_open(struct dentry *dentry, struct vfsmount *mnt,
 	}
 
 	f->f_mapping = inode->i_mapping;
-	f->f_path.dentry = dentry;
-	f->f_path.mnt = mnt;
+	f->f_path = *path;
 	f->f_pos = 0;
 	file_sb_list_add(f, inode->i_sb);
 
@@ -745,7 +744,7 @@ cleanup_all:
 			 * here, so just reset the state.
 			 */
 			file_reset_write(f);
-			mnt_drop_write(mnt);
+			mnt_drop_write(path->mnt);
 		}
 	}
 	file_sb_list_del(f);
@@ -753,8 +752,7 @@ cleanup_all:
 	f->f_path.mnt = NULL;
 cleanup_file:
 	put_filp(f);
-	dput(dentry);
-	mntput(mnt);
+	path_put(path);
 	return ERR_PTR(error);
 }
 
@@ -780,14 +778,14 @@ cleanup_file:
 struct file *lookup_instantiate_filp(struct nameidata *nd, struct dentry *dentry,
 		int (*open)(struct inode *, struct file *))
 {
+	struct path path = { .dentry = dentry, .mnt = nd->path.mnt };
 	const struct cred *cred = current_cred();
 
 	if (IS_ERR(nd->intent.open.file))
 		goto out;
 	if (IS_ERR(dentry))
 		goto out_err;
-	nd->intent.open.file = __dentry_open(dget(dentry), mntget(nd->path.mnt),
-					     nd->intent.open.file,
+	nd->intent.open.file = __dentry_open(&path, nd->intent.open.file,
 					     open, cred);
 out:
 	return nd->intent.open.file;
@@ -816,10 +814,17 @@ struct file *nameidata_to_filp(struct nameidata *nd)
 
 	/* Has the filesystem initialised the file for us? */
 	if (filp->f_path.dentry == NULL) {
-		path_get(&nd->path);
-		filp = __dentry_open(nd->path.dentry, nd->path.mnt, filp,
-				     NULL, cred);
+		struct inode *inode = nd->path.dentry->d_inode;
+
+		if (inode->i_op->open) {
+			int flags = filp->f_flags;
+			put_filp(filp);
+			filp = inode->i_op->open(nd->path.dentry, flags, cred);
+		} else {
+			filp = __dentry_open(&nd->path, filp, NULL, cred);
+		}
 	}
+
 	return filp;
 }
 
@@ -830,26 +835,45 @@ struct file *nameidata_to_filp(struct nameidata *nd)
 struct file *dentry_open(struct dentry *dentry, struct vfsmount *mnt, int flags,
 			 const struct cred *cred)
 {
-	int error;
-	struct file *f;
-
-	validate_creds(cred);
+	struct path path = { .dentry = dentry, .mnt = mnt };
+	struct file *ret;
 
 	/* We must always pass in a valid mount pointer. */
 	BUG_ON(!mnt);
 
-	error = -ENFILE;
+	ret = vfs_open(&path, flags, cred);
+	path_put(&path);
+
+	return ret;
+}
+EXPORT_SYMBOL(dentry_open);
+
+/**
+ * vfs_open - open the file at the given path
+ * @path: path to open
+ * @flags: open flags
+ * @cred: credentials to use
+ *
+ * Open the file.  If successful, the returned file will have acquired
+ * an additional reference for path.
+ */
+struct file *vfs_open(struct path *path, int flags, const struct cred *cred)
+{
+	struct file *f;
+	struct inode *inode = path->dentry->d_inode;
+
+	validate_creds(cred);
+
+	if (inode->i_op->open)
+		return inode->i_op->open(path->dentry, flags, cred);
 	f = get_empty_filp();
-	if (f == NULL) {
-		dput(dentry);
-		mntput(mnt);
-		return ERR_PTR(error);
-	}
+	if (f == NULL)
+		return ERR_PTR(-ENFILE);
 
 	f->f_flags = flags;
-	return __dentry_open(dentry, mnt, f, NULL, cred);
+	return __dentry_open(path, f, NULL, cred);
 }
-EXPORT_SYMBOL(dentry_open);
+EXPORT_SYMBOL(vfs_open);
 
 static void __put_unused_fd(struct files_struct *files, unsigned int fd)
 {
diff --git a/include/linux/fs.h b/include/linux/fs.h
index dbd860a..fa062a8 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1595,6 +1595,7 @@ struct inode_operations {
 	void (*truncate_range)(struct inode *, loff_t, loff_t);
 	int (*fiemap)(struct inode *, struct fiemap_extent_info *, u64 start,
 		      u64 len);
+	struct file *(*open)(struct dentry *, int flags, const struct cred *);
 } ____cacheline_aligned;
 
 struct seq_file;
@@ -1989,6 +1990,7 @@ extern long do_sys_open(int dfd, const char __user *filename, int flags,
 extern struct file *filp_open(const char *, int, int);
 extern struct file *file_open_root(struct dentry *, struct vfsmount *,
 				   const char *, int);
+extern struct file *vfs_open(struct path *, int flags, const struct cred *);
 extern struct file * dentry_open(struct dentry *, struct vfsmount *, int,
 				 const struct cred *);
 extern int filp_close(struct file *, fl_owner_t id);
-- 
1.7.3.4

[PATCH 3/7] vfs: export do_splice_direct() to modules

[Miklos Szeredi]

From: Miklos Szeredi <mszeredi@suse.cz>

Export do_splice_direct() to modules.  Needed by overlay filesystem.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 fs/splice.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/splice.c b/fs/splice.c
index 50a5d978..4f2fa05 100644
--- a/fs/splice.c
+++ b/fs/splice.c
@@ -1296,6 +1296,7 @@ long do_splice_direct(struct file *in, loff_t *ppos, struct file *out,
 
 	return ret;
 }
+EXPORT_SYMBOL(do_splice_direct);
 
 static int splice_pipe_to_pipe(struct pipe_inode_info *ipipe,
 			       struct pipe_inode_info *opipe,
-- 
1.7.3.4

[PATCH 4/7] vfs: introduce clone_private_mount()

[Miklos Szeredi]

From: Miklos Szeredi <mszeredi@suse.cz>

Overlayfs needs a private clone of the mount, so create a function for
this and export to modules.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 fs/namespace.c        |   17 +++++++++++++++++
 include/linux/mount.h |    3 +++
 2 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/fs/namespace.c b/fs/namespace.c
index d99bcf5..4bcc60e9 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -1494,6 +1494,23 @@ void drop_collected_mounts(struct vfsmount *mnt)
 	release_mounts(&umount_list);
 }
 
+struct vfsmount *clone_private_mount(struct path *path)
+{
+	struct vfsmount *mnt;
+
+	if (IS_MNT_UNBINDABLE(path->mnt))
+		return ERR_PTR(-EINVAL);
+
+	down_read(&namespace_sem);
+	mnt = clone_mnt(path->mnt, path->dentry, CL_PRIVATE);
+	up_read(&namespace_sem);
+	if (!mnt)
+		return ERR_PTR(-ENOMEM);
+
+	return mnt;
+}
+EXPORT_SYMBOL_GPL(clone_private_mount);
+
 int iterate_mounts(int (*f)(struct vfsmount *, void *), void *arg,
 		   struct vfsmount *root)
 {
diff --git a/include/linux/mount.h b/include/linux/mount.h
index 604f122..44e9bf4 100644
--- a/include/linux/mount.h
+++ b/include/linux/mount.h
@@ -100,6 +100,9 @@ extern void mnt_pin(struct vfsmount *mnt);
 extern void mnt_unpin(struct vfsmount *mnt);
 extern int __mnt_is_readonly(struct vfsmount *mnt);
 
+struct path;
+extern struct vfsmount *clone_private_mount(struct path *path);
+
 extern struct vfsmount *do_kern_mount(const char *fstype, int flags,
 				      const char *name, void *data);
 
-- 
1.7.3.4

[PATCH 5/7] overlay filesystem

[Miklos Szeredi]

From: Miklos Szeredi <mszeredi@suse.cz>

Overlayfs allows one, usually read-write, directory tree to be
overlaid onto another, read-only directory tree.  All modifications
go to the upper, writable layer.

This type of mechanism is most often used for live CDs but there's a
wide variety of other uses.

The implementation differs from other "union filesystem"
implementations in that after a file is opened all operations go
directly to the underlying, lower or upper, filesystems.  This
simplifies the implementation and allows native performance in these
cases.

The dentry tree is duplicated from the underlying filesystems, this
enables fast cached lookups without adding special support into the
VFS.  This uses slightly more memory than union mounts, but dentries
are relatively small.

Currently inodes are duplicated as well, but it is a possible
optimization to share inodes for non-directories.

Opening non directories results in the open forwarded to the
underlying filesystem.  This makes the behavior very similar to union
mounts (with the same limitations vs. fchmod/fchown on O_RDONLY file
descriptors).

Usage:

  mount -t overlay -olowerdir=/lower,upperdir=/upper overlay /mnt

Supported:

 - all operations

Missing:

 - ensure that filesystems part of the overlay are not modified outside
   the overlay

Neil Brown <neilb@suse.de>:
 - minimal remount support
 - use correct seek function for directories
 - initialise is_real before use
 - rename ovl_fill_cache to ovl_dir_read

Felix Fietkau <nbd@openwrt.org>:
 - fix a deadlock in ovl_dir_read_merged
 - fix a deadlock in ovl_remove_whiteouts

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 fs/Kconfig               |    1 +
 fs/Makefile              |    1 +
 fs/overlayfs/Kconfig     |    4 +
 fs/overlayfs/Makefile    |    7 +
 fs/overlayfs/copy_up.c   |  383 +++++++++++++++++++++++++++++
 fs/overlayfs/dir.c       |  594 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/overlayfs/inode.c     |  374 +++++++++++++++++++++++++++++
 fs/overlayfs/overlayfs.h |   62 +++++
 fs/overlayfs/readdir.c   |  560 +++++++++++++++++++++++++++++++++++++++++++
 fs/overlayfs/super.c     |  576 ++++++++++++++++++++++++++++++++++++++++++++
 10 files changed, 2562 insertions(+), 0 deletions(-)
 create mode 100644 fs/overlayfs/Kconfig
 create mode 100644 fs/overlayfs/Makefile
 create mode 100644 fs/overlayfs/copy_up.c
 create mode 100644 fs/overlayfs/dir.c
 create mode 100644 fs/overlayfs/inode.c
 create mode 100644 fs/overlayfs/overlayfs.h
 create mode 100644 fs/overlayfs/readdir.c
 create mode 100644 fs/overlayfs/super.c

diff --git a/fs/Kconfig b/fs/Kconfig
index 979992d..cb572d1 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -63,6 +63,7 @@ source "fs/quota/Kconfig"
 
 source "fs/autofs4/Kconfig"
 source "fs/fuse/Kconfig"
+source "fs/overlayfs/Kconfig"
 
 config CUSE
 	tristate "Character device in Userspace support"
diff --git a/fs/Makefile b/fs/Makefile
index fb68c2b..ac7402a 100644
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -105,6 +105,7 @@ obj-$(CONFIG_QNX4FS_FS)		+= qnx4/
 obj-$(CONFIG_AUTOFS4_FS)	+= autofs4/
 obj-$(CONFIG_ADFS_FS)		+= adfs/
 obj-$(CONFIG_FUSE_FS)		+= fuse/
+obj-$(CONFIG_OVERLAYFS_FS)	+= overlayfs/
 obj-$(CONFIG_UDF_FS)		+= udf/
 obj-$(CONFIG_SUN_OPENPROMFS)	+= openpromfs/
 obj-$(CONFIG_OMFS_FS)		+= omfs/
diff --git a/fs/overlayfs/Kconfig b/fs/overlayfs/Kconfig
new file mode 100644
index 0000000..c4517da
--- /dev/null
+++ b/fs/overlayfs/Kconfig
@@ -0,0 +1,4 @@
+config OVERLAYFS_FS
+	tristate "Overlay filesystem support"
+	help
+	  Add support for overlay filesystem.
diff --git a/fs/overlayfs/Makefile b/fs/overlayfs/Makefile
new file mode 100644
index 0000000..8f91889
--- /dev/null
+++ b/fs/overlayfs/Makefile
@@ -0,0 +1,7 @@
+#
+# Makefile for the overlay filesystem.
+#
+
+obj-$(CONFIG_OVERLAYFS_FS) += overlayfs.o
+
+overlayfs-objs := super.o inode.o dir.o readdir.o copy_up.o
diff --git a/fs/overlayfs/copy_up.c b/fs/overlayfs/copy_up.c
new file mode 100644
index 0000000..308a80a
--- /dev/null
+++ b/fs/overlayfs/copy_up.c
@@ -0,0 +1,383 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/file.h>
+#include <linux/splice.h>
+#include <linux/xattr.h>
+#include <linux/security.h>
+#include <linux/uaccess.h>
+#include "overlayfs.h"
+
+#define OVL_COPY_UP_CHUNK_SIZE (1 << 20)
+
+static int ovl_copy_up_xattr(struct dentry *old, struct dentry *new)
+{
+	ssize_t list_size, size;
+	char *buf, *name, *value;
+	int error;
+
+	if (!old->d_inode->i_op->getxattr ||
+	    !new->d_inode->i_op->getxattr)
+		return 0;
+
+	list_size = vfs_listxattr(old, NULL, 0);
+	if (list_size <= 0) {
+		if (list_size == -EOPNOTSUPP)
+			return 0;
+		return list_size;
+	}
+
+	buf = kzalloc(list_size, GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	error = -ENOMEM;
+	value = kmalloc(XATTR_SIZE_MAX, GFP_KERNEL);
+	if (!value)
+		goto out;
+
+	list_size = vfs_listxattr(old, buf, list_size);
+	if (list_size <= 0) {
+		error = list_size;
+		goto out_free_value;
+	}
+
+	for (name = buf; name < (buf + list_size); name += strlen(name) + 1) {
+		size = vfs_getxattr(old, name, value, XATTR_SIZE_MAX);
+		if (size <= 0) {
+			error = size;
+			goto out_free_value;
+		}
+		error = vfs_setxattr(new, name, value, size, 0);
+		if (error)
+			goto out_free_value;
+	}
+
+out_free_value:
+	kfree(value);
+out:
+	kfree(buf);
+	return error;
+}
+
+static int ovl_copy_up_data(struct path *old, struct path *new, loff_t len)
+{
+	struct file *old_file;
+	struct file *new_file;
+	int error = 0;
+
+	if (len == 0)
+		return 0;
+
+	old_file = vfs_open(old, O_RDONLY, current_cred());
+	if (IS_ERR(old_file))
+		return PTR_ERR(old_file);
+
+	new_file = vfs_open(new, O_WRONLY, current_cred());
+	if (IS_ERR(new_file)) {
+		error = PTR_ERR(new_file);
+		goto out_fput;
+	}
+
+	/* FIXME: copy up sparse files efficiently */
+	while (len) {
+		loff_t offset = new_file->f_pos;
+		size_t this_len = OVL_COPY_UP_CHUNK_SIZE;
+		long bytes;
+
+		if (len < this_len)
+			this_len = len;
+
+		if (signal_pending_state(TASK_KILLABLE, current)) {
+			error = -EINTR;
+			break;
+		}
+
+		bytes = do_splice_direct(old_file, &offset, new_file, this_len,
+				 SPLICE_F_MOVE);
+		if (bytes <= 0) {
+			error = bytes;
+			break;
+		}
+
+		len -= bytes;
+	}
+
+	fput(new_file);
+out_fput:
+	fput(old_file);
+	return error;
+}
+
+static char *ovl_read_symlink(struct dentry *realdentry)
+{
+	int res;
+	char *buf;
+	struct inode *inode = realdentry->d_inode;
+	mm_segment_t old_fs;
+
+	res = -EINVAL;
+	if (!inode->i_op->readlink)
+		goto err;
+
+	res = -ENOMEM;
+	buf = (char *) __get_free_page(GFP_KERNEL);
+	if (!buf)
+		goto err;
+
+	old_fs = get_fs();
+	set_fs(get_ds());
+	/* The cast to a user pointer is valid due to the set_fs() */
+	res = inode->i_op->readlink(realdentry,
+				    (char __user *)buf, PAGE_SIZE - 1);
+	set_fs(old_fs);
+	if (res < 0) {
+		free_page((unsigned long) buf);
+		goto err;
+	}
+	buf[res] = '\0';
+
+	return buf;
+
+err:
+	return ERR_PTR(res);
+}
+
+static int ovl_set_timestamps(struct dentry *upperdentry, struct kstat *stat)
+{
+	struct iattr attr = {
+		.ia_valid = ATTR_ATIME | ATTR_MTIME | ATTR_ATIME_SET | ATTR_MTIME_SET,
+		.ia_atime = stat->atime,
+		.ia_mtime = stat->mtime,
+	};
+
+	return notify_change(upperdentry, &attr);
+}
+
+static int ovl_set_mode(struct dentry *upperdentry, umode_t mode)
+{
+	struct iattr attr = {
+		.ia_valid = ATTR_MODE,
+		.ia_mode = mode,
+	};
+
+	return notify_change(upperdentry, &attr);
+}
+
+static int ovl_copy_up_locked(struct dentry *upperdir, struct dentry *dentry,
+			      struct path *lowerpath, struct kstat *stat,
+			      const char *link)
+{
+	int err;
+	struct path newpath;
+	umode_t mode = stat->mode;
+
+	/* Can't properly set mode on creation because of the umask */
+	stat->mode &= S_IFMT;
+
+	ovl_path_upper(dentry, &newpath);
+	WARN_ON(newpath.dentry);
+	newpath.dentry = ovl_upper_create(upperdir, dentry, stat, link);
+	if (IS_ERR(newpath.dentry))
+		return PTR_ERR(newpath.dentry);
+
+	if (S_ISREG(stat->mode)) {
+		err = ovl_copy_up_data(lowerpath, &newpath, stat->size);
+		if (err)
+			goto err_remove;
+	}
+
+	err = ovl_copy_up_xattr(lowerpath->dentry, newpath.dentry);
+	if (err)
+		goto err_remove;
+
+	mutex_lock(&newpath.dentry->d_inode->i_mutex);
+	if (!S_ISLNK(stat->mode))
+		err = ovl_set_mode(newpath.dentry, mode);
+	if (!err)
+		err = ovl_set_timestamps(newpath.dentry, stat);
+	mutex_unlock(&newpath.dentry->d_inode->i_mutex);
+	if (err)
+		goto err_remove;
+
+	ovl_dentry_update(dentry, newpath.dentry);
+
+	/*
+	 * Easiest way to get rid of the lower dentry reference is to
+	 * drop this dentry.  This is neither needed nor possible for
+	 * directories.
+	 */
+	if (!S_ISDIR(stat->mode))
+		d_drop(dentry);
+
+	return 0;
+
+err_remove:
+	if (S_ISDIR(stat->mode))
+		vfs_rmdir(upperdir->d_inode, newpath.dentry);
+	else
+		vfs_unlink(upperdir->d_inode, newpath.dentry);
+
+	dput(newpath.dentry);
+
+	return err;
+}
+
+/*
+ * Copy up a single dentry
+ *
+ * Directory renames only allowed on "pure upper" (already created on
+ * upper filesystem, never copied up).  Directories which are on lower or
+ * are merged may not be renamed.  For these -EXDEV is returned and
+ * userspace has to deal with it.  This means, when copying up a
+ * directory we can rely on it and ancestors being stable.
+ *
+ * Non-directory renames start with copy up of source if necessary.  The
+ * actual rename will only proceed once the copy up was successful.  Copy
+ * up uses upper parent i_mutex for exclusion.  Since rename can change
+ * d_parent it is possible that the copy up will lock the old parent.  At
+ * that point the file will have already been copied up anyway.
+ */
+static int ovl_copy_up_one(struct dentry *parent, struct dentry *dentry,
+			   struct path *lowerpath, struct kstat *stat)
+{
+	int err;
+	struct kstat pstat;
+	struct path parentpath;
+	struct dentry *upperdir;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+	char *link = NULL;
+
+	ovl_path_upper(parent, &parentpath);
+	upperdir = parentpath.dentry;
+
+	err = vfs_getattr(parentpath.mnt, parentpath.dentry, &pstat);
+	if (err)
+		return err;
+
+	if (S_ISLNK(stat->mode)) {
+		link = ovl_read_symlink(lowerpath->dentry);
+		if (IS_ERR(link))
+			return PTR_ERR(link);
+	}
+
+	err = -ENOMEM;
+	override_cred = prepare_creds();
+	if (!override_cred)
+		goto out_free_link;
+
+	override_cred->fsuid = stat->uid;
+	override_cred->fsgid = stat->gid;
+	/*
+	 * CAP_SYS_ADMIN for copying up extended attributes
+	 * CAP_DAC_OVERRIDE for create
+	 * CAP_FOWNER for chmod, timestamp update
+	 * CAP_FSETID for chmod
+	 * CAP_MKNOD for mknod
+	 */
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	cap_raise(override_cred->cap_effective, CAP_DAC_OVERRIDE);
+	cap_raise(override_cred->cap_effective, CAP_FOWNER);
+	cap_raise(override_cred->cap_effective, CAP_FSETID);
+	cap_raise(override_cred->cap_effective, CAP_MKNOD);
+	old_cred = override_creds(override_cred);
+
+	mutex_lock_nested(&upperdir->d_inode->i_mutex, I_MUTEX_PARENT);
+	if (ovl_path_type(dentry) != OVL_PATH_LOWER) {
+		err = 0;
+	} else {
+		err = ovl_copy_up_locked(upperdir, dentry, lowerpath,
+					 stat, link);
+		if (!err) {
+			/* Restore timestamps on parent (best effort) */
+			ovl_set_timestamps(upperdir, &pstat);
+		}
+	}
+
+	mutex_unlock(&upperdir->d_inode->i_mutex);
+
+	revert_creds(old_cred);
+	put_cred(override_cred);
+
+out_free_link:
+	if (link)
+		free_page((unsigned long) link);
+
+	return err;
+}
+
+int ovl_copy_up(struct dentry *dentry)
+{
+	int err;
+
+	err = 0;
+	while (!err) {
+		struct dentry *next;
+		struct dentry *parent;
+		struct path lowerpath;
+		struct kstat stat;
+		enum ovl_path_type type = ovl_path_type(dentry);
+
+		if (type != OVL_PATH_LOWER)
+			break;
+
+		next = dget(dentry);
+		/* find the topmost dentry not yet copied up */
+		for (;;) {
+			parent = dget_parent(next);
+
+			type = ovl_path_type(parent);
+			if (type != OVL_PATH_LOWER)
+				break;
+
+			dput(next);
+			next = parent;
+		}
+
+		ovl_path_lower(next, &lowerpath);
+		err = vfs_getattr(lowerpath.mnt, lowerpath.dentry, &stat);
+		if (!err)
+			err = ovl_copy_up_one(parent, next, &lowerpath, &stat);
+
+		dput(parent);
+		dput(next);
+	}
+
+	return err;
+}
+
+/* Optimize by not copying up the file first and truncating later */
+int ovl_copy_up_truncate(struct dentry *dentry, loff_t size)
+{
+	int err;
+	struct kstat stat;
+	struct path lowerpath;
+	struct dentry *parent = dget_parent(dentry);
+
+	err = ovl_copy_up(parent);
+	if (err)
+		goto out_dput_parent;
+
+	ovl_path_lower(dentry, &lowerpath);
+	err = vfs_getattr(lowerpath.mnt, lowerpath.dentry, &stat);
+	if (err)
+		goto out_dput_parent;
+
+	if (size < stat.size)
+		stat.size = size;
+
+	err = ovl_copy_up_one(parent, dentry, &lowerpath, &stat);
+
+out_dput_parent:
+	dput(parent);
+	return err;
+}
diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
new file mode 100644
index 0000000..e1c09c4
--- /dev/null
+++ b/fs/overlayfs/dir.c
@@ -0,0 +1,594 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/namei.h>
+#include <linux/xattr.h>
+#include <linux/security.h>
+#include "overlayfs.h"
+
+static const char *ovl_whiteout_symlink = "(overlay-whiteout)";
+
+static struct dentry *ovl_lookup(struct inode *dir, struct dentry *dentry,
+				 struct nameidata *nd)
+{
+	int err = ovl_do_lookup(dentry);
+
+	if (err)
+		return ERR_PTR(err);
+
+	return NULL;
+}
+
+static int ovl_whiteout(struct dentry *upperdir, struct dentry *dentry)
+{
+	int err;
+	struct dentry *newdentry;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+
+	/* FIXME: recheck lower dentry to see if whiteout is really needed */
+
+	err = -ENOMEM;
+	override_cred = prepare_creds();
+	if (!override_cred)
+		goto out;
+
+	/*
+	 * CAP_SYS_ADMIN for setxattr
+	 * CAP_DAC_OVERRIDE for symlink creation
+	 * CAP_FOWNER for unlink in sticky directory
+	 */
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	cap_raise(override_cred->cap_effective, CAP_DAC_OVERRIDE);
+	cap_raise(override_cred->cap_effective, CAP_FOWNER);
+	override_cred->fsuid = 0;
+	override_cred->fsgid = 0;
+	old_cred = override_creds(override_cred);
+
+	newdentry = lookup_one_len(dentry->d_name.name, upperdir,
+				   dentry->d_name.len);
+	err = PTR_ERR(newdentry);
+	if (IS_ERR(newdentry))
+		goto out_put_cred;
+
+	/* Just been removed within the same locked region */
+	WARN_ON(newdentry->d_inode);
+
+	err = vfs_symlink(upperdir->d_inode, newdentry, ovl_whiteout_symlink);
+	if (err)
+		goto out_dput;
+
+	ovl_dentry_version_inc(dentry->d_parent);
+
+	err = vfs_setxattr(newdentry, ovl_whiteout_xattr, "y", 1, 0);
+	if (err)
+		vfs_unlink(upperdir->d_inode, newdentry);
+
+out_dput:
+	dput(newdentry);
+out_put_cred:
+	revert_creds(old_cred);
+	put_cred(override_cred);
+out:
+	if (err) {
+		/*
+		 * There's no way to recover from failure to whiteout.
+		 * What should we do?  Log a big fat error and... ?
+		 */
+		printk(KERN_ERR "overlayfs: ERROR - failed to whiteout '%s'\n",
+		       dentry->d_name.name);
+	}
+
+	return err;
+}
+
+static struct dentry *ovl_lookup_create(struct dentry *upperdir,
+					struct dentry *template)
+{
+	int err;
+	struct dentry *newdentry;
+	struct qstr *name = &template->d_name;
+
+	newdentry = lookup_one_len(name->name, upperdir, name->len);
+	if (IS_ERR(newdentry))
+		return newdentry;
+
+	if (newdentry->d_inode) {
+		const struct cred *old_cred;
+		struct cred *override_cred;
+
+		/* No need to check whiteout if lower parent is non-existent */
+		err = -EEXIST;
+		if (!ovl_dentry_lower(template->d_parent))
+			goto out_dput;
+
+		if (!S_ISLNK(newdentry->d_inode->i_mode))
+			goto out_dput;
+
+		err = -ENOMEM;
+		override_cred = prepare_creds();
+		if (!override_cred)
+			goto out_dput;
+
+		/*
+		 * CAP_SYS_ADMIN for getxattr
+		 * CAP_FOWNER for unlink in sticky directory
+		 */
+		cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+		cap_raise(override_cred->cap_effective, CAP_FOWNER);
+		old_cred = override_creds(override_cred);
+
+		err = -EEXIST;
+		if (ovl_is_whiteout(newdentry))
+			err = vfs_unlink(upperdir->d_inode, newdentry);
+
+		revert_creds(old_cred);
+		put_cred(override_cred);
+		if (err)
+			goto out_dput;
+
+		dput(newdentry);
+		newdentry = lookup_one_len(name->name, upperdir, name->len);
+		if (IS_ERR(newdentry)) {
+			ovl_whiteout(upperdir, template);
+			return newdentry;
+		}
+
+		/*
+		 * Whiteout just been successfully removed, parent
+		 * i_mutex is still held, there's no way the lookup
+		 * could return positive.
+		 */
+		WARN_ON(newdentry->d_inode);
+	}
+
+	return newdentry;
+
+out_dput:
+	dput(newdentry);
+	return ERR_PTR(err);
+}
+
+struct dentry *ovl_upper_create(struct dentry *upperdir, struct dentry *dentry,
+				struct kstat *stat, const char *link)
+{
+	int err;
+	struct dentry *newdentry;
+	struct inode *dir = upperdir->d_inode;
+
+	newdentry = ovl_lookup_create(upperdir, dentry);
+	if (IS_ERR(newdentry))
+		goto out;
+
+	switch (stat->mode & S_IFMT) {
+	case S_IFREG:
+		err = vfs_create(dir, newdentry, stat->mode, NULL);
+		break;
+
+	case S_IFDIR:
+		err = vfs_mkdir(dir, newdentry, stat->mode);
+		break;
+
+	case S_IFCHR:
+	case S_IFBLK:
+	case S_IFIFO:
+	case S_IFSOCK:
+		err = vfs_mknod(dir, newdentry, stat->mode, stat->rdev);
+		break;
+
+	case S_IFLNK:
+		err = vfs_symlink(dir, newdentry, link);
+		break;
+
+	default:
+		err = -EPERM;
+	}
+	if (err) {
+		if (ovl_dentry_is_opaque(dentry))
+			ovl_whiteout(upperdir, dentry);
+		dput(newdentry);
+		newdentry = ERR_PTR(err);
+	}
+
+out:
+	return newdentry;
+
+}
+
+static int ovl_set_opaque(struct dentry *upperdentry)
+{
+	int err;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+
+	override_cred = prepare_creds();
+	if (!override_cred)
+		return -ENOMEM;
+
+	/* CAP_SYS_ADMIN for setxattr of "trusted" namespace */
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	old_cred = override_creds(override_cred);
+	err = vfs_setxattr(upperdentry, ovl_opaque_xattr, "y", 1, 0);
+	revert_creds(old_cred);
+	put_cred(override_cred);
+
+	return err;
+}
+
+static int ovl_remove_opaque(struct dentry *upperdentry)
+{
+	int err;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+
+	override_cred = prepare_creds();
+	if (!override_cred)
+		return -ENOMEM;
+
+	/* CAP_SYS_ADMIN for removexattr of "trusted" namespace */
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	old_cred = override_creds(override_cred);
+	err = vfs_removexattr(upperdentry, ovl_opaque_xattr);
+	revert_creds(old_cred);
+	put_cred(override_cred);
+
+	return err;
+}
+
+static int ovl_dir_getattr(struct vfsmount *mnt, struct dentry *dentry,
+			 struct kstat *stat)
+{
+	int err;
+	enum ovl_path_type type;
+	struct path realpath;
+
+	type = ovl_path_real(dentry, &realpath);
+	err = vfs_getattr(realpath.mnt, realpath.dentry, stat);
+	if (err)
+		return err;
+
+	stat->dev = dentry->d_sb->s_dev;
+	stat->ino = dentry->d_inode->i_ino;
+
+	/*
+	 * It's probably not worth it to count subdirs to get the
+	 * correct link count.  nlink=1 seems to pacify 'find' and
+	 * other utilities.
+	 */
+	if (type == OVL_PATH_MERGE)
+		stat->nlink = 1;
+
+	return 0;
+}
+
+static int ovl_create_object(struct dentry *dentry, int mode, dev_t rdev,
+			     const char *link)
+{
+	int err;
+	struct dentry *newdentry;
+	struct dentry *upperdir;
+	struct inode *inode;
+	struct kstat stat = {
+		.mode = mode,
+		.rdev = rdev,
+	};
+
+	err = -ENOMEM;
+	inode = ovl_new_inode(dentry->d_sb, mode, dentry->d_fsdata);
+	if (!inode)
+		goto out;
+
+	err = ovl_copy_up(dentry->d_parent);
+	if (err)
+		goto out_iput;
+
+	upperdir = ovl_dentry_upper(dentry->d_parent);
+	mutex_lock_nested(&upperdir->d_inode->i_mutex, I_MUTEX_PARENT);
+
+	newdentry = ovl_upper_create(upperdir, dentry, &stat, link);
+	err = PTR_ERR(newdentry);
+	if (IS_ERR(newdentry))
+		goto out_unlock;
+
+	ovl_dentry_version_inc(dentry->d_parent);
+	if (ovl_dentry_is_opaque(dentry) && S_ISDIR(mode)) {
+		err = ovl_set_opaque(newdentry);
+		if (err) {
+			vfs_rmdir(upperdir->d_inode, newdentry);
+			ovl_whiteout(upperdir, dentry);
+			goto out_dput;
+		}
+	}
+	ovl_dentry_update(dentry, newdentry);
+	d_instantiate(dentry, inode);
+	inode = NULL;
+	newdentry = NULL;
+	err = 0;
+
+out_dput:
+	dput(newdentry);
+out_unlock:
+	mutex_unlock(&upperdir->d_inode->i_mutex);
+out_iput:
+	iput(inode);
+out:
+	return err;
+}
+
+static int ovl_create(struct inode *dir, struct dentry *dentry, int mode,
+			struct nameidata *nd)
+{
+	return ovl_create_object(dentry, (mode & 07777) | S_IFREG, 0, NULL);
+}
+
+static int ovl_mkdir(struct inode *dir, struct dentry *dentry, int mode)
+{
+	return ovl_create_object(dentry, (mode & 07777) | S_IFDIR, 0, NULL);
+}
+
+static int ovl_mknod(struct inode *dir, struct dentry *dentry, int mode,
+		       dev_t rdev)
+{
+	return ovl_create_object(dentry, mode, rdev, NULL);
+}
+
+static int ovl_symlink(struct inode *dir, struct dentry *dentry,
+			 const char *link)
+{
+	return ovl_create_object(dentry, S_IFLNK, 0, link);
+}
+
+static int ovl_do_remove(struct dentry *dentry, bool is_dir)
+{
+	int err;
+	enum ovl_path_type type;
+	struct path realpath;
+	struct dentry *upperdir;
+
+	err = ovl_copy_up(dentry->d_parent);
+	if (err)
+		return err;
+
+	upperdir = ovl_dentry_upper(dentry->d_parent);
+	mutex_lock_nested(&upperdir->d_inode->i_mutex, I_MUTEX_PARENT);
+	type = ovl_path_real(dentry, &realpath);
+	if (type != OVL_PATH_LOWER) {
+		err = -ESTALE;
+		if (realpath.dentry->d_parent != upperdir)
+			goto out_d_drop;
+
+		/* FIXME: create whiteout up front and rename to target */
+
+		if (is_dir)
+			err = vfs_rmdir(upperdir->d_inode, realpath.dentry);
+		else
+			err = vfs_unlink(upperdir->d_inode, realpath.dentry);
+		if (err)
+			goto out_d_drop;
+
+		ovl_dentry_version_inc(dentry->d_parent);
+	}
+
+	if (type != OVL_PATH_UPPER || ovl_dentry_is_opaque(dentry))
+		err = ovl_whiteout(upperdir, dentry);
+
+	/*
+	 * Keeping this dentry hashed would mean having to release
+	 * upperpath/lowerpath, which could only be done if we are the
+	 * sole user of this dentry.  Too tricky...  Just unhash for
+	 * now.
+	 */
+out_d_drop:
+	d_drop(dentry);
+	mutex_unlock(&upperdir->d_inode->i_mutex);
+
+	return err;
+}
+
+static int ovl_unlink(struct inode *dir, struct dentry *dentry)
+{
+	return ovl_do_remove(dentry, false);
+}
+
+
+static int ovl_rmdir(struct inode *dir, struct dentry *dentry)
+{
+	int err;
+	enum ovl_path_type type;
+
+	type = ovl_path_type(dentry);
+	if (type != OVL_PATH_UPPER) {
+		err = ovl_check_empty_and_clear(dentry, type);
+		if (err)
+			return err;
+	}
+
+	return ovl_do_remove(dentry, true);
+}
+
+static int ovl_link(struct dentry *old, struct inode *newdir,
+		    struct dentry *new)
+{
+	int err;
+	struct dentry *olddentry;
+	struct dentry *newdentry;
+	struct dentry *upperdir;
+
+	err = ovl_copy_up(old);
+	if (err)
+		goto out;
+
+	err = ovl_copy_up(new->d_parent);
+	if (err)
+		goto out;
+
+	upperdir = ovl_dentry_upper(new->d_parent);
+	mutex_lock_nested(&upperdir->d_inode->i_mutex, I_MUTEX_PARENT);
+	newdentry = ovl_lookup_create(upperdir, new);
+	err = PTR_ERR(newdentry);
+	if (IS_ERR(newdentry))
+		goto out_unlock;
+
+	olddentry = ovl_dentry_upper(old);
+	err = vfs_link(olddentry, upperdir->d_inode, newdentry);
+	if (!err) {
+		ovl_dentry_version_inc(new->d_parent);
+		ovl_dentry_update(new, newdentry);
+
+		ihold(old->d_inode);
+		d_instantiate(new, old->d_inode);
+	} else {
+		if (ovl_dentry_is_opaque(new))
+			ovl_whiteout(upperdir, new);
+		dput(newdentry);
+	}
+out_unlock:
+	mutex_unlock(&upperdir->d_inode->i_mutex);
+out:
+	return err;
+
+}
+
+static int ovl_rename(struct inode *olddir, struct dentry *old,
+			struct inode *newdir, struct dentry *new)
+{
+	int err;
+	enum ovl_path_type old_type;
+	enum ovl_path_type new_type;
+	struct dentry *old_upperdir;
+	struct dentry *new_upperdir;
+	struct dentry *olddentry;
+	struct dentry *newdentry;
+	struct dentry *trap;
+	bool old_opaque;
+	bool new_opaque;
+	bool new_create = false;
+	bool is_dir = S_ISDIR(old->d_inode->i_mode);
+
+	/* Don't copy up directory trees */
+	old_type = ovl_path_type(old);
+	if (old_type != OVL_PATH_UPPER && is_dir)
+		return -EXDEV;
+
+	if (new->d_inode) {
+		new_type = ovl_path_type(new);
+
+		if (new_type == OVL_PATH_LOWER && old_type == OVL_PATH_LOWER) {
+			if (ovl_dentry_lower(old)->d_inode ==
+			    ovl_dentry_lower(new)->d_inode)
+				return 0;
+		}
+		if (new_type != OVL_PATH_LOWER && old_type != OVL_PATH_LOWER) {
+			if (ovl_dentry_upper(old)->d_inode ==
+			    ovl_dentry_upper(new)->d_inode)
+				return 0;
+		}
+
+		if (new_type != OVL_PATH_UPPER &&
+		    S_ISDIR(new->d_inode->i_mode)) {
+			err = ovl_check_empty_and_clear(new, new_type);
+			if (err)
+				return err;
+		}
+	} else {
+		new_type = OVL_PATH_UPPER;
+	}
+
+	err = ovl_copy_up(old);
+	if (err)
+		return err;
+
+	err = ovl_copy_up(new->d_parent);
+	if (err)
+		return err;
+
+	old_upperdir = ovl_dentry_upper(old->d_parent);
+	new_upperdir = ovl_dentry_upper(new->d_parent);
+
+	trap = lock_rename(new_upperdir, old_upperdir);
+
+	olddentry = ovl_dentry_upper(old);
+	newdentry = ovl_dentry_upper(new);
+	if (newdentry) {
+		dget(newdentry);
+	} else {
+		new_create = true;
+		newdentry = ovl_lookup_create(new_upperdir, new);
+		err = PTR_ERR(newdentry);
+		if (IS_ERR(newdentry))
+			goto out_unlock;
+	}
+
+	err = -ESTALE;
+	if (olddentry->d_parent != old_upperdir)
+		goto out_dput;
+	if (newdentry->d_parent != new_upperdir)
+		goto out_dput;
+	if (olddentry == trap)
+		goto out_dput;
+	if (newdentry == trap)
+		goto out_dput;
+
+	old_opaque = ovl_dentry_is_opaque(old);
+	new_opaque = ovl_dentry_is_opaque(new) || new_type != OVL_PATH_UPPER;
+
+	if (is_dir && !old_opaque && new_opaque) {
+		err = ovl_set_opaque(olddentry);
+		if (err)
+			goto out_dput;
+	}
+
+	err = vfs_rename(old_upperdir->d_inode, olddentry,
+			 new_upperdir->d_inode, newdentry);
+
+	if (err) {
+		if (new_create && ovl_dentry_is_opaque(new))
+			ovl_whiteout(new_upperdir, new);
+		if (is_dir && !old_opaque && new_opaque)
+			ovl_remove_opaque(olddentry);
+		goto out_dput;
+	}
+
+	if (old_type != OVL_PATH_UPPER || old_opaque)
+		err = ovl_whiteout(old_upperdir, old);
+	if (is_dir && old_opaque && !new_opaque)
+		ovl_remove_opaque(olddentry);
+
+	if (old_opaque != new_opaque)
+		ovl_dentry_set_opaque(old, new_opaque);
+
+	ovl_dentry_version_inc(old->d_parent);
+	ovl_dentry_version_inc(new->d_parent);
+
+out_dput:
+	dput(newdentry);
+out_unlock:
+	unlock_rename(new_upperdir, old_upperdir);
+	return err;
+}
+
+const struct inode_operations ovl_dir_inode_operations = {
+	.lookup		= ovl_lookup,
+	.mkdir		= ovl_mkdir,
+	.symlink	= ovl_symlink,
+	.unlink		= ovl_unlink,
+	.rmdir		= ovl_rmdir,
+	.rename		= ovl_rename,
+	.link		= ovl_link,
+	.setattr	= ovl_setattr,
+	.create		= ovl_create,
+	.mknod		= ovl_mknod,
+	.permission	= ovl_permission,
+	.getattr	= ovl_dir_getattr,
+	.setxattr	= ovl_setxattr,
+	.getxattr	= ovl_getxattr,
+	.listxattr	= ovl_listxattr,
+	.removexattr	= ovl_removexattr,
+};
diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
new file mode 100644
index 0000000..3c15d54
--- /dev/null
+++ b/fs/overlayfs/inode.c
@@ -0,0 +1,374 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/xattr.h>
+#include "overlayfs.h"
+
+int ovl_setattr(struct dentry *dentry, struct iattr *attr)
+{
+	struct dentry *upperdentry;
+	int err;
+
+	if ((attr->ia_valid & ATTR_SIZE) && !ovl_dentry_upper(dentry))
+		err = ovl_copy_up_truncate(dentry, attr->ia_size);
+	else
+		err = ovl_copy_up(dentry);
+	if (err)
+		return err;
+
+	upperdentry = ovl_dentry_upper(dentry);
+
+	if (attr->ia_valid & (ATTR_KILL_SUID|ATTR_KILL_SGID))
+		attr->ia_valid &= ~ATTR_MODE;
+
+	mutex_lock(&upperdentry->d_inode->i_mutex);
+	err = notify_change(upperdentry, attr);
+	mutex_unlock(&upperdentry->d_inode->i_mutex);
+
+	return err;
+}
+
+static int ovl_getattr(struct vfsmount *mnt, struct dentry *dentry,
+			 struct kstat *stat)
+{
+	struct path realpath;
+
+	ovl_path_real(dentry, &realpath);
+	return vfs_getattr(realpath.mnt, realpath.dentry, stat);
+}
+
+int ovl_permission(struct inode *inode, int mask, unsigned int flags)
+{
+	struct ovl_entry *oe;
+	struct dentry *alias = NULL;
+	struct inode *realinode;
+	struct dentry *realdentry;
+	bool is_upper;
+	int err;
+
+	if (S_ISDIR(inode->i_mode)) {
+		oe = inode->i_private;
+	} else if (flags & IPERM_FLAG_RCU) {
+		return -ECHILD;
+	} else {
+		/*
+		 * For non-directories find an alias and get the info
+		 * from there.
+		 */
+		spin_lock(&inode->i_lock);
+		if (WARN_ON(list_empty(&inode->i_dentry))) {
+			spin_unlock(&inode->i_lock);
+			return -ENOENT;
+		}
+		alias = list_entry(inode->i_dentry.next, struct dentry, d_alias);
+		dget(alias);
+		spin_unlock(&inode->i_lock);
+		oe = alias->d_fsdata;
+	}
+
+	realdentry = ovl_entry_real(oe, &is_upper);
+
+	/* Careful in RCU walk mode */
+	realinode = ACCESS_ONCE(realdentry->d_inode);
+	if (!realinode) {
+		WARN_ON(!(flags & IPERM_FLAG_RCU));
+		return -ENOENT;
+	}
+
+	if (mask & MAY_WRITE) {
+		umode_t mode = realinode->i_mode;
+
+		/*
+		 * Writes will always be redirected to upper layer, so
+		 * ignore lower layer being read-only.
+		 */
+		err = -EROFS;
+		if (is_upper && IS_RDONLY(realinode) &&
+		    (S_ISREG(mode) || S_ISDIR(mode) || S_ISLNK(mode)))
+			goto out_dput;
+
+		/*
+		 * Nobody gets write access to an immutable file.
+		 */
+		err = -EACCES;
+		if (IS_IMMUTABLE(realinode))
+			goto out_dput;
+	}
+
+	if (realinode->i_op->permission)
+		err = realinode->i_op->permission(realinode, mask, flags);
+	else
+		err = generic_permission(realinode, mask, flags,
+					 realinode->i_op->check_acl);
+out_dput:
+	dput(alias);
+	return err;
+}
+
+
+struct ovl_link_data {
+	struct dentry *realdentry;
+	void *cookie;
+};
+
+static void *ovl_follow_link(struct dentry *dentry, struct nameidata *nd)
+{
+	void *ret;
+	struct dentry *realdentry;
+	struct inode *realinode;
+
+	realdentry = ovl_dentry_real(dentry);
+	realinode = realdentry->d_inode;
+
+	if (WARN_ON(!realinode->i_op->follow_link))
+		return ERR_PTR(-EPERM);
+
+	ret = realinode->i_op->follow_link(realdentry, nd);
+	if (IS_ERR(ret))
+		return ret;
+
+	if (realinode->i_op->put_link) {
+		struct ovl_link_data *data;
+
+		data = kmalloc(sizeof(struct ovl_link_data), GFP_KERNEL);
+		if (!data) {
+			realinode->i_op->put_link(realdentry, nd, ret);
+			return ERR_PTR(-ENOMEM);
+		}
+		data->realdentry = realdentry;
+		data->cookie = ret;
+
+		return data;
+	} else {
+		return NULL;
+	}
+}
+
+static void ovl_put_link(struct dentry *dentry, struct nameidata *nd, void *c)
+{
+	struct inode *realinode;
+	struct ovl_link_data *data = c;
+
+	if (!data)
+		return;
+
+	realinode = data->realdentry->d_inode;
+	realinode->i_op->put_link(data->realdentry, nd, data->cookie);
+	kfree(data);
+}
+
+static int ovl_readlink(struct dentry *dentry, char __user *buf, int bufsiz)
+{
+	struct path realpath;
+	struct inode *realinode;
+
+	ovl_path_real(dentry, &realpath);
+	realinode = realpath.dentry->d_inode;
+
+	if (!realinode->i_op->readlink)
+		return -EINVAL;
+
+	touch_atime(realpath.mnt, realpath.dentry);
+
+	return realinode->i_op->readlink(realpath.dentry, buf, bufsiz);
+}
+
+
+static bool ovl_is_private_xattr(const char *name)
+{
+	return strncmp(name, "trusted.overlay.", 14) == 0;
+}
+
+int ovl_setxattr(struct dentry *dentry, const char *name,
+		 const void *value, size_t size, int flags)
+{
+	int err;
+	struct dentry *upperdentry;
+
+	if (ovl_is_private_xattr(name))
+		return -EPERM;
+
+	err = ovl_copy_up(dentry);
+	if (err)
+		return err;
+
+	upperdentry = ovl_dentry_upper(dentry);
+	return  vfs_setxattr(upperdentry, name, value, size, flags);
+}
+
+ssize_t ovl_getxattr(struct dentry *dentry, const char *name,
+		     void *value, size_t size)
+{
+	if (ovl_path_type(dentry->d_parent) == OVL_PATH_MERGE &&
+	    ovl_is_private_xattr(name))
+		return -ENODATA;
+
+	return vfs_getxattr(ovl_dentry_real(dentry), name, value, size);
+}
+
+ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size)
+{
+	ssize_t res;
+	int off;
+
+	res = vfs_listxattr(ovl_dentry_real(dentry), list, size);
+	if (res <= 0 || size == 0)
+		return res;
+
+	if (ovl_path_type(dentry->d_parent) != OVL_PATH_MERGE)
+		return res;
+
+	/* filter out private xattrs */
+	for (off = 0; off < res;) {
+		char *s = list + off;
+		size_t slen = strlen(s) + 1;
+
+		BUG_ON(off + slen > res);
+
+		if (ovl_is_private_xattr(s)) {
+			res -= slen;
+			memmove(s, s + slen, res - off);
+		} else {
+			off += slen;
+		}
+	}
+
+	return res;
+}
+
+int ovl_removexattr(struct dentry *dentry, const char *name)
+{
+	int err;
+	struct path realpath;
+	enum ovl_path_type type;
+
+	if (ovl_path_type(dentry->d_parent) == OVL_PATH_MERGE &&
+	    ovl_is_private_xattr(name))
+		return -ENODATA;
+
+	type = ovl_path_real(dentry, &realpath);
+	if (type == OVL_PATH_LOWER) {
+		err = vfs_getxattr(realpath.dentry, name, NULL, 0);
+		if (err < 0)
+			return err;
+
+		err = ovl_copy_up(dentry);
+		if (err)
+			return err;
+
+		ovl_path_upper(dentry, &realpath);
+	}
+
+	return vfs_removexattr(realpath.dentry, name);
+}
+
+static bool ovl_open_need_copy_up(int flags, enum ovl_path_type type,
+				  struct dentry *realdentry)
+{
+	if (type != OVL_PATH_LOWER)
+		return false;
+
+	if (special_file(realdentry->d_inode->i_mode))
+		return false;
+
+	if (!(OPEN_FMODE(flags) & FMODE_WRITE) && !(flags & O_TRUNC))
+		return false;
+
+	return true;
+}
+
+static struct file *ovl_open(struct dentry *dentry, int flags,
+			     const struct cred *cred)
+{
+	int err;
+	struct path realpath;
+	enum ovl_path_type type;
+
+	type = ovl_path_real(dentry, &realpath);
+	if (ovl_open_need_copy_up(flags, type, realpath.dentry)) {
+		if (flags & O_TRUNC)
+			err = ovl_copy_up_truncate(dentry, 0);
+		else
+			err = ovl_copy_up(dentry);
+		if (err)
+			return ERR_PTR(err);
+
+		ovl_path_upper(dentry, &realpath);
+	}
+
+	return vfs_open(&realpath, flags, cred);
+}
+
+static const struct inode_operations ovl_file_inode_operations = {
+	.setattr	= ovl_setattr,
+	.permission	= ovl_permission,
+	.getattr	= ovl_getattr,
+	.setxattr	= ovl_setxattr,
+	.getxattr	= ovl_getxattr,
+	.listxattr	= ovl_listxattr,
+	.removexattr	= ovl_removexattr,
+	.open		= ovl_open,
+};
+
+static const struct inode_operations ovl_symlink_inode_operations = {
+	.setattr	= ovl_setattr,
+	.follow_link	= ovl_follow_link,
+	.put_link	= ovl_put_link,
+	.readlink	= ovl_readlink,
+	.getattr	= ovl_getattr,
+	.setxattr	= ovl_setxattr,
+	.getxattr	= ovl_getxattr,
+	.listxattr	= ovl_listxattr,
+	.removexattr	= ovl_removexattr,
+};
+
+struct inode *ovl_new_inode(struct super_block *sb, umode_t mode,
+			    struct ovl_entry *oe)
+{
+	struct inode *inode;
+
+	inode = new_inode(sb);
+	if (!inode)
+		return NULL;
+
+	mode &= S_IFMT;
+
+	inode->i_ino = get_next_ino();
+	inode->i_mode = mode;
+	inode->i_flags |= S_NOATIME | S_NOCMTIME;
+
+	switch (mode) {
+	case S_IFDIR:
+		inode->i_private = oe;
+		inode->i_op = &ovl_dir_inode_operations;
+		inode->i_fop = &ovl_dir_operations;
+		break;
+
+	case S_IFLNK:
+		inode->i_op = &ovl_symlink_inode_operations;
+		break;
+
+	case S_IFREG:
+	case S_IFSOCK:
+	case S_IFBLK:
+	case S_IFCHR:
+	case S_IFIFO:
+		inode->i_op = &ovl_file_inode_operations;
+		break;
+
+	default:
+		WARN(1, "illegal file type: %i\n", mode);
+		inode = NULL;
+	}
+
+	return inode;
+
+}
diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h
new file mode 100644
index 0000000..10c4c36
--- /dev/null
+++ b/fs/overlayfs/overlayfs.h
@@ -0,0 +1,62 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+struct ovl_entry;
+
+enum ovl_path_type {
+	OVL_PATH_UPPER,
+	OVL_PATH_MERGE,
+	OVL_PATH_LOWER,
+};
+
+extern const char *ovl_opaque_xattr;
+extern const char *ovl_whiteout_xattr;
+extern const struct dentry_operations ovl_dentry_operations;
+
+enum ovl_path_type ovl_path_type(struct dentry *dentry);
+u64 ovl_dentry_version_get(struct dentry *dentry);
+void ovl_dentry_version_inc(struct dentry *dentry);
+void ovl_path_upper(struct dentry *dentry, struct path *path);
+void ovl_path_lower(struct dentry *dentry, struct path *path);
+enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path);
+struct dentry *ovl_dentry_upper(struct dentry *dentry);
+struct dentry *ovl_dentry_lower(struct dentry *dentry);
+struct dentry *ovl_dentry_real(struct dentry *dentry);
+struct dentry *ovl_entry_real(struct ovl_entry *oe, bool *is_upper);
+bool ovl_dentry_is_opaque(struct dentry *dentry);
+void ovl_dentry_set_opaque(struct dentry *dentry, bool opaque);
+bool ovl_is_whiteout(struct dentry *dentry);
+void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry);
+int ovl_do_lookup(struct dentry *dentry);
+
+struct dentry *ovl_upper_create(struct dentry *upperdir, struct dentry *dentry,
+				struct kstat *stat, const char *link);
+
+/* readdir.c */
+extern const struct file_operations ovl_dir_operations;
+int ovl_check_empty_and_clear(struct dentry *dentry, enum ovl_path_type type);
+
+/* inode.c */
+int ovl_setattr(struct dentry *dentry, struct iattr *attr);
+int ovl_permission(struct inode *inode, int mask, unsigned int flags);
+int ovl_setxattr(struct dentry *dentry, const char *name,
+		 const void *value, size_t size, int flags);
+ssize_t ovl_getxattr(struct dentry *dentry, const char *name,
+		     void *value, size_t size);
+ssize_t ovl_listxattr(struct dentry *dentry, char *list, size_t size);
+int ovl_removexattr(struct dentry *dentry, const char *name);
+
+struct inode *ovl_new_inode(struct super_block *sb, umode_t mode,
+			    struct ovl_entry *oe);
+/* dir.c */
+extern const struct inode_operations ovl_dir_inode_operations;
+
+/* copy_up.c */
+int ovl_copy_up(struct dentry *dentry);
+int ovl_copy_up_truncate(struct dentry *dentry, loff_t size);
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c
new file mode 100644
index 0000000..9b8004cf
--- /dev/null
+++ b/fs/overlayfs/readdir.c
@@ -0,0 +1,560 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/namei.h>
+#include <linux/file.h>
+#include <linux/xattr.h>
+#include <linux/rbtree.h>
+#include <linux/security.h>
+#include "overlayfs.h"
+
+struct ovl_cache_entry {
+	const char *name;
+	unsigned int len;
+	unsigned int type;
+	u64 ino;
+	bool is_whiteout;
+	struct list_head l_node;
+	struct rb_node node;
+};
+
+struct ovl_readdir_data {
+	struct rb_root *root;
+	struct list_head *list;
+	struct list_head *middle;
+	struct dentry *dir;
+	int count;
+	int err;
+};
+
+struct ovl_dir_file {
+	bool is_real;
+	bool is_cached;
+	struct list_head cursor;
+	u64 cache_version;
+	struct list_head cache;
+	struct file *realfile;
+};
+
+static struct ovl_cache_entry *ovl_cache_entry_from_node(struct rb_node *n)
+{
+	return container_of(n, struct ovl_cache_entry, node);
+}
+
+static struct ovl_cache_entry *ovl_cache_entry_find(struct rb_root *root,
+						    const char *name, int len)
+{
+	struct rb_node *node = root->rb_node;
+	int cmp;
+
+	while (node) {
+		struct ovl_cache_entry *p = ovl_cache_entry_from_node(node);
+
+		cmp = strncmp(name, p->name, len);
+		if (cmp > 0)
+			node = p->node.rb_right;
+		else if (cmp < 0 || len < p->len)
+			node = p->node.rb_left;
+		else
+			return p;
+	}
+
+	return NULL;
+}
+
+static struct ovl_cache_entry *ovl_cache_entry_new(const char *name, int len,
+						   u64 ino, unsigned int d_type)
+{
+	struct ovl_cache_entry *p;
+
+	p = kmalloc(sizeof(*p) + len + 1, GFP_KERNEL);
+	if (p) {
+		char *name_copy = (char *) (p + 1);
+		memcpy(name_copy, name, len);
+		name_copy[len] = '\0';
+		p->name = name_copy;
+		p->len = len;
+		p->type = d_type;
+		p->ino = ino;
+		p->is_whiteout = false;
+	}
+
+	return p;
+}
+
+static int ovl_cache_entry_add_rb(struct ovl_readdir_data *rdd,
+				  const char *name, int len, u64 ino,
+				  unsigned int d_type)
+{
+	struct rb_node **newp = &rdd->root->rb_node;
+	struct rb_node *parent = NULL;
+	struct ovl_cache_entry *p;
+
+	while (*newp) {
+		int cmp;
+		struct ovl_cache_entry *tmp;
+
+		parent = *newp;
+		tmp = ovl_cache_entry_from_node(*newp);
+		cmp = strncmp(name, tmp->name, len);
+		if (cmp > 0)
+			newp = &tmp->node.rb_right;
+		else if (cmp < 0 || len < tmp->len)
+			newp = &tmp->node.rb_left;
+		else
+			return 0;
+	}
+
+	p = ovl_cache_entry_new(name, len, ino, d_type);
+	if (p == NULL)
+		return -ENOMEM;
+
+	list_add_tail(&p->l_node, rdd->list);
+	rb_link_node(&p->node, parent, newp);
+	rb_insert_color(&p->node, rdd->root);
+
+	return 0;
+}
+
+static int ovl_fill_lower(void *buf, const char *name, int namelen,
+			    loff_t offset, u64 ino, unsigned int d_type)
+{
+	struct ovl_readdir_data *rdd = buf;
+	struct ovl_cache_entry *p;
+
+	rdd->count++;
+	p = ovl_cache_entry_find(rdd->root, name, namelen);
+	if (p) {
+		list_move_tail(&p->l_node, rdd->middle);
+	} else {
+		p = ovl_cache_entry_new(name, namelen, ino, d_type);
+		if (p == NULL)
+			rdd->err = -ENOMEM;
+		else
+			list_add_tail(&p->l_node, rdd->middle);
+	}
+
+	return rdd->err;
+}
+
+static void ovl_cache_free(struct list_head *list)
+{
+	struct ovl_cache_entry *p;
+	struct ovl_cache_entry *n;
+
+	list_for_each_entry_safe(p, n, list, l_node)
+		kfree(p);
+
+	INIT_LIST_HEAD(list);
+}
+
+static int ovl_fill_upper(void *buf, const char *name, int namelen,
+			  loff_t offset, u64 ino, unsigned int d_type)
+{
+	struct ovl_readdir_data *rdd = buf;
+
+	rdd->count++;
+	return ovl_cache_entry_add_rb(rdd, name, namelen, ino, d_type);
+}
+
+static int ovl_dir_read(struct path *realpath, struct ovl_readdir_data *rdd,
+			  filldir_t filler)
+{
+	struct file *realfile;
+	int err;
+
+	realfile = vfs_open(realpath, O_RDONLY | O_DIRECTORY, current_cred());
+	if (IS_ERR(realfile))
+		return PTR_ERR(realfile);
+
+	do {
+		rdd->count = 0;
+		rdd->err = 0;
+		err = vfs_readdir(realfile, filler, rdd);
+		if (err >= 0)
+			err = rdd->err;
+	} while (!err && rdd->count);
+	fput(realfile);
+
+	return 0;
+}
+
+static void ovl_dir_reset(struct file *file)
+{
+	struct ovl_dir_file *od = file->private_data;
+	enum ovl_path_type type = ovl_path_type(file->f_path.dentry);
+
+	if (ovl_dentry_version_get(file->f_path.dentry) != od->cache_version) {
+		list_del_init(&od->cursor);
+		ovl_cache_free(&od->cache);
+		od->is_cached = false;
+	}
+	WARN_ON(!od->is_real && type != OVL_PATH_MERGE);
+	if (od->is_real && type == OVL_PATH_MERGE) {
+		fput(od->realfile);
+		od->realfile = NULL;
+		od->is_real = false;
+	}
+}
+
+static int ovl_dir_mark_whiteouts(struct ovl_readdir_data *rdd)
+{
+	struct ovl_cache_entry *p;
+	struct dentry *dentry;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+
+	override_cred = prepare_creds();
+	if (!override_cred) {
+		ovl_cache_free(rdd->list);
+		return -ENOMEM;
+	}
+
+	/*
+	 * CAP_SYS_ADMIN for getxattr
+	 * CAP_DAC_OVERRIDE for lookup
+	 */
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	cap_raise(override_cred->cap_effective, CAP_DAC_OVERRIDE);
+	old_cred = override_creds(override_cred);
+
+	mutex_lock(&rdd->dir->d_inode->i_mutex);
+	list_for_each_entry(p, rdd->list, l_node) {
+		if (p->type != DT_LNK)
+			continue;
+
+		dentry = lookup_one_len(p->name, rdd->dir, p->len);
+		if (IS_ERR(dentry))
+			continue;
+
+		p->is_whiteout = ovl_is_whiteout(dentry);
+		dput(dentry);
+	}
+	mutex_unlock(&rdd->dir->d_inode->i_mutex);
+
+	revert_creds(old_cred);
+	put_cred(override_cred);
+
+	return 0;
+}
+
+static int ovl_dir_read_merged(struct path *upperpath, struct path *lowerpath,
+			       struct ovl_readdir_data *rdd)
+{
+	int err;
+	struct rb_root root = RB_ROOT;
+	struct list_head middle;
+
+	rdd->root = &root;
+	if (upperpath->dentry) {
+		rdd->dir = upperpath->dentry;
+		err = ovl_dir_read(upperpath, rdd, ovl_fill_upper);
+		if (err)
+			goto out;
+
+		err = ovl_dir_mark_whiteouts(rdd);
+		if (err)
+			goto out;
+	}
+	/*
+	 * Insert lowerpath entries before upperpath ones, this allows
+	 * offsets to be reasonably constant
+	 */
+	list_add(&middle, rdd->list);
+	rdd->middle = &middle;
+	err = ovl_dir_read(lowerpath, rdd, ovl_fill_lower);
+	list_del(&middle);
+out:
+	rdd->root = NULL;
+
+	return err;
+}
+
+static void ovl_seek_cursor(struct ovl_dir_file *od, loff_t pos)
+{
+	struct list_head *l;
+	loff_t off;
+
+	l = od->cache.next;
+	for (off = 0; off < pos; off++) {
+		if (l == &od->cache)
+			break;
+		l = l->next;
+	}
+	list_move_tail(&od->cursor, l);
+}
+
+static int ovl_readdir(struct file *file, void *buf, filldir_t filler)
+{
+	struct ovl_dir_file *od = file->private_data;
+	int res;
+
+	if (!file->f_pos)
+		ovl_dir_reset(file);
+
+	if (od->is_real) {
+		res = vfs_readdir(od->realfile, filler, buf);
+		file->f_pos = od->realfile->f_pos;
+
+		return res;
+	}
+
+	if (!od->is_cached) {
+		struct path lowerpath;
+		struct path upperpath;
+		struct ovl_readdir_data rdd = { .list = &od->cache };
+
+		ovl_path_lower(file->f_path.dentry, &lowerpath);
+		ovl_path_upper(file->f_path.dentry, &upperpath);
+
+		res = ovl_dir_read_merged(&upperpath, &lowerpath, &rdd);
+		if (res) {
+			ovl_cache_free(rdd.list);
+			return res;
+		}
+
+		od->cache_version = ovl_dentry_version_get(file->f_path.dentry);
+		od->is_cached = true;
+
+		ovl_seek_cursor(od, file->f_pos);
+	}
+
+	while (od->cursor.next != &od->cache) {
+		int over;
+		loff_t off;
+		struct ovl_cache_entry *p;
+
+		p = list_entry(od->cursor.next, struct ovl_cache_entry, l_node);
+		off = file->f_pos;
+		file->f_pos++;
+		list_move(&od->cursor, &p->l_node);
+
+		if (p->is_whiteout)
+			continue;
+
+		over = filler(buf, p->name, p->len, off, p->ino, p->type);
+		if (over)
+			break;
+	}
+
+	return 0;
+}
+
+static loff_t ovl_dir_llseek(struct file *file, loff_t offset, int origin)
+{
+	loff_t res;
+	struct ovl_dir_file *od = file->private_data;
+
+	mutex_lock(&file->f_dentry->d_inode->i_mutex);
+	if (!file->f_pos)
+		ovl_dir_reset(file);
+
+	if (od->is_real) {
+		res = vfs_llseek(od->realfile, offset, origin);
+		file->f_pos = od->realfile->f_pos;
+	} else {
+		res = -EINVAL;
+
+		switch (origin) {
+		case SEEK_CUR:
+			offset += file->f_pos;
+			break;
+		case SEEK_SET:
+			break;
+		default:
+			goto out_unlock;
+		}
+		if (offset < 0)
+			goto out_unlock;
+
+		if (offset != file->f_pos) {
+			file->f_pos = offset;
+			if (od->is_cached)
+				ovl_seek_cursor(od, offset);
+		}
+		res = offset;
+	}
+out_unlock:
+	mutex_unlock(&file->f_dentry->d_inode->i_mutex);
+
+	return res;
+}
+
+static int ovl_dir_fsync(struct file *file, int datasync)
+{
+	struct ovl_dir_file *od = file->private_data;
+
+	/* May need to reopen directory if it got copied up */
+	if (!od->realfile) {
+		struct path upperpath;
+
+		ovl_path_upper(file->f_path.dentry, &upperpath);
+		od->realfile = vfs_open(&upperpath, O_RDONLY, current_cred());
+		if (IS_ERR(od->realfile))
+			return PTR_ERR(od->realfile);
+	}
+
+	return vfs_fsync(od->realfile, datasync);
+}
+
+static int ovl_dir_release(struct inode *inode, struct file *file)
+{
+	struct ovl_dir_file *od = file->private_data;
+
+	list_del(&od->cursor);
+	ovl_cache_free(&od->cache);
+	if (od->realfile)
+		fput(od->realfile);
+	kfree(od);
+
+	return 0;
+}
+
+static int ovl_dir_open(struct inode *inode, struct file *file)
+{
+	struct path realpath;
+	struct file *realfile;
+	struct ovl_dir_file *od;
+	enum ovl_path_type type;
+
+	od = kzalloc(sizeof(struct ovl_dir_file), GFP_KERNEL);
+	if (!od)
+		return -ENOMEM;
+
+	type = ovl_path_real(file->f_path.dentry, &realpath);
+	realfile = vfs_open(&realpath, file->f_flags, current_cred());
+	if (IS_ERR(realfile)) {
+		kfree(od);
+		return PTR_ERR(realfile);
+	}
+	INIT_LIST_HEAD(&od->cache);
+	INIT_LIST_HEAD(&od->cursor);
+	od->is_cached = false;
+	od->realfile = realfile;
+	od->is_real = (type != OVL_PATH_MERGE);
+	file->private_data = od;
+
+	return 0;
+}
+
+const struct file_operations ovl_dir_operations = {
+	.read		= generic_read_dir,
+	.open		= ovl_dir_open,
+	.readdir	= ovl_readdir,
+	.llseek		= ovl_dir_llseek,
+	.fsync		= ovl_dir_fsync,
+	.release	= ovl_dir_release,
+};
+
+static int ovl_check_empty_dir(struct dentry *dentry, struct list_head *list)
+{
+	int err;
+	struct path lowerpath;
+	struct path upperpath;
+	struct ovl_cache_entry *p;
+	struct ovl_readdir_data rdd = { .list = list };
+
+	ovl_path_upper(dentry, &upperpath);
+	ovl_path_lower(dentry, &lowerpath);
+
+	err = ovl_dir_read_merged(&upperpath, &lowerpath, &rdd);
+	if (err)
+		return err;
+
+	err = 0;
+
+	list_for_each_entry(p, list, l_node) {
+		if (p->is_whiteout)
+			continue;
+
+		if (p->name[0] == '.') {
+			if (p->len == 1)
+				continue;
+			if (p->len == 2 && p->name[1] == '.')
+				continue;
+		}
+		err = -ENOTEMPTY;
+		break;
+	}
+
+	return err;
+}
+
+static int ovl_remove_whiteouts(struct dentry *dir, struct list_head *list)
+{
+	struct path upperpath;
+	struct dentry *upperdir;
+	struct ovl_cache_entry *p;
+	const struct cred *old_cred;
+	struct cred *override_cred;
+	int err;
+
+	ovl_path_upper(dir, &upperpath);
+	upperdir = upperpath.dentry;
+
+	override_cred = prepare_creds();
+	if (!override_cred)
+		return -ENOMEM;
+
+	/*
+	 * CAP_DAC_OVERRIDE for lookup and unlink
+	 * CAP_SYS_ADMIN for setxattr of "trusted" namespace
+	 * CAP_FOWNER for unlink in sticky directory
+	 */
+	cap_raise(override_cred->cap_effective, CAP_DAC_OVERRIDE);
+	cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+	cap_raise(override_cred->cap_effective, CAP_FOWNER);
+	old_cred = override_creds(override_cred);
+
+	err = vfs_setxattr(upperdir, ovl_opaque_xattr, "y", 1, 0);
+	if (err)
+		goto out_revert_creds;
+
+	mutex_lock_nested(&upperdir->d_inode->i_mutex, I_MUTEX_PARENT);
+	list_for_each_entry(p, list, l_node) {
+		struct dentry *dentry;
+		int ret;
+
+		if (!p->is_whiteout)
+			continue;
+
+		dentry = lookup_one_len(p->name, upperdir, p->len);
+		if (IS_ERR(dentry)) {
+			printk(KERN_WARNING "overlayfs: failed to lookup whiteout %.*s: %li\n", p->len, p->name, PTR_ERR(dentry));
+			continue;
+		}
+		ret = vfs_unlink(upperdir->d_inode, dentry);
+		dput(dentry);
+		if (ret)
+			printk(KERN_WARNING "overlayfs: failed to unlink whiteout %.*s: %i\n", p->len, p->name, ret);
+	}
+	mutex_unlock(&upperdir->d_inode->i_mutex);
+
+out_revert_creds:
+	revert_creds(old_cred);
+	put_cred(override_cred);
+
+	return err;
+}
+
+int ovl_check_empty_and_clear(struct dentry *dentry, enum ovl_path_type type)
+{
+	int err;
+	LIST_HEAD(list);
+
+	err = ovl_check_empty_dir(dentry, &list);
+	if (!err && type == OVL_PATH_MERGE)
+		err = ovl_remove_whiteouts(dentry, &list);
+
+	ovl_cache_free(&list);
+
+	return err;
+}
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
new file mode 100644
index 0000000..0901eda
--- /dev/null
+++ b/fs/overlayfs/super.c
@@ -0,0 +1,576 @@
+/*
+ *
+ * Copyright (C) 2011 Novell Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ */
+
+#include <linux/fs.h>
+#include <linux/namei.h>
+#include <linux/xattr.h>
+#include <linux/security.h>
+#include <linux/mount.h>
+#include <linux/slab.h>
+#include <linux/parser.h>
+#include <linux/module.h>
+#include "overlayfs.h"
+
+MODULE_AUTHOR("Miklos Szeredi <miklos@szeredi.hu>");
+MODULE_DESCRIPTION("Overlay filesystem");
+MODULE_LICENSE("GPL");
+
+struct ovl_fs {
+	struct vfsmount *upper_mnt;
+	struct vfsmount *lower_mnt;
+};
+
+struct ovl_entry {
+	struct dentry *__upperdentry;
+	struct dentry *lowerdentry;
+	union {
+		struct {
+			u64 version;
+			bool opaque;
+		};
+		struct rcu_head rcu;
+	};
+};
+
+const char *ovl_whiteout_xattr = "trusted.overlay.whiteout";
+const char *ovl_opaque_xattr = "trusted.overlay.opaque";
+
+
+enum ovl_path_type ovl_path_type(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	if (oe->__upperdentry) {
+		if (oe->lowerdentry && S_ISDIR(dentry->d_inode->i_mode))
+			return OVL_PATH_MERGE;
+		else
+			return OVL_PATH_UPPER;
+	} else {
+		return OVL_PATH_LOWER;
+	}
+}
+
+static struct dentry *ovl_upperdentry_dereference(struct ovl_entry *oe)
+{
+	struct dentry *upperdentry = ACCESS_ONCE(oe->__upperdentry);
+	smp_read_barrier_depends();
+	return upperdentry;
+}
+
+void ovl_path_upper(struct dentry *dentry, struct path *path)
+{
+	struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	path->mnt = ofs->upper_mnt;
+	path->dentry = ovl_upperdentry_dereference(oe);
+}
+
+void ovl_path_lower(struct dentry *dentry, struct path *path)
+{
+	struct ovl_fs *ofs = dentry->d_sb->s_fs_info;
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	path->mnt = ofs->lower_mnt;
+	path->dentry = oe->lowerdentry;
+}
+
+enum ovl_path_type ovl_path_real(struct dentry *dentry, struct path *path)
+{
+
+	enum ovl_path_type type = ovl_path_type(dentry);
+
+	if (type == OVL_PATH_LOWER)
+		ovl_path_lower(dentry, path);
+	else
+		ovl_path_upper(dentry, path);
+
+	return type;
+}
+
+struct dentry *ovl_dentry_upper(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	return ovl_upperdentry_dereference(oe);
+}
+
+struct dentry *ovl_dentry_lower(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	return oe->lowerdentry;
+}
+
+struct dentry *ovl_dentry_real(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+	struct dentry *realdentry;
+
+	realdentry = ovl_upperdentry_dereference(oe);
+	if (!realdentry)
+		realdentry = oe->lowerdentry;
+
+	return realdentry;
+}
+
+struct dentry *ovl_entry_real(struct ovl_entry *oe, bool *is_upper)
+{
+	struct dentry *realdentry;
+
+	realdentry = ovl_upperdentry_dereference(oe);
+	if (realdentry) {
+		*is_upper = true;
+	} else {
+		realdentry = oe->lowerdentry;
+		*is_upper = false;
+	}
+	return realdentry;
+}
+
+bool ovl_dentry_is_opaque(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+	return oe->opaque;
+}
+
+void ovl_dentry_set_opaque(struct dentry *dentry, bool opaque)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+	oe->opaque = opaque;
+}
+
+void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	WARN_ON(!mutex_is_locked(&upperdentry->d_parent->d_inode->i_mutex));
+	WARN_ON(oe->__upperdentry);
+	smp_wmb();
+	oe->__upperdentry = upperdentry;
+}
+
+void ovl_dentry_version_inc(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	WARN_ON(!mutex_is_locked(&dentry->d_inode->i_mutex));
+	oe->version++;
+}
+
+u64 ovl_dentry_version_get(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	WARN_ON(!mutex_is_locked(&dentry->d_inode->i_mutex));
+	return oe->version;
+}
+
+bool ovl_is_whiteout(struct dentry *dentry)
+{
+	int res;
+	char val;
+
+	if (!dentry)
+		return false;
+	if (!dentry->d_inode)
+		return false;
+	if (!S_ISLNK(dentry->d_inode->i_mode))
+		return false;
+
+	res = vfs_getxattr(dentry, ovl_whiteout_xattr, &val, 1);
+	if (res == 1 && val == 'y')
+		return true;
+
+	return false;
+}
+
+static bool ovl_is_opaquedir(struct dentry *dentry)
+{
+	int res;
+	char val;
+
+	if (!S_ISDIR(dentry->d_inode->i_mode))
+		return false;
+
+	res = vfs_getxattr(dentry, ovl_opaque_xattr, &val, 1);
+	if (res == 1 && val == 'y')
+		return true;
+
+	return false;
+}
+
+static void ovl_entry_free(struct rcu_head *head)
+{
+	struct ovl_entry *oe = container_of(head, struct ovl_entry, rcu);
+	kfree(oe);
+}
+
+static void ovl_dentry_release(struct dentry *dentry)
+{
+	struct ovl_entry *oe = dentry->d_fsdata;
+
+	if (oe) {
+		dput(oe->__upperdentry);
+		dput(oe->lowerdentry);
+		call_rcu(&oe->rcu, ovl_entry_free);
+	}
+}
+
+const struct dentry_operations ovl_dentry_operations = {
+	.d_release = ovl_dentry_release,
+};
+
+static struct ovl_entry *ovl_alloc_entry(void)
+{
+	return kzalloc(sizeof(struct ovl_entry), GFP_KERNEL);
+}
+
+static struct dentry *ovl_lookup_real(struct dentry *dir, struct qstr *name)
+{
+	struct dentry *dentry;
+
+	mutex_lock(&dir->d_inode->i_mutex);
+	dentry = lookup_one_len(name->name, dir, name->len);
+	mutex_unlock(&dir->d_inode->i_mutex);
+
+	if (IS_ERR(dentry)) {
+		if (PTR_ERR(dentry) == -ENOENT)
+			dentry = NULL;
+	} else if (!dentry->d_inode) {
+		dput(dentry);
+		dentry = NULL;
+	}
+	return dentry;
+}
+
+int ovl_do_lookup(struct dentry *dentry)
+{
+	struct ovl_entry *oe;
+	struct dentry *upperdir;
+	struct dentry *lowerdir;
+	struct dentry *upperdentry = NULL;
+	struct dentry *lowerdentry = NULL;
+	struct inode *inode = NULL;
+	int err;
+
+	err = -ENOMEM;
+	oe = ovl_alloc_entry();
+	if (!oe)
+		goto out;
+
+	upperdir = ovl_dentry_upper(dentry->d_parent);
+	lowerdir = ovl_dentry_lower(dentry->d_parent);
+
+	if (upperdir) {
+		upperdentry = ovl_lookup_real(upperdir, &dentry->d_name);
+		err = PTR_ERR(upperdentry);
+		if (IS_ERR(upperdentry))
+			goto out_put_dir;
+
+		if (lowerdir && upperdentry &&
+		    (S_ISLNK(upperdentry->d_inode->i_mode) ||
+		     S_ISDIR(upperdentry->d_inode->i_mode))) {
+			const struct cred *old_cred;
+			struct cred *override_cred;
+
+			err = -ENOMEM;
+			override_cred = prepare_creds();
+			if (!override_cred)
+				goto out_dput_upper;
+
+			/* CAP_SYS_ADMIN needed for getxattr */
+			cap_raise(override_cred->cap_effective, CAP_SYS_ADMIN);
+			old_cred = override_creds(override_cred);
+
+			if (ovl_is_opaquedir(upperdentry)) {
+				oe->opaque = true;
+			} else if (ovl_is_whiteout(upperdentry)) {
+				dput(upperdentry);
+				upperdentry = NULL;
+				oe->opaque = true;
+			}
+			revert_creds(old_cred);
+			put_cred(override_cred);
+		}
+	}
+	if (lowerdir && !oe->opaque) {
+		lowerdentry = ovl_lookup_real(lowerdir, &dentry->d_name);
+		err = PTR_ERR(lowerdentry);
+		if (IS_ERR(lowerdentry))
+			goto out_dput_upper;
+	}
+
+	if (lowerdentry && upperdentry &&
+	    (!S_ISDIR(upperdentry->d_inode->i_mode) ||
+	     !S_ISDIR(lowerdentry->d_inode->i_mode))) {
+		dput(lowerdentry);
+		lowerdentry = NULL;
+		oe->opaque = true;
+	}
+
+	if (lowerdentry || upperdentry) {
+		struct dentry *realdentry;
+
+		realdentry = upperdentry ? upperdentry : lowerdentry;
+		err = -ENOMEM;
+		inode = ovl_new_inode(dentry->d_sb, realdentry->d_inode->i_mode, oe);
+		if (!inode)
+			goto out_dput;
+	}
+
+	if (upperdentry)
+		oe->__upperdentry = upperdentry;
+
+	if (lowerdentry)
+		oe->lowerdentry = lowerdentry;
+
+	dentry->d_fsdata = oe;
+	dentry->d_op = &ovl_dentry_operations;
+	d_add(dentry, inode);
+
+	return 0;
+
+out_dput:
+	dput(lowerdentry);
+out_dput_upper:
+	dput(upperdentry);
+out_put_dir:
+	kfree(oe);
+out:
+	return err;
+}
+
+static void ovl_put_super(struct super_block *sb)
+{
+	struct ovl_fs *ufs = sb->s_fs_info;
+
+	if (!(sb->s_flags & MS_RDONLY))
+		mnt_drop_write(ufs->upper_mnt);
+
+	mntput(ufs->upper_mnt);
+	mntput(ufs->lower_mnt);
+
+	kfree(ufs);
+}
+
+static int ovl_remount_fs(struct super_block *sb, int *flagsp, char *data)
+{
+	int flags = *flagsp;
+	struct ovl_fs *ufs = sb->s_fs_info;
+
+	/* When remounting rw or ro, we need to adjust the write access to the
+	 * upper fs.
+	 */
+	if (((flags ^ sb->s_flags) & MS_RDONLY) == 0)
+		/* No change to readonly status */
+		return 0;
+
+	if (flags & MS_RDONLY) {
+		mnt_drop_write(ufs->upper_mnt);
+		return 0;
+	} else
+		return mnt_want_write(ufs->upper_mnt);
+}
+
+static const struct super_operations ovl_super_operations = {
+	.put_super	= ovl_put_super,
+	.remount_fs	= ovl_remount_fs,
+};
+
+struct ovl_config {
+	char *lowerdir;
+	char *upperdir;
+};
+
+enum {
+	Opt_lowerdir,
+	Opt_upperdir,
+	Opt_err,
+};
+
+static const match_table_t ovl_tokens = {
+	{Opt_lowerdir,			"lowerdir=%s"},
+	{Opt_upperdir,			"upperdir=%s"},
+	{Opt_err,			NULL}
+};
+
+static int ovl_parse_opt(char *opt, struct ovl_config *config)
+{
+	char *p;
+
+	config->upperdir = NULL;
+	config->lowerdir = NULL;
+
+	while ((p = strsep(&opt, ",")) != NULL) {
+		int token;
+		substring_t args[MAX_OPT_ARGS];
+
+		if (!*p)
+			continue;
+
+		token = match_token(p, ovl_tokens, args);
+		switch (token) {
+		case Opt_upperdir:
+			kfree(config->upperdir);
+			config->upperdir = match_strdup(&args[0]);
+			if (!config->upperdir)
+				return -ENOMEM;
+			break;
+
+		case Opt_lowerdir:
+			kfree(config->lowerdir);
+			config->lowerdir = match_strdup(&args[0]);
+			if (!config->lowerdir)
+				return -ENOMEM;
+			break;
+
+		default:
+			return -EINVAL;
+		}
+	}
+	return 0;
+}
+
+static int ovl_fill_super(struct super_block *sb, void *data, int silent)
+{
+	struct path lowerpath;
+	struct path upperpath;
+	struct inode *root_inode;
+	struct dentry *root_dentry;
+	struct ovl_entry *oe;
+	struct ovl_fs *ufs;
+	struct ovl_config config;
+	int err;
+
+	err = ovl_parse_opt((char *) data, &config);
+	if (err)
+		goto out;
+
+	err = -EINVAL;
+	if (!config.upperdir || !config.lowerdir) {
+		printk(KERN_ERR "overlayfs: missing upperdir or lowerdir\n");
+		goto out_free_config;
+	}
+
+	err = -ENOMEM;
+	ufs = kmalloc(sizeof(struct ovl_fs), GFP_KERNEL);
+	if (!ufs)
+		goto out_free_config;
+
+	oe = ovl_alloc_entry();
+	if (oe == NULL)
+		goto out_free_ufs;
+
+	root_inode = ovl_new_inode(sb, S_IFDIR, oe);
+	if (!root_inode)
+		goto out_free_oe;
+
+	err = kern_path(config.upperdir, LOOKUP_FOLLOW, &upperpath);
+	if (err)
+		goto out_put_root;
+
+	err = kern_path(config.lowerdir, LOOKUP_FOLLOW, &lowerpath);
+	if (err)
+		goto out_put_upperpath;
+
+	err = -ENOTDIR;
+	if (!S_ISDIR(upperpath.dentry->d_inode->i_mode) ||
+	    !S_ISDIR(lowerpath.dentry->d_inode->i_mode))
+		goto out_put_lowerpath;
+
+	ufs->upper_mnt = clone_private_mount(&upperpath);
+	err = PTR_ERR(ufs->upper_mnt);
+	if (IS_ERR(ufs->upper_mnt)) {
+		printk(KERN_ERR "overlayfs: failed to clone upperpath\n");
+		goto out_put_lowerpath;
+	}
+
+	ufs->lower_mnt = clone_private_mount(&lowerpath);
+	err = PTR_ERR(ufs->lower_mnt);
+	if (IS_ERR(ufs->lower_mnt)) {
+		printk(KERN_ERR "overlayfs: failed to clone lowerpath\n");
+		goto out_put_upper_mnt;
+	}
+
+	if (!(sb->s_flags & MS_RDONLY)) {
+		err = mnt_want_write(ufs->upper_mnt);
+		if (err)
+			goto out_put_lower_mnt;
+	}
+
+	err = -ENOMEM;
+	root_dentry = d_alloc_root(root_inode);
+	if (!root_dentry)
+		goto out_drop_write;
+
+	mntput(upperpath.mnt);
+	mntput(lowerpath.mnt);
+
+	oe->__upperdentry = upperpath.dentry;
+	oe->lowerdentry = lowerpath.dentry;
+
+	root_dentry->d_fsdata = oe;
+	root_dentry->d_op = &ovl_dentry_operations;
+
+	sb->s_op = &ovl_super_operations;
+	sb->s_root = root_dentry;
+	sb->s_fs_info = ufs;
+
+	return 0;
+
+out_drop_write:
+	if (!(sb->s_flags & MS_RDONLY))
+		mnt_drop_write(ufs->upper_mnt);
+out_put_lower_mnt:
+	mntput(ufs->lower_mnt);
+out_put_upper_mnt:
+	mntput(ufs->upper_mnt);
+out_put_lowerpath:
+	path_put(&lowerpath);
+out_put_upperpath:
+	path_put(&upperpath);
+out_put_root:
+	iput(root_inode);
+out_free_oe:
+	kfree(oe);
+out_free_ufs:
+	kfree(ufs);
+out_free_config:
+	kfree(config.lowerdir);
+	kfree(config.upperdir);
+out:
+	return err;
+}
+
+static struct dentry *ovl_mount(struct file_system_type *fs_type, int flags,
+				const char *dev_name, void *raw_data)
+{
+	return mount_nodev(fs_type, flags, raw_data, ovl_fill_super);
+}
+
+static struct file_system_type ovl_fs_type = {
+	.owner		= THIS_MODULE,
+	.name		= "overlayfs",
+	.mount		= ovl_mount,
+	.kill_sb	= kill_anon_super,
+};
+
+static int __init ovl_init(void)
+{
+	return register_filesystem(&ovl_fs_type);
+}
+
+static void __exit ovl_exit(void)
+{
+	unregister_filesystem(&ovl_fs_type);
+}
+
+module_init(ovl_init);
+module_exit(ovl_exit);
-- 
1.7.3.4

[PATCH 6/7] overlayfs: add statfs support

[Miklos Szeredi]

From: Andy Whitcroft <apw@canonical.com>

Add support for statfs to the overlayfs filesystem.  As the upper layer
is the target of all write operations assume that the space in that
filesystem is the space in the overlayfs.  There will be some inaccuracy as
overwriting a file will copy it up and consume space we were not expecting,
but it is better than nothing.

Use the upper layer dentry and mount from the overlayfs root inode,
passing the statfs call to that filesystem.

Signed-off-by: Andy Whitcroft <apw@canonical.com>
Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 fs/overlayfs/super.c |   20 ++++++++++++++++++++
 1 files changed, 20 insertions(+), 0 deletions(-)

diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index 0901eda..a9a09a6 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -379,9 +379,29 @@ static int ovl_remount_fs(struct super_block *sb, int *flagsp, char *data)
 		return mnt_want_write(ufs->upper_mnt);
 }
 
+/**
+ * ovl_statfs
+ * @sb: The overlayfs super block
+ * @buf: The struct kstatfs to fill in with stats
+ *
+ * Get the filesystem statistics.  As writes always target the upper layer
+ * filesystem pass the statfs to the same filesystem.
+ */
+static int ovl_statfs(struct dentry *dentry, struct kstatfs *buf)
+{
+	struct dentry *root_dentry = dentry->d_sb->s_root;
+	struct path path;
+	ovl_path_upper(root_dentry, &path);
+
+	if (!path.dentry->d_sb->s_op->statfs)
+		return -ENOSYS;
+	return path.dentry->d_sb->s_op->statfs(path.dentry, buf);
+}
+
 static const struct super_operations ovl_super_operations = {
 	.put_super	= ovl_put_super,
 	.remount_fs	= ovl_remount_fs,
+	.statfs		= ovl_statfs,
 };
 
 struct ovl_config {
-- 
1.7.3.4

[PATCH 7/7] overlay: overlay filesystem documentation

[Miklos Szeredi]

From: Neil Brown <neilb@suse.de>

Document the overlay filesystem.

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
---
 Documentation/filesystems/overlayfs.txt |  167 +++++++++++++++++++++++++++++++
 MAINTAINERS                             |    7 ++
 2 files changed, 174 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/filesystems/overlayfs.txt

diff --git a/Documentation/filesystems/overlayfs.txt b/Documentation/filesystems/overlayfs.txt
new file mode 100644
index 0000000..4bc0b34
--- /dev/null
+++ b/Documentation/filesystems/overlayfs.txt
@@ -0,0 +1,167 @@
+Written by: Neil Brown <neilb@suse.de>
+
+Overlay Filesystem
+==================
+
+This document describes a prototype for a new approach to providing
+overlay-filesystem functionality in Linux (sometimes referred to as
+union-filesystems).  An overlay-filesystem tries to present a
+filesystem which is the result over overlaying one filesystem on top
+of the other.
+
+The result will inevitably fail to look exactly like a normal
+filesystem for various technical reasons.  The expectation is that
+many use cases will be able to ignore these differences.
+
+This approach is 'hybrid' because the objects that appear in the
+filesystem do not all appear to belong to that filesystem.  In many
+cases an object accessed in the union will be indistinguishable
+from accessing the corresponding object from the original filesystem.
+This is most obvious from the 'st_dev' field returned by stat(2).
+
+While directories will report an st_dev from the overlay-filesystem,
+all non-directory objects will report an st_dev from the lower or
+upper filesystem that is providing the object.  Similarly st_ino will
+only be unique when combined with st_dev, and both of these can change
+over the lifetime of a non-directory object.  Many applications and
+tools ignore these values and will not be affected.
+
+Upper and Lower
+---------------
+
+An overlay filesystem combines two filesystems - an 'upper' filesystem
+and a 'lower' filesystem.  When a name exists in both filesystems, the
+object in the 'upper' filesystem is visible while the object in the
+'lower' filesystem is either hidden or, in the case of directories,
+merged with the 'upper' object.
+
+It would be more correct to refer to an upper and lower 'directory
+tree' rather than 'filesystem' as it is quite possible for both
+directory trees to be in the same filesystem and there is no
+requirement that the root of a filesystem be given for either upper or
+lower.
+
+The lower filesystem can be any filesystem supported by Linux and does
+not need to be writable.  The lower filesystem can even be another
+overlayfs.  The upper filesystem will normally be writable and if it
+is it must support the creation of trusted.* extended attributes, and
+must provide valid d_type in readdir responses, at least for symbolic
+links - so NFS is not suitable.
+
+A read-only overlay of two read-only filesystems may use any
+filesystem type.
+
+Directories
+-----------
+
+Overlaying mainly involved directories.  If a given name appears in both
+upper and lower filesystems and refers to a non-directory in either,
+then the lower object is hidden - the name refers only to the upper
+object.
+
+Where both upper and lower objects are directories, a merged directory
+is formed.
+
+At mount time, the two directories given as mount options are combined
+into a merged directory:
+
+  mount -t overlayfs overlayfs -olowerdir=/lower,upperdir=/upper /overlay
+
+Then whenever a lookup is requested in such a merged directory, the
+lookup is performed in each actual directory and the combined result
+is cached in the dentry belonging to the overlay filesystem.  If both
+actual lookups find directories, both are stored and a merged
+directory is created, otherwise only one is stored: the upper if it
+exists, else the lower.
+
+Only the lists of names from directories are merged.  Other content
+such as metadata and extended attributes are reported for the upper
+directory only.  These attributes of the lower directory are hidden.
+
+whiteouts and opaque directories
+--------------------------------
+
+In order to support rm and rmdir without changing the lower
+filesystem, an overlay filesystem needs to record in the upper filesystem
+that files have been removed.  This is done using whiteouts and opaque
+directories (non-directories are always opaque).
+
+The overlay filesystem uses extended attributes with a
+"trusted.overlay."  prefix to record these details.
+
+A whiteout is created as a symbolic link with target
+"(overlay-whiteout)" and with xattr "trusted.overlay.whiteout" set to "y".
+When a whiteout is found in the upper level of a merged directory, any
+matching name in the lower level is ignored, and the whiteout itself
+is also hidden.
+
+A directory is made opaque by setting the xattr "trusted.overlay.opaque"
+to "y".  Where the upper filesystem contains an opaque directory, any
+directory in the lower filesystem with the same name is ignored.
+
+readdir
+-------
+
+When a 'readdir' request is made on a merged directory, the upper and
+lower directories are each read and the name lists merged in the
+obvious way (upper is read first, then lower - entries that already
+exist are not re-added).  This merged name list is cached in the
+'struct file' and so remains as long as the file is kept open.  If the
+directory is opened and read by two processes at the same time, they
+will each have separate caches.  A seekdir to the start of the
+directory (offset 0) followed by a readdir will cause the cache to be
+discarded and rebuilt.
+
+This means that changes to the merged directory do not appear while a
+directory is being read.  This is unlikely to be noticed by many
+programs.
+
+seek offsets are assigned sequentially when the directories are read.
+Thus if
+  - read part of a directory
+  - remember an offset, and close the directory
+  - re-open the directory some time later
+  - seek to the remembered offset
+
+there may be little correlation between the old and new locations in
+the list of filenames, particularly if anything has changed in the
+directory.
+
+Readdir on directories that are not merged is simply handled by the
+underlying directory (upper or lower).
+
+
+Non-directories
+---------------
+
+Objects that are not directories (files, symlinks, device-special
+files etc.) are presented either from the upper or lower filesystem as
+appropriate.  When a file in the lower filesystem is accessed in a way
+the requires write-access, such as opening for write access, changing
+some metadata etc., the file is first copied from the lower filesystem
+to the upper filesystem (copy_up).  Note that creating a hard-link
+also requires copy_up, though of course creation of a symlink does
+not.
+
+The copy_up process first makes sure that the containing directory
+exists in the upper filesystem - creating it and any parents as
+necessary.  It then creates the object with the same metadata (owner,
+mode, mtime, symlink-target etc.) and then if the object is a file, the
+data is copied from the lower to the upper filesystem.  Finally any
+extended attributes are copied up.
+
+Once the copy_up is complete, the overlay filesystem simply
+provides direct access to the newly created file in the upper
+filesystem - future operations on the file are barely noticed by the
+overlay filesystem (though an operation on the name of the file such as
+rename or unlink will of course be noticed and handled).
+
+Changes to underlying filesystems
+---------------------------------
+
+Offline changes, when the overlay is not mounted, are allowed to either
+the upper or the lower trees.
+
+Changes to the underlying filesystems while part of a mounted overlay
+filesystem are not allowed.  This is not yet enforced, but will be in
+the future.
diff --git a/MAINTAINERS b/MAINTAINERS
index 16a5c5f..a3bf3ec 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4682,6 +4682,13 @@ F:	drivers/scsi/osd/
 F:	include/scsi/osd_*
 F:	fs/exofs/
 
+OVERLAYFS FILESYSTEM
+M:	Miklos Szeredi <miklos@szeredi.hu>
+L:	linux-fsdevel@vger.kernel.org
+S:	Supported
+F:	fs/overlayfs/*
+F:	Documentation/filesystems/overlayfs.txt
+
 P54 WIRELESS DRIVER
 M:	Christian Lamparter <chunkeey@googlemail.com>
 L:	linux-wireless@vger.kernel.org
-- 
1.7.3.4

Re: [PATCH 0/7] overlay filesystem v9

[Michal Suchanek]

Hello

On 17 May 2011 14:30, Miklos Szeredi <miklos@szeredi.hu> wrote:
> Here's an updated version of the overlay filesystem.
>
> Git tree is here:
>
>  git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9
>
>
> Changes from v8 to v9
>
> - support xattr on tmpfs
>
> - fix build after split-up
>
> - fix remove after rename (reported by Jordi Pujol)
>
> - fix rename failure case
>

I built a live CD with v9 and it works for me.

As I understand it v9 only folds in the extra v8 patches so I expected
no less ;-)

Thanks

Michal

Re: [PATCH 0/7] overlay filesystem v9

[Andy Whitcroft]

On Tue, May 17, 2011 at 02:30:45PM +0200, Miklos Szeredi wrote:

> Here's an updated version of the overlay filesystem.
> 
> Git tree is here:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9

Ok I pulled this into the Ubuntu kernel and made an Ubuntu Live CD
for testing.  Overall it worked pretty well, no hangs, no crashes,
performance seemed reasonable.  We hit one issue with hard links which
fail to be possible on overlayfs mounts when the Yama LSM (out of tree)
is enabled.  This module applies more aggressive checks on hard-links
preventing links to files you cannot read.

The bug seems to be related to the way we handle user and group owners for
the overlayfs inodes, which we do not initialise (and they remain as 0,0).
While these ownerships are never exposed to userspace they are exposed
to the LSM layer, and the LSM module checks the wrong owner and fails
to allow the links.

>From what I can see it is completly reasonable to initialise the ownership
fields in the overlayfs inode from the underlying inode, or for new files
ones initialise it in the normal way based on the containing directory.
Now I am nothing like a filesystems expert but looking at what other
filesystems do I think the patch below is sufficient, but certainly it
needs some sanity checking.  At least it fixes all the issues I see here.

With this in place I have been able to boot a Natty release live CD
image, and upgrade it in place to the latest and greatest of everything.
Very nice.  Thanks for your work continuing work on overlayfs.  It is
starting to look good enough to merge.

Comments?

-apw

Re: [PATCH 0/7] overlay filesystem v9

[Miklos Szeredi]

Andy Whitcroft <apw@canonical.com> writes:

> On Tue, May 17, 2011 at 02:30:45PM +0200, Miklos Szeredi wrote:
>
>> Here's an updated version of the overlay filesystem.
>> 
>> Git tree is here:
>> 
>>   git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9
>
> Ok I pulled this into the Ubuntu kernel and made an Ubuntu Live CD
> for testing.  Overall it worked pretty well, no hangs, no crashes,
> performance seemed reasonable.  We hit one issue with hard links which
> fail to be possible on overlayfs mounts when the Yama LSM (out of tree)
> is enabled.  This module applies more aggressive checks on hard-links
> preventing links to files you cannot read.

Great, thanks for testing.

> The bug seems to be related to the way we handle user and group owners for
> the overlayfs inodes, which we do not initialise (and they remain as 0,0).
> While these ownerships are never exposed to userspace they are exposed
> to the LSM layer, and the LSM module checks the wrong owner and fails
> to allow the links.

Hmm.

> From what I can see it is completly reasonable to initialise the ownership
> fields in the overlayfs inode from the underlying inode, or for new files
> ones initialise it in the normal way based on the containing directory.
> Now I am nothing like a filesystems expert but looking at what other
> filesystems do I think the patch below is sufficient, but certainly it
> needs some sanity checking.  At least it fixes all the issues I see
> here.

Forgot to attach the patch?

>
> With this in place I have been able to boot a Natty release live CD
> image, and upgrade it in place to the latest and greatest of everything.
> Very nice.  Thanks for your work continuing work on overlayfs.  It is
> starting to look good enough to merge.
>
> Comments?

I'd be happier if the LSM module were fixed not to peek inside the inode
structure.  Some filesystems (notably network ones) do not guarantee i_*
to be up-to-date, relying on these is not a good idea.

Thanks,
Miklos

Re: [PATCH 0/7] overlay filesystem v9

[Andy Whitcroft]

On Thu, May 19, 2011 at 05:37:09PM +0100, Andy Whitcroft wrote:

> Now I am nothing like a filesystems expert but looking at what other
> filesystems do I think the patch below is sufficient, but certainly it
> needs some sanity checking.  At least it fixes all the issues I see here.

Doh.  Below.

-apw

>From 8bab7242155e614d357e23132cc86964822300d0 Mon Sep 17 00:00:00 2001
From: Andy Whitcroft <apw@canonical.com>
Date: Thu, 19 May 2011 12:43:59 +0100
Subject: [PATCH 1/1] ovl: ensure overlayfs inodes have correct ownerships

Overlayfs builds internal inodes representing the intersection between
the upper and lower directories.  However these inodes do not inherit the
ownership of the underlying inodes, this is transparent in the normal case
as most operations apply to the real backing inodes.  However the LSM
hooks commonly are passed these inodes and may make erroneous decisions
based on the carried credentials.

Fix up the permissions in any new inode to either match the intended
ownership of the directory for new files, or the underlying file for
existing files.

Signed-off-by: Andy Whitcroft <apw@canonical.com>
---
 fs/overlayfs/dir.c   |   15 +++++++++------
 fs/overlayfs/super.c |    2 ++
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c
index e1c09c4..f0a672f 100644
--- a/fs/overlayfs/dir.c
+++ b/fs/overlayfs/dir.c
@@ -268,8 +268,8 @@ static int ovl_dir_getattr(struct vfsmount *mnt, struct dentry *dentry,
 	return 0;
 }
 
-static int ovl_create_object(struct dentry *dentry, int mode, dev_t rdev,
-			     const char *link)
+static int ovl_create_object(struct inode *dir, struct dentry *dentry, int mode,
+				dev_t rdev, const char *link)
 {
 	int err;
 	struct dentry *newdentry;
@@ -284,6 +284,7 @@ static int ovl_create_object(struct dentry *dentry, int mode, dev_t rdev,
 	inode = ovl_new_inode(dentry->d_sb, mode, dentry->d_fsdata);
 	if (!inode)
 		goto out;
+	inode_init_owner(inode, dir, mode);
 
 	err = ovl_copy_up(dentry->d_parent);
 	if (err)
@@ -325,24 +326,26 @@ out:
 static int ovl_create(struct inode *dir, struct dentry *dentry, int mode,
 			struct nameidata *nd)
 {
-	return ovl_create_object(dentry, (mode & 07777) | S_IFREG, 0, NULL);
+	return ovl_create_object(dir, dentry,
+					(mode & 07777) | S_IFREG, 0, NULL);
 }
 
 static int ovl_mkdir(struct inode *dir, struct dentry *dentry, int mode)
 {
-	return ovl_create_object(dentry, (mode & 07777) | S_IFDIR, 0, NULL);
+	return ovl_create_object(dir, dentry,
+					(mode & 07777) | S_IFDIR, 0, NULL);
 }
 
 static int ovl_mknod(struct inode *dir, struct dentry *dentry, int mode,
 		       dev_t rdev)
 {
-	return ovl_create_object(dentry, mode, rdev, NULL);
+	return ovl_create_object(dir, dentry, mode, rdev, NULL);
 }
 
 static int ovl_symlink(struct inode *dir, struct dentry *dentry,
 			 const char *link)
 {
-	return ovl_create_object(dentry, S_IFLNK, 0, link);
+	return ovl_create_object(dir, dentry, S_IFLNK, 0, link);
 }
 
 static int ovl_do_remove(struct dentry *dentry, bool is_dir)
diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index a9a09a6..e6b3af5 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -321,6 +321,8 @@ int ovl_do_lookup(struct dentry *dentry)
 		realdentry = upperdentry ? upperdentry : lowerdentry;
 		err = -ENOMEM;
 		inode = ovl_new_inode(dentry->d_sb, realdentry->d_inode->i_mode, oe);
+		inode->i_uid = realdentry->d_inode->i_uid;
+		inode->i_gid = realdentry->d_inode->i_gid;
 		if (!inode)
 			goto out_dput;
 	}
-- 
1.7.4.1

Re: [PATCH 0/7] overlay filesystem v9

[Andy Whitcroft]

On Thu, May 19, 2011 at 07:44:54PM +0200, Miklos Szeredi wrote:

> Forgot to attach the patch?

Yep, should be in your inbox now.

> >
> > With this in place I have been able to boot a Natty release live CD
> > image, and upgrade it in place to the latest and greatest of everything.
> > Very nice.  Thanks for your work continuing work on overlayfs.  It is
> > starting to look good enough to merge.
> >
> > Comments?
> 
> I'd be happier if the LSM module were fixed not to peek inside the inode
> structure.  Some filesystems (notably network ones) do not guarantee i_*
> to be up-to-date, relying on these is not a good idea.

I wonder what what the official way to get ownership information for an
inode is if we cannot rely on those.  I am happy to get the Yama LSM
fixed up if they are doing something wrong.

-apw

Re: [PATCH 0/7] overlay filesystem v9

[John Stoffel]

>>>>> "Miklos" == Miklos Szeredi <miklos@szeredi.hu> writes:

Miklos> Andy Whitcroft <apw@canonical.com> writes:
>> On Tue, May 17, 2011 at 02:30:45PM +0200, Miklos Szeredi wrote:
>> 
>>> Here's an updated version of the overlay filesystem.
>>> 
>>> Git tree is here:
>>> 
>>> git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9
>> 
>> Ok I pulled this into the Ubuntu kernel and made an Ubuntu Live CD
>> for testing.  Overall it worked pretty well, no hangs, no crashes,
>> performance seemed reasonable.  We hit one issue with hard links which
>> fail to be possible on overlayfs mounts when the Yama LSM (out of tree)
>> is enabled.  This module applies more aggressive checks on hard-links
>> preventing links to files you cannot read.

Miklos> Great, thanks for testing.

>> The bug seems to be related to the way we handle user and group owners for
>> the overlayfs inodes, which we do not initialise (and they remain as 0,0).
>> While these ownerships are never exposed to userspace they are exposed
>> to the LSM layer, and the LSM module checks the wrong owner and fails
>> to allow the links.

Miklos> Hmm.

>> From what I can see it is completly reasonable to initialise the ownership
>> fields in the overlayfs inode from the underlying inode, or for new files
>> ones initialise it in the normal way based on the containing directory.
>> Now I am nothing like a filesystems expert but looking at what other
>> filesystems do I think the patch below is sufficient, but certainly it
>> needs some sanity checking.  At least it fixes all the issues I see
>> here.

Miklos> Forgot to attach the patch?

>> 
>> With this in place I have been able to boot a Natty release live CD
>> image, and upgrade it in place to the latest and greatest of everything.
>> Very nice.  Thanks for your work continuing work on overlayfs.  It is
>> starting to look good enough to merge.
>> 
>> Comments?

Miklos> I'd be happier if the LSM module were fixed not to peek inside the inode
Miklos> structure.  Some filesystems (notably network ones) do not guarantee i_*
Miklos> to be up-to-date, relying on these is not a good idea.

One question I have is why is a LiveCD even using the LSM modules?
It's not like it tells you anything you don't know... unless you're
planning on mounting some external disks and then reading in the LSM
checksums from somewhere to confirm.

Just curious.  

But I am happy to see the testing happening and overlayFS looks really
interesting too.  I can see it being used for a bunch of compute nodes
that all use the same hardware and software, and only need minor local
customization.

John

Re: [PATCH 0/7] overlay filesystem v9

[Andy Whitcroft]

On Thu, May 19, 2011 at 03:04:37PM -0400, John Stoffel wrote:

> One question I have is why is a LiveCD even using the LSM modules?
> It's not like it tells you anything you don't know... unless you're
> planning on mounting some external disks and then reading in the LSM
> checksums from somewhere to confirm.
> 
> Just curious.  

Cause the LiveCD has the same kernel we want in the installed system on
it, so its there warts and all.

-apw

Re: [PATCH 0/7] overlay filesystem v9

[John Stoffel]

>>>>> "Andy" == Andy Whitcroft <apw@canonical.com> writes:

Andy> On Thu, May 19, 2011 at 03:04:37PM -0400, John Stoffel wrote:
>> One question I have is why is a LiveCD even using the LSM modules?
>> It's not like it tells you anything you don't know... unless you're
>> planning on mounting some external disks and then reading in the LSM
>> checksums from somewhere to confirm.
>> 
>> Just curious.  

Andy> Cause the LiveCD has the same kernel we want in the installed
Andy> system on it, so its there warts and all.

Can't argue with that then.  

Re: [PATCH 0/7] overlay filesystem v9

[NeilBrown]

On Thu, 19 May 2011 19:05:58 +0100 Andy Whitcroft <apw@canonical.com> wrote:

> On Thu, May 19, 2011 at 07:44:54PM +0200, Miklos Szeredi wrote:
> 
> > Forgot to attach the patch?
> 
> Yep, should be in your inbox now.
> 
> > >
> > > With this in place I have been able to boot a Natty release live CD
> > > image, and upgrade it in place to the latest and greatest of everything.
> > > Very nice.  Thanks for your work continuing work on overlayfs.  It is
> > > starting to look good enough to merge.
> > >
> > > Comments?
> > 
> > I'd be happier if the LSM module were fixed not to peek inside the inode
> > structure.  Some filesystems (notably network ones) do not guarantee i_*
> > to be up-to-date, relying on these is not a good idea.
> 
> I wonder what what the official way to get ownership information for an
> inode is if we cannot rely on those.  I am happy to get the Yama LSM
> fixed up if they are doing something wrong.
> 
> -apw

inode->getattr(...., &kstat);
look at kstat.uid, kstat.gid.

??

NeilBrown

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

Miklos,

I tried your overlayfs.v9 git repo w/ racer, using two separate ext3 filesystems (one for lowerdir and another for upperdir).  I got the WARN_ON in ovl_permission to trigger within about 10 minutes of testing.  Looking at the code, I see a problem in returning w/o cleaning up an dput-ing the alias dentry.  Simple patch enclosed below.

Cheers,
Erez.


diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 3c15d54..6c70f57 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -80,7 +82,8 @@ int ovl_permission(struct inode *inode, int mask, unsigned int flags)
 	realinode = ACCESS_ONCE(realdentry->d_inode);
 	if (!realinode) {
 		WARN_ON(!(flags & IPERM_FLAG_RCU));
-		return -ENOENT;
+		err = -ENOENT;
+		goto out_dput;
 	}
 
 	if (mask & MAY_WRITE) {

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

On May 20, 2011, at 1:39 AM, Erez Zadok wrote:

> Miklos,
> 
> I tried your overlayfs.v9 git repo w/ racer, using two separate ext3 filesystems (one for lowerdir and another for upperdir).  I got the WARN_ON in ovl_permission to trigger within about 10 minutes of testing.  Looking at the code, I see a problem in returning w/o cleaning up an dput-ing the alias dentry.  Simple patch enclosed below.
> 
> Cheers,
> Erez.

Niklos, I forgot to mention that I had to apply a small fix to a VFS bug in fs/namei.c:1362, where a mix of symlinks and renames, using racer, triggers an BUG_ON at the VFS layer (very reproducible).  Without this fix, racer oopses in the VFS well before it gets to trigger overlayfs bugs.  It's a pity 2.6.39 was released with this very reproducible *VFS* level bug (doesn't anyone run racer+fsx+ltp before releasing a new kernel?)

Anyway, here's the small patch here.  Clearly viro/hch need to review this "fix" b/c I'm not sure it's really the right one

Cheers,
Erez.


VFS: move BUG_ON test for symlink nd->depth after current->link_count test

This solves a bug in nested_symlink (which was rewritten from
do_follow_link), and follows the order of depth tests that existed before.
The bug triggers a BUG_ON in fs/namei.c:1346, when running racer with
symlink and rename ops.

Signed-off-by: Erez Zadok <ezk@cs.sunysb.edu>
diff --git a/fs/namei.c b/fs/namei.c
index 017c3fa..7a93387 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -1343,12 +1343,12 @@ static inline int nested_symlink(struct path *path, struct nameidata *nd)
 {
 	int res;
 
-	BUG_ON(nd->depth >= MAX_NESTED_LINKS);
 	if (unlikely(current->link_count >= MAX_NESTED_LINKS)) {
 		path_put_conditional(path, nd);
 		path_put(&nd->path);
 		return -ELOOP;
 	}
+	BUG_ON(nd->depth >= MAX_NESTED_LINKS);
 
 	nd->depth++;
 	current->link_count++;

Re: [PATCH 5/7] overlay filesystem (negative dentries cause OOPS on NULL inode)

[Erez Zadok]

Miklos,

I  continued to test your overlayfs.v9 git repo w/ racer, using two separate empty ext3 filesystems (one for lowerdir and another for upperdir).  I got a couple of NULL derefs in different places.  Both appear to be due to negative dentries being passed around, where the code using them assumes a positive dentry.

The first is in ovl_getattr().  The realpath.dentry->d_inode can apparently be NULL after calling ovl_path_real, and when  that's passed to vfs_getattr, the NULL inode is deref'ed in security_inode_getattr (calling IS_PRIVATE on the inode).  I verified it by sticking the BUG_ON(!inode), seeing it triggered, and then replaced it with "if (!inode) return -ENOENT".  After that, racer didn't trigger this bug, but I don't know if returning ENOENT is correct there.  There may be a more fundamental problem in that the lower dentry in ovl_getattr should never be negative.

The second instance of a similar problem is in ovl_follow_link.  Again, here realinode can be NULL, causing the WARN_ON(!realinode->…) to oops.  I verified this by placing a BUG_ON, triggering it, and then adding some code to return NULL if the realinode is NULL.  As with the first instance, I don't know if this is correct either.  But I include a simple patch below.

With these two fixes, the error-path bug in ovl_permission, and the fs/namei.c:1362 bug, racer ran for 60 minutes without triggering an OOPS.

Cheers,
Erez.


diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c
index 3c15d54..6c70f57 100644
--- a/fs/overlayfs/inode.c
+++ b/fs/overlayfs/inode.c
@@ -42,6 +42,8 @@ static int ovl_getattr(struct vfsmount *mnt, struct dentry *dentry,
 	struct path realpath;
 
 	ovl_path_real(dentry, &realpath);
+	if (!realpath.dentry->d_inode)
+		return -ENOENT;
 	return vfs_getattr(realpath.mnt, realpath.dentry, stat);
 }
 
@@ -127,6 +130,9 @@ static void *ovl_follow_link(struct dentry *dentry, struct nameidata *nd)
 
 	realdentry = ovl_dentry_real(dentry);
 	realinode = realdentry->d_inode;
+	if (!realinode)
+		return NULL;
 
 	if (WARN_ON(!realinode->i_op->follow_link))
 		return ERR_PTR(-EPERM);

Re: [PATCH 0/7] overlay filesystem v9

[Miklos Szeredi]

(Neil, sorry for the dup)

NeilBrown <neilb@suse.de> writes:

> On Thu, 19 May 2011 19:05:58 +0100 Andy Whitcroft <apw@canonical.com> wrote:
>
>> On Thu, May 19, 2011 at 07:44:54PM +0200, Miklos Szeredi wrote:
>> > I'd be happier if the LSM module were fixed not to peek inside the inode
>> > structure.  Some filesystems (notably network ones) do not guarantee i_*
>> > to be up-to-date, relying on these is not a good idea.
>> 
>> I wonder what what the official way to get ownership information for an
>> inode is if we cannot rely on those.  I am happy to get the Yama LSM
>> fixed up if they are doing something wrong.
>> 
>> -apw
>
> inode->getattr(...., &kstat);
> look at kstat.uid, kstat.gid.

Yeah, it should probably do something like:

	if (inode->i_op->getattr) {
		struct kstat stat;
		inode->i_op->getattr(...)
		uid = stat.uid;
	} else {
		uid = inode->i_uid;
	}

The problem is that getattr takes a vfsmount, which makes this
impossible to do cleanly.  And the solution is for ->getattr() to lose
that argument, which is a bit more involved :(

Thanks,
Miklos

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Miklos Szeredi]

Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:

> I tried your overlayfs.v9 git repo w/ racer, using two separate ext3
> filesystems (one for lowerdir and another for upperdir).  I got the
> WARN_ON in ovl_permission to trigger within about 10 minutes of
> testing.  Looking at the code, I see a problem in returning w/o
> cleaning up an dput-ing the alias dentry.  Simple patch enclosed
> below.

Hmm, thanks.  The more interesting question is: why does that WARN_ON
trigger?  I'll look into it.

Thanks,
Miklos

Re: [PATCH 0/7] overlay filesystem v9

[Michal Suchanek]

Hello

On 19 May 2011 18:37, Andy Whitcroft <apw@canonical.com> wrote:
> On Tue, May 17, 2011 at 02:30:45PM +0200, Miklos Szeredi wrote:
>
>> Here's an updated version of the overlay filesystem.
>>
>> Git tree is here:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v9
>
> Ok I pulled this into the Ubuntu kernel and made an Ubuntu Live CD
> for testing.  Overall it worked pretty well, no hangs, no crashes,
> performance seemed reasonable.  We hit one issue with hard links which
> fail to be possible on overlayfs mounts when the Yama LSM (out of tree)
> is enabled.  This module applies more aggressive checks on hard-links
> preventing links to files you cannot read.
>
> The bug seems to be related to the way we handle user and group owners for
> the overlayfs inodes, which we do not initialise (and they remain as 0,0).
> While these ownerships are never exposed to userspace they are exposed
> to the LSM layer, and the LSM module checks the wrong owner and fails
> to allow the links.
>
> From what I can see it is completly reasonable to initialise the ownership
> fields in the overlayfs inode from the underlying inode, or for new files
> ones initialise it in the normal way based on the containing directory.

If the ownership recorded in the inode is not user visible then the
ownership must be taken elsewhere (the lower filesystem?) . Since
deconstructing, modifying and reconstructing the overlay is supported
initializing the inode to some default gives you two possibly
inconsistent ownerships - one seen by LSM, the other by userspace.

So in general LSM should check whatever userspace would I guess.

Thanks

Michal

Re: [PATCH 0/7] overlay filesystem v9

[Andy Whitcroft]

On Fri, May 20, 2011 at 10:18:30AM +0200, Miklos Szeredi wrote:

> Yeah, it should probably do something like:
> 
> 	if (inode->i_op->getattr) {
> 		struct kstat stat;
> 		inode->i_op->getattr(...)
> 		uid = stat.uid;
> 	} else {
> 		uid = inode->i_uid;
> 	}
> 
> The problem is that getattr takes a vfsmount, which makes this
> impossible to do cleanly.  And the solution is for ->getattr() to lose
> that argument, which is a bit more involved :(

Ok.  I am going to take this as a bug in the LSM, so ignore my patch
here.  I have been poking at what the LSM is doing and actually I think
I see the flaw, so I will take the issue up with them.

In light of that take this as a "tested heavily and it worked without
issues" on the V9 version.

Tested-by: Andy Whitcroft <apw@canonical.com>

-apw

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Miklos Szeredi]

Miklos Szeredi <miklos@szeredi.hu> writes:

> Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
>
>> I tried your overlayfs.v9 git repo w/ racer, using two separate ext3
>> filesystems (one for lowerdir and another for upperdir).  I got the
>> WARN_ON in ovl_permission to trigger within about 10 minutes of
>> testing.  Looking at the code, I see a problem in returning w/o
>> cleaning up an dput-ing the alias dentry.  Simple patch enclosed
>> below.
>
> Hmm, thanks.  The more interesting question is: why does that WARN_ON
> trigger?  I'll look into it.

I think I found the cause of all the bug and oopsen you are seeing.

Overlayfs expects upper and lower dentries to be always positive, it
never stores negative dentries there, there's no point, instead it
stores NULL.

There are basically two ways a positive dentry can become negative:

 A) dentry becomes unhashed with d_count == 0

 B) d_delete with d_count == 1

Case A is not possible in our case, since overlayfs keeps a ref on the
upper/lower dentries for the lifetime of the overlayfs dentry.

Case B is however possible, since no extra ref is taken before calling
vfs_unlink/vfs_rmdir.  So it looks like this is being triggered.

This is easy to solve, just grab a ref to upperdentry before
unlink/rmdir.  Equivalent is if we grab an extra reference from the
start.  The below patch does this.

With the patch I can't trigger the bugs anymore.

Erez, could you please also check if reverting your patches and applying
this one fixes all the bugs?

Thanks,
Miklos



commit 9192816148e2c6b1d610226b1fc1c04c36216370
Author: Miklos Szeredi <mszeredi@suse.cz>
Date:   Fri May 20 16:07:34 2011 +0200

    ovl: don't allow upperdentry to go negative
    
    Upperdentry can become negative if the file/directory is removed,
    since d_delete checks for d_count == 1 (we are the only user of this
    dentry) and changes it to negative in that case.
    
    However users of overlayfs expect upper and lower dentries to be
    either NULL or positive, and finding a negative dentry will trigger a
    WARNING or Oops.
    
    Fix this by keeping double reference on upperdentry which prevents
    d_delete() turning the dentry into negative.
    
    Reported-by: Erez Zadok <ezk@fsl.cs.sunysb.edu>
    Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>

diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c
index a9a09a6..c9db954 100644
--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -27,6 +27,10 @@ struct ovl_fs {
 };
 
 struct ovl_entry {
+	/*
+	 * Keep "double reference" on upper dentries, so that
+	 * d_delete() doesn't think it's OK to reset d_inode to NULL.
+	 */
 	struct dentry *__upperdentry;
 	struct dentry *lowerdentry;
 	union {
@@ -152,8 +156,9 @@ void ovl_dentry_update(struct dentry *dentry, struct dentry *upperdentry)
 
 	WARN_ON(!mutex_is_locked(&upperdentry->d_parent->d_inode->i_mutex));
 	WARN_ON(oe->__upperdentry);
+	BUG_ON(!upperdentry->d_inode);
 	smp_wmb();
-	oe->__upperdentry = upperdentry;
+	oe->__upperdentry = dget(upperdentry);
 }
 
 void ovl_dentry_version_inc(struct dentry *dentry)
@@ -218,6 +223,7 @@ static void ovl_dentry_release(struct dentry *dentry)
 
 	if (oe) {
 		dput(oe->__upperdentry);
+		dput(oe->__upperdentry);
 		dput(oe->lowerdentry);
 		call_rcu(&oe->rcu, ovl_entry_free);
 	}
@@ -326,7 +332,7 @@ int ovl_do_lookup(struct dentry *dentry)
 	}
 
 	if (upperdentry)
-		oe->__upperdentry = upperdentry;
+		oe->__upperdentry = dget(upperdentry);
 
 	if (lowerdentry)
 		oe->lowerdentry = lowerdentry;
@@ -533,7 +539,7 @@ static int ovl_fill_super(struct super_block *sb, void *data, int silent)
 	mntput(upperpath.mnt);
 	mntput(lowerpath.mnt);
 
-	oe->__upperdentry = upperpath.dentry;
+	oe->__upperdentry = dget(upperpath.dentry);
 	oe->lowerdentry = lowerpath.dentry;
 
 	root_dentry->d_fsdata = oe;

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Miklos Szeredi]

Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
> Niklos, I forgot to mention that I had to apply a small fix to a VFS
> bug in fs/namei.c:1362, where a mix of symlinks and renames, using
> racer, triggers an BUG_ON at the VFS layer (very reproducible).
> Without this fix, racer oopses in the VFS well before it gets to
> trigger overlayfs bugs.  It's a pity 2.6.39 was released with this
> very reproducible *VFS* level bug (doesn't anyone run racer+fsx+ltp
> before releasing a new kernel?)
>
> Anyway, here's the small patch here.  Clearly viro/hch need to review
> this "fix" b/c I'm not sure it's really the right one

Patch looks good to me.

Acked-by: Miklos Szeredi <mszeredi@suse.cz>

If you've got a patch fixing a serious bug, you'd better send it to
Linus and Andrew (as well as the VFS maintainers), where it'll get more
prompt evaluation.

You can still do that, also adding "CC: stable@kernel.org" to the SOB
block in the patch so that the fix quickly makes it into the 2.6.39
stable series.

Thanks,
Miklos


>
> VFS: move BUG_ON test for symlink nd->depth after current->link_count test
>
> This solves a bug in nested_symlink (which was rewritten from
> do_follow_link), and follows the order of depth tests that existed before.
> The bug triggers a BUG_ON in fs/namei.c:1346, when running racer with
> symlink and rename ops.
>
> Signed-off-by: Erez Zadok <ezk@cs.sunysb.edu>
> diff --git a/fs/namei.c b/fs/namei.c
> index 017c3fa..7a93387 100644
> --- a/fs/namei.c
> +++ b/fs/namei.c
> @@ -1343,12 +1343,12 @@ static inline int nested_symlink(struct path *path, struct nameidata *nd)
>  {
>  	int res;
>  
> -	BUG_ON(nd->depth >= MAX_NESTED_LINKS);
>  	if (unlikely(current->link_count >= MAX_NESTED_LINKS)) {
>  		path_put_conditional(path, nd);
>  		path_put(&nd->path);
>  		return -ELOOP;
>  	}
> +	BUG_ON(nd->depth >= MAX_NESTED_LINKS);
>  
>  	nd->depth++;
>  	current->link_count++;

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

On May 20, 2011, at 10:17 AM, Miklos Szeredi wrote:

> Miklos Szeredi <miklos@szeredi.hu> writes:
> 
>> Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
>> 
>>> I tried your overlayfs.v9 git repo w/ racer, using two separate ext3
>>> filesystems (one for lowerdir and another for upperdir).  I got the
>>> WARN_ON in ovl_permission to trigger within about 10 minutes of
>>> testing.  Looking at the code, I see a problem in returning w/o
>>> cleaning up an dput-ing the alias dentry.  Simple patch enclosed
>>> below.
>> 
>> Hmm, thanks.  The more interesting question is: why does that WARN_ON
>> trigger?  I'll look into it.
> 
> I think I found the cause of all the bug and oopsen you are seeing.
> 
> Overlayfs expects upper and lower dentries to be always positive, it
> never stores negative dentries there, there's no point, instead it
> stores NULL.
> 
> There are basically two ways a positive dentry can become negative:
> 
> A) dentry becomes unhashed with d_count == 0
> 
> B) d_delete with d_count == 1
> 
> Case A is not possible in our case, since overlayfs keeps a ref on the
> upper/lower dentries for the lifetime of the overlayfs dentry.
> 
> Case B is however possible, since no extra ref is taken before calling
> vfs_unlink/vfs_rmdir.  So it looks like this is being triggered.
> 
> This is easy to solve, just grab a ref to upperdentry before
> unlink/rmdir.  Equivalent is if we grab an extra reference from the
> start.  The below patch does this.
> 
> With the patch I can't trigger the bugs anymore.
> 
> Erez, could you please also check if reverting your patches and applying
> this one fixes all the bugs?

Done: reverted my ovl patches and applied yours.  And indeed, the NULL oopses don't show up after 10 minutes of running.  Just to be sure, I ran overlayfs on top of ext2, ext3, xfs, reiserfs, and tmps — no oopses.  And I wasn't able to reproduce the WARN_ON in ovl_permission.

Are you interested in testing overlayfs on top of NFS at this time? You need xattr support right? Stacking over NFS takes extra effort, esp. b/c of the pesky silly deleted files and needing a nameidata everywhere (but I managed after a while to get Wrapfs working stably over nfs3).

> Thanks,
> Miklos
> 
> commit 9192816148e2c6b1d610226b1fc1c04c36216370
> Author: Miklos Szeredi <mszeredi@suse.cz>
> Date:   Fri May 20 16:07:34 2011 +0200
> 
>    ovl: don't allow upperdentry to go negative
[...]

Do you commit such patches to a master branch, the v9 branch, or some other one; I'd like to be able to git pull

Thanks,
Erez.

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

On May 20, 2011, at 10:17 AM, Miklos Szeredi wrote:

> Miklos Szeredi <miklos@szeredi.hu> writes:
> 
>> Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
>> 
>>> I tried your overlayfs.v9 git repo w/ racer, using two separate ext3
>>> filesystems (one for lowerdir and another for upperdir).  I got the
>>> WARN_ON in ovl_permission to trigger within about 10 minutes of
>>> testing.  Looking at the code, I see a problem in returning w/o
>>> cleaning up an dput-ing the alias dentry.  Simple patch enclosed
>>> below.
>> 
>> Hmm, thanks.  The more interesting question is: why does that WARN_ON
>> trigger?  I'll look into it.
> 
> I think I found the cause of all the bug and oopsen you are seeing.
> 
> Overlayfs expects upper and lower dentries to be always positive, it
> never stores negative dentries there, there's no point, instead it
> stores NULL.
> 
> There are basically two ways a positive dentry can become negative:
> 
> A) dentry becomes unhashed with d_count == 0
> 
> B) d_delete with d_count == 1
> 
> Case A is not possible in our case, since overlayfs keeps a ref on the
> upper/lower dentries for the lifetime of the overlayfs dentry.
> 
> Case B is however possible, since no extra ref is taken before calling
> vfs_unlink/vfs_rmdir.  So it looks like this is being triggered.
> 
> This is easy to solve, just grab a ref to upperdentry before
> unlink/rmdir.  Equivalent is if we grab an extra reference from the
> start.  The below patch does this.
> 
> With the patch I can't trigger the bugs anymore.
> 
> Erez, could you please also check if reverting your patches and applying
> this one fixes all the bugs?

Done: reverted my ovl patches and applied yours.  And indeed, the NULL oopses don't show up after 10 minutes of running.  Just to be sure, I ran overlayfs on top of ext2, ext3, xfs, reiserfs, and tmps — no oopses.  And I wasn't able to reproduce the WARN_ON in ovl_permission.

Are you interested in testing overlayfs on top of NFS at this time? You need xattr support right? Stacking over NFS takes extra effort, esp. b/c of the pesky silly deleted files and needing a nameidata everywhere (but I managed after a while to get Wrapfs working stably over nfs3).

> Thanks,
> Miklos
> 
> commit 9192816148e2c6b1d610226b1fc1c04c36216370
> Author: Miklos Szeredi <mszeredi@suse.cz>
> Date:   Fri May 20 16:07:34 2011 +0200
> 
>    ovl: don't allow upperdentry to go negative
[...]

Do you commit such patches to a master branch, the v9 branch, or some other one; I'd like to be able to git pull

Thanks,
Erez.

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

On May 20, 2011, at 10:25 AM, Miklos Szeredi wrote:

> Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
>> Niklos, I forgot to mention that I had to apply a small fix to a VFS
>> bug in fs/namei.c:1362, where a mix of symlinks and renames, using
>> racer, triggers an BUG_ON at the VFS layer (very reproducible).
>> Without this fix, racer oopses in the VFS well before it gets to
>> trigger overlayfs bugs.  It's a pity 2.6.39 was released with this
>> very reproducible *VFS* level bug (doesn't anyone run racer+fsx+ltp
>> before releasing a new kernel?)
>> 
>> Anyway, here's the small patch here.  Clearly viro/hch need to review
>> this "fix" b/c I'm not sure it's really the right one
> 
> Patch looks good to me.
> 
> Acked-by: Miklos Szeredi <mszeredi@suse.cz>
> 
> If you've got a patch fixing a serious bug, you'd better send it to
> Linus and Andrew (as well as the VFS maintainers), where it'll get more
> prompt evaluation.
> 
> You can still do that, also adding "CC: stable@kernel.org" to the SOB
> block in the patch so that the fix quickly makes it into the 2.6.39
> stable series.
> 
> Thanks,
> Miklos

Thanks.  I'll do that.

Erez.

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Miklos Szeredi]

Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:

> Done: reverted my ovl patches and applied yours.  And indeed, the NULL
> oopses don't show up after 10 minutes of running.  Just to be sure, I
> ran overlayfs on top of ext2, ext3, xfs, reiserfs, and tmps — no
> oopses.  And I wasn't able to reproduce the WARN_ON in ovl_permission.

Great, thanks for testing.  Pushed to .v9 branch.

> Are you interested in testing overlayfs on top of NFS at this time?

No, not as an upper layer.  NFS as a lower layer should work fine.

Thanks,
Miklos

Re: [PATCH 5/7] overlay filesystem (inode.c bad error path)

[Erez Zadok]

On May 23, 2011, at 4:57 AM, Miklos Szeredi wrote:

> Erez Zadok <ezk@fsl.cs.sunysb.edu> writes:
> 
>> Are you interested in testing overlayfs on top of NFS at this time?
> 
> No, not as an upper layer.  NFS as a lower layer should work fine.

OK, I'll give it a test as well (careful what you wish for :-)

BTW, when do you typically sync up w/ Linus's tree: is it only when you release the next ovl version?

> Thanks,
> Miklos

Erez.