How to cope with two incompatible overlayfs formats out in the wild

How to cope with two incompatible overlayfs formats out in the wild

[Miklos Szeredi]

[CC-ing mailing lists, Al and Linus for wider exposure]

This issue is this: Ubuntu and SUSE carry an "old" format of overlayfs
while mainline has a "new" format.  The differences are:

 - whiteouts are represented differently (symlink + xattr in the old
format, chardev in the new)

 - new one needs a "workdir" mount option, which points to a directory
on the same filesystem as upperdir.  If upperdir was the root of the
filesystem then it needs to be moved into a subdir to make space for
the work directory.

Migrating from old to new is not a big issue, but Ubuntu people have
expressed concerns about systems with mixed kernel versions and want
to support the old format alongside the new.

This can all be done with out-of-tree code.

So from mainline we need two things:

  - when mounting distinguish between old and new format.

  - userspace can detect which formats are supported by the kernel.

If we'd have a different filesystem type for the old and new formats,
then that would solve both (checking /proc/filesystems would indicate
which one is supported).

Unfortunately that would mean having to change "overlayfs" type to
something else in 3.18.  Question is, is there some sane name which
would fit?  "overlayfs2" is perhaps the best, but I'm not overly
enthusiastic about it.

Any other ideas?

Thanks,
Miklos

Re: How to cope with two incompatible overlayfs formats out in the wild

[Andy Whitcroft]

On Tue, Nov 18, 2014 at 03:28:03PM +0100, Miklos Szeredi wrote:
> [CC-ing mailing lists, Al and Linus for wider exposure]
> 
> This issue is this: Ubuntu and SUSE carry an "old" format of overlayfs
> while mainline has a "new" format.  The differences are:
> 
>  - whiteouts are represented differently (symlink + xattr in the old
> format, chardev in the new)
> 
>  - new one needs a "workdir" mount option, which points to a directory
> on the same filesystem as upperdir.  If upperdir was the root of the
> filesystem then it needs to be moved into a subdir to make space for
> the work directory.
> 
> Migrating from old to new is not a big issue, but Ubuntu people have
> expressed concerns about systems with mixed kernel versions and want
> to support the old format alongside the new.
> 
> This can all be done with out-of-tree code.
> 
> So from mainline we need two things:
> 
>   - when mounting distinguish between old and new format.
> 
>   - userspace can detect which formats are supported by the kernel.
> 
> If we'd have a different filesystem type for the old and new formats,
> then that would solve both (checking /proc/filesystems would indicate
> which one is supported).
> 
> Unfortunately that would mean having to change "overlayfs" type to
> something else in 3.18.  Question is, is there some sane name which
> would fit?  "overlayfs2" is perhaps the best, but I'm not overly
> enthusiastic about it.
> 
> Any other ideas?

ext4 makes use of feature flags in /sys/fs/ext4/features.  Perhaps we could
make use of this, say /sys/fs/overlayfs/features/{workdir,whiteout-chrdev},
or a even some kind of version in /sys/fs/overlayfs/version.

The presence of /sys/fs/overlayfs itself might be enought to assume the
presence of support for the new format.

-apw

Re: How to cope with two incompatible overlayfs formats out in the wild

[Al Viro]

On Tue, Nov 18, 2014 at 03:28:03PM +0100, Miklos Szeredi wrote:

> So from mainline we need two things:
> 
>   - when mounting distinguish between old and new format.
> 
>   - userspace can detect which formats are supported by the kernel.
> 
> If we'd have a different filesystem type for the old and new formats,
> then that would solve both (checking /proc/filesystems would indicate
> which one is supported).
> 
> Unfortunately that would mean having to change "overlayfs" type to
> something else in 3.18.  Question is, is there some sane name which
> would fit?  "overlayfs2" is perhaps the best, but I'm not overly
> enthusiastic about it.
> 
> Any other ideas?

Umm...  What does the old one do when it sees workdir=<something> in the
options?

Re: How to cope with two incompatible overlayfs formats out in the wild

[Miklos Szeredi]

On Wed, Nov 19, 2014 at 2:59 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Tue, Nov 18, 2014 at 03:28:03PM +0100, Miklos Szeredi wrote:
>
>> So from mainline we need two things:
>>
>>   - when mounting distinguish between old and new format.
>>
>>   - userspace can detect which formats are supported by the kernel.
>>
>> If we'd have a different filesystem type for the old and new formats,
>> then that would solve both (checking /proc/filesystems would indicate
>> which one is supported).
>>
>> Unfortunately that would mean having to change "overlayfs" type to
>> something else in 3.18.  Question is, is there some sane name which
>> would fit?  "overlayfs2" is perhaps the best, but I'm not overly
>> enthusiastic about it.
>>
>> Any other ideas?
>
> Umm...  What does the old one do when it sees workdir=<something> in the
> options?

Returns EINVAL.

Also the new one will fail without the "workdir=" option, as far as
3.18 is concerned at least.  With the multi layer support work that
option won't be mandatory any more.

But that still doesn't answer the question on how to select the format.

Thanks,
Miklos

Re: How to cope with two incompatible overlayfs formats out in the wild

[Miklos Szeredi]

On Tue, Nov 18, 2014 at 03:28:03PM +0100, Miklos Szeredi wrote:
> If we'd have a different filesystem type for the old and new formats,
> then that would solve both (checking /proc/filesystems would indicate
> which one is supported).
> 
> Unfortunately that would mean having to change "overlayfs" type to
> something else in 3.18.  Question is, is there some sane name which
> would fit?  "overlayfs2" is perhaps the best, but I'm not overly
> enthusiastic about it.

How 'bout just "overlay"?

Minimal patch follows.

I'd be happy to rename "fs/overlayfs" to "fs/overlay" and
"Documentation/filesystems/overlayfs.txt" to
"Documentation/filesystems/overlay.txt" as well, but that should be a separate
patch that just does renames and nothing else.

Thanks,
Miklos
----

Subject: ovl: rename filesystem type to "overlay"
From: Miklos Szeredi <mszeredi@suse.cz>

Some distributions carry an "old" format of overlayfs while mainline has a
"new" format.

The distros will possibly want to keep the old overlayfs alongside the new
for compatibility reasons.

To make it possible to differentiate the two versions change the name of
the new one from "overlayfs" to "overlay".

Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
Reported-by: Serge Hallyn <serge.hallyn@ubuntu.com>
Cc: Andy Whitcroft <apw@canonical.com>
---
 Documentation/filesystems/overlayfs.txt |    2 +-
 MAINTAINERS                             |    2 +-
 fs/Makefile                             |    2 +-
 fs/overlayfs/Kconfig                    |    2 +-
 fs/overlayfs/Makefile                   |    4 ++--
 fs/overlayfs/super.c                    |    6 +++---
 6 files changed, 9 insertions(+), 9 deletions(-)

--- a/fs/overlayfs/super.c
+++ b/fs/overlayfs/super.c
@@ -24,7 +24,7 @@ MODULE_AUTHOR("Miklos Szeredi <miklos@sz
 MODULE_DESCRIPTION("Overlay filesystem");
 MODULE_LICENSE("GPL");
 
-#define OVERLAYFS_SUPER_MAGIC 0x794c764f
+#define OVERLAYFS_SUPER_MAGIC 0x794c7630
 
 struct ovl_config {
 	char *lowerdir;
@@ -776,11 +776,11 @@ static struct dentry *ovl_mount(struct f
 
 static struct file_system_type ovl_fs_type = {
 	.owner		= THIS_MODULE,
-	.name		= "overlayfs",
+	.name		= "overlay",
 	.mount		= ovl_mount,
 	.kill_sb	= kill_anon_super,
 };
-MODULE_ALIAS_FS("overlayfs");
+MODULE_ALIAS_FS("overlay");
 
 static int __init ovl_init(void)
 {
--- a/Documentation/filesystems/overlayfs.txt
+++ b/Documentation/filesystems/overlayfs.txt
@@ -64,7 +64,7 @@ is formed.
 At mount time, the two directories given as mount options "lowerdir" and
 "upperdir" are combined into a merged directory:
 
-  mount -t overlayfs overlayfs -olowerdir=/lower,upperdir=/upper,\
+  mount -t overlay overlay -olowerdir=/lower,upperdir=/upper,\
 workdir=/work /merged
 
 The "workdir" needs to be an empty directory on the same filesystem
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6888,7 +6888,7 @@ F:	drivers/scsi/osd/
 F:	include/scsi/osd_*
 F:	fs/exofs/
 
-OVERLAYFS FILESYSTEM
+OVERLAY FILESYSTEM
 M:	Miklos Szeredi <miklos@szeredi.hu>
 L:	linux-fsdevel@vger.kernel.org
 S:	Supported
--- a/fs/Makefile
+++ b/fs/Makefile
@@ -104,7 +104,7 @@ obj-$(CONFIG_QNX6FS_FS)		+= qnx6/
 obj-$(CONFIG_AUTOFS4_FS)	+= autofs4/
 obj-$(CONFIG_ADFS_FS)		+= adfs/
 obj-$(CONFIG_FUSE_FS)		+= fuse/
-obj-$(CONFIG_OVERLAYFS_FS)	+= overlayfs/
+obj-$(CONFIG_OVERLAY_FS)	+= overlayfs/
 obj-$(CONFIG_UDF_FS)		+= udf/
 obj-$(CONFIG_SUN_OPENPROMFS)	+= openpromfs/
 obj-$(CONFIG_OMFS_FS)		+= omfs/
--- a/fs/overlayfs/Kconfig
+++ b/fs/overlayfs/Kconfig
@@ -1,4 +1,4 @@
-config OVERLAYFS_FS
+config OVERLAY_FS
 	tristate "Overlay filesystem support"
 	help
 	  An overlay filesystem combines two filesystems - an 'upper' filesystem
--- a/fs/overlayfs/Makefile
+++ b/fs/overlayfs/Makefile
@@ -2,6 +2,6 @@
 # Makefile for the overlay filesystem.
 #
 
-obj-$(CONFIG_OVERLAYFS_FS) += overlayfs.o
+obj-$(CONFIG_OVERLAY_FS) += overlay.o
 
-overlayfs-objs := super.o inode.o dir.o readdir.o copy_up.o
+overlay-objs := super.o inode.o dir.o readdir.o copy_up.o