archive mirror
 help / color / mirror / Atom feed
From: Vivek Goyal <>
To: James Bottomley <>
Cc: Christian Brauner <>,,
	Miklos Szeredi <>,
	"zhangyi (F)" <>,
	Netdev <>,
	overlayfs <>,,
	LSM List <>,,
	linux-fsdevel <>
Subject: Re: Overlayfs @ Containers and checkpoint/restart micro-conference at LPC2018
Date: Tue, 11 Sep 2018 11:36:30 -0400	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Tue, Sep 11, 2018 at 08:13:40AM -0700, James Bottomley wrote:
> On Tue, 2018-09-11 at 09:52 -0400, Vivek Goyal wrote:
> > On Sun, Sep 09, 2018 at 11:18:54AM +0200, Christian Brauner wrote:
> > [..]
> > > My team hast just started to be more involved with shifts
> > > development a few months back. Overlayfs is definitely an
> > > inspiration and we even once thought about making shifts an
> > > extension of overlayfs. Seth Forshee on my team is currently
> > > actively working on shifts and getting a POC ready.
> > > When he has a POC based on James' patchset there will be an RFC
> > > that will go to fsdevel and all parties of interest.
> > > There will also be an update on shifts development during the
> > > microconf. So even more reason for developers from overlayfs to
> > > stop by.
> > 
> > So we need both shiftfs and overlayfs in container deployments,
> > right?
> Well, no; only docker style containers need some form of overlay graph
> driver, but even there it doesn't have to be the overlayfs one.  When I
> build unprivileged containers, I never use overlays so for me having to
> use it will be problematic as it would be even in docker for the non-
> overlayfs graph drivers.

Hi James,

Ok. For us, overlayfs is now default for docker containers as it was
much faster as compared to devicemapper and vfs (due to page cache
sharing). So please keep in mind overlayfs graph driver use case 
as well while designing a solution.

For non docker containers, I am assuming all the image is in one directory
so no union is required. Also these probably are read-only containers
or this image directory is not shared with other containers for it to

> Perhaps we should consider this when we look at the use cases.
> > shiftfs to make sure each container can run in its own user namespace
> > and uid/gid mappings can be setup on the fly and overlayfs to provide
> > union of multiple layers and copy on write filesystem. I am assuming
> > that shiftfs is working on top of overlayfs here?
> > 
> > Doing shifting at VFS level using mount API was another idea
> > discussed at last plumbers. I saw David Howells was pushing all the
> > new mount API patches. Not sure if he ever got time to pursue
> > shifting at VFS level.
> I wasn't party to the conversation, but when I discussed it with Ted
> (who wants something similar for a feature changing bind mount) we need
> the entire VFS api to be struct path based instead of dentry/inode
> based.  That's the way it's going, but we'd need to get to the end
> point so we have a struct vfsmnt available for every VFS call.

Ok, thanks. So mappings will be per mount and available in vfsmnt and
hence pass around path so that one can get to vfsmnt (instead of
dentry/inode). Makes sense.

> > BTW, now we have metadata only copy up patches in overlayfs as
> > well(4.19-rc). That speeds up chown operation with overlayfs,
> > needed for changing ownership of files in images for making sure
> > they work fine with user namespaces. In my simple testing in a VM,
> > a fedora image was taking around 30 seconds to chown. With metadata
> > only copy up that time drops to around 2-3 seconds. So till shiftfs
> > or shiting at VFS level gets merged, it can be used as a stop gap
> > solution.
> Most of the snapshot based filesystem (btrfs, xfs) do this without any
> need for overlayfs.

Right. But they don't share page cache yet (same with devicemapper). So
till we get page cache sharing in these file systems, overlayfs still
has the advantage of being able to launch many more containers using
same image with smaller memory requirements (and its faster too as image
does not have to be read from disk).


  reply	other threads:[~2018-09-11 20:36 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-13 16:10 Containers and checkpoint/restart micro-conference at LPC2018 Stéphane Graber
2018-09-08  4:59 ` Stéphane Graber
2018-09-08  7:41   ` Amir Goldstein
2018-09-09  1:31     ` Christian Brauner
2018-09-09  6:31       ` Overlayfs @ " Amir Goldstein
2018-09-09  9:18         ` Christian Brauner
2018-09-11 13:52           ` Vivek Goyal
2018-09-11 15:13             ` James Bottomley
2018-09-11 15:36               ` Vivek Goyal [this message]
2018-09-09 19:08     ` [lxc-users] " Lucas Oketch
2018-09-09 15:30 ` James Bottomley
2018-09-09 17:38   ` Steve French

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).