linux-unionfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sargun Dhillon <sargun@sargun.me>
To: linux-unionfs@vger.kernel.org, miklos@szeredi.hu,
	Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Sargun Dhillon <sargun@sargun.me>,
	Giuseppe Scrivano <gscrivan@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>,
	Daniel J Walsh <dwalsh@redhat.com>,
	David Howells <dhowells@redhat.com>
Subject: [RFC PATCH 0/3] Make overlayfs volatile mounts reusable
Date: Sun, 15 Nov 2020 20:57:55 -0800	[thread overview]
Message-ID: <20201116045758.21774-1-sargun@sargun.me> (raw)

The volatile option is great for "ephemeral" containers. Unfortunately,
it doesn't capture all uses. There are two ways to use it safely right now:

1. Throw away the entire upperdir between mounts
2. Manually syncfs between mounts

For certain use-cases like serverless, or short-lived containers, it is
advantageous to be able to stop the container (runtime) and start it up on
demand / invocation of the function. Usually, there is some bootstrap
process which involves downloading some artifacts, or putting secrets on
disk, and then upon invocation of the function, you want to (re)start the
container.

If you have to syncfs every time you do this, it can lead to excess
filesystem overhead for all of the other containers on the machine, and
stall out every container who's upperdir is on the same underlying
filesystem, unless your filesystem offers something like subvolumes,
and if sync can be restricted to a subvolume.

The kernel has information that it can use to determine whether or not this
is safe -- primarily if the underlying FS has had writeback errors or not.
Overlayfs doesn't delay writes, so the consistency of the upperdir is not
contingent on the mount of overlayfs, but rather the mount of the
underlying filesystem. It can also make sure the underlying filesystem
wasn't remounted. Although, it was suggested that we use derive this
information from the upperdir's inode[1], we can checkpoint this data on
disk in an xattr.

Specifically we checkpoint:
  * Superblock "id": This is a new concept introduced in one of the patches
    which keeps track of (re)mounts of filesystems, by having a per boot
    monotonically increasing integer identifying the superblock. This is
    safer than trying to obfuscate the pointer and putting it into an
    xattr (due to leak risk, and address reuse), and during the course
    of a boot, the u64 should not wrap.
  * Overlay "boot id": This is a new UUID that is overlayfs specific,
    as overlayfs is a module that's independent from the rest of the
    system and can be (re)loaded independently -- thus it generates
    a UUID at load time which can be used to uniquely identify it.
  * upperdir / workdir errseq: A sample of the errseq_t on the workdir /
    upperdir's superblock. Since the errseq_t is implemented as a u32
    with errno + error counter, we can safely store it in a checkpoint.
    

[1]: https://lore.kernel.org/linux-unionfs/CAOQ4uxhadzC3-kh-igfxv3pAmC3ocDtAQTxByu4hrn8KtZuieQ@mail.gmail.com/

Sargun Dhillon (3):
  fs: Add s_instance_id field to superblock for unique identification
  overlay: Add ovl_do_getxattr helper
  overlay: Add the ability to remount volatile directories when safe

 Documentation/filesystems/overlayfs.rst |  5 +-
 fs/overlayfs/overlayfs.h                | 43 +++++++++++++
 fs/overlayfs/readdir.c                  | 86 +++++++++++++++++++++++--
 fs/overlayfs/super.c                    | 22 ++++++-
 fs/super.c                              |  3 +
 include/linux/fs.h                      |  7 ++
 include/uapi/linux/fs.h                 |  2 +
 7 files changed, 160 insertions(+), 8 deletions(-)

-- 
2.25.1


             reply	other threads:[~2020-11-16  4:58 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-16  4:57 Sargun Dhillon [this message]
2020-11-16  4:57 ` [RFC PATCH 1/3] fs: Add s_instance_id field to superblock for unique identification Sargun Dhillon
2020-11-16  5:07   ` Sargun Dhillon
2020-11-16  4:57 ` [RFC PATCH 2/3] overlay: Add ovl_do_getxattr helper Sargun Dhillon
2020-11-16 11:00   ` Amir Goldstein
2020-11-16  4:57 ` [RFC PATCH 3/3] overlay: Add the ability to remount volatile directories when safe Sargun Dhillon
2020-11-16  9:31   ` Amir Goldstein
2020-11-16 10:30     ` Sargun Dhillon
2020-11-16 11:17       ` Amir Goldstein
2020-11-16 12:52         ` Amir Goldstein
2020-11-16 14:42   ` Vivek Goyal
2020-11-16 14:45     ` Vivek Goyal
2020-11-16 15:20     ` Amir Goldstein
2020-11-16 16:36       ` Vivek Goyal
2020-11-16 18:25         ` Sargun Dhillon
2020-11-16 19:27           ` Vivek Goyal
2020-11-16 20:18         ` Amir Goldstein
2020-11-16 21:09           ` Vivek Goyal
2020-11-17  5:33             ` Amir Goldstein
2020-11-17 14:48               ` Vivek Goyal
2020-11-17 15:24                 ` Amir Goldstein
2020-11-17 15:40                   ` Vivek Goyal
2020-11-17 16:46                   ` Vivek Goyal
2020-11-17 18:03                     ` Amir Goldstein
2020-11-17 18:29                       ` Vivek Goyal
2020-11-18  7:24                         ` Amir Goldstein
2020-11-18  8:27                           ` Sargun Dhillon
2020-11-18 10:46                             ` Amir Goldstein
2020-11-18 14:55                           ` Vivek Goyal
2020-11-16 21:26           ` Vivek Goyal
2020-11-16 22:14             ` Sargun Dhillon
2020-11-17  5:41               ` Amir Goldstein
2020-11-17 17:05               ` Vivek Goyal
2020-11-16 17:38     ` Sargun Dhillon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201116045758.21774-1-sargun@sargun.me \
    --to=sargun@sargun.me \
    --cc=dhowells@redhat.com \
    --cc=dwalsh@redhat.com \
    --cc=gscrivan@redhat.com \
    --cc=linux-unionfs@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=vgoyal@redhat.com \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).