linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Miklos Szeredi <mszeredi@redhat.com>
To: linux-unionfs@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH 00/39] overlayfs: stack file operations
Date: Tue, 29 May 2018 16:43:00 +0200	[thread overview]
Message-ID: <20180529144339.16538-1-mszeredi@redhat.com> (raw)

Up till now overlayfs didn't stack regular file operations.  Instead, when
a file was opened on an overlay, the file from one of the underlying layers
would be opened and any file operations performed would directly go to the
underlying file on a real filesystem.

This works well mostly, but various hacks were added to the VFS to work
around issues with this:

 - d_path() and friends
 - relatime handling
 - file locking
 - fsnotify
 - writecount handling

There are also issues that are unresolved before this patchset:

 - ioctl's that need write access but can be performed on a O_RDONLY fd
 - ro/rw inconsistency: file on lower layer opened for read-only will
   return stale data on read after copy-up and modification
 - ro/rw inconsistency for mmap: file on lower layer mapped shared will
   contain stale data after copy-up and modification

This patch series reverts the VFS hacks (with the exception of d_path) and
fixes the unresoved issues.  We need to keep d_path related hacks, because
memory maps are still not stacked, yet d_path() should keep working on
vma->vm_file->f_path.

No regressions were observed after running various test suites (xfstests,
ltp, unionmount-testsuite, pjd-fstest).

Performance impact of stacking was found to be minimal.  Memory use for
open overlay files increases by about 256bytes or 12%.

Git tree is here:

  git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs-rorw

---
Miklos Szeredi (39):
  vfs: dedpue: return loff_t
  vfs: dedupe: rationalize args
  vfs: dedupe: extract helper for a single dedup
  vfs: add path_open()
  vfs: optionally don't account file in nr_files
  vfs: add f_op->pre_mmap()
  vfs: export vfs_ioctl() to modules
  vfs: export vfs_dedupe_file_range_one() to modules
  ovl: copy up times
  ovl: copy up inode flags
  Revert "Revert "ovl: get_write_access() in truncate""
  ovl: copy up file size as well
  ovl: deal with overlay files in ovl_d_real()
  ovl: stack file ops
  ovl: add helper to return real file
  ovl: add ovl_read_iter()
  ovl: add ovl_write_iter()
  ovl: add ovl_fsync()
  ovl: add ovl_mmap()
  ovl: add ovl_fallocate()
  ovl: add lsattr/chattr support
  ovl: add ovl_fiemap()
  ovl: add O_DIRECT support
  ovl: add reflink/copyfile/dedup support
  vfs: don't open real
  ovl: copy-up on MAP_SHARED
  ovl: obsolete "check_copy_up" module option
  ovl: fix documentation of non-standard behavior
  vfs: simplify dentry_open()
  Revert "ovl: fix may_write_real() for overlayfs directories"
  Revert "ovl: don't allow writing ioctl on lower layer"
  vfs: fix freeze protection in mnt_want_write_file() for overlayfs
  Revert "ovl: fix relatime for directories"
  Revert "vfs: update ovl inode before relatime check"
  Revert "vfs: add flags to d_real()"
  Revert "vfs: do get_write_access() on upper layer of overlayfs"
  Partially revert "locks: fix file locking on overlayfs"
  Revert "fsnotify: support overlayfs"
  vfs: remove open_flags from d_real()

 Documentation/filesystems/Locking       |   4 +-
 Documentation/filesystems/overlayfs.txt |  60 ++--
 Documentation/filesystems/vfs.txt       |  19 +-
 fs/btrfs/ctree.h                        |   5 +-
 fs/btrfs/ioctl.c                        |   7 +-
 fs/file_table.c                         |  13 +-
 fs/inode.c                              |  46 +--
 fs/internal.h                           |  17 +-
 fs/ioctl.c                              |   1 +
 fs/locks.c                              |  20 +-
 fs/namei.c                              |   2 +-
 fs/namespace.c                          |  69 +----
 fs/ocfs2/file.c                         |  10 +-
 fs/open.c                               |  74 ++---
 fs/overlayfs/Kconfig                    |  21 ++
 fs/overlayfs/Makefile                   |   4 +-
 fs/overlayfs/copy_up.c                  |  30 +-
 fs/overlayfs/dir.c                      |  31 +-
 fs/overlayfs/file.c                     | 509 ++++++++++++++++++++++++++++++++
 fs/overlayfs/inode.c                    |  63 +++-
 fs/overlayfs/overlayfs.h                |  21 +-
 fs/overlayfs/ovl_entry.h                |   1 +
 fs/overlayfs/super.c                    |  65 ++--
 fs/overlayfs/util.c                     |  11 +-
 fs/read_write.c                         |  91 +++---
 fs/xattr.c                              |   9 +-
 fs/xfs/xfs_file.c                       |   8 +-
 include/linux/dcache.h                  |  15 +-
 include/linux/fs.h                      |  32 +-
 include/linux/fsnotify.h                |  14 +-
 include/uapi/linux/fs.h                 |   1 -
 mm/util.c                               |   5 +
 32 files changed, 898 insertions(+), 380 deletions(-)
 create mode 100644 fs/overlayfs/file.c

-- 
2.14.3

             reply	other threads:[~2018-05-29 14:43 UTC|newest]

Thread overview: 83+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-29 14:43 Miklos Szeredi [this message]
2018-05-29 14:43 ` [PATCH 01/39] vfs: dedpue: return loff_t Miklos Szeredi
2018-06-04  8:43   ` Christoph Hellwig
2018-06-05  8:33     ` Miklos Szeredi
2018-06-06 15:09       ` Darrick J. Wong
2018-06-18 20:08         ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 02/39] vfs: dedupe: rationalize args Miklos Szeredi
2018-06-06 15:02   ` Darrick J. Wong
2018-05-29 14:43 ` [PATCH 03/39] vfs: dedupe: extract helper for a single dedup Miklos Szeredi
2018-05-29 15:41   ` Amir Goldstein
2018-05-29 16:04     ` Amir Goldstein
2018-06-04  8:44   ` Christoph Hellwig
2018-05-29 14:43 ` [PATCH 04/39] vfs: add path_open() Miklos Szeredi
2018-06-04  8:46   ` Christoph Hellwig
2018-06-10  4:36     ` Al Viro
2018-05-29 14:43 ` [PATCH 05/39] vfs: optionally don't account file in nr_files Miklos Szeredi
2018-06-04  8:47   ` Christoph Hellwig
2018-06-04  8:57     ` Miklos Szeredi
2018-06-10  4:41   ` Al Viro
2018-05-29 14:43 ` [PATCH 06/39] vfs: add f_op->pre_mmap() Miklos Szeredi
2018-06-04  8:48   ` Christoph Hellwig
2018-06-05 11:36     ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 07/39] vfs: export vfs_ioctl() to modules Miklos Szeredi
2018-06-04  8:49   ` Christoph Hellwig
2018-06-10  4:57     ` Al Viro
2018-06-11  7:19       ` Miklos Szeredi
2018-06-11 16:24         ` Christoph Hellwig
2018-06-19 14:04           ` Miklos Szeredi
2018-06-19 14:24             ` Christoph Hellwig
2018-06-19 14:34               ` Miklos Szeredi
2018-06-19 14:54                 ` Al Viro
2018-05-29 14:43 ` [PATCH 08/39] vfs: export vfs_dedupe_file_range_one() " Miklos Szeredi
2018-05-29 14:43 ` [PATCH 09/39] ovl: copy up times Miklos Szeredi
2018-05-29 14:43 ` [PATCH 10/39] ovl: copy up inode flags Miklos Szeredi
2018-05-29 14:43 ` [PATCH 11/39] Revert "Revert "ovl: get_write_access() in truncate"" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 12/39] ovl: copy up file size as well Miklos Szeredi
2018-05-29 14:43 ` [PATCH 13/39] ovl: deal with overlay files in ovl_d_real() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 14/39] ovl: stack file ops Miklos Szeredi
2018-06-10  4:13   ` Al Viro
2018-06-11  7:09     ` Miklos Szeredi
2018-06-12  2:29       ` Al Viro
2018-06-12  2:40         ` Al Viro
2018-06-12  9:24           ` Miklos Szeredi
2018-06-12 18:24             ` Al Viro
2018-06-12 18:31               ` Al Viro
2018-06-13  9:21                 ` Miklos Szeredi
2018-06-15  5:47                   ` Al Viro
2018-06-18 11:50                     ` Miklos Szeredi
2018-06-13 11:56               ` J. R. Okajima
2018-05-29 14:43 ` [PATCH 15/39] ovl: add helper to return real file Miklos Szeredi
2018-06-10  5:42   ` Al Viro
2018-06-11  8:11     ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 16/39] ovl: add ovl_read_iter() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 17/39] ovl: add ovl_write_iter() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 18/39] ovl: add ovl_fsync() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 19/39] ovl: add ovl_mmap() Miklos Szeredi
2018-06-10  5:24   ` Al Viro
2018-06-11  7:58     ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 20/39] ovl: add ovl_fallocate() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 21/39] ovl: add lsattr/chattr support Miklos Szeredi
2018-05-29 14:43 ` [PATCH 22/39] ovl: add ovl_fiemap() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 23/39] ovl: add O_DIRECT support Miklos Szeredi
2018-06-10  5:31   ` Al Viro
2018-06-11  8:08     ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 24/39] ovl: add reflink/copyfile/dedup support Miklos Szeredi
2018-05-29 14:43 ` [PATCH 25/39] vfs: don't open real Miklos Szeredi
2018-05-29 14:43 ` [PATCH 26/39] ovl: copy-up on MAP_SHARED Miklos Szeredi
2018-05-29 14:43 ` [PATCH 27/39] ovl: obsolete "check_copy_up" module option Miklos Szeredi
2018-05-29 15:13   ` Amir Goldstein
2018-05-30  8:26     ` Miklos Szeredi
2018-05-29 14:43 ` [PATCH 28/39] ovl: fix documentation of non-standard behavior Miklos Szeredi
2018-05-29 14:43 ` [PATCH 29/39] vfs: simplify dentry_open() Miklos Szeredi
2018-05-29 14:43 ` [PATCH 30/39] Revert "ovl: fix may_write_real() for overlayfs directories" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 31/39] Revert "ovl: don't allow writing ioctl on lower layer" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 32/39] vfs: fix freeze protection in mnt_want_write_file() for overlayfs Miklos Szeredi
2018-06-04  8:50   ` Christoph Hellwig
2018-05-29 14:43 ` [PATCH 33/39] Revert "ovl: fix relatime for directories" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 34/39] Revert "vfs: update ovl inode before relatime check" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 35/39] Revert "vfs: add flags to d_real()" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 36/39] Revert "vfs: do get_write_access() on upper layer of overlayfs" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 37/39] Partially revert "locks: fix file locking on overlayfs" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 38/39] Revert "fsnotify: support overlayfs" Miklos Szeredi
2018-05-29 14:43 ` [PATCH 39/39] vfs: remove open_flags from d_real() Miklos Szeredi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180529144339.16538-1-mszeredi@redhat.com \
    --to=mszeredi@redhat.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-unionfs@vger.kernel.org \
    --subject='Re: [PATCH 00/39] overlayfs: stack file operations' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).