linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH 00/35] overlayfs: stack file operations
@ 2018-04-12 15:07 Miklos Szeredi
  2018-04-12 15:07 ` [RFC PATCH 01/35] vfs: clean up dedup Miklos Szeredi
                   ` (36 more replies)
  0 siblings, 37 replies; 79+ messages in thread
From: Miklos Szeredi @ 2018-04-12 15:07 UTC (permalink / raw)
  To: linux-unionfs; +Cc: linux-fsdevel, linux-kernel

Up till now overlayfs didn't stack regular file operations.  Instead, when
a file was opened on an overlay, the file from one of the underlying layers
would be opened and any file operations performed would directly go to the
underlying file on a real filesystem.

This works well mostly, but various hacks were added to the VFS to work
around issues with this:

 - d_path() and friends
 - relatime handling
 - file locking
 - fsnotify
 - writecount handling

There are also issues that are unresolved before this patchset:

 - ioctl's that need write access but can be performed on a O_RDONLY fd
 - ro/rw inconsistency: file on lower layer opened for read-only will
   return stale data on read after copy-up and modification
 - ro/rw inconsistency for mmap: file on lower layer mapped shared will
   contain stale data after copy-up and modification

This patch series reverts the VFS hacks (with the exception of d_path) and
fixes the unresoved issues.  We need to keep d_path related hacks, because
memory maps are still not stacked, yet d_path() should keep working on
vma->vm_file->f_path.

No regressions were observed after running various test suites (xfstests,
ltp, unionmount-testsuite, pjd-fstest).

Performance impact of stacking was found to be minimal.  Memory use for
open overlay files increases by about 256bytes or 12%.

Git tree is here:

  git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs-rorw

Thanks,
Miklos
---

Miklos Szeredi (35):
  vfs: clean up dedup
  vfs: add path_open()
  vfs: optionally don't account file in nr_files
  ovl: copy up times
  ovl: copy up inode flags
  Revert "Revert "ovl: get_write_access() in truncate""
  ovl: copy up file size as well
  ovl: deal with overlay files in ovl_d_real()
  ovl: stack file ops
  ovl: add helper to return real file
  ovl: readd read_iter
  ovl: readd write_iter
  ovl: readd fsync
  ovl: readd mmap
  ovl: readd fallocate
  ovl: readd lsattr/chattr support
  ovl: readd fiemap
  ovl: readd O_DIRECT support
  ovl: readd reflink/copyfile/dedup support
  vfs: don't open real
  vfs: add f_op->pre_mmap()
  ovl: copy-up on MAP_SHARED
  vfs: simplify dentry_open()
  Revert "ovl: fix relatime for directories"
  Revert "vfs: update ovl inode before relatime check"
  Revert "ovl: fix may_write_real() for overlayfs directories"
  Revert "ovl: don't allow writing ioctl on lower layer"
  Revert "vfs: add flags to d_real()"
  Revert "vfs: do get_write_access() on upper layer of overlayfs"
  Revert "vfs: make argument of d_real_inode() const"
  Revert "vfs: add d_real_inode() helper"
  Partially revert "locks: fix file locking on overlayfs"
  Revert "fsnotify: support overlayfs"
  vfs: simplify d_op->d_real()
  ovl: fix documentation of non-standard behavior

 Documentation/filesystems/Locking       |   4 +-
 Documentation/filesystems/overlayfs.txt |  64 +++--
 Documentation/filesystems/vfs.txt       |  25 +-
 fs/file_table.c                         |  13 +-
 fs/inode.c                              |  46 +---
 fs/internal.h                           |  17 +-
 fs/ioctl.c                              |   1 +
 fs/locks.c                              |  20 +-
 fs/namei.c                              |   2 +-
 fs/namespace.c                          |  66 +----
 fs/open.c                               |  79 +++---
 fs/overlayfs/Kconfig                    |  21 ++
 fs/overlayfs/Makefile                   |   4 +-
 fs/overlayfs/dir.c                      |   5 +
 fs/overlayfs/file.c                     | 466 ++++++++++++++++++++++++++++++++
 fs/overlayfs/inode.c                    |  52 +++-
 fs/overlayfs/overlayfs.h                |  26 ++
 fs/overlayfs/ovl_entry.h                |   1 +
 fs/overlayfs/super.c                    |  82 +++---
 fs/overlayfs/util.c                     |  20 ++
 fs/read_write.c                         |  90 +++---
 fs/xattr.c                              |   9 +-
 include/linux/dcache.h                  |  41 +--
 include/linux/fs.h                      |  32 ++-
 include/linux/fsnotify.h                |  14 +-
 include/uapi/linux/fs.h                 |   1 -
 mm/util.c                               |   5 +
 27 files changed, 831 insertions(+), 375 deletions(-)
 create mode 100644 fs/overlayfs/file.c

-- 
2.14.3

^ permalink raw reply	[flat|nested] 79+ messages in thread

end of thread, other threads:[~2018-05-08 14:25 UTC | newest]

Thread overview: 79+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-12 15:07 [RFC PATCH 00/35] overlayfs: stack file operations Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 01/35] vfs: clean up dedup Miklos Szeredi
2018-04-12 16:25   ` Matthew Wilcox
2018-04-12 17:24     ` Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 02/35] vfs: add path_open() Miklos Szeredi
2018-04-12 16:38   ` Matthew Wilcox
2018-04-12 15:07 ` [RFC PATCH 03/35] vfs: optionally don't account file in nr_files Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 04/35] ovl: copy up times Miklos Szeredi
2018-04-13  8:25   ` Amir Goldstein
2018-04-13 14:23   ` Vivek Goyal
2018-04-12 15:07 ` [RFC PATCH 05/35] ovl: copy up inode flags Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 06/35] Revert "Revert "ovl: get_write_access() in truncate"" Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 07/35] ovl: copy up file size as well Miklos Szeredi
2018-04-24 18:10   ` Vivek Goyal
2018-04-12 15:07 ` [RFC PATCH 08/35] ovl: deal with overlay files in ovl_d_real() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 09/35] ovl: stack file ops Miklos Szeredi
2018-04-26 14:13   ` Vivek Goyal
2018-04-26 14:43     ` Miklos Szeredi
2018-04-26 14:56       ` Vivek Goyal
2018-04-26 15:01         ` Miklos Szeredi
2018-04-26 15:13           ` Vivek Goyal
2018-04-26 15:21             ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 10/35] ovl: add helper to return real file Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 11/35] ovl: readd read_iter Miklos Szeredi
2018-04-13 13:35   ` Amir Goldstein
2018-04-12 15:08 ` [RFC PATCH 12/35] ovl: readd write_iter Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 13/35] ovl: readd fsync Miklos Szeredi
2018-04-23 13:36   ` Vivek Goyal
2018-04-23 13:39     ` Miklos Szeredi
2018-04-23 13:53       ` Vivek Goyal
2018-04-23 14:09         ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 14/35] ovl: readd mmap Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 15/35] ovl: readd fallocate Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 16/35] ovl: readd lsattr/chattr support Miklos Szeredi
2018-04-13 14:48   ` Amir Goldstein
2018-04-17 19:51   ` Amir Goldstein
2018-04-22  8:35     ` Amir Goldstein
2018-04-22 15:18       ` Amir Goldstein
2018-04-23 10:21       ` Miklos Szeredi
2018-04-23 10:28         ` Miklos Szeredi
2018-04-23  6:11   ` Ritesh Harjani
2018-04-12 15:08 ` [RFC PATCH 17/35] ovl: readd fiemap Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 18/35] ovl: readd O_DIRECT support Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 19/35] ovl: readd reflink/copyfile/dedup support Miklos Szeredi
2018-04-17 20:31   ` Amir Goldstein
2018-04-18  8:39     ` Amir Goldstein
2018-05-03 16:04     ` Miklos Szeredi
2018-05-03 19:48       ` Amir Goldstein
2018-04-12 15:08 ` [RFC PATCH 20/35] vfs: don't open real Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 21/35] vfs: add f_op->pre_mmap() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 22/35] ovl: copy-up on MAP_SHARED Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 23/35] vfs: simplify dentry_open() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 24/35] Revert "ovl: fix relatime for directories" Miklos Szeredi
2018-04-13 14:02   ` Amir Goldstein
2018-04-13 15:55   ` Vivek Goyal
2018-04-12 15:08 ` [RFC PATCH 25/35] Revert "vfs: update ovl inode before relatime check" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 26/35] Revert "ovl: fix may_write_real() for overlayfs directories" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 27/35] Revert "ovl: don't allow writing ioctl on lower layer" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 28/35] Revert "vfs: add flags to d_real()" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 29/35] Revert "vfs: do get_write_access() on upper layer of overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 30/35] Revert "vfs: make argument of d_real_inode() const" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 31/35] Revert "vfs: add d_real_inode() helper" Miklos Szeredi
2018-04-18  8:19   ` Amir Goldstein
2018-04-18 11:42     ` Miklos Szeredi
2018-04-18 13:38       ` Steven Rostedt
2018-04-18 13:49         ` Miklos Szeredi
2018-04-18 13:56           ` Steven Rostedt
2018-04-19 19:54           ` Vivek Goyal
2018-04-20  9:14             ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 32/35] Partially revert "locks: fix file locking on overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 33/35] Revert "fsnotify: support overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 34/35] vfs: simplify d_op->d_real() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 35/35] ovl: fix documentation of non-standard behavior Miklos Szeredi
2018-04-13 11:23   ` Amir Goldstein
2018-04-25 14:49 ` [RFC PATCH 00/35] overlayfs: stack file operations J. R. Okajima
2018-04-25 19:44   ` Miklos Szeredi
2018-05-04 15:23 ` Miklos Szeredi
2018-05-05 16:37   ` Amir Goldstein
2018-05-08 14:25     ` Miklos Szeredi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).