From: Miklos Szeredi <mszeredi@redhat.com>
To: linux-unionfs@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH 00/35] overlayfs: stack file operations
Date: Thu, 12 Apr 2018 17:07:51 +0200 [thread overview]
Message-ID: <20180412150826.20988-1-mszeredi@redhat.com> (raw)
Up till now overlayfs didn't stack regular file operations. Instead, when
a file was opened on an overlay, the file from one of the underlying layers
would be opened and any file operations performed would directly go to the
underlying file on a real filesystem.
This works well mostly, but various hacks were added to the VFS to work
around issues with this:
- d_path() and friends
- relatime handling
- file locking
- fsnotify
- writecount handling
There are also issues that are unresolved before this patchset:
- ioctl's that need write access but can be performed on a O_RDONLY fd
- ro/rw inconsistency: file on lower layer opened for read-only will
return stale data on read after copy-up and modification
- ro/rw inconsistency for mmap: file on lower layer mapped shared will
contain stale data after copy-up and modification
This patch series reverts the VFS hacks (with the exception of d_path) and
fixes the unresoved issues. We need to keep d_path related hacks, because
memory maps are still not stacked, yet d_path() should keep working on
vma->vm_file->f_path.
No regressions were observed after running various test suites (xfstests,
ltp, unionmount-testsuite, pjd-fstest).
Performance impact of stacking was found to be minimal. Memory use for
open overlay files increases by about 256bytes or 12%.
Git tree is here:
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs-rorw
Thanks,
Miklos
---
Miklos Szeredi (35):
vfs: clean up dedup
vfs: add path_open()
vfs: optionally don't account file in nr_files
ovl: copy up times
ovl: copy up inode flags
Revert "Revert "ovl: get_write_access() in truncate""
ovl: copy up file size as well
ovl: deal with overlay files in ovl_d_real()
ovl: stack file ops
ovl: add helper to return real file
ovl: readd read_iter
ovl: readd write_iter
ovl: readd fsync
ovl: readd mmap
ovl: readd fallocate
ovl: readd lsattr/chattr support
ovl: readd fiemap
ovl: readd O_DIRECT support
ovl: readd reflink/copyfile/dedup support
vfs: don't open real
vfs: add f_op->pre_mmap()
ovl: copy-up on MAP_SHARED
vfs: simplify dentry_open()
Revert "ovl: fix relatime for directories"
Revert "vfs: update ovl inode before relatime check"
Revert "ovl: fix may_write_real() for overlayfs directories"
Revert "ovl: don't allow writing ioctl on lower layer"
Revert "vfs: add flags to d_real()"
Revert "vfs: do get_write_access() on upper layer of overlayfs"
Revert "vfs: make argument of d_real_inode() const"
Revert "vfs: add d_real_inode() helper"
Partially revert "locks: fix file locking on overlayfs"
Revert "fsnotify: support overlayfs"
vfs: simplify d_op->d_real()
ovl: fix documentation of non-standard behavior
Documentation/filesystems/Locking | 4 +-
Documentation/filesystems/overlayfs.txt | 64 +++--
Documentation/filesystems/vfs.txt | 25 +-
fs/file_table.c | 13 +-
fs/inode.c | 46 +---
fs/internal.h | 17 +-
fs/ioctl.c | 1 +
fs/locks.c | 20 +-
fs/namei.c | 2 +-
fs/namespace.c | 66 +----
fs/open.c | 79 +++---
fs/overlayfs/Kconfig | 21 ++
fs/overlayfs/Makefile | 4 +-
fs/overlayfs/dir.c | 5 +
fs/overlayfs/file.c | 466 ++++++++++++++++++++++++++++++++
fs/overlayfs/inode.c | 52 +++-
fs/overlayfs/overlayfs.h | 26 ++
fs/overlayfs/ovl_entry.h | 1 +
fs/overlayfs/super.c | 82 +++---
fs/overlayfs/util.c | 20 ++
fs/read_write.c | 90 +++---
fs/xattr.c | 9 +-
include/linux/dcache.h | 41 +--
include/linux/fs.h | 32 ++-
include/linux/fsnotify.h | 14 +-
include/uapi/linux/fs.h | 1 -
mm/util.c | 5 +
27 files changed, 831 insertions(+), 375 deletions(-)
create mode 100644 fs/overlayfs/file.c
--
2.14.3
next reply other threads:[~2018-04-12 15:08 UTC|newest]
Thread overview: 79+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-12 15:07 Miklos Szeredi [this message]
2018-04-12 15:07 ` [RFC PATCH 01/35] vfs: clean up dedup Miklos Szeredi
2018-04-12 16:25 ` Matthew Wilcox
2018-04-12 17:24 ` Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 02/35] vfs: add path_open() Miklos Szeredi
2018-04-12 16:38 ` Matthew Wilcox
2018-04-12 15:07 ` [RFC PATCH 03/35] vfs: optionally don't account file in nr_files Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 04/35] ovl: copy up times Miklos Szeredi
2018-04-13 8:25 ` Amir Goldstein
2018-04-13 14:23 ` Vivek Goyal
2018-04-12 15:07 ` [RFC PATCH 05/35] ovl: copy up inode flags Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 06/35] Revert "Revert "ovl: get_write_access() in truncate"" Miklos Szeredi
2018-04-12 15:07 ` [RFC PATCH 07/35] ovl: copy up file size as well Miklos Szeredi
2018-04-24 18:10 ` Vivek Goyal
2018-04-12 15:07 ` [RFC PATCH 08/35] ovl: deal with overlay files in ovl_d_real() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 09/35] ovl: stack file ops Miklos Szeredi
2018-04-26 14:13 ` Vivek Goyal
2018-04-26 14:43 ` Miklos Szeredi
2018-04-26 14:56 ` Vivek Goyal
2018-04-26 15:01 ` Miklos Szeredi
2018-04-26 15:13 ` Vivek Goyal
2018-04-26 15:21 ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 10/35] ovl: add helper to return real file Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 11/35] ovl: readd read_iter Miklos Szeredi
2018-04-13 13:35 ` Amir Goldstein
2018-04-12 15:08 ` [RFC PATCH 12/35] ovl: readd write_iter Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 13/35] ovl: readd fsync Miklos Szeredi
2018-04-23 13:36 ` Vivek Goyal
2018-04-23 13:39 ` Miklos Szeredi
2018-04-23 13:53 ` Vivek Goyal
2018-04-23 14:09 ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 14/35] ovl: readd mmap Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 15/35] ovl: readd fallocate Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 16/35] ovl: readd lsattr/chattr support Miklos Szeredi
2018-04-13 14:48 ` Amir Goldstein
2018-04-17 19:51 ` Amir Goldstein
2018-04-22 8:35 ` Amir Goldstein
2018-04-22 15:18 ` Amir Goldstein
2018-04-23 10:21 ` Miklos Szeredi
2018-04-23 10:28 ` Miklos Szeredi
2018-04-23 6:11 ` Ritesh Harjani
2018-04-12 15:08 ` [RFC PATCH 17/35] ovl: readd fiemap Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 18/35] ovl: readd O_DIRECT support Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 19/35] ovl: readd reflink/copyfile/dedup support Miklos Szeredi
2018-04-17 20:31 ` Amir Goldstein
2018-04-18 8:39 ` Amir Goldstein
2018-05-03 16:04 ` Miklos Szeredi
2018-05-03 19:48 ` Amir Goldstein
2018-04-12 15:08 ` [RFC PATCH 20/35] vfs: don't open real Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 21/35] vfs: add f_op->pre_mmap() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 22/35] ovl: copy-up on MAP_SHARED Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 23/35] vfs: simplify dentry_open() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 24/35] Revert "ovl: fix relatime for directories" Miklos Szeredi
2018-04-13 14:02 ` Amir Goldstein
2018-04-13 15:55 ` Vivek Goyal
2018-04-12 15:08 ` [RFC PATCH 25/35] Revert "vfs: update ovl inode before relatime check" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 26/35] Revert "ovl: fix may_write_real() for overlayfs directories" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 27/35] Revert "ovl: don't allow writing ioctl on lower layer" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 28/35] Revert "vfs: add flags to d_real()" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 29/35] Revert "vfs: do get_write_access() on upper layer of overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 30/35] Revert "vfs: make argument of d_real_inode() const" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 31/35] Revert "vfs: add d_real_inode() helper" Miklos Szeredi
2018-04-18 8:19 ` Amir Goldstein
2018-04-18 11:42 ` Miklos Szeredi
2018-04-18 13:38 ` Steven Rostedt
2018-04-18 13:49 ` Miklos Szeredi
2018-04-18 13:56 ` Steven Rostedt
2018-04-19 19:54 ` Vivek Goyal
2018-04-20 9:14 ` Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 32/35] Partially revert "locks: fix file locking on overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 33/35] Revert "fsnotify: support overlayfs" Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 34/35] vfs: simplify d_op->d_real() Miklos Szeredi
2018-04-12 15:08 ` [RFC PATCH 35/35] ovl: fix documentation of non-standard behavior Miklos Szeredi
2018-04-13 11:23 ` Amir Goldstein
2018-04-25 14:49 ` [RFC PATCH 00/35] overlayfs: stack file operations J. R. Okajima
2018-04-25 19:44 ` Miklos Szeredi
2018-05-04 15:23 ` Miklos Szeredi
2018-05-05 16:37 ` Amir Goldstein
2018-05-08 14:25 ` Miklos Szeredi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180412150826.20988-1-mszeredi@redhat.com \
--to=mszeredi@redhat.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-unionfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).