Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: david@fromorbit.com, darrick.wong@oracle.com
Cc: sandeen@redhat.com, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org, linux-unionfs@vger.kernel.org,
	linux-xfs@vger.kernel.org, linux-mm@kvack.org,
	linux-btrfs@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	ocfs2-devel@oss.oracle.com
Subject: [PATCH v5 00/26] fs: fixes for serious clone/dedupe problems
Date: Mon, 15 Oct 2018 20:09:55 -0700
Message-ID: <153965939489.1256.7400115244528045860.stgit@magnolia> (raw)

Hi all,

Dave, Eric, and I have been chasing a stale data exposure bug in the XFS
reflink implementation, and tracked it down to reflink forgetting to do
some of the file-extending activities that must happen for regular
writes.

We then started auditing the clone, dedupe, and copyfile code and
realized that from a file contents perspective, clonerange isn't any
different from a regular file write.  Unfortunately, we also noticed
that *unlike* a regular write, clonerange skips a ton of overflow
checks, such as validating the ranges against s_maxbytes, MAX_NON_LFS,
and RLIMIT_FSIZE.  We also observed that cloning into a file did not
strip security privileges (suid, capabilities) like a regular write
would.  I also noticed that xfs and ocfs2 need to dump the page cache
before remapping blocks, not after.

In fixing the range checking problems I also realized that both dedupe
and copyfile tell userspace how much of the requested operation was
acted upon.  Since the range validation can shorten a clone request (or
we can ENOSPC midway through), we might as well plumb the short
operation reporting back through the VFS indirection code to userspace.

So, here's the whole giant pile of patches[1] that fix all the problems.
This branch is against current upstream (4.19-rc8).  The patch
"generic: test reflink side effects" recently sent to fstests exercises
the fixes in this series.  Tests are in [2].

--D

[1] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=djwong-devel
[2] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=djwong-devel

             reply index

Thread overview: 39+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-16  3:09 Darrick J. Wong [this message]
2018-10-16  3:10 ` [PATCH 01/26] xfs: add a per-xfs trace_printk macro Darrick J. Wong
2018-10-16  3:10 ` [PATCH 02/26] vfs: vfs_clone_file_prep_inodes should return EINVAL for a clone from beyond EOF Darrick J. Wong
2018-10-16  3:10 ` [PATCH 03/26] vfs: check file ranges before cloning files Darrick J. Wong
2018-10-16  3:10 ` [PATCH 04/26] vfs: exit early from zero length remap operations Darrick J. Wong
2018-10-17  8:26   ` Christoph Hellwig
2018-10-16  3:10 ` [PATCH 05/26] vfs: strengthen checking of file range inputs to generic_remap_checks Darrick J. Wong
2018-10-16  3:10 ` [PATCH 06/26] vfs: avoid problematic remapping requests into partial EOF block Darrick J. Wong
2018-10-16  3:10 ` [PATCH 07/26] vfs: skip zero-length dedupe requests Darrick J. Wong
2018-10-16  3:10 ` [PATCH 08/26] vfs: rename vfs_clone_file_prep to be more descriptive Darrick J. Wong
2018-10-16  3:10 ` [PATCH 09/26] vfs: rename clone_verify_area to remap_verify_area Darrick J. Wong
2018-10-16  3:11 ` [PATCH 10/26] vfs: combine the clone and dedupe into a single remap_file_range Darrick J. Wong
2018-10-17  8:28   ` Christoph Hellwig
2018-10-16  3:11 ` [PATCH 11/26] vfs: pass remap flags to generic_remap_file_range_prep Darrick J. Wong
2018-10-17  8:28   ` Christoph Hellwig
2018-10-16  3:11 ` [PATCH 12/26] vfs: pass remap flags to generic_remap_checks Darrick J. Wong
2018-10-17  8:29   ` Christoph Hellwig
2018-10-16  3:19 ` [PATCH 13/26] vfs: create generic_remap_file_range_touch to update inode metadata Darrick J. Wong
2018-10-17  8:33   ` Christoph Hellwig
2018-10-16  3:19 ` [PATCH 14/26] vfs: make remap_file_range functions take and return bytes completed Darrick J. Wong
2018-10-16  3:19 ` [PATCH 15/26] vfs: plumb remap flags through the vfs clone functions Darrick J. Wong
2018-10-16  3:19 ` [PATCH 16/26] vfs: plumb remap flags through the vfs dedupe functions Darrick J. Wong
2018-10-16  3:19 ` [PATCH 17/26] vfs: enable remap callers that can handle short operations Darrick J. Wong
2018-10-17  8:36   ` Christoph Hellwig
2018-10-17 16:35     ` Darrick J. Wong
2018-10-16  3:20 ` [PATCH 18/26] vfs: hide file range comparison function Darrick J. Wong
2018-10-16  3:20 ` [PATCH 19/26] vfs: clean up generic_remap_file_range_prep return value Darrick J. Wong
2018-10-17  8:37   ` Christoph Hellwig
2018-10-16  3:20 ` [PATCH 20/26] ocfs2: truncate page cache for clone destination file before remapping Darrick J. Wong
2018-10-16  3:20 ` [PATCH 21/26] ocfs2: fix pagecache truncation prior to reflink Darrick J. Wong
2018-10-16  3:20 ` [PATCH 22/26] ocfs2: support partial clone range and dedupe range Darrick J. Wong
2018-10-16  3:20 ` [PATCH 23/26] ocfs2: remove ocfs2_reflink_remap_range Darrick J. Wong
2018-10-16  3:20 ` [PATCH 24/26] xfs: fix pagecache truncation prior to reflink Darrick J. Wong
2018-10-17  8:37   ` Christoph Hellwig
2018-10-16  3:20 ` [PATCH 25/26] xfs: support returning partial reflink results Darrick J. Wong
2018-10-17  8:40   ` Christoph Hellwig
2018-10-17 16:40     ` Darrick J. Wong
2018-10-16  3:21 ` [PATCH 26/26] xfs: remove redundant remap partial EOF block checks Darrick J. Wong
2018-10-17  8:40   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=153965939489.1256.7400115244528045860.stgit@magnolia \
    --to=darrick.wong@oracle.com \
    --cc=david@fromorbit.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-unionfs@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=ocfs2-devel@oss.oracle.com \
    --cc=sandeen@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org
	public-inbox-index linux-btrfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git