From: Dave Chinner <david@fromorbit.com> To: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: linux-btrfs@vger.kernel.org, fstests@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Date: Tue, 9 Feb 2016 18:21:06 +1100 [thread overview] Message-ID: <20160209072106.GH19486@dastard> (raw) In-Reply-To: <20160209011145.23099.95354.stgit@birch.djwong.org> On Mon, Feb 08, 2016 at 05:11:45PM -0800, Darrick J. Wong wrote: > Happy New Year! > > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > you'd like to pull. Can you include the commit ID I should see at the head of the tree so I can confirm I'm pulling the right branch? BTW, git doesn't like this: https://github.com/djwong/xfstests/tree/for-dave What git really wants is the tree url with a separate branch name like so: https://github.com/djwong/xfstests.git for-dave (i.e. the typical output from a git request-pull command) > This is a (no longer) small patch set against the reflink/dedupe test > cases in xfstests. The first four patches fix errors in the existing > reflink tests, some of which are from Christoph Hellwig. > > Patches 5-6 refactor the dmerror code so that we can use it to > simulate transient IO errors, then use this code to test that > unwritten extent conversion does NOT happen after a directio write to > an unwritten extent hits a disk error. Due to a bug in the VFS > directio code, ext4 can disclose stale disk contents if an aio dio > write fails; XFS suffers this problem for any failing dio write to an > unwritten extent. Christoph's kernel patchset titled "vfs/xfs: > directio updates to ease COW handling V2" (and a separate ext4 warning > cleanup) is needed to fix this. > > Patches 7-9, 13, 15, 17, 18, 20, 21, and 23 exercise various parts > of the copy on write behavior that are necessary to support shared > blocks. The earlier patches focus on correct CoW behavior in the > presence of IO errors during the copy-write, and the later patches > focus on XFS' new cow-extent-size hint that greatly reduces > fragmentation due to copy on write behavior by encouraging the > allocator to allocate larger extents of replacement blocks. > > Patches 10-12 and 14 perform stress testing on reflink and CoW to > check the behaviors when we get close to maximum refcount, when we > specify obnxiously large offsets and lengths, and when we try to > reflink millions of extents at a time. > > Patch 16 tests quota accounting behavior when reflink is enabled. > > Patch 19 adds a few tests for the XFS reverse mapping btree to ensure > that things like metadump and growfs work correctly. > > Patch 22 checks that get_bmapx and fiemap (on XFS) correctly flag > extents as having shared blocks. XFS now follows btrfs and ocfs2 > FIEMAP behavior such that if any blocks of a file's extent are shared, > the whole extent is marked shared. This is in contrast to earlier > XFS-only behavior that reported shared and non-shared regions as > separate extents. This may change - xfs_bmap doesn't combine extents in it's output even if they are adjacent. For debugging purposes (which is what xfs_bmap/fiemap is for), it's much better to be able to see the exact extent layout and block sharing. I suspect the solution of least surprise is to make fiemap behave like the other filesystems, and make xfs_bmap behave in a manner that is useful to us.... :P > If you're going to start using this mess, you probably ought to just > pull from my github trees for kernel[1], xfsprogs[2], xfstests[3], > xfs-docs[4], and man-pages[5]. All tests should pass on XFS. I > tried btrfs this weekend and it failed 166, 175, 182, 266, 271, 272, > 278, 281, 297, 298, 304, 333, and 334. ocfs2 (when I jury-rigged it > to run the cp_reflink tests) seemed to have a quota bug and crashes > hard in 284 (but was otherwise fine). Fun fun fun. I'll look through the patchs, and if there's nothing major I'll pull it in once I get a commit ID from you. Cheers, Dave. -- Dave Chinner david@fromorbit.com
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com> To: "Darrick J. Wong" <darrick.wong@oracle.com> Cc: fstests@vger.kernel.org, linux-btrfs@vger.kernel.org, xfs@oss.sgi.com Subject: Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Date: Tue, 9 Feb 2016 18:21:06 +1100 [thread overview] Message-ID: <20160209072106.GH19486@dastard> (raw) In-Reply-To: <20160209011145.23099.95354.stgit@birch.djwong.org> On Mon, Feb 08, 2016 at 05:11:45PM -0800, Darrick J. Wong wrote: > Happy New Year! > > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > you'd like to pull. Can you include the commit ID I should see at the head of the tree so I can confirm I'm pulling the right branch? BTW, git doesn't like this: https://github.com/djwong/xfstests/tree/for-dave What git really wants is the tree url with a separate branch name like so: https://github.com/djwong/xfstests.git for-dave (i.e. the typical output from a git request-pull command) > This is a (no longer) small patch set against the reflink/dedupe test > cases in xfstests. The first four patches fix errors in the existing > reflink tests, some of which are from Christoph Hellwig. > > Patches 5-6 refactor the dmerror code so that we can use it to > simulate transient IO errors, then use this code to test that > unwritten extent conversion does NOT happen after a directio write to > an unwritten extent hits a disk error. Due to a bug in the VFS > directio code, ext4 can disclose stale disk contents if an aio dio > write fails; XFS suffers this problem for any failing dio write to an > unwritten extent. Christoph's kernel patchset titled "vfs/xfs: > directio updates to ease COW handling V2" (and a separate ext4 warning > cleanup) is needed to fix this. > > Patches 7-9, 13, 15, 17, 18, 20, 21, and 23 exercise various parts > of the copy on write behavior that are necessary to support shared > blocks. The earlier patches focus on correct CoW behavior in the > presence of IO errors during the copy-write, and the later patches > focus on XFS' new cow-extent-size hint that greatly reduces > fragmentation due to copy on write behavior by encouraging the > allocator to allocate larger extents of replacement blocks. > > Patches 10-12 and 14 perform stress testing on reflink and CoW to > check the behaviors when we get close to maximum refcount, when we > specify obnxiously large offsets and lengths, and when we try to > reflink millions of extents at a time. > > Patch 16 tests quota accounting behavior when reflink is enabled. > > Patch 19 adds a few tests for the XFS reverse mapping btree to ensure > that things like metadump and growfs work correctly. > > Patch 22 checks that get_bmapx and fiemap (on XFS) correctly flag > extents as having shared blocks. XFS now follows btrfs and ocfs2 > FIEMAP behavior such that if any blocks of a file's extent are shared, > the whole extent is marked shared. This is in contrast to earlier > XFS-only behavior that reported shared and non-shared regions as > separate extents. This may change - xfs_bmap doesn't combine extents in it's output even if they are adjacent. For debugging purposes (which is what xfs_bmap/fiemap is for), it's much better to be able to see the exact extent layout and block sharing. I suspect the solution of least surprise is to make fiemap behave like the other filesystems, and make xfs_bmap behave in a manner that is useful to us.... :P > If you're going to start using this mess, you probably ought to just > pull from my github trees for kernel[1], xfsprogs[2], xfstests[3], > xfs-docs[4], and man-pages[5]. All tests should pass on XFS. I > tried btrfs this weekend and it failed 166, 175, 182, 266, 271, 272, > 278, 281, 297, 298, 304, 333, and 334. ocfs2 (when I jury-rigged it > to run the cp_reflink tests) seemed to have a quota bug and crashes > hard in 284 (but was otherwise fine). Fun fun fun. I'll look through the patchs, and if there's nothing major I'll pull it in once I get a commit ID from you. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2016-02-09 7:21 UTC|newest] Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-02-09 1:11 [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Darrick J. Wong 2016-02-09 1:11 ` Darrick J. Wong 2016-02-09 1:11 ` [PATCH 01/23] generic/182: this is a dedupe test, check for dedupe Darrick J. Wong 2016-02-09 1:11 ` Darrick J. Wong 2016-02-09 1:11 ` [PATCH 02/23] xfstests: filter whitespace in 128 and 132 Darrick J. Wong 2016-02-09 1:11 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 03/23] xfstests: make _scratch_mkfs_blocksized usable Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 04/23] reflink: remove redundant filesystem checks from the end of the tests Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 05/23] common/dmerror: add some more dmerror routines Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 06/23] dio unwritten conversion bug tests Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 7:37 ` Dave Chinner 2016-02-09 7:37 ` Dave Chinner 2016-02-09 8:08 ` Darrick J. Wong 2016-02-09 8:08 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 07/23] reflink: test intersecting CoW and falloc/fpunch/fzero/fcollapse/finsert/ftrunc Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 08/23] reflink: test CoW behavior with IO errors Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 09/23] reflink: test CoW operations against the source file Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 10/23] xfs: more reflink tests Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 7:36 ` Dave Chinner 2016-02-09 7:36 ` Dave Chinner 2016-02-09 8:16 ` Darrick J. Wong 2016-02-09 8:16 ` Darrick J. Wong 2016-02-09 1:12 ` [PATCH 11/23] reflink: ensure that we can handle reflinking a lot of extents Darrick J. Wong 2016-02-09 1:12 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 12/23] xfs/122: support refcount/rmap data structures Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 7:43 ` Dave Chinner 2016-02-09 7:43 ` Dave Chinner 2016-02-09 7:55 ` Darrick J. Wong 2016-02-09 7:55 ` Darrick J. Wong 2016-02-09 8:53 ` Dave Chinner 2016-02-09 8:53 ` Dave Chinner 2016-02-09 1:13 ` [PATCH 13/23] xfs: test fragmentation characteristics of copy-on-write Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 8:01 ` Dave Chinner 2016-02-09 8:01 ` Dave Chinner 2016-02-10 1:02 ` Darrick J. Wong 2016-02-10 1:02 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 14/23] reflink: high offset reflink and dedupe tests Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 15/23] reflink: test xfs cow behavior when the filesystem crashes Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 16/23] reflink: test quota accounting Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 17/23] reflink: test CoW across a mixed range of block types with cowextsize set Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 8:09 ` Dave Chinner 2016-02-09 8:09 ` Dave Chinner 2016-02-10 1:03 ` Darrick J. Wong 2016-02-10 1:03 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 18/23] xfs: test the automatic cowextsize extent garbage collector Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 8:15 ` Dave Chinner 2016-02-09 8:15 ` Dave Chinner 2016-02-10 1:06 ` Darrick J. Wong 2016-02-10 1:06 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 19/23] xfs: test rmapbt functionality Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 8:26 ` Dave Chinner 2016-02-09 8:26 ` Dave Chinner 2016-02-10 1:07 ` Darrick J. Wong 2016-02-10 1:07 ` Darrick J. Wong 2016-02-09 1:13 ` [PATCH 20/23] reflink: test aio copy on write Darrick J. Wong 2016-02-09 1:13 ` Darrick J. Wong 2016-02-09 1:14 ` [PATCH 21/23] xfs: aio cow tests Darrick J. Wong 2016-02-09 1:14 ` Darrick J. Wong 2016-02-09 8:32 ` Dave Chinner 2016-02-09 8:32 ` Dave Chinner 2016-02-09 21:51 ` Darrick J. Wong 2016-02-09 21:51 ` Darrick J. Wong 2016-02-09 1:14 ` [PATCH 22/23] xfs: test xfs_getbmapx behavior with shared extents Darrick J. Wong 2016-02-09 1:14 ` Darrick J. Wong 2016-02-09 1:14 ` [PATCH 23/23] reflink: test reflink+cow+enospc all at the same time Darrick J. Wong 2016-02-09 1:14 ` Darrick J. Wong 2016-02-09 7:21 ` Dave Chinner [this message] 2016-02-09 7:21 ` [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls Dave Chinner 2016-02-09 7:25 ` Darrick J. Wong 2016-02-09 7:25 ` Darrick J. Wong 2016-02-09 7:50 ` Darrick J. Wong 2016-02-09 7:50 ` Darrick J. Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20160209072106.GH19486@dastard \ --to=david@fromorbit.com \ --cc=darrick.wong@oracle.com \ --cc=fstests@vger.kernel.org \ --cc=linux-btrfs@vger.kernel.org \ --cc=xfs@oss.sgi.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.