linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: Shiyang Ruan <ruansy.fnst@fujitsu.com>
Cc: linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-nvdimm@lists.01.org, linux-fsdevel@vger.kernel.org,
	darrick.wong@oracle.com, dan.j.williams@intel.com,
	willy@infradead.org, viro@zeniv.linux.org.uk,
	david@fromorbit.com, hch@lst.de, rgoldwyn@suse.de
Subject: Re: [PATCH v6 0/7] fsdax,xfs: Add reflink&dedupe support for fsdax
Date: Tue, 25 May 2021 17:51:59 -0700	[thread overview]
Message-ID: <20210526005159.GF202144@locust> (raw)
In-Reply-To: <20210519060045.1051226-1-ruansy.fnst@fujitsu.com>

On Wed, May 19, 2021 at 02:00:38PM +0800, Shiyang Ruan wrote:
> This patchset is attempt to add CoW support for fsdax, and take XFS,
> which has both reflink and fsdax feature, as an example.

Soooo... how close are we to enabling reflink for DAX?

I <cough> got rid of the lockouts in xfs_super.c and ran a quick
fstests, which showed a number of odd regressions where dedupe tests
that were supposed to fail with EBADE didn't and a bunch of clonerange
tests failed with EINVAL:

generic/122     - output mismatch (see /var/tmp/fstests/generic/122.out.bad)
    --- tests/generic/122.out   2021-05-13 11:47:55.665860364 -0700
    +++ /var/tmp/fstests/generic/122.out.bad    2021-05-25 17:24:03.333270522 -0700
    @@ -4,7 +4,8 @@
     5e3501f97fd2669babfcbd3e1972e833  TEST_DIR/test-122/file2
     Files 1-2 do not match (intentional)
     (Fail to) dedupe the middle blocks together
    -XFS_IOC_FILE_EXTENT_SAME: Extents did not match.
    +deduped 131072/131072 bytes at offset 262144
    +128 KiB, 1 ops; 0.0000 sec (12.207 GiB/sec and 100000.0000 ops/sec)
     Compare sections
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/122.out /var/tmp/fstests/generic/122.out.bad'  to see the entire diff)
generic/136     - output mismatch (see /var/tmp/fstests/generic/136.out.bad)
    --- tests/generic/136.out   2021-05-13 11:47:55.668860355 -0700
    +++ /var/tmp/fstests/generic/136.out.bad    2021-05-25 17:24:05.773367756 -0700
    @@ -7,7 +7,8 @@
     Dedupe the last blocks together
     1->2
     1->3
    -XFS_IOC_FILE_EXTENT_SAME: Extents did not match.
    +deduped 37/37 bytes at offset 65536
    +37.000000 bytes, 1 ops; 0.0000 sec (1.960 MiB/sec and 55555.5556 ops/sec)
     c4fd505be25a0c91bcca9f502b9a8156  TEST_DIR/test-136/file1
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/136.out /var/tmp/fstests/generic/136.out.bad'  to see the entire diff)
generic/164     - output mismatch (see /var/tmp/fstests/generic/164.out.bad)
    --- tests/generic/164.out   2021-05-13 11:47:55.674860338 -0700
    +++ /var/tmp/fstests/generic/164.out.bad    2021-05-25 17:25:33.339738197 -0700
    @@ -2,4 +2,1028 @@
     Format and mount
     Initialize files
     Reflink and reread the files!
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/164.out /var/tmp/fstests/generic/164.out.bad'  to see the entire diff)
generic/165     - output mismatch (see /var/tmp/fstests/generic/165.out.bad)
    --- tests/generic/165.out   2021-05-13 11:47:55.674860338 -0700
    +++ /var/tmp/fstests/generic/165.out.bad    2021-05-25 17:25:45.247685323 -0700
    @@ -2,4 +2,1028 @@
     Format and mount
     Initialize files
     Reflink and dio reread the files!
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    +XFS_IOC_CLONE_RANGE: Invalid argument
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/165.out /var/tmp/fstests/generic/165.out.bad'  to see the entire diff)
generic/175     - output mismatch (see /var/tmp/fstests/generic/175.out.bad)
    --- tests/generic/175.out   2021-05-13 11:47:55.676860332 -0700
    +++ /var/tmp/fstests/generic/175.out.bad    2021-05-25 17:29:55.060917807 -0700
    @@ -3,3 +3,4 @@
     Create a one block file
     Create extents
     Reflink the big file
    +XFS_IOC_CLONE_RANGE: Invalid argument
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/175.out /var/tmp/fstests/generic/175.out.bad'  to see the entire diff)
generic/327     - output mismatch (see /var/tmp/fstests/generic/327.out.bad)
    --- tests/generic/327.out   2021-05-13 11:47:55.704860251 -0700
    +++ /var/tmp/fstests/generic/327.out.bad    2021-05-25 17:35:22.338448231 -0700
    @@ -7,6 +7,6 @@
     root 0 0 0
     fsgqa 2048 0 1024
     Try to reflink again
    -cp: failed to clone 'SCRATCH_MNT/test-327/file3' from 'SCRATCH_MNT/test-327/file1': Disk quota exceeded
    +cp: failed to clone 'SCRATCH_MNT/test-327/file3' from 'SCRATCH_MNT/test-327/file1': Invalid argument
     root 0 0 0
     fsgqa 2048 0 1024
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/327.out /var/tmp/fstests/generic/327.out.bad'  to see the entire diff)
generic/516     - output mismatch (see /var/tmp/fstests/generic/516.out.bad)
    --- tests/generic/516.out   2021-05-13 11:47:55.739860150 -0700
    +++ /var/tmp/fstests/generic/516.out.bad    2021-05-25 17:41:58.144177193 -0700
    @@ -4,7 +4,8 @@
     39578c21e2cb9f6049b1cf7fc7be12a6  TEST_DIR/test-516/file2
     Files 1-2 do not match (intentional)
     (partial) dedupe the middle blocks together
    -XFS_IOC_FILE_EXTENT_SAME: Extents did not match.
    +deduped XXXX/XXXX bytes at offset XXXX
    +XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
     Compare sections
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/516.out /var/tmp/fstests/generic/516.out.bad'  to see the entire diff)
generic/517     - output mismatch (see /var/tmp/fstests/generic/517.out.bad)
    --- tests/generic/517.out   2021-05-13 11:47:55.739860150 -0700
    +++ /var/tmp/fstests/generic/517.out.bad    2021-05-25 17:41:59.352000318 -0700
    @@ -33,8 +33,7 @@
     0786532
     wrote 100/100 bytes at offset 0
     XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    -deduped 100/100 bytes at offset 655360
    -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    +XFS_IOC_FILE_EXTENT_SAME: Invalid argument
     File content after second deduplication:
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/517.out /var/tmp/fstests/generic/517.out.bad'  to see the entire diff)
generic/518      1s
generic/540     - output mismatch (see /var/tmp/fstests/generic/540.out.bad)
    --- tests/generic/540.out   2021-05-13 11:47:55.743860139 -0700
    +++ /var/tmp/fstests/generic/540.out.bad    2021-05-25 17:42:01.999613949 -0700
    @@ -7,8 +7,9 @@
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-540/file3
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-540/file3.chk
     reflink across the transition
    +XFS_IOC_CLONE_RANGE: Invalid argument
     Compare files
     bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-540/file1
     5a5221017d3ab8fd7583312a14d2ba80  SCRATCH_MNT/test-540/file2
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/540.out /var/tmp/fstests/generic/540.out.bad'  to see the entire diff)
generic/541     - output mismatch (see /var/tmp/fstests/generic/541.out.bad)
    --- tests/generic/541.out   2021-05-13 11:47:55.743860139 -0700
    +++ /var/tmp/fstests/generic/541.out.bad    2021-05-25 17:42:03.623377997 -0700
    @@ -8,9 +8,10 @@
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-541/file3
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-541/file3.chk
     reflink across the transition
    +XFS_IOC_CLONE_RANGE: Invalid argument
     Compare files
     bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-541/file1
    -51a300aae3a4b4eaa023876a397e01ef  SCRATCH_MNT/test-541/file2
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/541.out /var/tmp/fstests/generic/541.out.bad'  to see the entire diff)
generic/542     - output mismatch (see /var/tmp/fstests/generic/542.out.bad)
    --- tests/generic/542.out   2021-05-13 11:47:55.743860139 -0700
    +++ /var/tmp/fstests/generic/542.out.bad    2021-05-25 17:42:05.487108030 -0700
    @@ -7,8 +7,9 @@
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-542/file3
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-542/file3.chk
     reflink across the transition
    +XFS_IOC_CLONE_RANGE: Invalid argument
     Compare files
     bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-542/file1
     5a5221017d3ab8fd7583312a14d2ba80  SCRATCH_MNT/test-542/file2
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/542.out /var/tmp/fstests/generic/542.out.bad'  to see the entire diff)
generic/543     - output mismatch (see /var/tmp/fstests/generic/543.out.bad)
    --- tests/generic/543.out   2021-05-13 11:47:55.744860136 -0700
    +++ /var/tmp/fstests/generic/543.out.bad    2021-05-25 17:42:07.386833815 -0700
    @@ -8,9 +8,10 @@
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-543/file3
     6366fd359371414186688a0ef6988893  SCRATCH_MNT/test-543/file3.chk
     reflink across the transition
    +XFS_IOC_CLONE_RANGE: Invalid argument
     Compare files
     bdbcf02ee0aa977795a79d25fcfdccb1  SCRATCH_MNT/test-543/file1
    -d93123af536c8c012f866ea383a905ce  SCRATCH_MNT/test-543/file2
    ...
    (Run 'diff -u /tmp/fstests/tests/generic/543.out /var/tmp/fstests/generic/543.out.bad'  to see the entire diff)

That's all the failures to the end of the generic group; I cut it off so
that I could schedule my regular nightly testing runs.

--D

> 
> Changes from V5:
>  - Fix the lock order of xfs_inode in xfs_mmaplock_two_inodes_and_break_dax_layout()
>  - move dax_remap_file_range_prep() to fs/dax.c
>  - change type of length to uint64_t in dax_iomap_cow_copy()
>  - fix mistake in dax_iomap_zero()
> 
> Changes from V4:
>  - Fix the mistake of breaking dax layout for two inodes
>  - Add CONFIG_FS_DAX judgement for fsdax code in remap_range.c
>  - Fix other small problems and mistakes
> 
> One of the key mechanism need to be implemented in fsdax is CoW.  Copy
> the data from srcmap before we actually write data to the destance
> iomap.  And we just copy range in which data won't be changed.
> 
> Another mechanism is range comparison.  In page cache case, readpage()
> is used to load data on disk to page cache in order to be able to
> compare data.  In fsdax case, readpage() does not work.  So, we need
> another compare data with direct access support.
> 
> With the two mechanisms implemented in fsdax, we are able to make reflink
> and fsdax work together in XFS.
> 
> Some of the patches are picked up from Goldwyn's patchset.  I made some
> changes to adapt to this patchset.
> 
> 
> (Rebased on v5.13-rc2 and patchset[1])
> [1]: https://lkml.org/lkml/2021/4/22/575
> 
> Shiyang Ruan (7):
>   fsdax: Introduce dax_iomap_cow_copy()
>   fsdax: Replace mmap entry in case of CoW
>   fsdax: Add dax_iomap_cow_copy() for dax_iomap_zero
>   iomap: Introduce iomap_apply2() for operations on two files
>   fsdax: Dedup file range to use a compare function
>   fs/xfs: Handle CoW for fsdax write() path
>   fs/xfs: Add dax dedupe support
> 
>  fs/dax.c               | 216 ++++++++++++++++++++++++++++++++++++-----
>  fs/iomap/apply.c       |  52 ++++++++++
>  fs/iomap/buffered-io.c |   2 +-
>  fs/remap_range.c       |  36 +++++--
>  fs/xfs/xfs_bmap_util.c |   3 +-
>  fs/xfs/xfs_file.c      |  11 +--
>  fs/xfs/xfs_inode.c     |  57 +++++++++++
>  fs/xfs/xfs_inode.h     |   1 +
>  fs/xfs/xfs_iomap.c     |  38 +++++++-
>  fs/xfs/xfs_iomap.h     |  24 +++++
>  fs/xfs/xfs_iops.c      |   7 +-
>  fs/xfs/xfs_reflink.c   |  15 +--
>  include/linux/dax.h    |  11 ++-
>  include/linux/fs.h     |  12 ++-
>  include/linux/iomap.h  |   7 +-
>  15 files changed, 431 insertions(+), 61 deletions(-)
> 
> -- 
> 2.31.1
> 
> 
> 

      parent reply	other threads:[~2021-05-26  0:52 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-19  6:00 [PATCH v6 0/7] fsdax,xfs: Add reflink&dedupe support for fsdax Shiyang Ruan
2021-05-19  6:00 ` [PATCH v6 1/7] fsdax: Introduce dax_iomap_cow_copy() Shiyang Ruan
2021-05-19  6:00 ` [PATCH v6 2/7] fsdax: Replace mmap entry in case of CoW Shiyang Ruan
2021-05-19  6:00 ` [PATCH v6 3/7] fsdax: Add dax_iomap_cow_copy() for dax_iomap_zero Shiyang Ruan
2021-05-25 22:17   ` Darrick J. Wong
2021-05-19  6:00 ` [PATCH v6 4/7] iomap: Introduce iomap_apply2() for operations on two files Shiyang Ruan
2021-05-19  6:00 ` [PATCH v6 5/7] fsdax: Dedup file range to use a compare function Shiyang Ruan
2021-05-25 23:29   ` Darrick J. Wong
2021-05-19  6:00 ` [PATCH v6 6/7] fs/xfs: Handle CoW for fsdax write() path Shiyang Ruan
2021-05-26  0:21   ` Darrick J. Wong
2021-06-09  2:28     ` ruansy.fnst
2021-06-15  7:21       ` [PATCH v6.1 " Shiyang Ruan
2021-06-24  8:49         ` ruansy.fnst
2021-06-25 22:18           ` Darrick J. Wong
2021-06-28  2:55             ` ruansy.fnst
2021-06-28  5:09               ` Darrick J. Wong
2021-06-29 11:25                 ` ruansy.fnst
2021-06-29 21:01                   ` Darrick J. Wong
2021-07-08 23:16                 ` Dave Chinner
2021-07-09 12:36               ` [PATCH v6.2 6/7] dax: Introduce dax_iomap_ops for end of reflink Shiyang Ruan
2021-05-19  6:00 ` [PATCH v6 7/7] fs/xfs: Add dax dedupe support Shiyang Ruan
2021-05-26  0:31   ` Darrick J. Wong
2021-05-26  0:51 ` Darrick J. Wong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210526005159.GF202144@locust \
    --to=djwong@kernel.org \
    --cc=dan.j.williams@intel.com \
    --cc=darrick.wong@oracle.com \
    --cc=david@fromorbit.com \
    --cc=hch@lst.de \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvdimm@lists.01.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=rgoldwyn@suse.de \
    --cc=ruansy.fnst@fujitsu.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).