From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: Brian Foster <bfoster@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>,
linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: Re: [PATCH 23/24] iomap: add support for sub-pagesize buffered I/O without buffer heads
Date: Wed, 20 Jun 2018 12:02:30 -0700 [thread overview]
Message-ID: <20180620190230.GB4838@magnolia> (raw)
In-Reply-To: <20180620181259.GD4493@bfoster>
On Wed, Jun 20, 2018 at 02:12:59PM -0400, Brian Foster wrote:
> On Wed, Jun 20, 2018 at 09:08:03AM -0700, Darrick J. Wong wrote:
> > On Wed, Jun 20, 2018 at 10:32:53AM -0400, Brian Foster wrote:
> > > Sending again without the attachment... Christoph, let me know if it
> > > didn't hit your mbox at least.
> > >
> > > On Wed, Jun 20, 2018 at 09:56:55AM +0200, Christoph Hellwig wrote:
> > > > On Tue, Jun 19, 2018 at 12:52:11PM -0400, Brian Foster wrote:
> > > > > > + /*
> > > > > > + * Move the caller beyond our range so that it keeps making progress.
> > > > > > + * For that we have to include any leading non-uptodate ranges, but
> > > > >
> > > > > Do you mean "leading uptodate ranges" here? E.g., pos is pushed forward
> > > > > past those ranges we don't have to read, so (pos - orig_pos) reflects
> > > > > the initial uptodate range while plen reflects the length we have to
> > > > > read..?
> > > >
> > > > Yes.
> > > >
> > > > > > +
> > > > > > + do {
> > > > >
> > > > > Kind of a nit, but this catches my eye and manages to confuse me every
> > > > > time I look at it. A comment along the lines of:
> > > > >
> > > > > /*
> > > > > * Pass in the block aligned start/end so we get back block
> > > > > * aligned/adjusted poff/plen and can compare with unaligned
> > > > > * from/to below.
> > > > > */
> > > > >
> > > > > ... would be nice here, IMO.
> > > >
> > > > Fine with me.
> > > >
> > > > > > + iomap_adjust_read_range(inode, iop, &block_start,
> > > > > > + block_end - block_start, &poff, &plen);
> > > > > > + if (plen == 0)
> > > > > > + break;
> > > > > > +
> > > > > > + if ((from > poff && from < poff + plen) ||
> > > > > > + (to > poff && to < poff + plen)) {
> > > > > > + status = iomap_read_page_sync(inode, block_start, page,
> > > > > > + poff, plen, from, to, iomap);
> > > > >
> > > > > After taking another look at the buffer head path, it does look like we
> > > > > have slightly different behavior here. IIUC, the former reads only the
> > > > > !uptodate blocks that fall along the from/to boundaries. Here, if say
> > > > > from = 1, to = PAGE_SIZE and the page is fully !uptodate, it looks like
> > > > > we'd read the entire page worth of blocks (assuming contiguous 512b
> > > > > blocks, for example). Intentional? Doesn't seem like a big deal, but
> > > > > could be worth a followup fix.
> > > >
> > > > It wasn't actuall intentional, but I actually think it is the right thing
> > > > in then end, as it means we'll often do a single read instead of two
> > > > separate ones.
> > >
> > > Ok, but if that's the argument, then shouldn't we not be doing two
> > > separate I/Os if the middle range of a write happens to be already
> > > uptodate? Or more for that matter, if the page happens to be sparsely
> > > uptodate for whatever reason..?
> > >
> > > OTOH, I also do wonder a bit whether that may always be the right thing
> > > if we consider cases like 64k page size arches and whatnot. It seems
> > > like we could end up consuming more bandwidth for reads than we
> > > typically have in the past. That said, unless there's a functional
> > > reason to change this I think it's fine to optimize this path for these
> > > kinds of corner cases in follow on patches.
> > >
> > > Finally, this survived xfstests on a sub-page block size fs but I
> > > managed to hit an fsx error:
> > >
> > > Mapped Read: non-zero data past EOF (0x21a1f) page offset 0xc00 is
> > > 0xc769
> > >
> > > It repeats 100% of the time for me using the attached fsxops file (with
> > > --replay-ops) on XFS w/ -bsize=1k. It doesn't occur without the final
> > > patch to enable sub-page block iomap on XFS.
> >
> > Funny, because I saw the exact same complaint from generic/127 last
> > night on my development tree that doesn't include hch's patches and was
> > going to see if I could figure out what's going on.
> >
> > FWIW it's been happening sporadically for a few weeks now but every time
> > I've tried to analyze it I (of course) couldn't get it to reproduce. :)
> >
> > I also ran this series (all of it, including the subpagesize config)
> > last night and aside from it stumbling over an unrelated locking problem
> > seemed fine....
> >
>
> That's interesting. Perhaps it's a pre-existing issue in that case and
> the iomap stuff just changes the timing to make it reliably reproducible
> on this particular system.
>
> I only ran it a handful of times in both cases and now have lost access
> to the server. Once I regain access, I'll try running for longer on
> for-next to see if the same thing eventually triggers.
I managed to cut the testcase down to a nine-line fsx script and so
turned it into a fstests regression case. It seems to reproduce 100% on
scsi disks and doesn't at all on pmem.
Note that changing the second to last line of the fsxops script to call
punch_hole instead of zero_range triggers it too.
I've also narrowed it down to something going wrong w.r.t. handling the
page cache somewhere under xfs_free_file_space.
(See attached diff...)
--D
generic: mread past eof shows nonzero contents
Certain sequences of generic/127 invocations complain about being able
to mread nonzero contents past eof. Replicate that here as a regression
test.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
tests/generic/708 | 54 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/708.out | 2 ++
tests/generic/group | 1 +
3 files changed, 57 insertions(+)
create mode 100755 tests/generic/708
create mode 100644 tests/generic/708.out
diff --git a/tests/generic/708 b/tests/generic/708
new file mode 100755
index 00000000..fa5584f5
--- /dev/null
+++ b/tests/generic/708
@@ -0,0 +1,54 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2018 Oracle. All Rights Reserved.
+#
+# FS QA Test No. 708
+#
+# Test a specific sequence of fsx operations that causes an mmap read past
+# eof to return nonzero contents.
+#
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+
+# real QA test starts here
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+cat >> $tmp.fsxops << ENDL
+fallocate 0x77e2 0x5f06 0x269a2 keep_size
+mapwrite 0x2e7fc 0x42ba 0x3f989
+write 0x67a9 0x714e 0x3f989
+write 0x39f96 0x185a 0x3f989
+collapse_range 0x36000 0x8000 0x3f989
+mapread 0x74c0 0x1bb3 0x3e2d0
+truncate 0x0 0x8aa2 0x3e2d0
+zero_range 0x1265 0x783d 0x8aa2
+mapread 0x7bd8 0xeca 0x8aa2
+ENDL
+
+victim=$SCRATCH_MNT/a
+touch $victim
+$here/ltp/fsx --replay-ops $tmp.fsxops $victim > $tmp.output || cat $tmp.output
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/generic/708.out b/tests/generic/708.out
new file mode 100644
index 00000000..33c478ad
--- /dev/null
+++ b/tests/generic/708.out
@@ -0,0 +1,2 @@
+QA output created by 708
+Silence is golden
diff --git a/tests/generic/group b/tests/generic/group
index 83a6fdab..1a1a0a6e 100644
--- a/tests/generic/group
+++ b/tests/generic/group
@@ -501,3 +501,4 @@
496 auto quick swap
497 auto quick swap collapse
498 auto quick log
+708 auto quick rw collapse
next prev parent reply other threads:[~2018-06-20 19:02 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-15 13:01 stop using buffer heads in xfs v6 Christoph Hellwig
2018-06-15 13:01 ` [PATCH 01/24] iomap: add an iomap-based readpage and readpages implementation Christoph Hellwig
2018-06-29 14:44 ` [PATCH] iomap: Add inline data support to iomap_readpage_actor Andreas Gruenbacher
2018-07-01 6:21 ` Christoph Hellwig
2018-07-01 21:43 ` Andreas Gruenbacher
2018-07-02 12:52 ` Christoph Hellwig
2018-07-02 15:05 ` Andreas Gruenbacher
2018-06-15 13:01 ` [PATCH 02/24] xfs: use iomap for blocksize == PAGE_SIZE readpage and readpages Christoph Hellwig
2018-06-15 13:01 ` [PATCH 03/24] iomap: add initial support for writes without buffer heads Christoph Hellwig
2018-06-15 13:01 ` [PATCH 04/24] xfs: simplify xfs_bmap_punch_delalloc_range Christoph Hellwig
2018-06-15 13:01 ` [PATCH 05/24] xfs: simplify xfs_aops_discard_page Christoph Hellwig
2018-06-15 13:01 ` [PATCH 06/24] xfs: move locking into xfs_bmap_punch_delalloc_range Christoph Hellwig
2018-06-19 5:26 ` Darrick J. Wong
2018-06-15 13:01 ` [PATCH 07/24] xfs: do not set the page uptodate in xfs_writepage_map Christoph Hellwig
2018-06-15 13:01 ` [PATCH 08/24] xfs: don't clear imap_valid for a non-uptodate buffers Christoph Hellwig
2018-06-15 13:01 ` [PATCH 09/24] xfs: don't use XFS_BMAPI_IGSTATE in xfs_map_blocks Christoph Hellwig
2018-06-19 5:27 ` Darrick J. Wong
2018-06-15 13:01 ` [PATCH 10/24] xfs: remove xfs_reflink_trim_irec_to_next_cow Christoph Hellwig
2018-06-19 5:30 ` Darrick J. Wong
2018-06-15 13:01 ` [PATCH 11/24] xfs: remove xfs_map_cow Christoph Hellwig
2018-06-18 17:38 ` Brian Foster
2018-06-19 5:35 ` Darrick J. Wong
2018-06-19 16:53 ` Christoph Hellwig
2018-06-20 0:37 ` Darrick J. Wong
2018-06-15 13:01 ` [PATCH 12/24] xfs: rename the offset variable in xfs_writepage_map Christoph Hellwig
2018-06-19 5:37 ` Darrick J. Wong
2018-06-15 13:01 ` [PATCH 13/24] xfs: make xfs_writepage_map extent map centric Christoph Hellwig
2018-06-18 17:38 ` Brian Foster
2018-06-19 5:43 ` Darrick J. Wong
2018-06-19 16:52 ` Christoph Hellwig
2018-06-15 13:01 ` [PATCH 14/24] xfs: remove the now unused XFS_BMAPI_IGSTATE flag Christoph Hellwig
2018-06-15 13:02 ` [PATCH 15/24] xfs: remove xfs_reflink_find_cow_mapping Christoph Hellwig
2018-06-15 13:02 ` [PATCH 16/24] xfs: simplify xfs_map_blocks by using xfs_iext_lookup_extent directly Christoph Hellwig
2018-06-15 13:02 ` [PATCH 17/24] xfs: remove the imap_valid flag Christoph Hellwig
2018-06-15 13:02 ` [PATCH 18/24] xfs: don't look at buffer heads in xfs_add_to_ioend Christoph Hellwig
2018-06-15 13:02 ` [PATCH 19/24] xfs: move all writeback buffer_head manipulation into xfs_map_at_offset Christoph Hellwig
2018-06-15 13:02 ` [PATCH 20/24] xfs: remove xfs_start_page_writeback Christoph Hellwig
2018-06-15 13:02 ` [PATCH 21/24] xfs: refactor the tail of xfs_writepage_map Christoph Hellwig
2018-06-15 13:02 ` [PATCH 22/24] xfs: allow writeback on pages without buffer heads Christoph Hellwig
2018-06-15 13:02 ` [PATCH 23/24] iomap: add support for sub-pagesize buffered I/O " Christoph Hellwig
2018-06-19 16:52 ` Brian Foster
2018-06-20 7:56 ` Christoph Hellwig
2018-06-20 14:32 ` Brian Foster
2018-06-20 16:08 ` Darrick J. Wong
2018-06-20 18:12 ` Brian Foster
2018-06-20 19:02 ` Darrick J. Wong [this message]
2018-06-21 8:46 ` Christoph Hellwig
2018-06-23 13:06 ` Brian Foster
2018-06-29 15:59 ` Christoph Hellwig
2018-07-02 12:50 ` Christoph Hellwig
2018-07-02 18:16 ` Brian Foster
2018-06-21 7:53 ` Christoph Hellwig
2018-06-15 13:02 ` [PATCH 24/24] xfs: add support for sub-pagesize writeback without buffer_heads Christoph Hellwig
2018-06-19 6:15 ` Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180620190230.GB4838@magnolia \
--to=darrick.wong@oracle.com \
--cc=bfoster@redhat.com \
--cc=hch@lst.de \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).