From: David Howells <dhowells@redhat.com>
To: Dave Chinner <david@fromorbit.com>
Cc: dhowells@redhat.com, Amir Goldstein <amir73il@gmail.com>,
linux-cachefs@redhat.com, Jeff Layton <jlayton@redhat.com>,
David Wysochanski <dwysocha@redhat.com>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
"J. Bruce Fields" <bfields@fieldses.org>,
Christoph Hellwig <hch@infradead.org>,
Dave Chinner <dchinner@redhat.com>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-afs@lists.infradead.org,
Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
CIFS <linux-cifs@vger.kernel.org>,
ceph-devel <ceph-devel@vger.kernel.org>,
v9fs-developer@lists.sourceforge.net,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
linux-kernel <linux-kernel@vger.kernel.org>,
Miklos Szeredi <miklos@szeredi.hu>
Subject: Re: fscache: Redesigning the on-disk cache
Date: Tue, 09 Mar 2021 09:21:45 +0000 [thread overview]
Message-ID: <152281.1615281705@warthog.procyon.org.uk> (raw)
In-Reply-To: <20210308215535.GA63242@dread.disaster.area>
Dave Chinner <david@fromorbit.com> wrote:
> > > With ->fiemap() you can at least make the distinction between a non
> > > existing and an UNWRITTEN extent.
> >
> > I can't use that for XFS, Ext4 or btrfs, I suspect. Christoph and Dave's
> > assertion is that the cache can't rely on the backing filesystem's metadata
> > because these can arbitrarily insert or remove blocks of zeros to bridge or
> > split extents.
>
> Well, that's not the big problem. The issue that makes FIEMAP
> unusable for determining if there is user data present in a file is
> that on-disk extent maps aren't exactly coherent with in-memory user
> data state.
>
> That is, we can have a hole on disk with delalloc user data in
> memory. There's user data in the file, just not on disk. Same goes
> for unwritten extents - there can be dirty data in memory over an
> unwritten extent, and it won't get converted to written until the
> data is written back and the filesystem runs a conversion
> transaction.
>
> So, yeah, if you use FIEMAP to determine where data lies in a file
> that is being actively modified, you're going get corrupt data
> sooner rather than later. SEEK_HOLE/DATA are coherent with in
> memory user data, so don't have this problem.
I thought you and/or Christoph said it *was* a problem to use the backing
filesystem's metadata to track presence of data in the cache because the
filesystem (or its tools) can arbitrarily insert blocks of zeros to
bridge/break up extents.
If that is the case, then that is a big problem, and SEEK_HOLE/DATA won't
suffice.
If it's not a problem - maybe if I can set a mark on a file to tell the
filesystem and tools not to do that - then that would obviate the need for me
to store my own maps.
David
prev parent reply other threads:[~2021-03-09 9:22 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-03 23:20 fscache: Redesigning the on-disk cache David Howells
2021-03-04 13:47 ` fscache: Redesigning the on-disk cache - LRU handling David Howells
2021-03-05 9:46 ` fscache: Redesigning the on-disk cache Amir Goldstein
2021-03-08 9:13 ` David Howells
2021-03-08 10:35 ` Amir Goldstein
2021-03-08 11:28 ` Metadata writtenback notification? -- was " David Howells
2021-03-08 22:32 ` Dave Chinner
2021-03-08 23:20 ` Matthew Wilcox
2021-03-09 11:27 ` David Howells
2021-03-08 18:54 ` J. Bruce Fields
2021-03-08 19:08 ` David Howells
2021-03-08 21:55 ` Dave Chinner
2021-03-09 9:21 ` David Howells [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=152281.1615281705@warthog.procyon.org.uk \
--to=dhowells@redhat.com \
--cc=amir73il@gmail.com \
--cc=bfields@fieldses.org \
--cc=ceph-devel@vger.kernel.org \
--cc=david@fromorbit.com \
--cc=dchinner@redhat.com \
--cc=dwysocha@redhat.com \
--cc=hch@infradead.org \
--cc=jlayton@redhat.com \
--cc=linux-afs@lists.infradead.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=v9fs-developer@lists.sourceforge.net \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).