All of
 help / color / mirror / Atom feed
From: "Theodore Y. Ts'o" <>
To: David Howells <>
Subject: Re: Problems with determining data presence by examining extents?
Date: Tue, 14 Jan 2020 17:49:17 -0500	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Tue, Jan 14, 2020 at 04:48:29PM +0000, David Howells wrote:
> Again with regard to my rewrite of fscache and cachefiles:
> I've got rid of my use of bmap()!  Hooray!
> However, I'm informed that I can't trust the extent map of a backing file to
> tell me accurately whether content exists in a file because:
>  (a) Not-quite-contiguous extents may be joined by insertion of blocks of
>      zeros by the filesystem optimising itself.  This would give me a false
>      positive when trying to detect the presence of data.
>  (b) Blocks of zeros that I write into the file may get punched out by
>      filesystem optimisation since a read back would be expected to read zeros
>      there anyway, provided it's below the EOF.  This would give me a false
>      negative.
> Is there some setting I can use to prevent these scenarios on a file - or can
> one be added?

I don't think there's any way to do this in a portable way, at least
today.  There is a hack we could be use that would work for ext4
today, at least with respect to (a), but I'm not sure we would want to
make any guarantees with respect to (b).

I suspect I understand why you want this; I've fielded some requests
for people wanting to do something very like this at $WORK, for what I
assume to be for the same reason you're seeking to do this; to create
do incremental caching of files and letting the file system track what
has and hasn't been cached yet.

If we were going to add such a facility, what we could perhaps do is
to define a new flag indicating that a particular file should have no
extent mapping optimization applied, such that FIEMAP would return a
mapping if and only if userspace had written to a particular block, or
had requested that a block be preallocated using fallocate().  The
flag could only be set on a zero-length file, and this might disable
certain advanced file system features, such as reflink, at the file
system's discretion; and there might be unspecified performance
impacts if this flag is set on a file.

File systems which do not support this feature would not allow this
flag to be set.

				- Ted

  reply	other threads:[~2020-01-14 22:51 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-14 16:48 Problems with determining data presence by examining extents? David Howells
2020-01-14 22:49 ` Theodore Y. Ts'o [this message]
2020-01-15  3:54 ` Qu Wenruo
2020-01-15 12:46   ` Andreas Dilger
2020-01-15 13:10     ` Qu Wenruo
2020-01-15 13:31       ` Christoph Hellwig
2020-01-15 19:48         ` Andreas Dilger
2020-01-16 10:16           ` Christoph Hellwig
2020-01-15 20:55         ` David Howells
2020-01-15 22:11           ` Andreas Dilger
2020-01-15 23:09           ` David Howells
2020-01-26 18:19             ` Zygo Blaxell
2020-01-15 14:35       ` David Howells
2020-01-15 14:48         ` Christoph Hellwig
2020-01-15 14:59         ` David Howells
2020-01-16 10:13           ` Christoph Hellwig
2020-01-17 16:43           ` David Howells
2020-01-15 14:20   ` David Howells
2020-01-15  8:38 ` Christoph Hellwig
2020-01-15 13:50 ` David Howells
2020-01-15 14:05 ` David Howells
2020-01-15 14:24   ` Qu Wenruo
2020-01-15 14:50   ` David Howells
2020-01-15 14:15 ` David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.