Linux-BTRFS Archive on lore.kernel.org
 help / Atom feed
From: Christoph Anton Mitterer <calestyo@scientia.net>
To: "Austin S. Hemmelgarn" <ahferroin7@gmail.com>,
	Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Cc: linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7
Date: Thu, 14 Mar 2019 19:58:56 +0100
Message-ID: <6c388ca5944d8e679e66d2d889e594f6fc3ef68b.camel@scientia.net> (raw)
In-Reply-To: <b701dc28-a703-7e7c-2ced-aad07e70850a@gmail.com>

On Fri, 2019-03-08 at 07:20 -0500, Austin S. Hemmelgarn wrote:
> On 2019-03-07 15:07, Zygo Blaxell wrote:
> > Legacy POSIX doesn't have the hole-punching concept, so legacy
> > tools won't do it; however, people add features to GNU tools all
> > the
> > time, so it's hard to be 100% sure without downloading the code and
> > reading/auditing/scanning it.  I'm 99% sure cp and tar are OK.
> > 
> They are, the only things they do with sparse files are creating new 
> ones from scratch using the standard seek then write method.  The
> same 
> is true of a vast majority of applications as well.

Thanks for your confirmation.


>   The stuff most 
> people would have to worry about largely comes down to:
> 
> * VM software.  Some hypervisors such as QEMU can be configured to 
> translate discard commands issued against the emulated block devices
> to 
> fallocate calls to punch holes in the VM disk image file (and QEMU
> can 
> be configured to translate block writes of null bytes to this too), 
> though I know of none that do this by default.
> * Database software.  This is what stuff like punching holes
> originated 
> for, so it's obviously a potential source of this issue.
> * FUSE filesystem drivers.  Most of them that support the required 
> fallocate flag to punch holes pass it down directly.  Some make use
> of 
> it themselves too.
> * Userspace distributed storage systems.  Stuff like Ceph or
> Gluster. 
> Same arguments as above for FUSE filesystem drivers.

These do at least not affect me personally, though only because I
didn't use compress, where I use qemu (which I have configured to pass
on the TRIMs).


> > 'btrfs' (the command-line utility) doesn't do these operations as
> > far
> > as I can tell.  The kernel only does these when requested by
> > applications.
> The receive command will issue clone operations if the sent
> subvolume 
> requires it to get the correct block layout, so there is a 'regular' 
> BTRFS operation that can in theory set things up such that the
> required 
> patterns are more likely to happen.

As long as snapshoting itself doesn't create the issue, I should be
still safe at least on my master disks (which were always only the
source or send/receive), which I'll now compare to the backup disks


Thanks,
Chris.


  reply index

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-23  3:11 Reproducer for "compressed data + hole data corruption bug, 2018 editiion" Zygo Blaxell
2018-08-23  5:10 ` Qu Wenruo
2018-08-23 16:44   ` Zygo Blaxell
2018-08-23 23:50     ` Qu Wenruo
2019-02-12  3:09 ` Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7 Zygo Blaxell
2019-02-12 15:33   ` Christoph Anton Mitterer
2019-02-12 15:35   ` Filipe Manana
2019-02-12 17:01     ` Zygo Blaxell
2019-02-12 17:56       ` Filipe Manana
2019-02-12 18:13         ` Zygo Blaxell
2019-02-13  7:24           ` Qu Wenruo
2019-02-13 17:36           ` Filipe Manana
2019-02-13 18:14             ` Filipe Manana
2019-02-14  1:22               ` Filipe Manana
2019-02-14  5:00                 ` Zygo Blaxell
2019-02-14 12:21                 ` Christoph Anton Mitterer
2019-02-15  5:40                   ` Zygo Blaxell
2019-03-04 15:34                     ` Christoph Anton Mitterer
2019-03-07 20:07                       ` Zygo Blaxell
2019-03-08 10:37                         ` Filipe Manana
2019-03-14 18:58                           ` Christoph Anton Mitterer
2019-03-14 20:22                           ` Christoph Anton Mitterer
2019-03-14 22:39                             ` Filipe Manana
2019-03-08 12:20                         ` Austin S. Hemmelgarn
2019-03-14 18:58                           ` Christoph Anton Mitterer [this message]
2019-03-14 18:58                         ` Christoph Anton Mitterer
2019-03-15  5:28                           ` Zygo Blaxell
2019-03-16 22:11                             ` Christoph Anton Mitterer
2019-03-17  2:54                               ` Zygo Blaxell
2019-02-15 12:02                   ` Filipe Manana
2019-03-04 15:46                     ` Christoph Anton Mitterer
2019-02-12 18:58       ` Andrei Borzenkov
2019-02-12 21:48         ` Chris Murphy
2019-02-12 22:11           ` Zygo Blaxell
2019-02-12 22:53             ` Chris Murphy
2019-02-13  2:46               ` Zygo Blaxell
2019-02-13  7:47   ` Roman Mamedov
2019-02-13  8:04     ` Qu Wenruo

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6c388ca5944d8e679e66d2d889e594f6fc3ef68b.camel@scientia.net \
    --to=calestyo@scientia.net \
    --cc=ahferroin7@gmail.com \
    --cc=ce3g8jdj@umail.furryterror.org \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox