Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
To: Christoph Anton Mitterer <calestyo@scientia.net>
Cc: linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7
Date: Fri, 15 Mar 2019 01:28:45 -0400
Message-ID: <20190315052827.GH23918@hungrycats.org> (raw)
In-Reply-To: <beebed47702d783e00d3478f217ffc646d47bb92.camel@scientia.net>

[-- Attachment #1: Type: text/plain, Size: 3384 bytes --]

On Thu, Mar 14, 2019 at 07:58:45PM +0100, Christoph Anton Mitterer wrote:
> Phew... too much [silent] corruption bugs in btrfs... :-(
> 
> Actually I didn't even notice the others (which unfortunately doesn't
> mean I'm definitely not affected), so I probably cannot much do/check
> about them now... but only about the "recent" one that was fixed now.
> 
> But maybe there should be something like a btrfs-announce list, i.e. a
> low volume mailing list, in which (interested) users are informed about
> more grave issues.
> Such things can happen and there's no one to blame about that... but if
> they happen it would be good for users to get notified so that they can
> check their systems and possibly recover data from (still existing)
> other sources.

I don't know if it would be a low-volume list...every kernel release
includes fixes for _some_ exotic corner case.

> What about the direct IO issues that may be still present and which
> you've mentioned above... is this used somewhere per default / under
> normal circumstances?

Direct IO is an odd case because it's not all that well understood
what the correct behavior is.  You can't prevent the kernel from making
copies of data and also expect full data integrity and also lock-free
performance, all at the same time.  Pick any two, and pay for it with
losses in the third.

The bug fixes here are more along the lines of "OK so you're using direct
IO which means you've basically admitted you don't care about *your* data,
let's try not to corrupt *other* data on the filesystem at the same time."

> I think by now I'm pretty confident that I, personally, am safe.

It took me two years to find this bug, and I had to write a tool to
encounter it often enough to notice.  A lot of people are safe.

> > If you had asked questions like "is this bug the reason why I've been
> > seeing random SHA hash verification failures for several years?" then
> > you should worry about this bug; otherwise, it probably didn't affect
> > you.
> 
> I think you're right... but my data with many thousands of pictures,
> etc. from all life is really precious to me, so I better wanted to
> understand the issue in "depth"... and I think these questions and your
> answers may still benefit others who may also want to find out whether
> they could have been silently affected :-)

I found the 2017 compression bug in a lot of digital photographs.
It turns out that several popular cameras (including some of the ones I
own) put a big chunk of zeros near the beginnings of JPG files, and when
rsync copies those it will insert a hole instead of copying the zeros.
The 2017 bug affected "ordinary" holes so standard tools like cp and
rsync could trigger it.  Most photo tools ignore this data completely,
so when garbage appears there, nobody notices.

A similar thing happens to .o files:  ld aligns things to 4K block
boundaries, triggering the 2017 compressed read bug.  Nobody reads that
data either--it's just alignment padding.

I don't think I found an application that cared about the 2017 bug at all.
Only backup verifications.

The 2018 bug is a different story--when it hits, it's obvious, and
ordinary application things break--but it won't happen to typical photo
image files, even with aggressive dedupe.

> 
> Cheers and thanks,
> Chris.
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

  reply index

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-23  3:11 Reproducer for "compressed data + hole data corruption bug, 2018 editiion" Zygo Blaxell
2018-08-23  5:10 ` Qu Wenruo
2018-08-23 16:44   ` Zygo Blaxell
2018-08-23 23:50     ` Qu Wenruo
2019-02-12  3:09 ` Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7 Zygo Blaxell
2019-02-12 15:33   ` Christoph Anton Mitterer
2019-02-12 15:35   ` Filipe Manana
2019-02-12 17:01     ` Zygo Blaxell
2019-02-12 17:56       ` Filipe Manana
2019-02-12 18:13         ` Zygo Blaxell
2019-02-13  7:24           ` Qu Wenruo
2019-02-13 17:36           ` Filipe Manana
2019-02-13 18:14             ` Filipe Manana
2019-02-14  1:22               ` Filipe Manana
2019-02-14  5:00                 ` Zygo Blaxell
2019-02-14 12:21                 ` Christoph Anton Mitterer
2019-02-15  5:40                   ` Zygo Blaxell
2019-03-04 15:34                     ` Christoph Anton Mitterer
2019-03-07 20:07                       ` Zygo Blaxell
2019-03-08 10:37                         ` Filipe Manana
2019-03-14 18:58                           ` Christoph Anton Mitterer
2019-03-14 20:22                           ` Christoph Anton Mitterer
2019-03-14 22:39                             ` Filipe Manana
2019-03-08 12:20                         ` Austin S. Hemmelgarn
2019-03-14 18:58                           ` Christoph Anton Mitterer
2019-03-14 18:58                         ` Christoph Anton Mitterer
2019-03-15  5:28                           ` Zygo Blaxell [this message]
2019-03-16 22:11                             ` Christoph Anton Mitterer
2019-03-17  2:54                               ` Zygo Blaxell
2019-02-15 12:02                   ` Filipe Manana
2019-03-04 15:46                     ` Christoph Anton Mitterer
2019-02-12 18:58       ` Andrei Borzenkov
2019-02-12 21:48         ` Chris Murphy
2019-02-12 22:11           ` Zygo Blaxell
2019-02-12 22:53             ` Chris Murphy
2019-02-13  2:46               ` Zygo Blaxell
2019-02-13  7:47   ` Roman Mamedov
2019-02-13  8:04     ` Qu Wenruo

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190315052827.GH23918@hungrycats.org \
    --to=ce3g8jdj@umail.furryterror.org \
    --cc=calestyo@scientia.net \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox