Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
To: Christoph Anton Mitterer <calestyo@scientia.net>
Cc: linux-btrfs <linux-btrfs@vger.kernel.org>
Subject: Re: Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7
Date: Fri, 15 Feb 2019 00:40:45 -0500
Message-ID: <20190215054031.GC9995@hungrycats.org> (raw)
In-Reply-To: <95e4d9825c2565473184765c4d77ae0015d01580.camel@scientia.net>

[-- Attachment #1: Type: text/plain, Size: 4815 bytes --]

On Thu, Feb 14, 2019 at 01:21:29PM +0100, Christoph Anton Mitterer wrote:
> On Thu, 2019-02-14 at 01:22 +0000, Filipe Manana wrote:
> > The following one liner fixes it:
> > https://friendpaste.com/22t4OdktHQTl0aMGxcWLj3
> 
> Great to see that fixed... is there any advise that can be given for
> users/admins?
> 
> 
> Like whether and how any occurred corruptions can be detected (right
> now, people may still have backups)?

The problem occurs only on reads.  Data that is written to disk will
be OK, and can be read correctly by a fixed kernel.

A kernel without the fix will give corrupt data on reads with no
indication of corruption other than the changes to the data itself.

Applications that copy data may read corrupted data and write it back
to the filesystem.  This will make the corruption permanent in the
copied data.

Given the age of the bug, backups that can be corrupted by this bug
probably already are.  Verify files against internal CRC/hashes where
possible.  The original files are likely to be OK, since the bug does
not affect writes.  If your situation has the risk factors listed below,
it may be worthwhile to create a fresh set of non-incremental backups
after applying the kernel fix.

> Or under which exact circumstances did the corruption happen? And under
> which was one safe?

Compression is required to trigger the bug, so you are safe if you (or
the applications you run) never enabled filesystem compression.  Even if
compression is enabled, the file data must be compressed for the bug to
corrupt it.  Incompressible data extents will never be affected by
this bug.

If you do use compression, you are still safe if:

	- you never punch holes in files

	- you never dedupe or clone files

If you do use compression and do the other things, the probability of
corruption by this particular bug is non-zero.  Whether you get corruption
and how often depends on the technical details of what you're doing.

To get corruption you have to have one data extent that is split in
two parts by punching a hole, or an extent that is cloned/deduped in
two parts to adjacent logical offsets in the same file.  Both of these
methods create the pattern on disk which triggers the bug.

Files that consist entirely of unique data will not be affected by dedupe
so will not trigger the bug that way.  Files that consist partially of
unique data may or may not be affected depending on the dedupe tool,
data alignment, etc.

> E.g. only on specific compression algos (I've been using -o compress
> (which should be zlib) for quite a while but never found any

All decompress algorithms are affected.  The bug is in the generic btrfs
decompression handling, so it is not limited to any single algorithm.

Compression (i.e. writing) is not affected--whatever data is written to
disk should be readable correctly with a fixed kernel.

> compression),... or only when specific file operations were done (I did
> e.g. cp with refcopy, but I think none of the standard tools does hole-
> punching)?

That depends on whether you consider fallocate or qemu to be standard
tools.  The hole-punching function has been a feature of several Linux
filesystems for some years now, so we can expect it to be more widely
adopted over time.  You'd have to do an audit to be sure none of the
tools you use are punching holes.

"Ordinary" sparse files (made by seeking forward while writing, as done
by older Unix utilities including cp, tar, rsync, cpio, binutils) do not
trigger this bug.  An ordinary sparse file has two distinct data extents
from two different writes separated by a hole which has never contained
file data.  A punched hole splits an existing single data extent into two
pieces with a newly created hole between them that replaces previously
existing file data.  These actions create different extent reference
patterns and only the hole-punching one is affected by the bug.

Files that contain no blocks full of zeros will not be affected by
fallocate-d-style hole punching (it searches for existing zeros and
punches holes over them--no zeros, no holes).  If the the hole punching
intentionally introduces zeros where zeros did not exist before (e.g. qemu
discard operations on raw image files) then it may trigger the bug.

btrfs send and receive may be affected, but I don't use them so I don't
have any experience of the bug related to these tools.  It seems from
reading the btrfs receive code that it lacks any code capable of punching
a hole, but I'm only doing a quick search for words like "punch", not
a detailed code analysis.

bees continues to be an awesome tool for discovering btrfs kernel bugs.
It compresses, dedupes, *and* punches holes.

> 
> Cheers,
> Chris.
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

  reply index

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-23  3:11 Reproducer for "compressed data + hole data corruption bug, 2018 editiion" Zygo Blaxell
2018-08-23  5:10 ` Qu Wenruo
2018-08-23 16:44   ` Zygo Blaxell
2018-08-23 23:50     ` Qu Wenruo
2019-02-12  3:09 ` Reproducer for "compressed data + hole data corruption bug, 2018 edition" still works on 4.20.7 Zygo Blaxell
2019-02-12 15:33   ` Christoph Anton Mitterer
2019-02-12 15:35   ` Filipe Manana
2019-02-12 17:01     ` Zygo Blaxell
2019-02-12 17:56       ` Filipe Manana
2019-02-12 18:13         ` Zygo Blaxell
2019-02-13  7:24           ` Qu Wenruo
2019-02-13 17:36           ` Filipe Manana
2019-02-13 18:14             ` Filipe Manana
2019-02-14  1:22               ` Filipe Manana
2019-02-14  5:00                 ` Zygo Blaxell
2019-02-14 12:21                 ` Christoph Anton Mitterer
2019-02-15  5:40                   ` Zygo Blaxell [this message]
2019-03-04 15:34                     ` Christoph Anton Mitterer
2019-03-07 20:07                       ` Zygo Blaxell
2019-03-08 10:37                         ` Filipe Manana
2019-03-14 18:58                           ` Christoph Anton Mitterer
2019-03-14 20:22                           ` Christoph Anton Mitterer
2019-03-14 22:39                             ` Filipe Manana
2019-03-08 12:20                         ` Austin S. Hemmelgarn
2019-03-14 18:58                           ` Christoph Anton Mitterer
2019-03-14 18:58                         ` Christoph Anton Mitterer
2019-03-15  5:28                           ` Zygo Blaxell
2019-03-16 22:11                             ` Christoph Anton Mitterer
2019-03-17  2:54                               ` Zygo Blaxell
2019-02-15 12:02                   ` Filipe Manana
2019-03-04 15:46                     ` Christoph Anton Mitterer
2019-02-12 18:58       ` Andrei Borzenkov
2019-02-12 21:48         ` Chris Murphy
2019-02-12 22:11           ` Zygo Blaxell
2019-02-12 22:53             ` Chris Murphy
2019-02-13  2:46               ` Zygo Blaxell
2019-02-13  7:47   ` Roman Mamedov
2019-02-13  8:04     ` Qu Wenruo

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190215054031.GC9995@hungrycats.org \
    --to=ce3g8jdj@umail.furryterror.org \
    --cc=calestyo@scientia.net \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox