From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ot0-f181.google.com ([74.125.82.181]:55753 "EHLO mail-ot0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752565AbdKDROf (ORCPT ); Sat, 4 Nov 2017 13:14:35 -0400 Received: by mail-ot0-f181.google.com with SMTP id u41so5183420otf.12 for ; Sat, 04 Nov 2017 10:14:35 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20171104044634.thg7mnchm4hvzdic@angband.pl> References: <9871a669-141b-ac64-9da6-9050bcad7640@cn.fujitsu.com> <10fb0b92-bc93-a217-0608-5284ac1a05cd@rqc.ru> <20171104044634.thg7mnchm4hvzdic@angband.pl> From: Chris Murphy Date: Sat, 4 Nov 2017 11:14:33 -0600 Message-ID: Subject: Re: Problem with file system To: Adam Borowski Cc: Chris Murphy , "Austin S. Hemmelgarn" , Marat Khalili , Dave , Linux fs Btrfs , Fred Van Andel Content-Type: text/plain; charset="UTF-8" Sender: linux-btrfs-owner@vger.kernel.org List-ID: On Fri, Nov 3, 2017 at 10:46 PM, Adam Borowski wrote: > On Fri, Nov 03, 2017 at 04:03:44PM -0600, Chris Murphy wrote: >> On Tue, Oct 31, 2017 at 5:28 AM, Austin S. Hemmelgarn >> wrote: >> >> > If you're running on an SSD (or thinly provisioned storage, or something >> > else which supports discards) and have the 'discard' mount option enabled, >> > then there is no backup metadata tree (this issue was mentioned on the list >> > a while ago, but nobody ever replied), >> >> >> This is a really good point. I've been running discard mount option >> for some time now without problems, in a laptop with Samsung >> Electronics Co Ltd NVMe SSD Controller SM951/PM951. >> >> However, just trying btrfs-debug-tree -b on a specific block address >> for any of the backup root trees listed in the super, only the current >> one returns a valid result. All others fail with checksum errors. And >> even the good one fails with checksum errors within seconds as a new >> tree is created, the super updated, and Btrfs considers the old root >> tree disposable and subject to discard. >> >> So absolutely if I were to have a problem, probably no rollback for >> me. This seems to totally obviate a fundamental part of Btrfs design. > > How is this an issue? Discard is issued only once we're positive there's no > reference to the freed blocks anywhere. At that point, they're also open > for reuse, thus they can be arbitrarily scribbled upon. If it's not an issue, then no one should ever need those backup slots in the super and we should just remove them. But in fact, we know people end up situations where they're needed for either automatic recovery at mount time or explicitly calling --usebackuproot. And in some cases we're seeing users using discard who have a borked root tree, and none of the backup roots are present so they're fucked. Their file system is fucked. Now again, maybe this means the hardware is misbehaving, and honored the discard out of order, and did that and wrote the new supers before it had completely committed all the metadata? I have no idea, but the evidence is present in the list that some people run into this and when they do the file system is beyond repair even t hough it can usually be scraped with btrfs restore. > Unless your hardware is seriously broken (such as lying about barriers, > which is nearly-guaranteed data loss on btrfs anyway), there's no way the > filesystem will ever reference such blocks. The corpses of old trees that > are left lying around with no discard can at most be used for manual > forensics, but whether a given block will have been overwritten or not is > a matter of pure luck. File systems that overwrite, are hinting the intent in the journal what's about to happen. So if there's a partial overwrite of metadata, it's fine. The journal can help recover. But Btrfs without a journal, has a major piece of information required to bootstrap the file system at mount time, that's damaged, and then every backup has been discarded. So it actually makes Btrfs more fragile than other file systems in the same situation. > > For rollbacks, there are snapshots. Once a transaction has been fully > committed, the old version is considered gone. Yeah well snapshots do not cause root trees to stick around. > >> because it's already been discarded. >> > This is ideally something which should be addressed (we need some sort of >> > discard queue for handling in-line discards), but it's not easy to address. >> >> Discard data extents, don't discard metadata extents? Or put them on a >> substantial delay. > > Why would you special-case metadata? Metadata that points to overwritten or > discarded blocks is of no use either. I would rather lose 30 seconds, 1 minute, or even 2 minutes of writes, than lose an entire file system. That's why. Anyway right now I consider discard mount option fundamentally broken on Btrfs for SSDs. I haven't tested this on LVM thinp, maybe it's broken there too. Even fstrim leaves a tiny window open for a few minutes every time it gets called, where if the root tree is corrupted for any reason, you're fucked because all the backup roots are already gone. -- Chris Murphy