Linux-BTRFS Archive on
 help / color / Atom feed
From: Timothy Pearson <>
To: linux-btrfs <>
Subject: Re: Unusual crash -- data rolled back ~2 weeks?
Date: Sat, 9 Nov 2019 16:48:05 -0600 (CST)
Message-ID: <> (raw)
In-Reply-To: <>

One item I did forget to mention here is that the underlying device was expanded online using "btrfs fi resize max /mount/path" at most a month before the failure -- I don't have the exact timestamps available, so there remains a possibility that the latest files on the currently mounted filesystem correspond to the filesystem as it was immediately prior to the resize operation.

Again, any suggestions welcome.  It took us a bit of time (and several large file restores) to realize the filesystem had rolled back vs just corrupted a few files, so while there does exist a raw copy of the filesystem it is tainted by being mounted and written to before the copy was taken.

----- Original Message -----
> From: "Timothy Pearson" <>
> To: "linux-btrfs" <>
> Sent: Saturday, November 9, 2019 4:33:29 PM
> Subject: Unusual crash -- data rolled back ~2 weeks?

> We just experienced a very unusual crash on a Linux 5.3 file server using NFS to
> serve a BTRFS filesystem.  NFS went into deadlock (D wait) with no apparent
> underlying disk subsystem problems, and when the server was hard rebooted to
> clear the D wait the BTRFS filesystem remounted itself in the state that it was
> in approximately two weeks earlier (!).  There was also significant corruption
> of certain files (e.g. LDAP MDB and MySQL InnoDB) noted -- we restored from
> backup for those files, but are concerned about the status of the entire
> filesystem at this point.
> We do not use subvolumes, snapshots, or any of the advanced features of BTRFS
> beyond the data checksumming.  I am at a loss as to how BTRFS could suddenly
> just "forget" about the past two weeks of written data and (mostly) cleanly
> roll back on the next mount without even throwing any warnings in dmesg.
> Any thoughts on how this is possible, and if there is any chance of getting the
> lost couple weeks of data back, would be appreciated.
> Thank you!

  reply index

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-09 22:33 Timothy Pearson
2019-11-09 22:48 ` Timothy Pearson [this message]
2019-11-10  3:38 ` Qu Wenruo
2019-11-10  6:47   ` Timothy Pearson
2019-11-10  6:54     ` Qu Wenruo
2019-11-10  7:18       ` Timothy Pearson
2019-11-10  7:45         ` Qu Wenruo
2019-11-10  7:48           ` Timothy Pearson
2019-11-10 10:02           ` Timothy Pearson
2019-11-10 20:10             ` Zygo Blaxell
2019-11-11 23:28           ` Timothy Pearson
2019-11-11 23:33             ` Timothy Pearson
2019-11-12 11:30             ` Chris Murphy
2019-11-10  8:04         ` Andrei Borzenkov

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on

Archives are clonable:
	git clone --mirror linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ \
	public-inbox-index linux-btrfs

Example config snippet for mirrors

Newsgroup available over NNTP:

AGPL code for this site: git clone