[PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes

[PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes

[Qu Wenruo]

There is an issue reported in github, where an fs get corrupted
extent tree initialy, then I recommended --init-extent-tree.

Although --init-extent-tree indeed fixed the original problem, it caused
new problems, quite a lot of EXTENT_ITEMs now get bad generation number,
which failed to mount with v5.4.

The problem turns out to be a bug in backref repair code, which doesn't
initialize extent_record::generation, causing garbage in EXTENT_ITEMs.

This patch will:
- Fix the problem
  Patch 1

- Enhance EXTENT_ITEM generation repair
  Patch 2

- Make `btrfs check` able to detect such bad generation
  Patch 3~4

- Add new test case for above ability
  Patch 5

Qu Wenruo (5):
  btrfs-progs: check: Initialize extent_record::generation member
  btrfs-progs: check: Populate extent generation correctly for data
    extents
  btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA
    generation
  btrfs-progs: check/original: Detect invalid extent generation
  btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent
    item generation

 check/main.c                                  |  36 ++++++++++++++----
 check/mode-lowmem.c                           |  19 +++++++++
 .../bad_extent_item_gen.img.xz                | Bin 0 -> 1916 bytes
 .../test.sh                                   |  19 +++++++++
 4 files changed, 67 insertions(+), 7 deletions(-)
 create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz
 create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh

-- 
2.24.1

[PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member

[Qu Wenruo]

[BUG]
When using `btrfs check --init-extent-tree`, there is a pretty high
chance that the result fs can't pass tree-checker:

  BTRFS critical (device dm-3): corrupt leaf: block=5390336 slot=149 extent bytenr=20115456 len=4096 invalid generation, have 16384 expect (0, 360]
  BTRFS error (device dm-3): block=5390336 read time tree block corruption detected
  BTRFS error (device dm-3): failed to read block groups: -5
  BTRFS error (device dm-3): open_ctree failed

[CAUSE]
The result fs has a pretty screwed up EXTENT_ITEMs for data extents:

        item 148 key (20111360 EXTENT_ITEM 4096) itemoff 8777 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 841 offset 0 count 1
        item 149 key (20115456 EXTENT_ITEM 4096) itemoff 8724 itemsize 53
                refs 1 gen 16384 flags DATA
                extent data backref root FS_TREE objectid 906 offset 0 count 1

Kernel tree-checker will accept 0 generation, but that 16384 generation
is definitely going to trigger the alarm.

Looking into the code, it's add_extent_rec_nolookup() allocating a new
extent_rec, but not copying all members from parameter @tmpl, resulting
generation not properly initialized.

[FIX]
Just copy tmpl->generation in add_extent_rec_nolookup(). And since all
call sites have set all members of @tmpl to 0 before
add_extent_rec_nolookup(), we shouldn't get garbage values.

For the 0 generation problem, it will be solved in another patch.

Issue: 225 (Not the initial report, but extent tree rebuild result)
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/check/main.c b/check/main.c
index 08dc9e66..2dbed091 100644
--- a/check/main.c
+++ b/check/main.c
@@ -4605,6 +4605,7 @@ static int add_extent_rec_nolookup(struct cache_tree *extent_cache,
 	rec->refs = tmpl->refs;
 	rec->extent_item_refs = tmpl->extent_item_refs;
 	rec->parent_generation = tmpl->parent_generation;
+	rec->generation = tmpl->generation;
 	INIT_LIST_HEAD(&rec->backrefs);
 	INIT_LIST_HEAD(&rec->dups);
 	INIT_LIST_HEAD(&rec->list);
-- 
2.24.1

[PATCH 2/5] btrfs-progs: check: Populate extent generation correctly for data extents

[Qu Wenruo]

[BUG]
When using `btrfs check --init-extent-tree`, we will create incorrect
generation number for data extents in extent tree:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

[CAUSE]
Since data extent generation is not as obvious as tree blocks, which has
header containing its generations, so for data extents, its
extent_record::generation is not really updated, resulting such 0
generation.

[FIX]
To get generation of a data extent, there are two sources we can rely:
- EXTENT_ITEM
  There is always a btrfs_extent_item::generation can be utilized.
  Although this is not possible for --init-extent-tree use case.

- EXTENT_DATA
  We have btrfs_file_extent_item::generation for regular and
  preallocated data extents.
  Since --init-extent-tree will go through subvolume trees, this would
  be the main source for extent data generation.

Then we only need to make add_data_backref() to accept @gen parameter,
and pass it down to extent_record structure.

And for the final extent item generation update, here we add extra
fallback values, if we can't find FILE_EXTENT items.
In that case, we just fall back to current transid.

With this modification, recreated data EXTENT_ITEM now has correct
generation number:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 6 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/check/main.c b/check/main.c
index 2dbed091..88b174ab 100644
--- a/check/main.c
+++ b/check/main.c
@@ -4810,7 +4810,7 @@ static int add_tree_backref(struct cache_tree *extent_cache, u64 bytenr,
 
 static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr,
 			    u64 parent, u64 root, u64 owner, u64 offset,
-			    u32 num_refs, int found_ref, u64 max_size)
+			    u32 num_refs, u64 gen, int found_ref, u64 max_size)
 {
 	struct extent_record *rec;
 	struct data_backref *back;
@@ -4826,6 +4826,7 @@ static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr,
 		tmpl.start = bytenr;
 		tmpl.nr = 1;
 		tmpl.max_size = max_size;
+		tmpl.generation = gen;
 
 		ret = add_extent_rec_nolookup(extent_cache, &tmpl);
 		if (ret)
@@ -4840,6 +4841,8 @@ static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr,
 	if (rec->max_size < max_size)
 		rec->max_size = max_size;
 
+	if (rec->generation < gen)
+		rec->generation = gen;
 	/*
 	 * If found_ref is set then max_size is the real size and must match the
 	 * existing refs.  So if we have already found a ref then we need to
@@ -5315,6 +5318,7 @@ static int process_extent_item(struct btrfs_root *root,
 	u64 refs = 0;
 	u64 offset;
 	u64 num_bytes;
+	u64 gen;
 	int metadata = 0;
 
 	btrfs_item_key_to_cpu(eb, &key, slot);
@@ -5340,6 +5344,7 @@ static int process_extent_item(struct btrfs_root *root,
 
 	ei = btrfs_item_ptr(eb, slot, struct btrfs_extent_item);
 	refs = btrfs_extent_refs(eb, ei);
+	gen = btrfs_extent_generation(eb, ei);
 	if (btrfs_extent_flags(eb, ei) & BTRFS_EXTENT_FLAG_TREE_BLOCK)
 		metadata = 1;
 	else
@@ -5362,6 +5367,7 @@ static int process_extent_item(struct btrfs_root *root,
 	tmpl.metadata = metadata;
 	tmpl.found_rec = 1;
 	tmpl.max_size = num_bytes;
+	tmpl.generation = gen;
 	add_extent_rec(extent_cache, &tmpl);
 
 	ptr = (unsigned long)(ei + 1);
@@ -5401,14 +5407,14 @@ static int process_extent_item(struct btrfs_root *root,
 								       dref),
 					btrfs_extent_data_ref_offset(eb, dref),
 					btrfs_extent_data_ref_count(eb, dref),
-					0, num_bytes);
+					gen, 0, num_bytes);
 			break;
 		case BTRFS_SHARED_DATA_REF_KEY:
 			sref = (struct btrfs_shared_data_ref *)(iref + 1);
 			add_data_backref(extent_cache, key.objectid, offset,
 					0, 0, 0,
 					btrfs_shared_data_ref_count(eb, sref),
-					0, num_bytes);
+					gen, 0, num_bytes);
 			break;
 		default:
 			fprintf(stderr,
@@ -6352,7 +6358,7 @@ static int run_next_block(struct btrfs_root *root,
 								       ref),
 					btrfs_extent_data_ref_offset(buf, ref),
 					btrfs_extent_data_ref_count(buf, ref),
-					0, root->fs_info->sectorsize);
+					0, 0, root->fs_info->sectorsize);
 				continue;
 			}
 			if (key.type == BTRFS_SHARED_DATA_REF_KEY) {
@@ -6363,7 +6369,7 @@ static int run_next_block(struct btrfs_root *root,
 				add_data_backref(extent_cache,
 					key.objectid, key.offset, 0, 0, 0,
 					btrfs_shared_data_ref_count(buf, ref),
-					0, root->fs_info->sectorsize);
+					0, 0, root->fs_info->sectorsize);
 				continue;
 			}
 			if (key.type == BTRFS_ORPHAN_ITEM_KEY) {
@@ -6403,7 +6409,8 @@ static int run_next_block(struct btrfs_root *root,
 			add_data_backref(extent_cache,
 				btrfs_file_extent_disk_bytenr(buf, fi),
 				parent, owner, key.objectid, key.offset -
-				btrfs_file_extent_offset(buf, fi), 1, 1,
+				btrfs_file_extent_offset(buf, fi), 1,
+				btrfs_file_extent_generation(buf, fi), 1,
 				btrfs_file_extent_disk_num_bytes(buf, fi));
 		}
 	} else {
@@ -6706,7 +6713,10 @@ static int record_extent(struct btrfs_trans_handle *trans,
 				    struct btrfs_extent_item);
 
 		btrfs_set_extent_refs(leaf, ei, 0);
-		btrfs_set_extent_generation(leaf, ei, rec->generation);
+		if (rec->generation)
+			btrfs_set_extent_generation(leaf, ei, rec->generation);
+		else
+			btrfs_set_extent_generation(leaf, ei, trans->transid);
 
 		if (back->is_data) {
 			btrfs_set_extent_flags(leaf, ei,
-- 
2.24.1

[PATCH 3/5] btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation

[Qu Wenruo]

Since older `btrfs check --init-extent-tree` could cause invalid
EXTENT_ITEM generation for data extents, add such check to lowmem mode
check.

Also add such generation check to file extents too.

For the repair part, I don't have any good idea yet. So affected user
may depend on --init-extent-tree again.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/mode-lowmem.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
index f53a0c39..aad30c28 100644
--- a/check/mode-lowmem.c
+++ b/check/mode-lowmem.c
@@ -2041,6 +2041,8 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path,
 	u64 csum_found;		/* In byte size, sectorsize aligned */
 	u64 search_start;	/* Logical range start we search for csum */
 	u64 search_len;		/* Logical range len we search for csum */
+	u64 gen;
+	u64 super_gen = btrfs_super_generation(root->fs_info->super_copy);
 	unsigned int extent_type;
 	unsigned int is_hole;
 	int slot = path->slots[0];
@@ -2067,6 +2069,7 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path,
 		return check_file_extent_inline(root, path, size, end);
 
 	/* Check REG_EXTENT/PREALLOC_EXTENT */
+	gen = btrfs_file_extent_generation(node, fi);
 	disk_bytenr = btrfs_file_extent_disk_bytenr(node, fi);
 	disk_num_bytes = btrfs_file_extent_disk_num_bytes(node, fi);
 	extent_num_bytes = btrfs_file_extent_num_bytes(node, fi);
@@ -2074,6 +2077,13 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path,
 	compressed = btrfs_file_extent_compression(node, fi);
 	is_hole = (disk_bytenr == 0) && (disk_num_bytes == 0);
 
+	if (gen > super_gen + 1) {
+		error(
+		"vainlid file extent generation, have %llu expect (0, %llu]",
+			gen, super_gen + 1);
+		err |= INVALID_GENERATION;
+	}
+
 	/*
 	 * Check EXTENT_DATA csum
 	 *
@@ -4152,8 +4162,10 @@ static int check_extent_item(struct btrfs_fs_info *fs_info,
 	u64 parent;
 	u64 num_bytes;
 	u64 root_objectid;
+	u64 gen;
 	u64 owner;
 	u64 owner_offset;
+	u64 super_gen = btrfs_super_generation(fs_info->super_copy);
 	int metadata = 0;
 	int level;
 	struct btrfs_key key;
@@ -4183,6 +4195,13 @@ static int check_extent_item(struct btrfs_fs_info *fs_info,
 
 	ei = btrfs_item_ptr(eb, slot, struct btrfs_extent_item);
 	flags = btrfs_extent_flags(eb, ei);
+	gen = btrfs_extent_generation(eb, ei);
+	if (gen > super_gen + 1) {
+		error(
+		"invalid generation for extent %llu, have %llu expect (0, %llu]",
+			key.objectid, gen, super_gen + 1);
+		err |= INVALID_GENERATION;
+	}
 
 	if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK)
 		metadata = 1;
-- 
2.24.1

[PATCH 4/5] btrfs-progs: check/original: Detect invalid extent generation

[Qu Wenruo]

Much like what we have done in lowmem mode, also detect and report
invalid extent generation in original mode.

Unlike lowmem mode, we have extent_record::generation, which is the
higher number of generations in EXTENT_ITEM, EXTENT_DATA or tree block
header, so there is no need to check generations in different locations.

For repair, we still need to depend on --init-extent-tree, as directly
modifying extent items can easily cause conflicts with delayed refs,
thus it should be avoided.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/check/main.c b/check/main.c
index 88b174ab..a9a236a4 100644
--- a/check/main.c
+++ b/check/main.c
@@ -4021,10 +4021,13 @@ static void free_extent_record_cache(struct cache_tree *extent_cache)
 static int maybe_free_extent_rec(struct cache_tree *extent_cache,
 				 struct extent_record *rec)
 {
+	u64 super_gen = btrfs_super_generation(global_info->super_copy);
+
 	if (rec->content_checked && rec->owner_ref_checked &&
 	    rec->extent_item_refs == rec->refs && rec->refs > 0 &&
 	    rec->num_duplicates == 0 && !all_backpointers_checked(rec, 0) &&
 	    !rec->bad_full_backref && !rec->crossing_stripes &&
+	    rec->generation <= super_gen + 1 &&
 	    !rec->wrong_chunk_type) {
 		remove_cache_extent(extent_cache, &rec->cache);
 		free_all_extent_backrefs(rec);
@@ -7857,6 +7860,7 @@ static int check_extent_refs(struct btrfs_root *root,
 {
 	struct extent_record *rec;
 	struct cache_extent *cache;
+	u64 super_gen = btrfs_super_generation(root->fs_info->super_copy);
 	int ret = 0;
 	int had_dups = 0;
 	int err = 0;
@@ -7939,6 +7943,13 @@ static int check_extent_refs(struct btrfs_root *root,
 			cur_err = 1;
 		}
 
+		if (rec->generation > super_gen + 1) {
+			error(
+	"invalid generation for extent %llu, have %llu expect (0, %llu]",
+				rec->start, rec->generation,
+				super_gen + 1);
+			cur_err = 1;
+		}
 		if (rec->refs != rec->extent_item_refs) {
 			fprintf(stderr, "ref mismatch on [%llu %llu] ",
 				(unsigned long long)rec->start,
-- 
2.24.1

[PATCH 5/5] btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation

[Qu Wenruo]

The new image has a bad data extent item generation:

        item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16230 itemsize 53
                refs 1 gen 16384 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

Just like what older `btrfs check --init-extent-tree` could cause.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../bad_extent_item_gen.img.xz                | Bin 0 -> 1916 bytes
 .../test.sh                                   |  19 ++++++++++++++++++
 2 files changed, 19 insertions(+)
 create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz
 create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh

diff --git a/tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz b/tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz
new file mode 100644
index 0000000000000000000000000000000000000000..c3e30313538558384352d4987acef69f56acea2d
GIT binary patch
literal 1916
zcmV-?2ZQ+iH+ooF000E$*0e?f03iV!0000G&sfah3;zc?T>wRyj;C3^v%$$4d1r37
zhA1?_LT>Wbx+L2{5ct*#2hYYGZ$^%K*7}6#@?zh(OKyt!ovg(ZXD3s(w#SpNSsRep
zn;&Hxy|d8!Hi<RRaJ(=+T!Nf*Grt0$9YKM_uNCH&f@?oOPNUPM#dHBgutu$*YH7Sd
z;8vxqgB?;P8%c$1w4ktrsMRs{rq$7d%o`v|q>*o#PdRXN)B+ivWj$v1%hPZx8E=T<
zIFi++(r$u*0A%YuW7aB1tv#h~la(SW&<75ZbX{2HpaX7<+lhn{3MIgh{Z)k&XH7!-
z$i?+3e{kf1rujr8NoiytrfWvbj+^erx%LwZ6`4?RpV8Uj2%8PN6}TKB72stHk=erF
zZfbD^3`i=rDXXQqVgn?n3)M_=zMVg460_al*1EQe6RehzF>**=@B3DZM^tdid(Av&
zv|+=Xx*;MKXd`nuxO6M=Gu{<nMFP2`CfHR(zeIi>c4coH69RgW26`P$AZ|-HLCS+e
zeQ+IA0>~0ckk&~DR#h{#Lp@mbC@Ee29#5kC5@O{=_t7tZ>H1Nh>jBd8C9(CT$kAIA
z7N+}Kp`|CSujkwTDP(M;#QF)=>_SC=^65fQf+GYi>GVs5(%FRmcL!ay1An28JAMk9
zjF005pXjxPsZF>PAgk_{*)^zrWuOGYCr%m;Z5Iij7$)S9V}N6eTIG4xL9&v%JbXRN
z!Hso@wG^`ob`AffkCnd`Hq4{5KcX>fJ^e$rcHH8Vt&)x$9OPVOMxN(qb_|1}&%GF0
z^(YphZ4V5@1-TMwXWqaOpZfoJQo+W)#S=v;wkY}p>|u*6uX|37vL`vo58!U0>kdzd
zF)xr?nN{c|Z|FhwP?&?Sb)cnhgGQRDRvhYE3f~?Lx#nb(g>g~ulpZ74ywFa*bMr<8
zw1dO*Si2v{xcVNYXD(I{O`qzgT_6mJl2>ZW!KqB~!ZD7pmsg=|9;s8j4A(pTixI+T
z>-*2VM+)UY=qM+OKyI)*4IOp-GgTH{U~;T1(vr-uvOhlmhg%X%qdhaZ-z1*p%unU^
z+Gm-zwG_iMJ%jD{@MrryJ-7x-lk_fc*8o9CKOs@mOA+&`I;q7-`+HWC4D4Pyk0m#;
zE&%t+T#KL|eD~39LL~%KEG=Q))Jo7|{|tF|A^CKck~~fr?d?bLZRt{bW}E#7$g6*u
zlwJj~7_!f1LK~Qd`bwbgd+m0S#kFNhm*dMOi5|fFYW>4}hNhZiYMJBX1DW-+FC3>>
zKuEv?mDdP`0a_@$pvSdrC4Kf-2q7IVCR?v9%A-APiGsO_+8)i`KufH<L&=cpE&*CO
zmexp=`$4a)^7l^>1@WbrDXdUBrD>y(hTNdxMbV#PXijZ=v{s+Fm*c|Mz^W7OeAM?0
z<>ZpcI$ZZH<e!m`Sjehu(2{|;^gfYc976$TBC@?SUR>CGbrG-abJ+$@cfUk$sElUJ
zeJ4G+K-_tP^o;FLZ9?_g!!`ZZhx?*HSeSaM^YNfVQQ+I<OXm2ZEY5v}aR+XgPGL4^
znX9pTiZSMVDu5knh#?$4$En+S-i^6O?}Ch4LVZylEj`5s?5hTHtSZgJXOInlu@~_Y
zRkYV<52qb0%p|f_KUfDAUr-kP*)!bx|BEGV=+G&~2jzenP4`5SMD+G9w{w}UXHazP
zcoZ%_r^64L_ByX%`pG?^$_Tout5bnJ!1%p>AB_k?aw=c6dB+mBIk|s)L%u*2@Rg^h
zq*qPE$vg%N0_Ul)p)m9GEUa5QRUvQb)TX@L{cy;H@2HiFWhQIun8M=P>%M|FA94VF
z6Z<MCz?ueUp$9AAC%1N<!tL2~LD_eytni8i`+LG=)7;TcHHi#N&4yX(hEgT+ZprNu
zIbs`uKXvCj_rf)Y9mU5l)!vIT)7}J3+gSyCaTCm^Y(^mGzNuL1Czz>XciBSk=z^i#
z*Ewxo65u1o;3PFe;aIh84y?2eNdGvz5{&Or82y7KJ4tIK-j)u&BUg&xc}}Q!MS#6r
z<#h&3ZEYc6NV!?kl1N5BUZyp*yxWYL1hELeBKWO(j`a^#hX-2Kh%Q-3<s&UsQ=SL9
zi%Ku&({<6VGdX5}Ojk*sVj_T!)D)3zgoJn!=fmrtXWRune6!1AXPA(vrdrPqFnb<H
zf+>9amd!CJ)4HXuyfdA_lS0b*zdo<v<3`%1mRm|DI|GN`j$1B!!-0nwB{IJX@~Slb
zvYVN~pn0i35$7Z(ecPteuLdb}kz*upp810p``+>(aF?s0XE~EZ=Lw_I2Hbr*H|jWf
z3jC6`6x+nW2DNkTh(HE!jp(YB12(98r24u9t`g4>DF;L3({!kWwIV@;?O0aVl$}e;
zEdcyEP+0;aOD}}5hEIO$Och-)`a(0J$#4gX1AcuZzQTUEnjCiFSz4d=Km48y^Ea{U
z?cPJ9qXiVtI@2tQ(-*G*00021-<@0)+Cvur0oD$H7ytl$(QZev#Ao{g000001X)_c
C2dftV

literal 0
HcmV?d00001

diff --git a/tests/fsck-tests/044-invalid-extent-item-generation/test.sh b/tests/fsck-tests/044-invalid-extent-item-generation/test.sh
new file mode 100755
index 00000000..2b88a3c7
--- /dev/null
+++ b/tests/fsck-tests/044-invalid-extent-item-generation/test.sh
@@ -0,0 +1,19 @@
+#!/bin/bash
+#
+# Due to a bug in --init-extent-tree option, we may create bad generation
+# number for data extents.
+#
+# This test case will ensure btrfs check can at least detect such problem,
+# just like kernel tree-checker.
+
+source "$TEST_TOP/common"
+
+check_prereq btrfs
+
+check_image() {
+	run_mustfail \
+		"btrfs check failed to detect invalid extent item generation" \
+		"$TOP/btrfs" check "$1"
+}
+
+check_all_images
-- 
2.24.1

Re: [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes

[Su Yue]

On 2019/12/31 3:12 PM, Qu Wenruo wrote:
> There is an issue reported in github, where an fs get corrupted
> extent tree initialy, then I recommended --init-extent-tree.
>
> Although --init-extent-tree indeed fixed the original problem, it caused
> new problems, quite a lot of EXTENT_ITEMs now get bad generation number,
> which failed to mount with v5.4.
>
> The problem turns out to be a bug in backref repair code, which doesn't
> initialize extent_record::generation, causing garbage in EXTENT_ITEMs.
>
> This patch will:
> - Fix the problem
>    Patch 1
>
> - Enhance EXTENT_ITEM generation repair
>    Patch 2
>
> - Make `btrfs check` able to detect such bad generation
>    Patch 3~4
>
> - Add new test case for above ability
>    Patch 5
>
> Qu Wenruo (5):
>    btrfs-progs: check: Initialize extent_record::generation member
>    btrfs-progs: check: Populate extent generation correctly for data
>      extents
>    btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA
>      generation
>    btrfs-progs: check/original: Detect invalid extent generation
>    btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent
>      item generation
>

Nice fixes.

Reviewed-by: Su Yue <Damenly_Su@gmx.com>

>   check/main.c                                  |  36 ++++++++++++++----
>   check/mode-lowmem.c                           |  19 +++++++++
>   .../bad_extent_item_gen.img.xz                | Bin 0 -> 1916 bytes
>   .../test.sh                                   |  19 +++++++++
>   4 files changed, 67 insertions(+), 7 deletions(-)
>   create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz
>   create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh
>

Re: [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes

[David Sterba]

On Tue, Dec 31, 2019 at 03:12:15PM +0800, Qu Wenruo wrote:
> There is an issue reported in github, where an fs get corrupted
> extent tree initialy, then I recommended --init-extent-tree.
> 
> Although --init-extent-tree indeed fixed the original problem, it caused
> new problems, quite a lot of EXTENT_ITEMs now get bad generation number,
> which failed to mount with v5.4.
> 
> The problem turns out to be a bug in backref repair code, which doesn't
> initialize extent_record::generation, causing garbage in EXTENT_ITEMs.
> 
> This patch will:
> - Fix the problem
>   Patch 1
> 
> - Enhance EXTENT_ITEM generation repair
>   Patch 2
> 
> - Make `btrfs check` able to detect such bad generation
>   Patch 3~4
> 
> - Add new test case for above ability
>   Patch 5
> 
> Qu Wenruo (5):
>   btrfs-progs: check: Initialize extent_record::generation member
>   btrfs-progs: check: Populate extent generation correctly for data
>     extents
>   btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA
>     generation
>   btrfs-progs: check/original: Detect invalid extent generation
>   btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent
>     item generation

Thanks for the fixes, added to devel.