* [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes @ 2019-12-31 7:12 Qu Wenruo 2019-12-31 7:12 ` [PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member Qu Wenruo ` (6 more replies) 0 siblings, 7 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs There is an issue reported in github, where an fs get corrupted extent tree initialy, then I recommended --init-extent-tree. Although --init-extent-tree indeed fixed the original problem, it caused new problems, quite a lot of EXTENT_ITEMs now get bad generation number, which failed to mount with v5.4. The problem turns out to be a bug in backref repair code, which doesn't initialize extent_record::generation, causing garbage in EXTENT_ITEMs. This patch will: - Fix the problem Patch 1 - Enhance EXTENT_ITEM generation repair Patch 2 - Make `btrfs check` able to detect such bad generation Patch 3~4 - Add new test case for above ability Patch 5 Qu Wenruo (5): btrfs-progs: check: Initialize extent_record::generation member btrfs-progs: check: Populate extent generation correctly for data extents btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation btrfs-progs: check/original: Detect invalid extent generation btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation check/main.c | 36 ++++++++++++++---- check/mode-lowmem.c | 19 +++++++++ .../bad_extent_item_gen.img.xz | Bin 0 -> 1916 bytes .../test.sh | 19 +++++++++ 4 files changed, 67 insertions(+), 7 deletions(-) create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh -- 2.24.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo @ 2019-12-31 7:12 ` Qu Wenruo 2019-12-31 7:12 ` [PATCH 2/5] btrfs-progs: check: Populate extent generation correctly for data extents Qu Wenruo ` (5 subsequent siblings) 6 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs [BUG] When using `btrfs check --init-extent-tree`, there is a pretty high chance that the result fs can't pass tree-checker: BTRFS critical (device dm-3): corrupt leaf: block=5390336 slot=149 extent bytenr=20115456 len=4096 invalid generation, have 16384 expect (0, 360] BTRFS error (device dm-3): block=5390336 read time tree block corruption detected BTRFS error (device dm-3): failed to read block groups: -5 BTRFS error (device dm-3): open_ctree failed [CAUSE] The result fs has a pretty screwed up EXTENT_ITEMs for data extents: item 148 key (20111360 EXTENT_ITEM 4096) itemoff 8777 itemsize 53 refs 1 gen 0 flags DATA extent data backref root FS_TREE objectid 841 offset 0 count 1 item 149 key (20115456 EXTENT_ITEM 4096) itemoff 8724 itemsize 53 refs 1 gen 16384 flags DATA extent data backref root FS_TREE objectid 906 offset 0 count 1 Kernel tree-checker will accept 0 generation, but that 16384 generation is definitely going to trigger the alarm. Looking into the code, it's add_extent_rec_nolookup() allocating a new extent_rec, but not copying all members from parameter @tmpl, resulting generation not properly initialized. [FIX] Just copy tmpl->generation in add_extent_rec_nolookup(). And since all call sites have set all members of @tmpl to 0 before add_extent_rec_nolookup(), we shouldn't get garbage values. For the 0 generation problem, it will be solved in another patch. Issue: 225 (Not the initial report, but extent tree rebuild result) Signed-off-by: Qu Wenruo <wqu@suse.com> --- check/main.c | 1 + 1 file changed, 1 insertion(+) diff --git a/check/main.c b/check/main.c index 08dc9e66..2dbed091 100644 --- a/check/main.c +++ b/check/main.c @@ -4605,6 +4605,7 @@ static int add_extent_rec_nolookup(struct cache_tree *extent_cache, rec->refs = tmpl->refs; rec->extent_item_refs = tmpl->extent_item_refs; rec->parent_generation = tmpl->parent_generation; + rec->generation = tmpl->generation; INIT_LIST_HEAD(&rec->backrefs); INIT_LIST_HEAD(&rec->dups); INIT_LIST_HEAD(&rec->list); -- 2.24.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/5] btrfs-progs: check: Populate extent generation correctly for data extents 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo 2019-12-31 7:12 ` [PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member Qu Wenruo @ 2019-12-31 7:12 ` Qu Wenruo 2019-12-31 7:12 ` [PATCH 3/5] btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation Qu Wenruo ` (4 subsequent siblings) 6 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs [BUG] When using `btrfs check --init-extent-tree`, we will create incorrect generation number for data extents in extent tree: item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53 refs 1 gen 0 flags DATA extent data backref root FS_TREE objectid 257 offset 0 count 1 [CAUSE] Since data extent generation is not as obvious as tree blocks, which has header containing its generations, so for data extents, its extent_record::generation is not really updated, resulting such 0 generation. [FIX] To get generation of a data extent, there are two sources we can rely: - EXTENT_ITEM There is always a btrfs_extent_item::generation can be utilized. Although this is not possible for --init-extent-tree use case. - EXTENT_DATA We have btrfs_file_extent_item::generation for regular and preallocated data extents. Since --init-extent-tree will go through subvolume trees, this would be the main source for extent data generation. Then we only need to make add_data_backref() to accept @gen parameter, and pass it down to extent_record structure. And for the final extent item generation update, here we add extra fallback values, if we can't find FILE_EXTENT items. In that case, we just fall back to current transid. With this modification, recreated data EXTENT_ITEM now has correct generation number: item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53 refs 1 gen 6 flags DATA extent data backref root FS_TREE objectid 257 offset 0 count 1 Signed-off-by: Qu Wenruo <wqu@suse.com> --- check/main.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/check/main.c b/check/main.c index 2dbed091..88b174ab 100644 --- a/check/main.c +++ b/check/main.c @@ -4810,7 +4810,7 @@ static int add_tree_backref(struct cache_tree *extent_cache, u64 bytenr, static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr, u64 parent, u64 root, u64 owner, u64 offset, - u32 num_refs, int found_ref, u64 max_size) + u32 num_refs, u64 gen, int found_ref, u64 max_size) { struct extent_record *rec; struct data_backref *back; @@ -4826,6 +4826,7 @@ static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr, tmpl.start = bytenr; tmpl.nr = 1; tmpl.max_size = max_size; + tmpl.generation = gen; ret = add_extent_rec_nolookup(extent_cache, &tmpl); if (ret) @@ -4840,6 +4841,8 @@ static int add_data_backref(struct cache_tree *extent_cache, u64 bytenr, if (rec->max_size < max_size) rec->max_size = max_size; + if (rec->generation < gen) + rec->generation = gen; /* * If found_ref is set then max_size is the real size and must match the * existing refs. So if we have already found a ref then we need to @@ -5315,6 +5318,7 @@ static int process_extent_item(struct btrfs_root *root, u64 refs = 0; u64 offset; u64 num_bytes; + u64 gen; int metadata = 0; btrfs_item_key_to_cpu(eb, &key, slot); @@ -5340,6 +5344,7 @@ static int process_extent_item(struct btrfs_root *root, ei = btrfs_item_ptr(eb, slot, struct btrfs_extent_item); refs = btrfs_extent_refs(eb, ei); + gen = btrfs_extent_generation(eb, ei); if (btrfs_extent_flags(eb, ei) & BTRFS_EXTENT_FLAG_TREE_BLOCK) metadata = 1; else @@ -5362,6 +5367,7 @@ static int process_extent_item(struct btrfs_root *root, tmpl.metadata = metadata; tmpl.found_rec = 1; tmpl.max_size = num_bytes; + tmpl.generation = gen; add_extent_rec(extent_cache, &tmpl); ptr = (unsigned long)(ei + 1); @@ -5401,14 +5407,14 @@ static int process_extent_item(struct btrfs_root *root, dref), btrfs_extent_data_ref_offset(eb, dref), btrfs_extent_data_ref_count(eb, dref), - 0, num_bytes); + gen, 0, num_bytes); break; case BTRFS_SHARED_DATA_REF_KEY: sref = (struct btrfs_shared_data_ref *)(iref + 1); add_data_backref(extent_cache, key.objectid, offset, 0, 0, 0, btrfs_shared_data_ref_count(eb, sref), - 0, num_bytes); + gen, 0, num_bytes); break; default: fprintf(stderr, @@ -6352,7 +6358,7 @@ static int run_next_block(struct btrfs_root *root, ref), btrfs_extent_data_ref_offset(buf, ref), btrfs_extent_data_ref_count(buf, ref), - 0, root->fs_info->sectorsize); + 0, 0, root->fs_info->sectorsize); continue; } if (key.type == BTRFS_SHARED_DATA_REF_KEY) { @@ -6363,7 +6369,7 @@ static int run_next_block(struct btrfs_root *root, add_data_backref(extent_cache, key.objectid, key.offset, 0, 0, 0, btrfs_shared_data_ref_count(buf, ref), - 0, root->fs_info->sectorsize); + 0, 0, root->fs_info->sectorsize); continue; } if (key.type == BTRFS_ORPHAN_ITEM_KEY) { @@ -6403,7 +6409,8 @@ static int run_next_block(struct btrfs_root *root, add_data_backref(extent_cache, btrfs_file_extent_disk_bytenr(buf, fi), parent, owner, key.objectid, key.offset - - btrfs_file_extent_offset(buf, fi), 1, 1, + btrfs_file_extent_offset(buf, fi), 1, + btrfs_file_extent_generation(buf, fi), 1, btrfs_file_extent_disk_num_bytes(buf, fi)); } } else { @@ -6706,7 +6713,10 @@ static int record_extent(struct btrfs_trans_handle *trans, struct btrfs_extent_item); btrfs_set_extent_refs(leaf, ei, 0); - btrfs_set_extent_generation(leaf, ei, rec->generation); + if (rec->generation) + btrfs_set_extent_generation(leaf, ei, rec->generation); + else + btrfs_set_extent_generation(leaf, ei, trans->transid); if (back->is_data) { btrfs_set_extent_flags(leaf, ei, -- 2.24.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 3/5] btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo 2019-12-31 7:12 ` [PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member Qu Wenruo 2019-12-31 7:12 ` [PATCH 2/5] btrfs-progs: check: Populate extent generation correctly for data extents Qu Wenruo @ 2019-12-31 7:12 ` Qu Wenruo 2019-12-31 7:12 ` [PATCH 4/5] btrfs-progs: check/original: Detect invalid extent generation Qu Wenruo ` (3 subsequent siblings) 6 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs Since older `btrfs check --init-extent-tree` could cause invalid EXTENT_ITEM generation for data extents, add such check to lowmem mode check. Also add such generation check to file extents too. For the repair part, I don't have any good idea yet. So affected user may depend on --init-extent-tree again. Signed-off-by: Qu Wenruo <wqu@suse.com> --- check/mode-lowmem.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c index f53a0c39..aad30c28 100644 --- a/check/mode-lowmem.c +++ b/check/mode-lowmem.c @@ -2041,6 +2041,8 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, u64 csum_found; /* In byte size, sectorsize aligned */ u64 search_start; /* Logical range start we search for csum */ u64 search_len; /* Logical range len we search for csum */ + u64 gen; + u64 super_gen = btrfs_super_generation(root->fs_info->super_copy); unsigned int extent_type; unsigned int is_hole; int slot = path->slots[0]; @@ -2067,6 +2069,7 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, return check_file_extent_inline(root, path, size, end); /* Check REG_EXTENT/PREALLOC_EXTENT */ + gen = btrfs_file_extent_generation(node, fi); disk_bytenr = btrfs_file_extent_disk_bytenr(node, fi); disk_num_bytes = btrfs_file_extent_disk_num_bytes(node, fi); extent_num_bytes = btrfs_file_extent_num_bytes(node, fi); @@ -2074,6 +2077,13 @@ static int check_file_extent(struct btrfs_root *root, struct btrfs_path *path, compressed = btrfs_file_extent_compression(node, fi); is_hole = (disk_bytenr == 0) && (disk_num_bytes == 0); + if (gen > super_gen + 1) { + error( + "vainlid file extent generation, have %llu expect (0, %llu]", + gen, super_gen + 1); + err |= INVALID_GENERATION; + } + /* * Check EXTENT_DATA csum * @@ -4152,8 +4162,10 @@ static int check_extent_item(struct btrfs_fs_info *fs_info, u64 parent; u64 num_bytes; u64 root_objectid; + u64 gen; u64 owner; u64 owner_offset; + u64 super_gen = btrfs_super_generation(fs_info->super_copy); int metadata = 0; int level; struct btrfs_key key; @@ -4183,6 +4195,13 @@ static int check_extent_item(struct btrfs_fs_info *fs_info, ei = btrfs_item_ptr(eb, slot, struct btrfs_extent_item); flags = btrfs_extent_flags(eb, ei); + gen = btrfs_extent_generation(eb, ei); + if (gen > super_gen + 1) { + error( + "invalid generation for extent %llu, have %llu expect (0, %llu]", + key.objectid, gen, super_gen + 1); + err |= INVALID_GENERATION; + } if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) metadata = 1; -- 2.24.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 4/5] btrfs-progs: check/original: Detect invalid extent generation 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo ` (2 preceding siblings ...) 2019-12-31 7:12 ` [PATCH 3/5] btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation Qu Wenruo @ 2019-12-31 7:12 ` Qu Wenruo 2019-12-31 7:12 ` [PATCH 5/5] btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation Qu Wenruo ` (2 subsequent siblings) 6 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs Much like what we have done in lowmem mode, also detect and report invalid extent generation in original mode. Unlike lowmem mode, we have extent_record::generation, which is the higher number of generations in EXTENT_ITEM, EXTENT_DATA or tree block header, so there is no need to check generations in different locations. For repair, we still need to depend on --init-extent-tree, as directly modifying extent items can easily cause conflicts with delayed refs, thus it should be avoided. Signed-off-by: Qu Wenruo <wqu@suse.com> --- check/main.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/check/main.c b/check/main.c index 88b174ab..a9a236a4 100644 --- a/check/main.c +++ b/check/main.c @@ -4021,10 +4021,13 @@ static void free_extent_record_cache(struct cache_tree *extent_cache) static int maybe_free_extent_rec(struct cache_tree *extent_cache, struct extent_record *rec) { + u64 super_gen = btrfs_super_generation(global_info->super_copy); + if (rec->content_checked && rec->owner_ref_checked && rec->extent_item_refs == rec->refs && rec->refs > 0 && rec->num_duplicates == 0 && !all_backpointers_checked(rec, 0) && !rec->bad_full_backref && !rec->crossing_stripes && + rec->generation <= super_gen + 1 && !rec->wrong_chunk_type) { remove_cache_extent(extent_cache, &rec->cache); free_all_extent_backrefs(rec); @@ -7857,6 +7860,7 @@ static int check_extent_refs(struct btrfs_root *root, { struct extent_record *rec; struct cache_extent *cache; + u64 super_gen = btrfs_super_generation(root->fs_info->super_copy); int ret = 0; int had_dups = 0; int err = 0; @@ -7939,6 +7943,13 @@ static int check_extent_refs(struct btrfs_root *root, cur_err = 1; } + if (rec->generation > super_gen + 1) { + error( + "invalid generation for extent %llu, have %llu expect (0, %llu]", + rec->start, rec->generation, + super_gen + 1); + cur_err = 1; + } if (rec->refs != rec->extent_item_refs) { fprintf(stderr, "ref mismatch on [%llu %llu] ", (unsigned long long)rec->start, -- 2.24.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 5/5] btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo ` (3 preceding siblings ...) 2019-12-31 7:12 ` [PATCH 4/5] btrfs-progs: check/original: Detect invalid extent generation Qu Wenruo @ 2019-12-31 7:12 ` Qu Wenruo 2019-12-31 8:05 ` [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Su Yue 2020-01-02 18:36 ` David Sterba 6 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:12 UTC (permalink / raw) To: linux-btrfs The new image has a bad data extent item generation: item 0 key (13631488 EXTENT_ITEM 4096) itemoff 16230 itemsize 53 refs 1 gen 16384 flags DATA extent data backref root FS_TREE objectid 257 offset 0 count 1 Just like what older `btrfs check --init-extent-tree` could cause. Signed-off-by: Qu Wenruo <wqu@suse.com> --- .../bad_extent_item_gen.img.xz | Bin 0 -> 1916 bytes .../test.sh | 19 ++++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh diff --git a/tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz b/tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz new file mode 100644 index 0000000000000000000000000000000000000000..c3e30313538558384352d4987acef69f56acea2d GIT binary patch literal 1916 zcmV-?2ZQ+iH+ooF000E$*0e?f03iV!0000G&sfah3;zc?T>wRyj;C3^v%$$4d1r37 zhA1?_LT>Wbx+L2{5ct*#2hYYGZ$^%K*7}6#@?zh(OKyt!ovg(ZXD3s(w#SpNSsRep zn;&Hxy|d8!Hi<RRaJ(=+T!Nf*Grt0$9YKM_uNCH&f@?oOPNUPM#dHBgutu$*YH7Sd z;8vxqgB?;P8%c$1w4ktrsMRs{rq$7d%o`v|q>*o#PdRXN)B+ivWj$v1%hPZx8E=T< zIFi++(r$u*0A%YuW7aB1tv#h~la(SW&<75ZbX{2HpaX7<+lhn{3MIgh{Z)k&XH7!- z$i?+3e{kf1rujr8NoiytrfWvbj+^erx%LwZ6`4?RpV8Uj2%8PN6}TKB72stHk=erF zZfbD^3`i=rDXXQqVgn?n3)M_=zMVg460_al*1EQe6RehzF>**=@B3DZM^tdid(Av& zv|+=Xx*;MKXd`nuxO6M=Gu{<nMFP2`CfHR(zeIi>c4coH69RgW26`P$AZ|-HLCS+e zeQ+IA0>~0ckk&~DR#h{#Lp@mbC@Ee29#5kC5@O{=_t7tZ>H1Nh>jBd8C9(CT$kAIA z7N+}Kp`|CSujkwTDP(M;#QF)=>_SC=^65fQf+GYi>GVs5(%FRmcL!ay1An28JAMk9 zjF005pXjxPsZF>PAgk_{*)^zrWuOGYCr%m;Z5Iij7$)S9V}N6eTIG4xL9&v%JbXRN z!Hso@wG^`ob`AffkCnd`Hq4{5KcX>fJ^e$rcHH8Vt&)x$9OPVOMxN(qb_|1}&%GF0 z^(YphZ4V5@1-TMwXWqaOpZfoJQo+W)#S=v;wkY}p>|u*6uX|37vL`vo58!U0>kdzd zF)xr?nN{c|Z|FhwP?&?Sb)cnhgGQRDRvhYE3f~?Lx#nb(g>g~ulpZ74ywFa*bMr<8 zw1dO*Si2v{xcVNYXD(I{O`qzgT_6mJl2>ZW!KqB~!ZD7pmsg=|9;s8j4A(pTixI+T z>-*2VM+)UY=qM+OKyI)*4IOp-GgTH{U~;T1(vr-uvOhlmhg%X%qdhaZ-z1*p%unU^ z+Gm-zwG_iMJ%jD{@MrryJ-7x-lk_fc*8o9CKOs@mOA+&`I;q7-`+HWC4D4Pyk0m#; zE&%t+T#KL|eD~39LL~%KEG=Q))Jo7|{|tF|A^CKck~~fr?d?bLZRt{bW}E#7$g6*u zlwJj~7_!f1LK~Qd`bwbgd+m0S#kFNhm*dMOi5|fFYW>4}hNhZiYMJBX1DW-+FC3>> zKuEv?mDdP`0a_@$pvSdrC4Kf-2q7IVCR?v9%A-APiGsO_+8)i`KufH<L&=cpE&*CO zmexp=`$4a)^7l^>1@WbrDXdUBrD>y(hTNdxMbV#PXijZ=v{s+Fm*c|Mz^W7OeAM?0 z<>ZpcI$ZZH<e!m`Sjehu(2{|;^gfYc976$TBC@?SUR>CGbrG-abJ+$@cfUk$sElUJ zeJ4G+K-_tP^o;FLZ9?_g!!`ZZhx?*HSeSaM^YNfVQQ+I<OXm2ZEY5v}aR+XgPGL4^ znX9pTiZSMVDu5knh#?$4$En+S-i^6O?}Ch4LVZylEj`5s?5hTHtSZgJXOInlu@~_Y zRkYV<52qb0%p|f_KUfDAUr-kP*)!bx|BEGV=+G&~2jzenP4`5SMD+G9w{w}UXHazP zcoZ%_r^64L_ByX%`pG?^$_Tout5bnJ!1%p>AB_k?aw=c6dB+mBIk|s)L%u*2@Rg^h zq*qPE$vg%N0_Ul)p)m9GEUa5QRUvQb)TX@L{cy;H@2HiFWhQIun8M=P>%M|FA94VF z6Z<MCz?ueUp$9AAC%1N<!tL2~LD_eytni8i`+LG=)7;TcHHi#N&4yX(hEgT+ZprNu zIbs`uKXvCj_rf)Y9mU5l)!vIT)7}J3+gSyCaTCm^Y(^mGzNuL1Czz>XciBSk=z^i# z*Ewxo65u1o;3PFe;aIh84y?2eNdGvz5{&Or82y7KJ4tIK-j)u&BUg&xc}}Q!MS#6r z<#h&3ZEYc6NV!?kl1N5BUZyp*yxWYL1hELeBKWO(j`a^#hX-2Kh%Q-3<s&UsQ=SL9 zi%Ku&({<6VGdX5}Ojk*sVj_T!)D)3zgoJn!=fmrtXWRune6!1AXPA(vrdrPqFnb<H zf+>9amd!CJ)4HXuyfdA_lS0b*zdo<v<3`%1mRm|DI|GN`j$1B!!-0nwB{IJX@~Slb zvYVN~pn0i35$7Z(ecPteuLdb}kz*upp810p``+>(aF?s0XE~EZ=Lw_I2Hbr*H|jWf z3jC6`6x+nW2DNkTh(HE!jp(YB12(98r24u9t`g4>DF;L3({!kWwIV@;?O0aVl$}e; zEdcyEP+0;aOD}}5hEIO$Och-)`a(0J$#4gX1AcuZzQTUEnjCiFSz4d=Km48y^Ea{U z?cPJ9qXiVtI@2tQ(-*G*00021-<@0)+Cvur0oD$H7ytl$(QZev#Ao{g000001X)_c C2dftV literal 0 HcmV?d00001 diff --git a/tests/fsck-tests/044-invalid-extent-item-generation/test.sh b/tests/fsck-tests/044-invalid-extent-item-generation/test.sh new file mode 100755 index 00000000..2b88a3c7 --- /dev/null +++ b/tests/fsck-tests/044-invalid-extent-item-generation/test.sh @@ -0,0 +1,19 @@ +#!/bin/bash +# +# Due to a bug in --init-extent-tree option, we may create bad generation +# number for data extents. +# +# This test case will ensure btrfs check can at least detect such problem, +# just like kernel tree-checker. + +source "$TEST_TOP/common" + +check_prereq btrfs + +check_image() { + run_mustfail \ + "btrfs check failed to detect invalid extent item generation" \ + "$TOP/btrfs" check "$1" +} + +check_all_images -- 2.24.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo ` (4 preceding siblings ...) 2019-12-31 7:12 ` [PATCH 5/5] btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation Qu Wenruo @ 2019-12-31 8:05 ` Su Yue 2020-01-02 18:36 ` David Sterba 6 siblings, 0 replies; 9+ messages in thread From: Su Yue @ 2019-12-31 8:05 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs On 2019/12/31 3:12 PM, Qu Wenruo wrote: > There is an issue reported in github, where an fs get corrupted > extent tree initialy, then I recommended --init-extent-tree. > > Although --init-extent-tree indeed fixed the original problem, it caused > new problems, quite a lot of EXTENT_ITEMs now get bad generation number, > which failed to mount with v5.4. > > The problem turns out to be a bug in backref repair code, which doesn't > initialize extent_record::generation, causing garbage in EXTENT_ITEMs. > > This patch will: > - Fix the problem > Patch 1 > > - Enhance EXTENT_ITEM generation repair > Patch 2 > > - Make `btrfs check` able to detect such bad generation > Patch 3~4 > > - Add new test case for above ability > Patch 5 > > Qu Wenruo (5): > btrfs-progs: check: Initialize extent_record::generation member > btrfs-progs: check: Populate extent generation correctly for data > extents > btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA > generation > btrfs-progs: check/original: Detect invalid extent generation > btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent > item generation > Nice fixes. Reviewed-by: Su Yue <Damenly_Su@gmx.com> > check/main.c | 36 ++++++++++++++---- > check/mode-lowmem.c | 19 +++++++++ > .../bad_extent_item_gen.img.xz | Bin 0 -> 1916 bytes > .../test.sh | 19 +++++++++ > 4 files changed, 67 insertions(+), 7 deletions(-) > create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz > create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo ` (5 preceding siblings ...) 2019-12-31 8:05 ` [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Su Yue @ 2020-01-02 18:36 ` David Sterba 6 siblings, 0 replies; 9+ messages in thread From: David Sterba @ 2020-01-02 18:36 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs On Tue, Dec 31, 2019 at 03:12:15PM +0800, Qu Wenruo wrote: > There is an issue reported in github, where an fs get corrupted > extent tree initialy, then I recommended --init-extent-tree. > > Although --init-extent-tree indeed fixed the original problem, it caused > new problems, quite a lot of EXTENT_ITEMs now get bad generation number, > which failed to mount with v5.4. > > The problem turns out to be a bug in backref repair code, which doesn't > initialize extent_record::generation, causing garbage in EXTENT_ITEMs. > > This patch will: > - Fix the problem > Patch 1 > > - Enhance EXTENT_ITEM generation repair > Patch 2 > > - Make `btrfs check` able to detect such bad generation > Patch 3~4 > > - Add new test case for above ability > Patch 5 > > Qu Wenruo (5): > btrfs-progs: check: Initialize extent_record::generation member > btrfs-progs: check: Populate extent generation correctly for data > extents > btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA > generation > btrfs-progs: check/original: Detect invalid extent generation > btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent > item generation Thanks for the fixes, added to devel. ^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes @ 2019-12-31 7:17 Qu Wenruo 0 siblings, 0 replies; 9+ messages in thread From: Qu Wenruo @ 2019-12-31 7:17 UTC (permalink / raw) To: linux-btrfs There is an issue reported in github, where an fs get corrupted extent tree initialy, then I recommended --init-extent-tree. Although --init-extent-tree indeed fixed the original problem, it caused new problems, quite a lot of EXTENT_ITEMs now get bad generation number, which failed to mount with v5.4. The problem turns out to be a bug in backref repair code, which doesn't initialize extent_record::generation, causing garbage in EXTENT_ITEMs. This patch will: - Fix the problem Patch 1 - Enhance EXTENT_ITEM generation repair Patch 2 - Make `btrfs check` able to detect such bad generation Patch 3~4 - Add new test case for above ability Patch 5 Qu Wenruo (5): btrfs-progs: check: Initialize extent_record::generation member btrfs-progs: check: Populate extent generation correctly for data extents btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation btrfs-progs: check/original: Detect invalid extent generation btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation check/main.c | 36 ++++++++++++++---- check/mode-lowmem.c | 19 +++++++++ .../bad_extent_item_gen.img.xz | Bin 0 -> 1916 bytes .../test.sh | 19 +++++++++ 4 files changed, 67 insertions(+), 7 deletions(-) create mode 100644 tests/fsck-tests/044-invalid-extent-item-generation/bad_extent_item_gen.img.xz create mode 100755 tests/fsck-tests/044-invalid-extent-item-generation/test.sh -- 2.24.1 ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-01-02 18:36 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-31 7:12 [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Qu Wenruo 2019-12-31 7:12 ` [PATCH 1/5] btrfs-progs: check: Initialize extent_record::generation member Qu Wenruo 2019-12-31 7:12 ` [PATCH 2/5] btrfs-progs: check: Populate extent generation correctly for data extents Qu Wenruo 2019-12-31 7:12 ` [PATCH 3/5] btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation Qu Wenruo 2019-12-31 7:12 ` [PATCH 4/5] btrfs-progs: check/original: Detect invalid extent generation Qu Wenruo 2019-12-31 7:12 ` [PATCH 5/5] btrfs-progs: fsck-tests: Make sure btrfs check can detect bad extent item generation Qu Wenruo 2019-12-31 8:05 ` [PATCH 0/5] btrfs-progs: Bad extent item generation related bug fixes Su Yue 2020-01-02 18:36 ` David Sterba 2019-12-31 7:17 Qu Wenruo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).