linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation
@ 2019-08-12  6:34 Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 1/3] btrfs-progs: check/lowmem: Check and repair root generation Qu Wenruo
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Qu Wenruo @ 2019-08-12  6:34 UTC (permalink / raw)
  To: linux-btrfs

Kernel is going to reject invalid root generation.

Consider the existing checks are causing some error reports, we should
handle such problem in advance, so that's the patchset is going to do,
check and repair such invalid root generation.

Changelog:
v2:
- Use existing recow_extent_buffer() to do the repair

v2.1:
- Add beacon file to allow lowmem mode repair for newly added test case.

Qu Wenruo (3):
  btrfs-progs: check/lowmem: Check and repair root generation
  btrfs-progs: check/original: Check and repair root item geneartion
  btrfs-progs: fsck-tests: Add test case for invalid root generation

 check/main.c                                  |  57 +++++++-----------
 check/mode-common.c                           |  36 +++++++++++
 check/mode-common.h                           |   1 +
 check/mode-lowmem.c                           |  17 ++++++
 check/mode-lowmem.h                           |   1 +
 .../.lowmem_repairable                        |   0
 .../default_case.img                          | Bin 0 -> 3072 bytes
 7 files changed, 76 insertions(+), 36 deletions(-)
 create mode 100644 tests/fsck-tests/041-invalid-root-generation/.lowmem_repairable
 create mode 100644 tests/fsck-tests/041-invalid-root-generation/default_case.img

-- 
2.22.0


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v2.1 1/3] btrfs-progs: check/lowmem: Check and repair root generation
  2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
@ 2019-08-12  6:34 ` Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 2/3] btrfs-progs: check/original: Check and repair root item geneartion Qu Wenruo
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Qu Wenruo @ 2019-08-12  6:34 UTC (permalink / raw)
  To: linux-btrfs

Since kernel is going to reject any root item which is newer than super
block generation, we need to provide a way to fix such problem in
btrfs-check.

This patch addes the ability to report and repair root generation in
lowmem mode.

This is done by cowing the root node, so we will update the root
generation along with the root node generation at commit transaction
time.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c        | 36 ------------------------------------
 check/mode-common.c | 36 ++++++++++++++++++++++++++++++++++++
 check/mode-common.h |  1 +
 check/mode-lowmem.c | 17 +++++++++++++++++
 check/mode-lowmem.h |  1 +
 5 files changed, 55 insertions(+), 36 deletions(-)

diff --git a/check/main.c b/check/main.c
index 0cc6fdba0289..98b07fcb58c8 100644
--- a/check/main.c
+++ b/check/main.c
@@ -9024,42 +9024,6 @@ again:
 	return ret;
 }
 
-static int recow_extent_buffer(struct btrfs_root *root, struct extent_buffer *eb)
-{
-	struct btrfs_path path;
-	struct btrfs_trans_handle *trans;
-	struct btrfs_key key;
-	int ret;
-
-	printf("Recowing metadata block %llu\n", eb->start);
-	key.objectid = btrfs_header_owner(eb);
-	key.type = BTRFS_ROOT_ITEM_KEY;
-	key.offset = (u64)-1;
-
-	root = btrfs_read_fs_root(root->fs_info, &key);
-	if (IS_ERR(root)) {
-		fprintf(stderr, "Couldn't find owner root %llu\n",
-			key.objectid);
-		return PTR_ERR(root);
-	}
-
-	trans = btrfs_start_transaction(root, 1);
-	if (IS_ERR(trans))
-		return PTR_ERR(trans);
-
-	btrfs_init_path(&path);
-	path.lowest_level = btrfs_header_level(eb);
-	if (path.lowest_level)
-		btrfs_node_key_to_cpu(eb, &key, 0);
-	else
-		btrfs_item_key_to_cpu(eb, &key, 0);
-
-	ret = btrfs_search_slot(trans, root, &key, &path, 0, 1);
-	btrfs_commit_transaction(trans, root);
-	btrfs_release_path(&path);
-	return ret;
-}
-
 static int delete_bad_item(struct btrfs_root *root, struct bad_item *bad)
 {
 	struct btrfs_path path;
diff --git a/check/mode-common.c b/check/mode-common.c
index d5f6c8ef97b1..195b6efaa7aa 100644
--- a/check/mode-common.c
+++ b/check/mode-common.c
@@ -924,3 +924,39 @@ int check_repair_free_space_inode(struct btrfs_fs_info *fs_info,
 	}
 	return ret;
 }
+
+int recow_extent_buffer(struct btrfs_root *root, struct extent_buffer *eb)
+{
+	struct btrfs_path path;
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key key;
+	int ret;
+
+	printf("Recowing metadata block %llu\n", eb->start);
+	key.objectid = btrfs_header_owner(eb);
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = (u64)-1;
+
+	root = btrfs_read_fs_root(root->fs_info, &key);
+	if (IS_ERR(root)) {
+		fprintf(stderr, "Couldn't find owner root %llu\n",
+			key.objectid);
+		return PTR_ERR(root);
+	}
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans))
+		return PTR_ERR(trans);
+
+	btrfs_init_path(&path);
+	path.lowest_level = btrfs_header_level(eb);
+	if (path.lowest_level)
+		btrfs_node_key_to_cpu(eb, &key, 0);
+	else
+		btrfs_item_key_to_cpu(eb, &key, 0);
+
+	ret = btrfs_search_slot(trans, root, &key, &path, 0, 1);
+	btrfs_commit_transaction(trans, root);
+	btrfs_release_path(&path);
+	return ret;
+}
diff --git a/check/mode-common.h b/check/mode-common.h
index 4c169c6e3b29..161b84a8deb0 100644
--- a/check/mode-common.h
+++ b/check/mode-common.h
@@ -155,4 +155,5 @@ static inline bool is_valid_imode(u32 imode)
 	return true;
 }
 
+int recow_extent_buffer(struct btrfs_root *root, struct extent_buffer *eb);
 #endif
diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
index a2be0e6d7034..da6b6fd86ae3 100644
--- a/check/mode-lowmem.c
+++ b/check/mode-lowmem.c
@@ -4957,6 +4957,7 @@ static int check_btrfs_root(struct btrfs_root *root, int check_all)
 	struct btrfs_path path;
 	struct node_refs nrefs;
 	struct btrfs_root_item *root_item = &root->root_item;
+	u64 super_generation = btrfs_super_generation(root->fs_info->super_copy);
 	int ret;
 	int level;
 	int err = 0;
@@ -4978,6 +4979,22 @@ static int check_btrfs_root(struct btrfs_root *root, int check_all)
 	level = btrfs_header_level(root->node);
 	btrfs_init_path(&path);
 
+	if (btrfs_root_generation(root_item) > super_generation + 1) {
+		error(
+	"invalid root generation for root %llu, have %llu expect (0, %llu)",
+		      root->root_key.objectid, btrfs_root_generation(root_item),
+		      super_generation + 1);
+		err |= INVALID_GENERATION;
+		if (repair) {
+			root->node->flags |= EXTENT_BAD_TRANSID;
+			ret = recow_extent_buffer(root, root->node);
+			if (!ret) {
+				printf("Reset generation for root %llu\n",
+					root->root_key.objectid);
+				err &= ~INVALID_GENERATION;
+			}
+		}
+	}
 	if (btrfs_root_refs(root_item) > 0 ||
 	    btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
 		path.nodes[level] = root->node;
diff --git a/check/mode-lowmem.h b/check/mode-lowmem.h
index d2983fd12eb4..0361fb3382b1 100644
--- a/check/mode-lowmem.h
+++ b/check/mode-lowmem.h
@@ -47,6 +47,7 @@
 #define INODE_FLAGS_ERROR	(1<<23) /* Invalid inode flags */
 #define DIR_ITEM_HASH_MISMATCH	(1<<24) /* Dir item hash mismatch */
 #define INODE_MODE_ERROR	(1<<25) /* Bad inode mode */
+#define INVALID_GENERATION	(1<<26)	/* Generation is too new */
 
 /*
  * Error bit for low memory mode check.
-- 
2.22.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH v2.1 2/3] btrfs-progs: check/original: Check and repair root item geneartion
  2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 1/3] btrfs-progs: check/lowmem: Check and repair root generation Qu Wenruo
@ 2019-08-12  6:34 ` Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 3/3] btrfs-progs: fsck-tests: Add test case for invalid root generation Qu Wenruo
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Qu Wenruo @ 2019-08-12  6:34 UTC (permalink / raw)
  To: linux-btrfs

Add such ability to original mode to fix root generation mismatch, which
can be rejected by kernel.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/check/main.c b/check/main.c
index 98b07fcb58c8..ae3ce13e9466 100644
--- a/check/main.c
+++ b/check/main.c
@@ -3437,8 +3437,10 @@ static int check_fs_root(struct btrfs_root *root,
 {
 	int ret = 0;
 	int err = 0;
+	bool generation_err = false;
 	int wret;
 	int level;
+	u64 super_generation;
 	struct btrfs_path path;
 	struct shared_node root_node;
 	struct root_record *rec;
@@ -3449,6 +3451,23 @@ static int check_fs_root(struct btrfs_root *root,
 	struct unaligned_extent_rec_t *urec;
 	struct unaligned_extent_rec_t *tmp;
 
+	super_generation = btrfs_super_generation(root->fs_info->super_copy);
+	if (btrfs_root_generation(root_item) > super_generation + 1) {
+		error(
+	"invalid generation for root %llu, have %llu expect (0, %llu]",
+		      root->root_key.objectid, btrfs_root_generation(root_item),
+		      super_generation + 1);
+		generation_err = true;
+		if (repair) {
+			root->node->flags |= EXTENT_BAD_TRANSID;
+			ret = recow_extent_buffer(root, root->node);
+			if (!ret) {
+				printf("Reset generation for root %llu\n",
+					root->root_key.objectid);
+				generation_err = false;
+			}
+		}
+	}
 	/*
 	 * Reuse the corrupt_block cache tree to record corrupted tree block
 	 *
@@ -3597,6 +3616,8 @@ skip_walking:
 
 	free_corrupt_blocks_tree(&corrupt_blocks);
 	root->fs_info->corrupt_blocks = NULL;
+	if (!ret && generation_err)
+		ret = -1;
 	return ret;
 }
 
-- 
2.22.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* [PATCH v2.1 3/3] btrfs-progs: fsck-tests: Add test case for invalid root generation
  2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 1/3] btrfs-progs: check/lowmem: Check and repair root generation Qu Wenruo
  2019-08-12  6:34 ` [PATCH v2.1 2/3] btrfs-progs: check/original: Check and repair root item geneartion Qu Wenruo
@ 2019-08-12  6:34 ` Qu Wenruo
  2019-08-12  8:22 ` [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Nikolay Borisov
  2019-08-26 17:07 ` David Sterba
  4 siblings, 0 replies; 6+ messages in thread
From: Qu Wenruo @ 2019-08-12  6:34 UTC (permalink / raw)
  To: linux-btrfs

The image contains a fs tree whose generation is over 100 larger than
super block generation.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../.lowmem_repairable                           |   0
 .../041-invalid-root-generation/default_case.img | Bin 0 -> 3072 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/041-invalid-root-generation/.lowmem_repairable
 create mode 100644 tests/fsck-tests/041-invalid-root-generation/default_case.img

diff --git a/tests/fsck-tests/041-invalid-root-generation/.lowmem_repairable b/tests/fsck-tests/041-invalid-root-generation/.lowmem_repairable
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tests/fsck-tests/041-invalid-root-generation/default_case.img b/tests/fsck-tests/041-invalid-root-generation/default_case.img
new file mode 100644
index 0000000000000000000000000000000000000000..4a2c8d88a1a885fb9bcae56adc98dcbe75681441
GIT binary patch
literal 3072
zcmeB9n_BcpHD)gZ6tDnkMlb?0gn%pthX4@Czy>BSfXQ_r0fviUb|5pDGfIqxz-R~z
zy%4ClwfpC>Btd4mfEQECdaGAmZV8>XXi|>uFI66%o`gcT6*YS%y9Y!S?9tcuX+Esl
zF~yldnNQhLK}2{)N;B6XQSOE>PKFciax94`lz7sS!>ruK%Dbk3IZ;A?N9EVRNcQIV
zGhgNx#`NdqtUUDP*9o@&3lom8F_?7PCMGm2eap6FN>9MTmP6?*>b7cARJbno&wC*7
zNqobp%ZHl+4Y=G6YDlmyP7qnZlyO<?;uT(pr}8^GPPJd#q*GuwtFEm(<I@V?WVcPS
zQVrL11f8Cmb1XW^z35gsgYP{@foksEjT4Q^<r;QOp2@iLZU2#chw1z-Ppuc6ynJdd
z-;0kD4X1!2?vLdirY{fO)87>6pv1r+b1Y$D#jUru?XwR#NF4w8{7?HWRd4q2!fQ8O
zUAOb8K8VsPd7-vqiqlocgN_;=(=RQpUNHNC^ox53mnMZ57M7F*7i)QXD8IYUAM>v0
z{N2BC2QRPvdqvgZ^RvRTuW@z1PsJ}!YQ8ghiqF>j;k(wX34CsPb=8-(AIraO-nGu1
z|NEhK%SRSEpC2p5i&PebOj8Q3-l)|v<6YCa>YJic^Q6BS73$zfV0uDUwK==yT0j1E
zpVxVQ;PTqb`#yc&T_Jbs?cL+g>~_xZztvZDWWC-c;r)BQ{In6BV;(yFiROHpr?;-Z
zegFTtXN<q?-GqD#H--1z_v-%X&Yc(S1Pq0<>${vBMc5zg?3!kp^?{Y;^j)=xBlfw8
zlW%4jzu9zFB}3>)lJO0_8#OHV4{XU-?7peA=#c4}2b(_rKK19EWpP^A)+JJMKPq0=
z@7KBYxpZ&Hx&HY5PB&bi*Of@RpUkn)tuJ#snI^5jyZzvkUDJ2Zn>k@$ZS~h19Enq3
ziR3+8UZ{uW*9Y>cPBQ8B(q`{ZYfSyVcK`oK&3)hJom4*%A?XWr?H&INnNS(_58G$^
zUYjAqW0K&&)wQf+$5(+VztVSKj9waWA-lCnS0ubE$Un|3g5Ocg+ti$Yuc)Yt|D%(k
z0{mKPf-Dp7?73z?UF4-=UfHwUvh>aMCdbTv{FE`$|6E(2_Kf$<OLP0$+Ar%24rr^N
z`)jfFQ_0`v<juvx)BECh5_PxN+D$*QGdq9(ziW3^pVx`~{iOR+_4&Uaixg$UdC<Ia
zU`5@JXD2=_*t7iKexs<r|9jWE*p+<i+k8)_y<qF(GbK5H!tVWJN7MAr^=E*VsN0*_
zuQTG~|2)q=CvR0*6|wvBnLFiA9Y24)^8dQq>i_fK>{<W$>fzI~H+M^$hnq=pf7SlF
zw7jpqpj{kPvm=3q|9d}vsd#eTes%LNpk{Nn>;I2eetub~Ww~bCKQpHlTcz@4_#eIt
zdmXY)yF!y~s!D}@-0{w|f_$UU@(&4W8Apz$-^yN7+W%wkoW2z)P4ANfPP51#T`-YT
z@3_L6a`vm=q-H+2`Hcg~Mj*4HS$C~9|4pmU{l$l^MA}MEq+5w>l2vOj-LY}%i`xq&
z@)u8hJ?quKMB{$`-9I*36ffQVytY;@Va|(%Q)_ZBe15m`zrXg~_N)KJt@KUwzUBTA
zI8Zk8zw%wjp!fe>@+VCB>i^bgQR!#<>DA5G9?35Qw@;89z#h1}&+gjf<zJsGuHAb6
z+V1yz|F7H<%KiIty(!PD{eKS4JnMLVF(gXrWB#4{HaTOLNc!KWD+TP%zO7MRAZzm{
zMd(Q8GWAQxY|kdBy<vGPm@e<{=9BXN_v&QD7+!0~>Q(1w9V~JG@aNa``SJJi_T1)?
z%aH57X=*0teQm}Ww$J@)j(<KlTkqH~0VOgHFbc1pcWbNH)UEG=7M^<dxX5|g<NY_c
ztw_6FwsY@ByZ*(S?|(fSy(9PFw$?tUQ(H<3v(>fQ{&c6GpIf=M%5dJv_Mpr)_kDhE
zIsducZ{9!ax3~Y=)@**x6!!g-+_9JE!uj>{&*|l@$uSXo75>V*dheOK{__2z&cDvx
nefQhqUU6Ej{%?zS=?|{uL?@r!*7`x*L{ifYDbz-p!#M;1<_P#~

literal 0
HcmV?d00001

-- 
2.22.0


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation
  2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
                   ` (2 preceding siblings ...)
  2019-08-12  6:34 ` [PATCH v2.1 3/3] btrfs-progs: fsck-tests: Add test case for invalid root generation Qu Wenruo
@ 2019-08-12  8:22 ` Nikolay Borisov
  2019-08-26 17:07 ` David Sterba
  4 siblings, 0 replies; 6+ messages in thread
From: Nikolay Borisov @ 2019-08-12  8:22 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs



On 12.08.19 г. 9:34 ч., Qu Wenruo wrote:
> Kernel is going to reject invalid root generation.
> 
> Consider the existing checks are causing some error reports, we should
> handle such problem in advance, so that's the patchset is going to do,
> check and repair such invalid root generation.
> 
> Changelog:
> v2:
> - Use existing recow_extent_buffer() to do the repair
> 
> v2.1:
> - Add beacon file to allow lowmem mode repair for newly added test case.
> 
> Qu Wenruo (3):
>   btrfs-progs: check/lowmem: Check and repair root generation
>   btrfs-progs: check/original: Check and repair root item geneartion
>   btrfs-progs: fsck-tests: Add test case for invalid root generation
> 
>  check/main.c                                  |  57 +++++++-----------
>  check/mode-common.c                           |  36 +++++++++++
>  check/mode-common.h                           |   1 +
>  check/mode-lowmem.c                           |  17 ++++++
>  check/mode-lowmem.h                           |   1 +
>  .../.lowmem_repairable                        |   0
>  .../default_case.img                          | Bin 0 -> 3072 bytes
>  7 files changed, 76 insertions(+), 36 deletions(-)
>  create mode 100644 tests/fsck-tests/041-invalid-root-generation/.lowmem_repairable
>  create mode 100644 tests/fsck-tests/041-invalid-root-generation/default_case.img
> 

The patchset looks rather trivial - just use recow_extent_buffer when
you find a root with larger generation. So:

Reviewed-by: Nikolay Borisov <nborisov@suse.com>


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation
  2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
                   ` (3 preceding siblings ...)
  2019-08-12  8:22 ` [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Nikolay Borisov
@ 2019-08-26 17:07 ` David Sterba
  4 siblings, 0 replies; 6+ messages in thread
From: David Sterba @ 2019-08-26 17:07 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs

On Mon, Aug 12, 2019 at 02:34:19PM +0800, Qu Wenruo wrote:
> Kernel is going to reject invalid root generation.
> 
> Consider the existing checks are causing some error reports, we should
> handle such problem in advance, so that's the patchset is going to do,
> check and repair such invalid root generation.
> 
> Changelog:
> v2:
> - Use existing recow_extent_buffer() to do the repair
> 
> v2.1:
> - Add beacon file to allow lowmem mode repair for newly added test case.
> 
> Qu Wenruo (3):
>   btrfs-progs: check/lowmem: Check and repair root generation
>   btrfs-progs: check/original: Check and repair root item geneartion
>   btrfs-progs: fsck-tests: Add test case for invalid root generation

Added to devel, thanks!

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2019-08-26 17:06 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-12  6:34 [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Qu Wenruo
2019-08-12  6:34 ` [PATCH v2.1 1/3] btrfs-progs: check/lowmem: Check and repair root generation Qu Wenruo
2019-08-12  6:34 ` [PATCH v2.1 2/3] btrfs-progs: check/original: Check and repair root item geneartion Qu Wenruo
2019-08-12  6:34 ` [PATCH v2.1 3/3] btrfs-progs: fsck-tests: Add test case for invalid root generation Qu Wenruo
2019-08-12  8:22 ` [PATCH v2.1 0/3] btrfs-progs: Check and repair invalid root item generation Nikolay Borisov
2019-08-26 17:07 ` David Sterba

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).