[PATCH 0/3] btrfs-progs: detect and fix orphan roots without orphan items

[PATCH 0/3] btrfs-progs: detect and fix orphan roots without orphan items

[Qu Wenruo]

There is bug report from Zhenyu Wu <wuzy001@gmail.com>, where even there
is only one subvolume, his filesystem still consumes way more space than
expected.

The problem is, there are 58 snapshots, but all of them have no
ROOT_REF/ROOT_BACKREF, and all their root refs is 0.

But strangely there is no orphan items for them.

This means, btrfs kernel module won't detect them as orphan nor queue
them to be removed.

"btrfs subvolume delete -i" do no helps, as it still requires
ROOT_BACKREF to resolve the dentry.

For now, we should teach btrfs check to detect and repair the case by
adding new orphan items for them.

The root cause I guess is some strange behavior for orphan item insert
and subvolume unlink.
As the report mentioned that he had one forced power off.

Qu Wenruo (3):
  btrfs-progs: check/lowmem: add the ability to detect and repair orphan
    subvolume trees which doesn't have orphan item
  btrfs-progs: check/original: detect and repair orphan subvolume
    without orphan item
  btrfs-progs: fsck-tests: add test image for orphan roots without an
    orphan item

 check/main.c                                  |  37 ++++--------
 check/mode-common.c                           |  56 ++++++++++++++++++
 check/mode-common.h                           |   3 +
 check/mode-lowmem.c                           |  42 +++++++------
 .../048-orphan-roots/.lowmem_repairable       |   0
 .../048-orphan-roots/default.img.xz           | Bin 0 -> 2584 bytes
 6 files changed, 93 insertions(+), 45 deletions(-)
 create mode 100644 tests/fsck-tests/048-orphan-roots/.lowmem_repairable
 create mode 100644 tests/fsck-tests/048-orphan-roots/default.img.xz

-- 
2.32.0

[PATCH 1/3] btrfs-progs: check/lowmem: add the ability to detect and repair orphan subvolume trees which doesn't have orphan item

[Qu Wenruo]

There is a bug report from the mail list that, after certain
btrfs-zero-log operation, the filesystem has inaccessible subvolumes,
and there is no way to delete them.

Such subvolumes have no ROOT_REF/ROOT_BACKREF, and their root_refs is 0.
Without an orphan item, kernel won't detect them as orphan, thus no
cleanup will happen.

Btrfs check won't report this as a problem either.

Such ghost subvolumes will just waste disk space, and can be pretty hard
to detect without proper btrfs check support.

For the repair part, we just add orphan item so later kernel can handle
it.

Since the check for orphan item and repair can be reused between
original and lowmem mode, move those functions to mode-common.[ch].

Reported-by: Zhenyu Wu <wuzy001@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/mode-common.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
 check/mode-common.h |  3 +++
 check/mode-lowmem.c | 42 +++++++++++++++++++---------------
 3 files changed, 82 insertions(+), 19 deletions(-)

diff --git a/check/mode-common.c b/check/mode-common.c
index cb22f3233c00..d8c18f6603bf 100644
--- a/check/mode-common.c
+++ b/check/mode-common.c
@@ -1243,3 +1243,59 @@ out:
 	btrfs_release_path(&path);
 	return ret;
 }
+
+/*
+ * Check if we have orphan item for @objectid.
+ *
+ * Note: if something wrong happened during search, we consider it as no orphan
+ * item.
+ */
+bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid)
+{
+	struct btrfs_path path;
+	struct btrfs_key key;
+	int ret;
+
+	btrfs_init_path(&path);
+	key.objectid = BTRFS_ORPHAN_OBJECTID;
+	key.type = BTRFS_ORPHAN_ITEM_KEY;
+	key.offset = objectid;
+	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
+	btrfs_release_path(&path);
+	if (ret == 0)
+		return true;
+	return false;
+}
+
+int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid)
+{
+	struct btrfs_path path;
+	struct btrfs_trans_handle *trans;
+	int ret;
+
+	btrfs_init_path(&path);
+
+	trans = btrfs_start_transaction(fs_info->tree_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error("failed to start transaction: %m");
+		return ret;
+	}
+	ret = btrfs_add_orphan_item(trans, fs_info->tree_root, &path, rootid);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to insert orphan item: %m");
+		goto error;
+	}
+	btrfs_release_path(&path);
+	ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+	if (ret < 0)
+		goto error;
+	printf("Inserted orphan root item for root %llu\n", rootid);
+	return ret;
+error:
+	btrfs_release_path(&path);
+	btrfs_abort_transaction(trans, ret);
+	return ret;
+}
diff --git a/check/mode-common.h b/check/mode-common.h
index 3ba29ecab6cd..c44815a171ac 100644
--- a/check/mode-common.h
+++ b/check/mode-common.h
@@ -192,4 +192,7 @@ static inline void btrfs_check_subpage_eb_alignment(u64 start, u32 len)
 			start, start + len);
 }
 
+bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid);
+int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid);
+
 #endif
diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
index 2f736712bc7f..1ef781ad20bc 100644
--- a/check/mode-lowmem.c
+++ b/check/mode-lowmem.c
@@ -2500,24 +2500,6 @@ out:
 	return ret;
 }
 
-static bool has_orphan_item(struct btrfs_root *root, u64 ino)
-{
-	struct btrfs_path path;
-	struct btrfs_key key;
-	int ret;
-
-	btrfs_init_path(&path);
-	key.objectid = BTRFS_ORPHAN_OBJECTID;
-	key.type = BTRFS_ORPHAN_ITEM_KEY;
-	key.offset = ino;
-
-	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
-	btrfs_release_path(&path);
-	if (ret == 0)
-		return true;
-	return false;
-}
-
 static int repair_inode_gen_lowmem(struct btrfs_root *root,
 				   struct btrfs_path *path)
 {
@@ -2622,7 +2604,7 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
 		return err;
 	}
 
-	is_orphan = has_orphan_item(root, inode_id);
+	is_orphan = btrfs_has_orphan_item(root, inode_id);
 	ii = btrfs_item_ptr(node, slot, struct btrfs_inode_item);
 	isize = btrfs_inode_size(node, ii);
 	nbytes = btrfs_inode_nbytes(node, ii);
@@ -5209,6 +5191,28 @@ static int check_btrfs_root(struct btrfs_root *root, int check_all)
 			}
 		}
 	}
+
+	/* Make sure orphan subvolume tree has proper orphan item for it */
+	if (btrfs_root_refs(root_item) == 0 &&
+	    is_fstree(root->root_key.objectid)) {
+		bool is_orphan;
+		is_orphan = btrfs_has_orphan_item(root->fs_info->tree_root,
+						  root->root_key.objectid);
+
+		if (!is_orphan) {
+			error("orphan root %llu doesn't have orphan item",
+			      root->root_key.objectid);
+			if (repair) {
+				ret = repair_root_orphan_item(root->fs_info,
+						root->root_key.objectid);
+				if (!ret)
+					is_orphan = true;
+			}
+			if (!is_orphan)
+				err |= ORPHAN_ITEM;
+		}
+	}
+
 	if (btrfs_root_refs(root_item) > 0 ||
 	    btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
 		path.nodes[level] = root->node;
-- 
2.32.0

[PATCH 2/3] btrfs-progs: check/original: detect and repair orphan subvolume without orphan item

[Qu Wenruo]

For the detection part, just change how we handle root_item::refs == 0
case, so that even we find a root_item whose refs == 0, we still set
root_rec::found_root_item, then existing code can handle it well.

For repair part, use the repair_root_orphan_item() function.

Also since we're here, use the common orphan item search function to
replace the original mode specific function.

Reported-by: Zhenyu Wu <wuzy001@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 check/main.c | 37 +++++++++++--------------------------
 1 file changed, 11 insertions(+), 26 deletions(-)

diff --git a/check/main.c b/check/main.c
index ee6cf793251c..62bfc00ab12e 100644
--- a/check/main.c
+++ b/check/main.c
@@ -862,24 +862,6 @@ static void maybe_free_inode_rec(struct cache_tree *inode_cache,
 	}
 }
 
-static int check_orphan_item(struct btrfs_root *root, u64 ino)
-{
-	struct btrfs_path path;
-	struct btrfs_key key;
-	int ret;
-
-	key.objectid = BTRFS_ORPHAN_OBJECTID;
-	key.type = BTRFS_ORPHAN_ITEM_KEY;
-	key.offset = ino;
-
-	btrfs_init_path(&path);
-	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
-	btrfs_release_path(&path);
-	if (ret > 0)
-		ret = -ENOENT;
-	return ret;
-}
-
 static int process_inode_item(struct extent_buffer *eb,
 			      int slot, struct btrfs_key *key,
 			      struct shared_node *active_node)
@@ -3063,8 +3045,7 @@ static int check_inode_recs(struct btrfs_root *root,
 		}
 
 		if (rec->errors & I_ERR_NO_ORPHAN_ITEM) {
-			ret = check_orphan_item(root, rec->ino);
-			if (ret == 0)
+			if (btrfs_has_orphan_item(root, rec->ino))
 				rec->errors &= ~I_ERR_NO_ORPHAN_ITEM;
 			if (can_free_inode_rec(rec)) {
 				free_inode_rec(rec);
@@ -3339,8 +3320,8 @@ static int check_root_refs(struct btrfs_root *root,
 		if (rec->found_ref == 0 &&
 		    rec->objectid >= BTRFS_FIRST_FREE_OBJECTID &&
 		    rec->objectid <= BTRFS_LAST_FREE_OBJECTID) {
-			ret = check_orphan_item(gfs_info->tree_root, rec->objectid);
-			if (ret == 0)
+			if (btrfs_has_orphan_item(root->fs_info->tree_root,
+						  rec->objectid))
 				continue;
 
 			/*
@@ -3350,9 +3331,14 @@ static int check_root_refs(struct btrfs_root *root,
 			 */
 			if (!rec->found_root_item)
 				continue;
-			errors++;
 			fprintf(stderr, "fs tree %llu not referenced\n",
-				(unsigned long long)rec->objectid);
+				rec->objectid);
+
+			if (repair)
+				ret = repair_root_orphan_item(root->fs_info,
+							      rec->objectid);
+			if (!repair || ret)
+				errors++;
 		}
 
 		error = 0;
@@ -3579,8 +3565,7 @@ static int check_fs_root(struct btrfs_root *root,
 	if (root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID) {
 		rec = get_root_rec(root_cache, root->root_key.objectid);
 		BUG_ON(IS_ERR(rec));
-		if (btrfs_root_refs(root_item) > 0)
-			rec->found_root_item = 1;
+		rec->found_root_item = 1;
 	}
 
 	btrfs_init_path(&path);
-- 
2.32.0

[PATCH 3/3] btrfs-progs: fsck-tests: add test image for orphan roots without an orphan item

[Qu Wenruo]

The image is manually crafted so that:

- Root 257/258 has no ROOT_BACKREF/ROOT_REF
- Root 257/258 has root refs 0
- No DIR_INDEX/DIR_ITEM points to root 257/258

Reported-by: Zhenyu Wu <wuzy001@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 .../048-orphan-roots/.lowmem_repairable          |   0
 tests/fsck-tests/048-orphan-roots/default.img.xz | Bin 0 -> 2584 bytes
 2 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 tests/fsck-tests/048-orphan-roots/.lowmem_repairable
 create mode 100644 tests/fsck-tests/048-orphan-roots/default.img.xz

diff --git a/tests/fsck-tests/048-orphan-roots/.lowmem_repairable b/tests/fsck-tests/048-orphan-roots/.lowmem_repairable
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/tests/fsck-tests/048-orphan-roots/default.img.xz b/tests/fsck-tests/048-orphan-roots/default.img.xz
new file mode 100644
index 0000000000000000000000000000000000000000..80da92c6c0edbf194d2f6fc8c1c0c149983b605e
GIT binary patch
literal 2584
zcmV+z3g`9xH+ooF000E$*0e?f03iVu0001VFXf})5B~|+T>wRyj;C3^v%$$4d1zEs
zh6euaev|Jls6|E?YAIeZep}MC;>QI7ji|$M!RSVBT>d%0JyqUTS^z;wmfuXdMRJwy
zL(+flKLC1VnM891mowG0)H}nqq1ZX|6xB=~H{PmdBRqgV7<xJcCYDfo`ajm;DR4D=
zz;!7+it3Vl8u3|k_F*SHg>AC&YaW?{GmsCQn7vmeGgY$URB+~=HLeq(N1s>sQr_{r
zwTCey#_(q$C)w;%?0(YNZ_RApYqC@>Q&=nVO}W<3dA>m>8-CZsnO*Mg&zXlimqo4j
zC>Lq$I+q3O5KUtJ3U7Oo7rXRCF;r&opm^dL{3Q^V1z-a3_;@P{5cd`-a~}pk?Cp41
zW7-hZu(?BCRA+|ljIj|u@dSjEJOX4ky697+K|1#1<PLv#+eL2~!_u?XlzNG_Q*u^-
zJR@dlwfROb@$DZmES{#`-+`+!#CMM{7}J&#04IDjLpL~r0ie`nVIKB!sVrsqj7EMz
zd<{9X#ZAr|s!;)ky!&HKapn(chMf_~0LAR3n0N}E9;}DYr?63SC{c4h7hN14rL6vW
zXCY7ajK2|yk{2`>xOgt)Ku{L@{cR>`ZG5o!SRt5W829-ayco8Xa8$!#*=?OurVbtT
zcZLZUVBvWS0Ib+C)4=?k^zLoKi-Iy*Cx)YUx8eMlz%YsuiS9GGZcHrm+Au6Qxz>kH
zO96M*{D$cy!Ot9HOy?1SuTVzxm3DfY{SE+FX;ZcoXn1A6$&X?_kg8U{&0-r^VtWcZ
z8C_<-48B&`-fH0D+TCJ7>-^~|#Pj{if+sY;DpLI^p?E(UuU_gb&x7tX+38^^JUj}z
z*-Y6Yo+I?;p32b1YL|6Oj3!H)R?_1Os0XB|ShblP2LUr=qxox88g%T@T6Kx^paJD{
z!Sxbt<xzk9O-bLfRRW**eOiDXtKte*djnCqd1c}tar|rvhOT7jf=m3^z}})df0T7e
z1~*o40-wo*egc&|qhk9hpaGh;JBbdtSlp^nBcgG0eTG`z9y-`{u_V>5$CSK?e1~pJ
zBkE5#wXU3sz5<6g=GfW$NWN2MNgW??(yA*=yLhgR#Y-*>ei}-r_$76zw;}7s0OqsX
z>~tdcpQ>rugx2Kq_+KkqY<hsvWa?l^I%iKr9abztuPXX2ct)guFJnh)vq)fc$sU8h
zPI0zJ(xA^SweC0SaD!!R*g?G1XVAEc8ttB!KE2k+233ny;Bx@uPAoa_Mi(!){QhvS
zeyI{^4?mjJy;iVW%i8a2##i4pes1tQeSfBcPGdtHP%>rQ=DxltEidgg{9|Q_7^Y!&
z<VL7<@)^co+XcDBcS+>j=h8ET_YX@b;0pcy&r4}p69E!|7tpZu-?8Oj)^1KUs@P}>
zcrqq90Bsq`MOOZv&>IV`##x6;CrnM59aOyO=^()-L#5Zl!HtNFjI|mcS+kA=XfVV(
zqZcP8o<|<#_pHm3>*ZFTLo3G|$A#A6sY4<u)$58DgIu^+*4KnkcbSf4GN3d<lQm1g
znr>+K;@MaP(qxbdMS!zX+J&1>Zxp&T=Y%slp?DoS-e6o}g~}ZMCz@njS#%8ZoY;$R
z@&x__2ReblogMzb<EYJ|arP|x*~Geg1uOZ^ksD1{aVH}^s9Imb%+sN_>H?VnNhCi^
zmRnpgYQ)i4M*`}9P_@d<KVi{E{`+c`dri4dM<vlgZv*+mcw16Uy0R%P>(M0-41ZNN
zzS9w8$zmgjnxQ+)z7cSjjY#BVkVuPumF6p3QP`vHTkD9{DN+cXjiW*iEl^BinAcK2
zN1cs7!8Mlky2?Qc#Z`SM)0hlqaYV*bIuP9w)GRAW(mX^l@P_(&Q;R^QT0%=|(TcBs
z@9p%__LK^*A;4~uq3QiQ^o{`6G!obzI%JUW?)=zGWU13P2m!i?#;#iYbn-kE*tk$(
zKuAs5PUAWFWWXk);UlAp;$;^7ir7|7HE|=nvK1PaoaJan0&>OsZ76lEWuZQ;D1ec(
zR<>{VfC_z;Dq@J`mr_W&>WnzY?PaodIE#ZG{GDZ`Emh>{S?<eRzP(yiCL&v#ES&S?
zSQms|gkOFfqz4FpNL9LhD&s~CJbo}6CwYA2aut>W)89a|4B>;WT@qXUk0q1sS5w)r
zI;?;qGG|~xxzGm`b147zTpq}EPb<<#_k@<l7mSZ8k&!qJ2G^sbpQ4dJGX$NqGtwq$
zLIeogdtI$y3E!DG*`M{W>jPK^8GXZxJSV;Egg!ugT0CdpsA!HV@hy*4kBezJx0K3t
zE_s@GA2DF=VCusE(IC!*RvAGp_5Bf65S}W#C_kY+HBE>k>gCP*Zi~B{?WN$ADbw9#
zOB8zcR7vv2CYtB#Yybe$C&Dw(j`m<Vz0BtefUq^44x6~=`gSxYGw7=VSj&cKnf0Rg
zz2dh{D~1_%D(z>4;R`}#or~8O)Kj*Pf?r2l5naid@<Quh^FWR*Y#JoT7i=i!1h4d8
zdOlT7izs)u;v<Gu$#w5S$>FY>M@z3Wb45V%7s4&O!SGInq|d7wgpxZ3$1DnXi&39b
zpHSR0fnP@rfj&t|O>PXOF5Zw26mxdr$!(6YKWQQBu!VvOUpP)slTW!p2q7}cJRQC5
zz{!G=gl*W~+uP_D@V4Xvd64G6#>LJ|z7OZy&4TKb)Ht&~27my9ODaA<z9^&RJ<vfd
zNpz7!gyqGsEN|fp#zxwMP^3_$p?(pJb@y>c!gl^-Ww}bmxT80Z78iC;L@S;KVdTl8
zYgj(vu>FHcfF02Ol@)^)adOllP&g3M%uJ-qJO0LmiqqBGViH;fq66>^cn9S3G<8wx
z;z|<Ak;q@c#D@ac4&MeBet>vv&=>99AEb6Z#8QTT!GOMpt3LY$R^~6+&(qH_kJr>c
zgN-$k=>CM~>qi1F6}b0Y=?F4zb{(MS)(YwFd`5P)3M!WIu^Ud%6x`EAgcJ22VP+W5
z#D%yjca+Y|is6}Uz~jgbwFtQ8J7%$Xv|o5X%2{ZC*Y_4F${{4cK+DeZyMN=zEy_~U
z6~D<7qCmbkDP)*=z>o6t%bAEpI<4kRG%1|dZwbjvofqSeOeoE_0`0)z6m2H&QmFN~
z?;1p2aFp=1(BYVM8*w4b|7x?S4^+ROq~|jn9i0~b@y%#sbsV_;ZzSJj*LfeEqgp7p
zpG$gtzxcRvd0DMGNz*Sb8aB=KppREd_;N%yH31_X^|y-!PT@k%(}J&5{9*(6T1-39
zWi{w_-I21vY?VYP`B&~SIJfwCp;fyEm_be!rKaxNDX+b_`%p2a)RHGUX?<(gV!{!2
uC3yG_GPy@Ij{pGr@dylV-a6_40rV4qAOHYck^;Z6#Ao{g000001X)_&H}_Tm

literal 0
HcmV?d00001

-- 
2.32.0

Re: [PATCH 1/3] btrfs-progs: check/lowmem: add the ability to detect and repair orphan subvolume trees which doesn't have orphan item

[Anand Jain]

On 25/06/2021 15:13, Qu Wenruo wrote:
> There is a bug report from the mail list that, after certain
> btrfs-zero-log operation, the filesystem has inaccessible subvolumes,
> and there is no way to delete them.
> 
> Such subvolumes have no ROOT_REF/ROOT_BACKREF, and their root_refs is 0.


> Without an orphan item, kernel won't detect them as orphan, thus no
> cleanup will happen.


Changes here looks good. But what if, in another case, the ghost 
subvolume isn't an orphan but a corruption? is it possible?

Thanks, Anand


> Btrfs check won't report this as a problem either.
> 
> Such ghost subvolumes will just waste disk space, and can be pretty hard
> to detect without proper btrfs check support.
> 
> For the repair part, we just add orphan item so later kernel can handle
> it.
> 
> Since the check for orphan item and repair can be reused between
> original and lowmem mode, move those functions to mode-common.[ch].
> 
> Reported-by: Zhenyu Wu <wuzy001@gmail.com>
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>   check/mode-common.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
>   check/mode-common.h |  3 +++
>   check/mode-lowmem.c | 42 +++++++++++++++++++---------------
>   3 files changed, 82 insertions(+), 19 deletions(-)
> 
> diff --git a/check/mode-common.c b/check/mode-common.c
> index cb22f3233c00..d8c18f6603bf 100644
> --- a/check/mode-common.c
> +++ b/check/mode-common.c
> @@ -1243,3 +1243,59 @@ out:
>   	btrfs_release_path(&path);
>   	return ret;
>   }
> +
> +/*
> + * Check if we have orphan item for @objectid.
> + *
> + * Note: if something wrong happened during search, we consider it as no orphan
> + * item.
> + */
> +bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid)
> +{
> +	struct btrfs_path path;
> +	struct btrfs_key key;
> +	int ret;
> +
> +	btrfs_init_path(&path);
> +	key.objectid = BTRFS_ORPHAN_OBJECTID;
> +	key.type = BTRFS_ORPHAN_ITEM_KEY;
> +	key.offset = objectid;
> +	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
> +	btrfs_release_path(&path);
> +	if (ret == 0)
> +		return true;
> +	return false;
> +}
> +
> +int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid)
> +{
> +	struct btrfs_path path;
> +	struct btrfs_trans_handle *trans;
> +	int ret;
> +
> +	btrfs_init_path(&path);
> +
> +	trans = btrfs_start_transaction(fs_info->tree_root, 1);
> +	if (IS_ERR(trans)) {
> +		ret = PTR_ERR(trans);
> +		errno = -ret;
> +		error("failed to start transaction: %m");
> +		return ret;
> +	}
> +	ret = btrfs_add_orphan_item(trans, fs_info->tree_root, &path, rootid);
> +	if (ret < 0) {
> +		errno = -ret;
> +		error("failed to insert orphan item: %m");
> +		goto error;
> +	}
> +	btrfs_release_path(&path);
> +	ret = btrfs_commit_transaction(trans, fs_info->tree_root);
> +	if (ret < 0)
> +		goto error;
> +	printf("Inserted orphan root item for root %llu\n", rootid);
> +	return ret;
> +error:
> +	btrfs_release_path(&path);
> +	btrfs_abort_transaction(trans, ret);
> +	return ret;
> +}
> diff --git a/check/mode-common.h b/check/mode-common.h
> index 3ba29ecab6cd..c44815a171ac 100644
> --- a/check/mode-common.h
> +++ b/check/mode-common.h
> @@ -192,4 +192,7 @@ static inline void btrfs_check_subpage_eb_alignment(u64 start, u32 len)
>   			start, start + len);
>   }
>   
> +bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid);
> +int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid);
> +
>   #endif
> diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
> index 2f736712bc7f..1ef781ad20bc 100644
> --- a/check/mode-lowmem.c
> +++ b/check/mode-lowmem.c
> @@ -2500,24 +2500,6 @@ out:
>   	return ret;
>   }
>   
> -static bool has_orphan_item(struct btrfs_root *root, u64 ino)
> -{
> -	struct btrfs_path path;
> -	struct btrfs_key key;
> -	int ret;
> -
> -	btrfs_init_path(&path);
> -	key.objectid = BTRFS_ORPHAN_OBJECTID;
> -	key.type = BTRFS_ORPHAN_ITEM_KEY;
> -	key.offset = ino;
> -
> -	ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
> -	btrfs_release_path(&path);
> -	if (ret == 0)
> -		return true;
> -	return false;
> -}
> -
>   static int repair_inode_gen_lowmem(struct btrfs_root *root,
>   				   struct btrfs_path *path)
>   {
> @@ -2622,7 +2604,7 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
>   		return err;
>   	}
>   
> -	is_orphan = has_orphan_item(root, inode_id);
> +	is_orphan = btrfs_has_orphan_item(root, inode_id);
>   	ii = btrfs_item_ptr(node, slot, struct btrfs_inode_item);
>   	isize = btrfs_inode_size(node, ii);
>   	nbytes = btrfs_inode_nbytes(node, ii);
> @@ -5209,6 +5191,28 @@ static int check_btrfs_root(struct btrfs_root *root, int check_all)
>   			}
>   		}
>   	}
> +
> +	/* Make sure orphan subvolume tree has proper orphan item for it */
> +	if (btrfs_root_refs(root_item) == 0 &&
> +	    is_fstree(root->root_key.objectid)) {
> +		bool is_orphan;
> +		is_orphan = btrfs_has_orphan_item(root->fs_info->tree_root,
> +						  root->root_key.objectid);
> +
> +		if (!is_orphan) {
> +			error("orphan root %llu doesn't have orphan item",
> +			      root->root_key.objectid);
> +			if (repair) {
> +				ret = repair_root_orphan_item(root->fs_info,
> +						root->root_key.objectid);
> +				if (!ret)
> +					is_orphan = true;
> +			}
> +			if (!is_orphan)
> +				err |= ORPHAN_ITEM;
> +		}
> +	}
> +
>   	if (btrfs_root_refs(root_item) > 0 ||
>   	    btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
>   		path.nodes[level] = root->node;
> 

Re: [PATCH 1/3] btrfs-progs: check/lowmem: add the ability to detect and repair orphan subvolume trees which doesn't have orphan item

[Qu Wenruo]

On 2021/6/25 下午4:58, Anand Jain wrote:
> On 25/06/2021 15:13, Qu Wenruo wrote:
>> There is a bug report from the mail list that, after certain
>> btrfs-zero-log operation, the filesystem has inaccessible subvolumes,
>> and there is no way to delete them.
>>
>> Such subvolumes have no ROOT_REF/ROOT_BACKREF, and their root_refs is 0.
> 
> 
>> Without an orphan item, kernel won't detect them as orphan, thus no
>> cleanup will happen.
> 
> 
> Changes here looks good. But what if, in another case, the ghost 
> subvolume isn't an orphan but a corruption? is it possible?

Firstly it needs to pass all the other root item check, including 
ROOT_BACKREF/ROOT_REF, and DIR_INDEX/DIR_ITEM.

This means if there is any ROOT_BACKREF/ROOT_REF, or DIR_INDEX/DIR_ITEM 
referring to this inode, it should not be repaired.

Furthermore, after passing above checks, it must have a root refs as 0.

With all conditions met, it's really a ghost subvolume.
In fact I have to do all above works to craft such test case, thus it's 
not resulted by some simple corruption.

Thanks,
Qu
> 
> Thanks, Anand
> 
> 
>> Btrfs check won't report this as a problem either.
>>
>> Such ghost subvolumes will just waste disk space, and can be pretty hard
>> to detect without proper btrfs check support.
>>
>> For the repair part, we just add orphan item so later kernel can handle
>> it.
>>
>> Since the check for orphan item and repair can be reused between
>> original and lowmem mode, move those functions to mode-common.[ch].
>>
>> Reported-by: Zhenyu Wu <wuzy001@gmail.com>
>> Signed-off-by: Qu Wenruo <wqu@suse.com>
>> ---
>>   check/mode-common.c | 56 +++++++++++++++++++++++++++++++++++++++++++++
>>   check/mode-common.h |  3 +++
>>   check/mode-lowmem.c | 42 +++++++++++++++++++---------------
>>   3 files changed, 82 insertions(+), 19 deletions(-)
>>
>> diff --git a/check/mode-common.c b/check/mode-common.c
>> index cb22f3233c00..d8c18f6603bf 100644
>> --- a/check/mode-common.c
>> +++ b/check/mode-common.c
>> @@ -1243,3 +1243,59 @@ out:
>>       btrfs_release_path(&path);
>>       return ret;
>>   }
>> +
>> +/*
>> + * Check if we have orphan item for @objectid.
>> + *
>> + * Note: if something wrong happened during search, we consider it as 
>> no orphan
>> + * item.
>> + */
>> +bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid)
>> +{
>> +    struct btrfs_path path;
>> +    struct btrfs_key key;
>> +    int ret;
>> +
>> +    btrfs_init_path(&path);
>> +    key.objectid = BTRFS_ORPHAN_OBJECTID;
>> +    key.type = BTRFS_ORPHAN_ITEM_KEY;
>> +    key.offset = objectid;
>> +    ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
>> +    btrfs_release_path(&path);
>> +    if (ret == 0)
>> +        return true;
>> +    return false;
>> +}
>> +
>> +int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid)
>> +{
>> +    struct btrfs_path path;
>> +    struct btrfs_trans_handle *trans;
>> +    int ret;
>> +
>> +    btrfs_init_path(&path);
>> +
>> +    trans = btrfs_start_transaction(fs_info->tree_root, 1);
>> +    if (IS_ERR(trans)) {
>> +        ret = PTR_ERR(trans);
>> +        errno = -ret;
>> +        error("failed to start transaction: %m");
>> +        return ret;
>> +    }
>> +    ret = btrfs_add_orphan_item(trans, fs_info->tree_root, &path, 
>> rootid);
>> +    if (ret < 0) {
>> +        errno = -ret;
>> +        error("failed to insert orphan item: %m");
>> +        goto error;
>> +    }
>> +    btrfs_release_path(&path);
>> +    ret = btrfs_commit_transaction(trans, fs_info->tree_root);
>> +    if (ret < 0)
>> +        goto error;
>> +    printf("Inserted orphan root item for root %llu\n", rootid);
>> +    return ret;
>> +error:
>> +    btrfs_release_path(&path);
>> +    btrfs_abort_transaction(trans, ret);
>> +    return ret;
>> +}
>> diff --git a/check/mode-common.h b/check/mode-common.h
>> index 3ba29ecab6cd..c44815a171ac 100644
>> --- a/check/mode-common.h
>> +++ b/check/mode-common.h
>> @@ -192,4 +192,7 @@ static inline void 
>> btrfs_check_subpage_eb_alignment(u64 start, u32 len)
>>               start, start + len);
>>   }
>> +bool btrfs_has_orphan_item(struct btrfs_root *root, u64 objectid);
>> +int repair_root_orphan_item(struct btrfs_fs_info *fs_info, u64 rootid);
>> +
>>   #endif
>> diff --git a/check/mode-lowmem.c b/check/mode-lowmem.c
>> index 2f736712bc7f..1ef781ad20bc 100644
>> --- a/check/mode-lowmem.c
>> +++ b/check/mode-lowmem.c
>> @@ -2500,24 +2500,6 @@ out:
>>       return ret;
>>   }
>> -static bool has_orphan_item(struct btrfs_root *root, u64 ino)
>> -{
>> -    struct btrfs_path path;
>> -    struct btrfs_key key;
>> -    int ret;
>> -
>> -    btrfs_init_path(&path);
>> -    key.objectid = BTRFS_ORPHAN_OBJECTID;
>> -    key.type = BTRFS_ORPHAN_ITEM_KEY;
>> -    key.offset = ino;
>> -
>> -    ret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
>> -    btrfs_release_path(&path);
>> -    if (ret == 0)
>> -        return true;
>> -    return false;
>> -}
>> -
>>   static int repair_inode_gen_lowmem(struct btrfs_root *root,
>>                      struct btrfs_path *path)
>>   {
>> @@ -2622,7 +2604,7 @@ static int check_inode_item(struct btrfs_root 
>> *root, struct btrfs_path *path)
>>           return err;
>>       }
>> -    is_orphan = has_orphan_item(root, inode_id);
>> +    is_orphan = btrfs_has_orphan_item(root, inode_id);
>>       ii = btrfs_item_ptr(node, slot, struct btrfs_inode_item);
>>       isize = btrfs_inode_size(node, ii);
>>       nbytes = btrfs_inode_nbytes(node, ii);
>> @@ -5209,6 +5191,28 @@ static int check_btrfs_root(struct btrfs_root 
>> *root, int check_all)
>>               }
>>           }
>>       }
>> +
>> +    /* Make sure orphan subvolume tree has proper orphan item for it */
>> +    if (btrfs_root_refs(root_item) == 0 &&
>> +        is_fstree(root->root_key.objectid)) {
>> +        bool is_orphan;
>> +        is_orphan = btrfs_has_orphan_item(root->fs_info->tree_root,
>> +                          root->root_key.objectid);
>> +
>> +        if (!is_orphan) {
>> +            error("orphan root %llu doesn't have orphan item",
>> +                  root->root_key.objectid);
>> +            if (repair) {
>> +                ret = repair_root_orphan_item(root->fs_info,
>> +                        root->root_key.objectid);
>> +                if (!ret)
>> +                    is_orphan = true;
>> +            }
>> +            if (!is_orphan)
>> +                err |= ORPHAN_ITEM;
>> +        }
>> +    }
>> +
>>       if (btrfs_root_refs(root_item) > 0 ||
>>           btrfs_disk_key_objectid(&root_item->drop_progress) == 0) {
>>           path.nodes[level] = root->node;
>>
> 

Re: [PATCH 0/3] btrfs-progs: detect and fix orphan roots without orphan items

[Qu Wenruo]

Hi David,

Any idea why the patchest is not merged yet?

In fact we got a newer report on the same problem, which this patchset 
is definitely going to help:

https://lore.kernel.org/linux-btrfs/c75599fa-3b4e-5a5a-c695-75c99b315a06@gmail.com/

It has two subvolumes which has 0 refs, but no ORPHAN item for it.

Thanks,
Qu

On 2021/6/25 15:13, Qu Wenruo wrote:
> There is bug report from Zhenyu Wu <wuzy001@gmail.com>, where even there
> is only one subvolume, his filesystem still consumes way more space than
> expected.
> 
> The problem is, there are 58 snapshots, but all of them have no
> ROOT_REF/ROOT_BACKREF, and all their root refs is 0.
> 
> But strangely there is no orphan items for them.
> 
> This means, btrfs kernel module won't detect them as orphan nor queue
> them to be removed.
> 
> "btrfs subvolume delete -i" do no helps, as it still requires
> ROOT_BACKREF to resolve the dentry.
> 
> For now, we should teach btrfs check to detect and repair the case by
> adding new orphan items for them.
> 
> The root cause I guess is some strange behavior for orphan item insert
> and subvolume unlink.
> As the report mentioned that he had one forced power off.
> 
> Qu Wenruo (3):
>    btrfs-progs: check/lowmem: add the ability to detect and repair orphan
>      subvolume trees which doesn't have orphan item
>    btrfs-progs: check/original: detect and repair orphan subvolume
>      without orphan item
>    btrfs-progs: fsck-tests: add test image for orphan roots without an
>      orphan item
> 
>   check/main.c                                  |  37 ++++--------
>   check/mode-common.c                           |  56 ++++++++++++++++++
>   check/mode-common.h                           |   3 +
>   check/mode-lowmem.c                           |  42 +++++++------
>   .../048-orphan-roots/.lowmem_repairable       |   0
>   .../048-orphan-roots/default.img.xz           | Bin 0 -> 2584 bytes
>   6 files changed, 93 insertions(+), 45 deletions(-)
>   create mode 100644 tests/fsck-tests/048-orphan-roots/.lowmem_repairable
>   create mode 100644 tests/fsck-tests/048-orphan-roots/default.img.xz
>