[PATCH v3] btrfs: rescue: allow ibadroots to skip bad extent tree when reading block group items

[PATCH v3] btrfs: rescue: allow ibadroots to skip bad extent tree when reading block group items

[Qu Wenruo]

When extent tree gets corrupted, normally it's not extent tree root, but
one toasted tree leaf/node.

In that case, rescue=ibadroots mount option won't help as it can only
handle the extent tree root corruption.

This patch will enhance the behavior by:

- Allow fill_dummy_bgs() to ignore -EEXIST error

  This means we may have some block group items read from disk, but
  then hit some error halfway.

- Fallback to fill_dummy_bgs() if any error gets hit in
  btrfs_read_block_groups()

  Of course, this still needs rescue=ibadroots mount option.

With that, rescue=ibadroots can handle extent tree corruption more
gracefully and allow a better recover chance.

Reported-by: Zhenyu Wu <wuzy001@gmail.com>
Link: https://www.spinics.net/lists/linux-btrfs/msg114424.html
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
Changelog:
v2:
- Don't try to fill with dummy block groups when we hit ENOMEM
v3:
- Remove a dead condition
  The empty fs_info->extent_root case has already been handled.
---
 fs/btrfs/block-group.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 5bd76a45037e..9bc68515bc4a 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2105,11 +2105,16 @@ static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
 		bg->used = em->len;
 		bg->flags = map->type;
 		ret = btrfs_add_block_group_cache(fs_info, bg);
-		if (ret) {
+		/*
+		 * We may have some block groups filled already, thus ignore
+		 * the -EEXIST error.
+		 */
+		if (ret && ret != -EEXIST) {
 			btrfs_remove_free_space_cache(bg);
 			btrfs_put_block_group(bg);
 			break;
 		}
+		ret = 0;
 		btrfs_update_space_info(fs_info, bg->flags, em->len, em->len,
 					0, 0, &space_info);
 		bg->space_info = space_info;
@@ -2212,6 +2217,14 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
 	ret = check_chunk_block_group_mappings(info);
 error:
 	btrfs_free_path(path);
+	/*
+	 * We hit some error reading the extent tree, and have rescue=ibadroots
+	 * mount option.
+	 * Try to fill using dummy block groups so that the user can continue
+	 * to mount and grab their data.
+	 */
+	if (ret && btrfs_test_opt(info, IGNOREBADROOTS))
+		ret = fill_dummy_bgs(info);
 	return ret;
 }
 
-- 
2.32.0

Re: [PATCH v3] btrfs: rescue: allow ibadroots to skip bad extent tree when reading block group items

[Anand Jain]

On 15/07/2021 13:00, Qu Wenruo wrote:
> When extent tree gets corrupted, normally it's not extent tree root, but
> one toasted tree leaf/node.
> 
> In that case, rescue=ibadroots mount option won't help as it can only
> handle the extent tree root corruption.
> 
> This patch will enhance the behavior by:
> 
> - Allow fill_dummy_bgs() to ignore -EEXIST error
> 
>    This means we may have some block group items read from disk, but
>    then hit some error halfway.
> 
> - Fallback to fill_dummy_bgs() if any error gets hit in
>    btrfs_read_block_groups()
> 
>    Of course, this still needs rescue=ibadroots mount option.
> 
> With that, rescue=ibadroots can handle extent tree corruption more
> gracefully and allow a better recover chance.
> 
> Reported-by: Zhenyu Wu <wuzy001@gmail.com>
> Link: https://www.spinics.net/lists/linux-btrfs/msg114424.html
> Signed-off-by: Qu Wenruo <wqu@suse.com>

  Reviewed-by: Anand Jain <anand.jain@oracle.com>

Thanks, Anand

> ---
> Changelog:
> v2:
> - Don't try to fill with dummy block groups when we hit ENOMEM
> v3:
> - Remove a dead condition
>    The empty fs_info->extent_root case has already been handled.
> ---
>   fs/btrfs/block-group.c | 15 ++++++++++++++-
>   1 file changed, 14 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
> index 5bd76a45037e..9bc68515bc4a 100644
> --- a/fs/btrfs/block-group.c
> +++ b/fs/btrfs/block-group.c
> @@ -2105,11 +2105,16 @@ static int fill_dummy_bgs(struct btrfs_fs_info *fs_info)
>   		bg->used = em->len;
>   		bg->flags = map->type;
>   		ret = btrfs_add_block_group_cache(fs_info, bg);
> -		if (ret) {
> +		/*
> +		 * We may have some block groups filled already, thus ignore
> +		 * the -EEXIST error.
> +		 */
> +		if (ret && ret != -EEXIST) {
>   			btrfs_remove_free_space_cache(bg);
>   			btrfs_put_block_group(bg);
>   			break;
>   		}
> +		ret = 0;
>   		btrfs_update_space_info(fs_info, bg->flags, em->len, em->len,
>   					0, 0, &space_info);
>   		bg->space_info = space_info;
> @@ -2212,6 +2217,14 @@ int btrfs_read_block_groups(struct btrfs_fs_info *info)
>   	ret = check_chunk_block_group_mappings(info);
>   error:
>   	btrfs_free_path(path);
> +	/*
> +	 * We hit some error reading the extent tree, and have rescue=ibadroots
> +	 * mount option.
> +	 * Try to fill using dummy block groups so that the user can continue
> +	 * to mount and grab their data.
> +	 */
> +	if (ret && btrfs_test_opt(info, IGNOREBADROOTS))
> +		ret = fill_dummy_bgs(info);
>   	return ret;
>   }
>   
> 

Re: [PATCH v3] btrfs: rescue: allow ibadroots to skip bad extent tree when reading block group items

[Su Yue]

On Thu 15 Jul 2021 at 13:00, Qu Wenruo <wqu@suse.com> wrote:

> When extent tree gets corrupted, normally it's not extent tree 
> root, but
> one toasted tree leaf/node.
>
> In that case, rescue=ibadroots mount option won't help as it can 
> only
> handle the extent tree root corruption.
>
> This patch will enhance the behavior by:
>
> - Allow fill_dummy_bgs() to ignore -EEXIST error
>
>   This means we may have some block group items read from disk, 
>   but
>   then hit some error halfway.
>
> - Fallback to fill_dummy_bgs() if any error gets hit in
>   btrfs_read_block_groups()
>
>   Of course, this still needs rescue=ibadroots mount option.
>
> With that, rescue=ibadroots can handle extent tree corruption 
> more
> gracefully and allow a better recover chance.
>
> Reported-by: Zhenyu Wu <wuzy001@gmail.com>
> Link: https://www.spinics.net/lists/linux-btrfs/msg114424.html
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> Changelog:
> v2:
> - Don't try to fill with dummy block groups when we hit ENOMEM
> v3:
> - Remove a dead condition
>   The empty fs_info->extent_root case has already been handled.
> ---
>  fs/btrfs/block-group.c | 15 ++++++++++++++-
>  1 file changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
> index 5bd76a45037e..9bc68515bc4a 100644
> --- a/fs/btrfs/block-group.c
> +++ b/fs/btrfs/block-group.c
> @@ -2105,11 +2105,16 @@ static int fill_dummy_bgs(struct 
> btrfs_fs_info *fs_info)
>  		bg->used = em->len;
>  		bg->flags = map->type;
>  		ret = btrfs_add_block_group_cache(fs_info, bg);
> -		if (ret) {
> +		/*
> +		 * We may have some block groups filled already, thus 
> ignore
> +		 * the -EEXIST error.
> +		 */
> +		if (ret && ret != -EEXIST) {
>  			btrfs_remove_free_space_cache(bg);
>  			btrfs_put_block_group(bg);
>  			break;
>  		}
>
So we continue to link_block_group() bellow even -EEXIST. The new
allocated bg will be inserted into 
&space_info->block_groups[index].
Then while calling close_ctree(), it only frees bgs not allocated 
by
fill_dummy_bgs(). The bgs still exist in
&space_info->block_groups[index]. Memory leaks!

--
Su

> +		ret = 0;
>  		btrfs_update_space_info(fs_info, bg->flags, em->len, 
>  em->len,
>  					0, 0, &space_info);
>  		bg->space_info = space_info;
> @@ -2212,6 +2217,14 @@ int btrfs_read_block_groups(struct 
> btrfs_fs_info *info)
>  	ret = check_chunk_block_group_mappings(info);
>  error:
>  	btrfs_free_path(path);
> +	/*
> +	 * We hit some error reading the extent tree, and have 
> rescue=ibadroots
> +	 * mount option.
> +	 * Try to fill using dummy block groups so that the user can 
> continue
> +	 * to mount and grab their data.
> +	 */
> +	if (ret && btrfs_test_opt(info, IGNOREBADROOTS))
> +		ret = fill_dummy_bgs(info);
>  	return ret;
>  }

Re: [PATCH v3] btrfs: rescue: allow ibadroots to skip bad extent tree when reading block group items

[Qu Wenruo]

On 2021/7/15 下午3:50, Su Yue wrote:
>
> On Thu 15 Jul 2021 at 13:00, Qu Wenruo <wqu@suse.com> wrote:
>
>> When extent tree gets corrupted, normally it's not extent tree root, but
>> one toasted tree leaf/node.
>>
>> In that case, rescue=ibadroots mount option won't help as it can only
>> handle the extent tree root corruption.
>>
>> This patch will enhance the behavior by:
>>
>> - Allow fill_dummy_bgs() to ignore -EEXIST error
>>
>>   This means we may have some block group items read from disk,   but
>>   then hit some error halfway.
>>
>> - Fallback to fill_dummy_bgs() if any error gets hit in
>>   btrfs_read_block_groups()
>>
>>   Of course, this still needs rescue=ibadroots mount option.
>>
>> With that, rescue=ibadroots can handle extent tree corruption more
>> gracefully and allow a better recover chance.
>>
>> Reported-by: Zhenyu Wu <wuzy001@gmail.com>
>> Link: https://www.spinics.net/lists/linux-btrfs/msg114424.html
>> Signed-off-by: Qu Wenruo <wqu@suse.com>
>> ---
>> Changelog:
>> v2:
>> - Don't try to fill with dummy block groups when we hit ENOMEM
>> v3:
>> - Remove a dead condition
>>   The empty fs_info->extent_root case has already been handled.
>> ---
>>  fs/btrfs/block-group.c | 15 ++++++++++++++-
>>  1 file changed, 14 insertions(+), 1 deletion(-)
>>
>> diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
>> index 5bd76a45037e..9bc68515bc4a 100644
>> --- a/fs/btrfs/block-group.c
>> +++ b/fs/btrfs/block-group.c
>> @@ -2105,11 +2105,16 @@ static int fill_dummy_bgs(struct btrfs_fs_info
>> *fs_info)
>>          bg->used = em->len;
>>          bg->flags = map->type;
>>          ret = btrfs_add_block_group_cache(fs_info, bg);
>> -        if (ret) {
>> +        /*
>> +         * We may have some block groups filled already, thus ignore
>> +         * the -EEXIST error.
>> +         */
>> +        if (ret && ret != -EEXIST) {
>>              btrfs_remove_free_space_cache(bg);
>>              btrfs_put_block_group(bg);
>>              break;
>>          }
>>
> So we continue to link_block_group() bellow even -EEXIST. The new
> allocated bg will be inserted into &space_info->block_groups[index].
> Then while calling close_ctree(), it only frees bgs not allocated by
> fill_dummy_bgs(). The bgs still exist in
> &space_info->block_groups[index]. Memory leaks!

Right, when -EEXIST is hit, we should skip to next chunk, not continuing
the remaining works.

Thanks,
Qu
>
> --
> Su
>
>> +        ret = 0;
>>          btrfs_update_space_info(fs_info, bg->flags, em->len,  em->len,
>>                      0, 0, &space_info);
>>          bg->space_info = space_info;
>> @@ -2212,6 +2217,14 @@ int btrfs_read_block_groups(struct
>> btrfs_fs_info *info)
>>      ret = check_chunk_block_group_mappings(info);
>>  error:
>>      btrfs_free_path(path);
>> +    /*
>> +     * We hit some error reading the extent tree, and have
>> rescue=ibadroots
>> +     * mount option.
>> +     * Try to fill using dummy block groups so that the user can
>> continue
>> +     * to mount and grab their data.
>> +     */
>> +    if (ret && btrfs_test_opt(info, IGNOREBADROOTS))
>> +        ret = fill_dummy_bgs(info);
>>      return ret;
>>  }