[PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[Anand Jain]

Fixes BUG_ON in find_next_devid() and fixes to return real error in
clone_fs_devices(). These two patches can be send to be independent.

Anand Jain (2):
  btrfs: fix BUG_ON with proper error handle in find_next_devid
  btrfs: fix error return on alloc fail in clone_fs_devices

 fs/btrfs/volumes.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

-- 
2.21.0 (Apple Git-120)

[PATCH 1/2] btrfs: fix BUG_ON with proper error handle in find_next_devid

[Anand Jain]

In a corrupted tree if search for next devid finds the device with
devid = -1, then report the error -EUCLEAN back to the parent
function to fail gracefully.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 fs/btrfs/volumes.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 4db4a100c05b..36aa5f79fb6c 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1849,7 +1849,12 @@ static noinline int find_next_devid(struct btrfs_fs_info *fs_info,
 	if (ret < 0)
 		goto error;
 
-	BUG_ON(ret == 0); /* Corruption */
+	if (ret == 0) {
+		/* Corruption */
+		btrfs_err(fs_info, "corrupted chunk tree devid -1 matched");
+		ret = -EUCLEAN;
+		goto error;
+	}
 
 	ret = btrfs_previous_item(fs_info->chunk_root, path,
 				  BTRFS_DEV_ITEMS_OBJECTID,
-- 
2.21.0 (Apple Git-120)

[PATCH 2/2] btrfs: fix error return on alloc fail in clone_fs_devices

[Anand Jain]

Fix the fake ENOMEM return error code to the actual error in
clone_fs_devices().

Signed-off-by: Anand Jain <anand.jain@oracle.com>
---
 fs/btrfs/volumes.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 36aa5f79fb6c..8d72098ccb4b 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1115,6 +1115,7 @@ static struct btrfs_fs_devices *clone_fs_devices(struct btrfs_fs_devices *orig)
 	struct btrfs_fs_devices *fs_devices;
 	struct btrfs_device *device;
 	struct btrfs_device *orig_dev;
+	int ret = 0;
 
 	fs_devices = alloc_fs_devices(orig->fsid, NULL);
 	if (IS_ERR(fs_devices))
@@ -1128,8 +1129,10 @@ static struct btrfs_fs_devices *clone_fs_devices(struct btrfs_fs_devices *orig)
 
 		device = btrfs_alloc_device(NULL, &orig_dev->devid,
 					    orig_dev->uuid);
-		if (IS_ERR(device))
+		if (IS_ERR(device)) {
+			ret = PTR_ERR(device);
 			goto error;
+		}
 
 		/*
 		 * This is ok to do without rcu read locked because we hold the
@@ -1140,6 +1143,7 @@ static struct btrfs_fs_devices *clone_fs_devices(struct btrfs_fs_devices *orig)
 					GFP_KERNEL);
 			if (!name) {
 				btrfs_free_device(device);
+				ret = -ENOMEM;
 				goto error;
 			}
 			rcu_assign_pointer(device->name, name);
@@ -1154,7 +1158,7 @@ static struct btrfs_fs_devices *clone_fs_devices(struct btrfs_fs_devices *orig)
 error:
 	mutex_unlock(&orig->device_list_mutex);
 	free_fs_devices(fs_devices);
-	return ERR_PTR(-ENOMEM);
+	return ERR_PTR(ret);
 }
 
 /*
-- 
2.21.0 (Apple Git-120)

Re: [PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[Nikolay Borisov]

On 27.08.19 г. 10:40 ч., Anand Jain wrote:
> Fixes BUG_ON in find_next_devid() and fixes to return real error in
> clone_fs_devices(). These two patches can be send to be independent.
> 
> Anand Jain (2):
>   btrfs: fix BUG_ON with proper error handle in find_next_devid
>   btrfs: fix error return on alloc fail in clone_fs_devices
> 
>  fs/btrfs/volumes.c | 15 ++++++++++++---
>  1 file changed, 12 insertions(+), 3 deletions(-)

Reviewed-by: Nikolay Borisov <nborisov@suse.com>

> 

Re: [PATCH 1/2] btrfs: fix BUG_ON with proper error handle in find_next_devid

[Johannes Thumshirn]

Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5
90409 Nürnberg
Germany
(HRB 247165, AG München)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH 2/2] btrfs: fix error return on alloc fail in clone_fs_devices

[Johannes Thumshirn]

Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5
90409 Nürnberg
Germany
(HRB 247165, AG München)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH 1/2] btrfs: fix BUG_ON with proper error handle in find_next_devid

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 1366 bytes --]



On 2019/8/27 下午3:40, Anand Jain wrote:
> In a corrupted tree if search for next devid finds the device with
> devid = -1, then report the error -EUCLEAN back to the parent
> function to fail gracefully.
> 
> Signed-off-by: Anand Jain <anand.jain@oracle.com>
> ---
>  fs/btrfs/volumes.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index 4db4a100c05b..36aa5f79fb6c 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -1849,7 +1849,12 @@ static noinline int find_next_devid(struct btrfs_fs_info *fs_info,
>  	if (ret < 0)
>  		goto error;
>  
> -	BUG_ON(ret == 0); /* Corruption */
> +	if (ret == 0) {
> +		/* Corruption */
> +		btrfs_err(fs_info, "corrupted chunk tree devid -1 matched");

It will never hit this branch.

As in tree checker, we have checked if the devid is so large that a
chunk item or system chunk array can't contain one.

That limit is way smaller than (u64)-1.
Thus if we really have a key (DEV_ITEMS DEV_ITEM -1), it will be
rejected by tree-checker in the first place, thus you will get a ret ==
-EUCLEAN from previous btrfs_search_slot() call.

Thanks,
Qu
> +		ret = -EUCLEAN;
> +		goto error;
> +	}
>  
>  	ret = btrfs_previous_item(fs_info->chunk_root, path,
>  				  BTRFS_DEV_ITEMS_OBJECTID,
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] btrfs: fix BUG_ON with proper error handle in find_next_devid

[Anand Jain]

On 27/8/19 4:12 PM, Qu Wenruo wrote:
> 
> 
> On 2019/8/27 下午3:40, Anand Jain wrote:
>> In a corrupted tree if search for next devid finds the device with
>> devid = -1, then report the error -EUCLEAN back to the parent
>> function to fail gracefully.
>>
>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>> ---
>>   fs/btrfs/volumes.c | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
>> index 4db4a100c05b..36aa5f79fb6c 100644
>> --- a/fs/btrfs/volumes.c
>> +++ b/fs/btrfs/volumes.c
>> @@ -1849,7 +1849,12 @@ static noinline int find_next_devid(struct btrfs_fs_info *fs_info,
>>   	if (ret < 0)
>>   		goto error;
>>   
>> -	BUG_ON(ret == 0); /* Corruption */
>> +	if (ret == 0) {
>> +		/* Corruption */
>> +		btrfs_err(fs_info, "corrupted chunk tree devid -1 matched");
> 
> It will never hit this branch.
> 
> As in tree checker, we have checked if the devid is so large that a
> chunk item or system chunk array can't contain one.

  That check is buggy. It assumes devid represents the num_devices,
  it does not account for the possible devid hole as created in the
  below script.

$ cat t

umount /btrfs
dev1=/dev/sdb
dev2=/dev/sdc
mkfs.btrfs -fq -dsingle -msingle $dev1
mount $dev1 /btrfs

_fail()
{
	echo $1
	exit 1
}

while true; do
	btrfs dev add -f $dev2 /btrfs || _fail "add failed"
	btrfs dev del $dev1 /btrfs || _fail "del failed"
	dev_tmp=$dev1
	dev1=$dev2
	dev2=$dev_tmp
done

-----------------------
[  185.446441] BTRFS critical (device sdb): corrupt leaf: root=3 
block=313739198464 slot=1 devid=1 invalid devid: has=507 expect=[0, 506]
[  185.446446] BTRFS error (device sdb): block=313739198464 write time 
tree block corruption detected
[  185.446556] BTRFS: error (device sdb) in 
btrfs_commit_transaction:2268: errno=-5 IO failure (Error while writing 
out transaction)
[  185.446559] BTRFS warning (device sdb): Skipping commit of aborted 
transaction.
[  185.446561] BTRFS: error (device sdb) in cleanup_transaction:1827: 
errno=-5 IO failure
-----------------------


Thanks, Anand


> That limit is way smaller than (u64)-1.
> Thus if we really have a key (DEV_ITEMS DEV_ITEM -1), it will be
> rejected by tree-checker in the first place, thus you will get a ret ==
> -EUCLEAN from previous btrfs_search_slot() call.
> 
> Thanks,
> Qu
>> +		ret = -EUCLEAN;
>> +		goto error;
>> +	}
>>   
>>   	ret = btrfs_previous_item(fs_info->chunk_root, path,
>>   				  BTRFS_DEV_ITEMS_OBJECTID,
>>
> 

Re: [PATCH 1/2] btrfs: fix BUG_ON with proper error handle in find_next_devid

[Qu Wenruo]

[-- Attachment #1.1: Type: text/plain, Size: 3139 bytes --]



On 2019/8/27 下午5:58, Anand Jain wrote:
> On 27/8/19 4:12 PM, Qu Wenruo wrote:
>>
>>
>> On 2019/8/27 下午3:40, Anand Jain wrote:
>>> In a corrupted tree if search for next devid finds the device with
>>> devid = -1, then report the error -EUCLEAN back to the parent
>>> function to fail gracefully.
>>>
>>> Signed-off-by: Anand Jain <anand.jain@oracle.com>
>>> ---
>>>   fs/btrfs/volumes.c | 7 ++++++-
>>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
>>> index 4db4a100c05b..36aa5f79fb6c 100644
>>> --- a/fs/btrfs/volumes.c
>>> +++ b/fs/btrfs/volumes.c
>>> @@ -1849,7 +1849,12 @@ static noinline int find_next_devid(struct
>>> btrfs_fs_info *fs_info,
>>>       if (ret < 0)
>>>           goto error;
>>>   -    BUG_ON(ret == 0); /* Corruption */
>>> +    if (ret == 0) {
>>> +        /* Corruption */
>>> +        btrfs_err(fs_info, "corrupted chunk tree devid -1 matched");
>>
>> It will never hit this branch.
>>
>> As in tree checker, we have checked if the devid is so large that a
>> chunk item or system chunk array can't contain one.
> 
>  That check is buggy. It assumes devid represents the num_devices,
>  it does not account for the possible devid hole as created in the
>  below script.
> 
> $ cat t
> 
> umount /btrfs
> dev1=/dev/sdb
> dev2=/dev/sdc
> mkfs.btrfs -fq -dsingle -msingle $dev1
> mount $dev1 /btrfs
> 
> _fail()
> {
>     echo $1
>     exit 1
> }
> 
> while true; do
>     btrfs dev add -f $dev2 /btrfs || _fail "add failed"
>     btrfs dev del $dev1 /btrfs || _fail "del failed"
>     dev_tmp=$dev1
>     dev1=$dev2
>     dev2=$dev_tmp
> done
> 
> -----------------------
> [  185.446441] BTRFS critical (device sdb): corrupt leaf: root=3
> block=313739198464 slot=1 devid=1 invalid devid: has=507 expect=[0, 506]
> [  185.446446] BTRFS error (device sdb): block=313739198464 write time
> tree block corruption detected
> [  185.446556] BTRFS: error (device sdb) in
> btrfs_commit_transaction:2268: errno=-5 IO failure (Error while writing
> out transaction)
> [  185.446559] BTRFS warning (device sdb): Skipping commit of aborted
> transaction.
> [  185.446561] BTRFS: error (device sdb) in cleanup_transaction:1827:
> errno=-5 IO failure
> -----------------------

Oh, that's a case I haven't considered.

Great we can find a bug in a seemingly unrelated patch.

So the patch itself is OK.

Reviewed-by: Qu Wenruo <wqu@suse.com>

Thanks,
Qu
> 
> 
> Thanks, Anand
> 
> 
>> That limit is way smaller than (u64)-1.
>> Thus if we really have a key (DEV_ITEMS DEV_ITEM -1), it will be
>> rejected by tree-checker in the first place, thus you will get a ret ==
>> -EUCLEAN from previous btrfs_search_slot() call.
>>
>> Thanks,
>> Qu
>>> +        ret = -EUCLEAN;
>>> +        goto error;
>>> +    }
>>>         ret = btrfs_previous_item(fs_info->chunk_root, path,
>>>                     BTRFS_DEV_ITEMS_OBJECTID,
>>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[David Sterba]

On Tue, Aug 27, 2019 at 03:40:43PM +0800, Anand Jain wrote:
> Fixes BUG_ON in find_next_devid() and fixes to return real error in
> clone_fs_devices(). These two patches can be send to be independent.
> 
> Anand Jain (2):
>   btrfs: fix BUG_ON with proper error handle in find_next_devid
>   btrfs: fix error return on alloc fail in clone_fs_devices

Added to misc-next, thanks. If you have script that can reproduce the
problem, please add them to the changelog. I've added more from the
discussion and questions from Qu.

Re: [PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[Nikolay Borisov]

On 27.08.19 г. 16:25 ч., David Sterba wrote:
> On Tue, Aug 27, 2019 at 03:40:43PM +0800, Anand Jain wrote:
>> Fixes BUG_ON in find_next_devid() and fixes to return real error in
>> clone_fs_devices(). These two patches can be send to be independent.
>>
>> Anand Jain (2):
>>   btrfs: fix BUG_ON with proper error handle in find_next_devid
>>   btrfs: fix error return on alloc fail in clone_fs_devices
> 
> Added to misc-next, thanks. If you have script that can reproduce the
> problem, please add them to the changelog. I've added more from the
> discussion and questions from Qu.
> 

Actually such a script should ideally be turned into an fstest testcase

Re: [PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[Anand Jain]

On 27/8/19 9:25 PM, David Sterba wrote:
> On Tue, Aug 27, 2019 at 03:40:43PM +0800, Anand Jain wrote:
>> Fixes BUG_ON in find_next_devid() and fixes to return real error in
>> clone_fs_devices(). These two patches can be send to be independent.
>>
>> Anand Jain (2):
>>    btrfs: fix BUG_ON with proper error handle in find_next_devid
>>    btrfs: fix error return on alloc fail in clone_fs_devices
> 
> Added to misc-next, thanks. 



> If you have script that can reproduce the
> problem, please add them to the changelog. I've added more from the
> discussion and questions from Qu.
> 

  The script and output rather makes senses in the patch [1] it doesn't 
here.
   [1] [PATCH v2] btrfs: tree-checker: Fix wrong check on max devid

Thanks, Anand

Re: [PATCH 0/2] fix BUG_ON and retun real error in find_next_devid() and clone_fs_devices()

[Anand Jain]

On 28/8/19 7:12 AM, Anand Jain wrote:
> On 27/8/19 9:25 PM, David Sterba wrote:
>> On Tue, Aug 27, 2019 at 03:40:43PM +0800, Anand Jain wrote:
>>> Fixes BUG_ON in find_next_devid() and fixes to return real error in
>>> clone_fs_devices(). These two patches can be send to be independent.
>>>
>>> Anand Jain (2):
>>>    btrfs: fix BUG_ON with proper error handle in find_next_devid
>>>    btrfs: fix error return on alloc fail in clone_fs_devices
>>
>> Added to misc-next, thanks. 
> 
> 
> 
>> If you have script that can reproduce the
>> problem, please add them to the changelog. I've added more from the
>> discussion and questions from Qu.
>>
> 
>   The script and output rather makes senses in the patch [1] it doesn't 
> here.
>    [1] [PATCH v2] btrfs: tree-checker: Fix wrong check on max devid

David,

  The script does not make sense in this patch. I still see it in misc-next.

Thanks, Anand