Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

[Pete]

I recently recovered created a fresh filesystem on one disk and
recovered from backups with data as SINGLE and metadata as DUP.  I added
a second disk yesterday and ran a balance with -dconvert=raid1
-mconvert=raid1.  I did reboot during the process for a couple of
reasons, putting the sides on the PC case, putting it back under the
desk and I updated the kernel from 5.3.9 to 5.2.13 at some point during
this process. Balance resumed as one would expect.  Balance has now
completed:

root@phoenix:~# btrfs balance status /home_data
No balance found on '/home_data'

However, some metadata remains as DUP which does not seem right:

root@phoenix:~# btrfs fi usage  /home_data/
Overall:
    Device size:                  10.92TiB
    Device allocated:              4.69TiB
    Device unallocated:            6.23TiB
    Device missing:                  0.00B
    Used:                          4.61TiB
    Free (estimated):              3.15TiB      (min: 3.15TiB)
    Data ratio:                       2.00
    Metadata ratio:                   2.00
    Global reserve:              512.00MiB      (used: 0.00B)

Data,RAID1: Size:2.34TiB, Used:2.30TiB
   /dev/mapper/data_disk_ESFH      2.34TiB
   /dev/mapper/data_disk_EVPC      2.34TiB

Metadata,RAID1: Size:7.00GiB, Used:4.48GiB
   /dev/mapper/data_disk_ESFH      7.00GiB
   /dev/mapper/data_disk_EVPC      7.00GiB

Metadata,DUP: Size:1.00GiB, Used:257.22MiB
   /dev/mapper/data_disk_ESFH      2.00GiB

System,RAID1: Size:32.00MiB, Used:368.00KiB
   /dev/mapper/data_disk_ESFH     32.00MiB
   /dev/mapper/data_disk_EVPC     32.00MiB

Unallocated:
   /dev/mapper/data_disk_ESFH      3.11TiB
   /dev/mapper/data_disk_EVPC      3.11TiB
root@phoenix:~#

root@phoenix:~# btrfs --version
btrfs-progs v5.1


I presume running another balance will fix this, but surely all metadata
should have been converted?  Is there a way to only balance the DUP
metadata?

Re: Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

[Holger Hoffstätte]

On 9/8/19 9:09 AM, Pete wrote:
(snip)
> I presume running another balance will fix this, but surely all metadata
> should have been converted?  Is there a way to only balance the DUP
> metadata?

Adding "soft" to -mconvert should do exactly that; it will then skip
any chunks that are already in the target profile.

-h

Re: Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

[Pete]

On 9/8/19 8:57 AM, Holger Hoffstätte wrote:
> On 9/8/19 9:09 AM, Pete wrote:
> (snip)
>> I presume running another balance will fix this, but surely all metadata
>> should have been converted?  Is there a way to only balance the DUP
>> metadata?
> 
> Adding "soft" to -mconvert should do exactly that; it will then skip
> any chunks that are already in the target profile.
> 
> -h

Appreciated.  Fixed it, very rapidly, with:

btrfs bal start -mconvert=raid1,soft  /home_data

I think a few examples on the wiki page would be helpful.  As I don't do
this sort of maintenance every day I looked at the filter section on the
wiki / man pages online following your prompting and was adding
'type=soft' in various places with no success and it was about the 3rd
reading of the relevant area where I cam up with the above which worked.

Pete

Re: Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

[Holger Hoffstätte]

On 9/8/19 11:18 AM, Pete wrote:
> On 9/8/19 8:57 AM, Holger Hoffstätte wrote:
>> On 9/8/19 9:09 AM, Pete wrote:
>> (snip)
>>> I presume running another balance will fix this, but surely all metadata
>>> should have been converted?  Is there a way to only balance the DUP
>>> metadata?
>>
>> Adding "soft" to -mconvert should do exactly that; it will then skip
>> any chunks that are already in the target profile.
>>
>> -h
> 
> Appreciated.  Fixed it, very rapidly, with:
> 
> btrfs bal start -mconvert=raid1,soft  /home_data
> 
> I think a few examples on the wiki page would be helpful.  As I don't do
> this sort of maintenance every day I looked at the filter section on the
> wiki / man pages online following your prompting and was adding
> 'type=soft' in various places with no success and it was about the 3rd
> reading of the relevant area where I cam up with the above which worked.

IMHO 'soft' should just be the implicit default behaviour for convert,
since it almost always does what one would expect when converting.
I can't think of a good reason why it shouldn't be, but maybe there is
one - Dave?

-h

Re: Balance conversion to metadata RAID1, data RAID1 leaves some metadata as DUP

[Andrei Borzenkov]

On Mon, Sep 9, 2019 at 11:03 AM Pete <pete@petezilla.co.uk> wrote:
>
> I recently recovered created a fresh filesystem on one disk and
> recovered from backups with data as SINGLE and metadata as DUP.  I added
> a second disk yesterday and ran a balance with -dconvert=raid1
> -mconvert=raid1.  I did reboot during the process for a couple of
> reasons, putting the sides on the PC case, putting it back under the
> desk and I updated the kernel from 5.3.9 to 5.2.13 at some point during
> this process. Balance resumed as one would expect.  Balance has now
> completed:
>
> root@phoenix:~# btrfs balance status /home_data
> No balance found on '/home_data'
>
> However, some metadata remains as DUP which does not seem right:
>
> root@phoenix:~# btrfs fi usage  /home_data/
> Overall:
>     Device size:                  10.92TiB
>     Device allocated:              4.69TiB
>     Device unallocated:            6.23TiB
>     Device missing:                  0.00B
>     Used:                          4.61TiB
>     Free (estimated):              3.15TiB      (min: 3.15TiB)
>     Data ratio:                       2.00
>     Metadata ratio:                   2.00
>     Global reserve:              512.00MiB      (used: 0.00B)
>
> Data,RAID1: Size:2.34TiB, Used:2.30TiB
>    /dev/mapper/data_disk_ESFH      2.34TiB
>    /dev/mapper/data_disk_EVPC      2.34TiB
>
> Metadata,RAID1: Size:7.00GiB, Used:4.48GiB
>    /dev/mapper/data_disk_ESFH      7.00GiB
>    /dev/mapper/data_disk_EVPC      7.00GiB
>
> Metadata,DUP: Size:1.00GiB, Used:257.22MiB
>    /dev/mapper/data_disk_ESFH      2.00GiB
>
> System,RAID1: Size:32.00MiB, Used:368.00KiB
>    /dev/mapper/data_disk_ESFH     32.00MiB
>    /dev/mapper/data_disk_EVPC     32.00MiB
>
> Unallocated:
>    /dev/mapper/data_disk_ESFH      3.11TiB
>    /dev/mapper/data_disk_EVPC      3.11TiB
> root@phoenix:~#
>
> root@phoenix:~# btrfs --version
> btrfs-progs v5.1
>
>
> I presume running another balance will fix this, but surely all metadata
> should have been converted?  Is there a way to only balance the DUP
> metadata?


btrfs balance start -m convert=raid1,soft

should do it.