* Btrfs not using all devices in raid1
@ 2021-05-21 16:34 Leszek Dubiel
2021-05-21 20:11 ` Forza
2021-05-22 2:08 ` Zygo Blaxell
0 siblings, 2 replies; 6+ messages in thread
From: Leszek Dubiel @ 2021-05-21 16:34 UTC (permalink / raw)
To: linux-btrfs
Hello!
Why Btrfs is not using /dev/sdc2?
There is no line "Data,RAID1" for this disk.
Isn't it supposed to use disk that has most of free space?
Thanks for help :) :)
Using Btrfs in production.
Here are some command outputs:
### btrfs fi show /
Label: none uuid: ea6ae51d-d9b0-4628-a8f3-3406e1dc59c6
Total devices 4 FS bytes used 2.96TiB
devid 1 size 7.25TiB used 3.20TiB path /dev/sda2
devid 2 size 7.25TiB used 3.20TiB path /dev/sdb2
devid 3 size 7.25TiB used 3.21TiB path /dev/sdd2
devid 4 size 7.25TiB used 32.00MiB path /dev/sdc2
### btrfs fi df /
Data, RAID1: total=4.49TiB, used=2.90TiB
System, RAID1: total=64.00MiB, used=784.00KiB
Metadata, RAID1: total=321.00GiB, used=56.08GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
### btrfs dev usa /
/dev/sda2, ID: 1
Device size: 7.25TiB
Device slack: 0.00B
Data,RAID1: 2.99TiB
Metadata,RAID1: 210.00GiB
System,RAID1: 64.00MiB
Unallocated: 4.05TiB
/dev/sdb2, ID: 2
Device size: 7.25TiB
Device slack: 0.00B
Data,RAID1: 3.00TiB
Metadata,RAID1: 210.00GiB
Unallocated: 4.04TiB
/dev/sdc2, ID: 4
Device size: 7.25TiB
Device slack: 0.00B ... no Data/RAID1
System,RAID1: 32.00MiB
Unallocated: 7.25TiB
/dev/sdd2, ID: 3
Device size: 7.25TiB
Device slack: 0.00B
Data,RAID1: 2.99TiB
Metadata,RAID1: 222.00GiB
System,RAID1: 32.00MiB
Unallocated: 4.04TiB
### time btrfs balance start -dconvert=raid1,soft -mconvert=raid1,soft /
Done, had to relocate 0 out of 4922 chunks
real 0m0,522s
user 0m0,000s
sys 0m0,033s
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Btrfs not using all devices in raid1
2021-05-21 16:34 Btrfs not using all devices in raid1 Leszek Dubiel
@ 2021-05-21 20:11 ` Forza
2021-05-21 21:10 ` Leszek Dubiel
2021-05-22 2:08 ` Zygo Blaxell
1 sibling, 1 reply; 6+ messages in thread
From: Forza @ 2021-05-21 20:11 UTC (permalink / raw)
To: Leszek Dubiel, linux-btrfs
---- From: Leszek Dubiel <leszek@dubiel.pl> -- Sent: 2021-05-21 - 18:34 ----
>
> Hello!
>
> Why Btrfs is not using /dev/sdc2?
> There is no line "Data,RAID1" for this disk.
> Isn't it supposed to use disk that has most of free space?
>
> Thanks for help :) :)
> Using Btrfs in production.
>
>
> Here are some command outputs:
>
>
>
> ### btrfs fi show /
>
> Label: none uuid: ea6ae51d-d9b0-4628-a8f3-3406e1dc59c6
> Total devices 4 FS bytes used 2.96TiB
> devid 1 size 7.25TiB used 3.20TiB path /dev/sda2
> devid 2 size 7.25TiB used 3.20TiB path /dev/sdb2
> devid 3 size 7.25TiB used 3.21TiB path /dev/sdd2
> devid 4 size 7.25TiB used 32.00MiB path /dev/sdc2
>
>
>
> ### btrfs fi df /
>
> Data, RAID1: total=4.49TiB, used=2.90TiB
> System, RAID1: total=64.00MiB, used=784.00KiB
> Metadata, RAID1: total=321.00GiB, used=56.08GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
>
> ### btrfs dev usa /
>
> /dev/sda2, ID: 1
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 2.99TiB
> Metadata,RAID1: 210.00GiB
> System,RAID1: 64.00MiB
> Unallocated: 4.05TiB
>
> /dev/sdb2, ID: 2
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 3.00TiB
> Metadata,RAID1: 210.00GiB
> Unallocated: 4.04TiB
>
> /dev/sdc2, ID: 4
> Device size: 7.25TiB
> Device slack: 0.00B ... no Data/RAID1
> System,RAID1: 32.00MiB
> Unallocated: 7.25TiB
>
> /dev/sdd2, ID: 3
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 2.99TiB
> Metadata,RAID1: 222.00GiB
> System,RAID1: 32.00MiB
> Unallocated: 4.04TiB
>
>
>
> ### time btrfs balance start -dconvert=raid1,soft -mconvert=raid1,soft /
>
> Done, had to relocate 0 out of 4922 chunks
>
> real 0m0,522s
> user 0m0,000s
> sys 0m0,033s
>
>
Raid1 means two copies on different devices. This is fulfilled with the previous 3 drives so the 'soft' keyword is not going to help here.
You can do a full data balance (-dusage=100) to move some data across to the new disk. There is no need to do a metadata balance in this case, unless you want to convert to raid1c3 to have three copies of metadata.
If you do nothing, the filesystem will eventually balance itself as you add abs delete data.
https://btrfs.wiki.kernel.org/index.php/Balance_Filters
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Btrfs not using all devices in raid1
2021-05-21 20:11 ` Forza
@ 2021-05-21 21:10 ` Leszek Dubiel
2021-05-21 21:53 ` Forza
0 siblings, 1 reply; 6+ messages in thread
From: Leszek Dubiel @ 2021-05-21 21:10 UTC (permalink / raw)
To: Forza, linux-btrfs
> Raid1 means two copies on different devices. This is fulfilled with the previous 3 drives so the 'soft' keyword is not going to help here.
That's right.
> You can do a full data balance (-dusage=100) to move some data across to the new disk. There is no need to do a metadata balance in this case, unless you want to convert to raid1c3 to have three copies of metadata.
This is production server, and I don't want to do full balance, because it will hit performance for users.
I hoped that when data gets written to filesysystem BTRFS will choose drive that has most free space, that is /dev/sdc2.
Is there any way to tell BTRFS to use /dev/sdc2?
> If you do nothing, the filesystem will eventually balance itself as you add abs delete data.
If I do nothing then /dev/sd{a,b,d}2 would get almost full, and only
after /dev/sdc2 would be used?
Not using /dev/sdc2 is bad because:
-- maybe this drive is already failed, but nothing is written to it so I
have no reports of errors
-- other drives get all the data, so if one of them fails, then I will
have to take longer time to replace it
I would prefer to have all drives EQUAL in Raid1.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Btrfs not using all devices in raid1
2021-05-21 21:10 ` Leszek Dubiel
@ 2021-05-21 21:53 ` Forza
0 siblings, 0 replies; 6+ messages in thread
From: Forza @ 2021-05-21 21:53 UTC (permalink / raw)
To: Leszek Dubiel, linux-btrfs
---- From: Leszek Dubiel <leszek@dubiel.pl> -- Sent: 2021-05-21 - 23:10 ----
>
>
>> Raid1 means two copies on different devices. This is fulfilled with the previous 3 drives so the 'soft' keyword is not going to help here.
>
> That's right.
>
>
>
>> You can do a full data balance (-dusage=100) to move some data across to the new disk. There is no need to do a metadata balance in this case, unless you want to convert to raid1c3 to have three copies of metadata.
>
>
> This is production server, and I don't want to do full balance, because it will hit performance for users.
> I hoped that when data gets written to filesysystem BTRFS will choose drive that has most free space, that is /dev/sdc2.
>
> Is there any way to tell BTRFS to use /dev/sdc2?
>
>
>
>
>> If you do nothing, the filesystem will eventually balance itself as you add abs delete data.
>
>
> If I do nothing then /dev/sd{a,b,d}2 would get almost full, and only
> after /dev/sdc2 would be used?
>
>
Btrfs will fill the disk with most unallocated space first. So eventually they will end up almost equal.
>
>
> Not using /dev/sdc2 is bad because:
>
> -- maybe this drive is already failed, but nothing is written to it so I
> have no reports of errors
>
> -- other drives get all the data, so if one of them fails, then I will
> have to take longer time to replace it
>
> I would prefer to have all drives EQUAL in Raid1.
>
>
>
You can run balance with filters to do a little at a time and only use blocks on the other disks. Look at devid and limit https://btrfs.wiki.kernel.org/index.php/Balance_filters
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Btrfs not using all devices in raid1
2021-05-21 16:34 Btrfs not using all devices in raid1 Leszek Dubiel
2021-05-21 20:11 ` Forza
@ 2021-05-22 2:08 ` Zygo Blaxell
2021-05-22 8:52 ` Leszek Dubiel
1 sibling, 1 reply; 6+ messages in thread
From: Zygo Blaxell @ 2021-05-22 2:08 UTC (permalink / raw)
To: Leszek Dubiel; +Cc: linux-btrfs
On Fri, May 21, 2021 at 06:34:27PM +0200, Leszek Dubiel wrote:
>
> Hello!
>
> Why Btrfs is not using /dev/sdc2?
> There is no line "Data,RAID1" for this disk.
> Isn't it supposed to use disk that has most of free space?
>
> Thanks for help :) :)
> Using Btrfs in production.
>
>
> Here are some command outputs:
>
>
>
> ### btrfs fi show /
>
> Label: none uuid: ea6ae51d-d9b0-4628-a8f3-3406e1dc59c6
> Total devices 4 FS bytes used 2.96TiB
> devid 1 size 7.25TiB used 3.20TiB path /dev/sda2
> devid 2 size 7.25TiB used 3.20TiB path /dev/sdb2
> devid 3 size 7.25TiB used 3.21TiB path /dev/sdd2
> devid 4 size 7.25TiB used 32.00MiB path /dev/sdc2
>
>
>
> ### btrfs fi df /
>
> Data, RAID1: total=4.49TiB, used=2.90TiB
There are about 1.5TB of available space in existing block groups, so
those will be filled in first. Block group allocations will start on
sdc2 after the existing block groups are filled.
> System, RAID1: total=64.00MiB, used=784.00KiB
> Metadata, RAID1: total=321.00GiB, used=56.08GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
>
> ### btrfs dev usa /
>
> /dev/sda2, ID: 1
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 2.99TiB
> Metadata,RAID1: 210.00GiB
> System,RAID1: 64.00MiB
> Unallocated: 4.05TiB
>
> /dev/sdb2, ID: 2
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 3.00TiB
> Metadata,RAID1: 210.00GiB
> Unallocated: 4.04TiB
>
> /dev/sdc2, ID: 4
> Device size: 7.25TiB
> Device slack: 0.00B ... no Data/RAID1
> System,RAID1: 32.00MiB
> Unallocated: 7.25TiB
>
> /dev/sdd2, ID: 3
> Device size: 7.25TiB
> Device slack: 0.00B
> Data,RAID1: 2.99TiB
> Metadata,RAID1: 222.00GiB
> System,RAID1: 32.00MiB
> Unallocated: 4.04TiB
The disks in decreasing size order have 7.25TB, 4.05 TB, 4.04 TB, and 4.04
TB. RAID1 requires the sum of all disks after the largest be equal or
greater in size to the largest, and 12.13 >= 7.25, so you're good to go.
> ### time btrfs balance start -dconvert=raid1,soft -mconvert=raid1,soft /
>
> Done, had to relocate 0 out of 4922 chunks
>
> real 0m0,522s
> user 0m0,000s
> sys 0m0,033s
You might want something more like;
time btrfs balance start -dlimit=1000,devid=1 /
time btrfs balance start -dlimit=1000,devid=2 /
time btrfs balance start -dlimit=1000,devid=3 /
but that will take a long time and isn't strictly necessary.
Alternatively, use 'btrfs-balance-least-used' from the 'python-btrfs'
package, with a usage limit of about 90%. That will repack all of the
data into existing block groups first, so future allocations will use
sdc2.
Unfortunately balance is the only way to redistribute the existing data
onto new drives, so if you don't want to run balance, then you'll just
have to wait until sdc2 fills in naturally.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Btrfs not using all devices in raid1
2021-05-22 2:08 ` Zygo Blaxell
@ 2021-05-22 8:52 ` Leszek Dubiel
0 siblings, 0 replies; 6+ messages in thread
From: Leszek Dubiel @ 2021-05-22 8:52 UTC (permalink / raw)
To: Zygo Blaxell, linux-btrfs
>> ### btrfs fi df /
>>
>> Data, RAID1: total=4.49TiB, used=2.90TiB
> There are about 1.5TB of available space in existing block groups, so
> those will be filled in first. Block group allocations will start on
> sdc2 after the existing block groups are filled.
Ok! This is what I didn't know.
Now it's clear why Btrfs is not using /dev/sdc2 currently.
New blocks would be put on /dev/sdc2, but only after
currently allocated blocks got filled.
> Unfortunately balance is the only way to redistribute the existing data
> onto new drives, so if you don't want to run balance, then you'll just
> have to wait until sdc2 fills in naturally.
Your answer above solved my problem.
Thank you.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-05-22 8:53 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-21 16:34 Btrfs not using all devices in raid1 Leszek Dubiel
2021-05-21 20:11 ` Forza
2021-05-21 21:10 ` Leszek Dubiel
2021-05-21 21:53 ` Forza
2021-05-22 2:08 ` Zygo Blaxell
2021-05-22 8:52 ` Leszek Dubiel
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.