Issue with btrfs balance

Issue with btrfs balance

[Austin S Hemmelgarn]

I just recently discovered something about btrfs filesystem balance that
(as far as I can see) isn't documented anywhere, and doesn't necessarily
have an obvious (to the average user) explanation.

Apparently, trying to use -mconvert=dup or -sconvert=dup on a
multi-device filesystem using one of the RAID profiles for metadata
fails with a statement to look at the kernel log, which doesn't show
anything at all about the failure.

Based on what I've been able to understand from the source, it appears
that the kernel stops you from converting to a dup profile for metadata
in this case because it thinks that such a profile doesn't work on
multiple devices, despite the fact that you can take a single device
filesystem, and a device, and it will still work fine even without
converting the metadata/system profiles.

I feel at the very least, this should be documented, and the kernel
should give at least some indication of what went wrong.  Ideally, this
should be changed to allow converting to dup so that when converting a
multi-device filesystem to single-device, you never have to have
metadata or system chunks use a single profile.

Re: Issue with btrfs balance

[Imran Geriskovan]

I've experienced the following with balance:

Setup:
- Kernel 3.12.9
- 11 DVD sized (4.3GB) loopback devices.
(9 Read-Only Seed devices + 2 Read/Write devices)
- 9 device seed created with "-m single -d single" and made
Read-only with "btrfstune -S 1 ..."
- 2 devices was added at different dates. NO balance performed until now.
- NOW add 1 more device to the array and perform a balance.

Result:
Balance did run for a while and exited displaying "Process Killed".
Any attempt to unmount the array failed, preventing any
shutdown. Hence I had no option other than hard reset.

After reboot, issuing balance command gave the message
"Balance in progress".

I cancelled the balance and tried to remove the last device
which ended with a kernel crash. So I dumped 2 + 1 normal devices.

Former 9 multi device seed was OK and was mountable.

Regards,
Imran

Re: Issue with btrfs balance

[Brendan Hide]

On 2014/02/10 04:33 AM, Austin S Hemmelgarn wrote:
> <snip>
> Apparently, trying to use -mconvert=dup or -sconvert=dup on a
> multi-device filesystem using one of the RAID profiles for metadata
> fails with a statement to look at the kernel log, which doesn't show
> anything at all about the failure.
^ If this is the case then it is definitely a bug. Can you provide some 
version info? Specifically kernel, btrfs-tools, and Distro.
> <snip> it appears
> that the kernel stops you from converting to a dup profile for metadata
> in this case because it thinks that such a profile doesn't work on
> multiple devices, despite the fact that you can take a single device
> filesystem, and a device, and it will still work fine even without
> converting the metadata/system profiles.
I believe dup used to work on multiple devices but the facility was 
removed. In the standard case it doesn't make sense to use dup with 
multiple devices: It uses the same amount of diskspace but is more 
vulnerable than the RAID1 alternative.
> <snip> Ideally, this
> should be changed to allow converting to dup so that when converting a
> multi-device filesystem to single-device, you never have to have
> metadata or system chunks use a single profile.
This is a good use-case for having the facility. I'm thinking that, if 
it is brought back in, the only caveat is that appropriate warnings 
should be put in place to indicate that it is inappropriate.

My guess on how you'd like to migrate from raid1/raid1 to single/dup, 
assuming sda and sdb:
btrfs balance start -dconvert=single -mconvert=dup /
btrfs device delete /dev/sdb /

-- 
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97

Re: Issue with btrfs balance

[Austin S Hemmelgarn]

On 2014-02-10 08:41, Brendan Hide wrote:
> On 2014/02/10 04:33 AM, Austin S Hemmelgarn wrote:
>> <snip>
>> Apparently, trying to use -mconvert=dup or -sconvert=dup on a
>> multi-device filesystem using one of the RAID profiles for metadata
>> fails with a statement to look at the kernel log, which doesn't show
>> anything at all about the failure.
> ^ If this is the case then it is definitely a bug. Can you provide some
> version info? Specifically kernel, btrfs-tools, and Distro.
In this case, btrfs-progs 3.12, kernel 3.13.2, and Gentoo.
>> <snip> it appears
>> that the kernel stops you from converting to a dup profile for metadata
>> in this case because it thinks that such a profile doesn't work on
>> multiple devices, despite the fact that you can take a single device
>> filesystem, and a device, and it will still work fine even without
>> converting the metadata/system profiles.
> I believe dup used to work on multiple devices but the facility was
> removed. In the standard case it doesn't make sense to use dup with
> multiple devices: It uses the same amount of diskspace but is more
> vulnerable than the RAID1 alternative.
>> <snip> Ideally, this
>> should be changed to allow converting to dup so that when converting a
>> multi-device filesystem to single-device, you never have to have
>> metadata or system chunks use a single profile.
> This is a good use-case for having the facility. I'm thinking that, if
> it is brought back in, the only caveat is that appropriate warnings
> should be put in place to indicate that it is inappropriate.
> 
> My guess on how you'd like to migrate from raid1/raid1 to single/dup,
> assuming sda and sdb:
> btrfs balance start -dconvert=single -mconvert=dup /
> btrfs device delete /dev/sdb /
> 
Ideally, yes.  The exact command I tried to use was:
btrfs balance start -dconvert=single -mconvert=dup -sconvert=dup -f -v /
Trying again without the system chunk conversion also failed.

Re: Issue with btrfs balance

[Austin S. Hemmelgarn]

On 02/10/2014 08:41 AM, Brendan Hide wrote:
> On 2014/02/10 04:33 AM, Austin S Hemmelgarn wrote:
>> <snip>
>> Apparently, trying to use -mconvert=dup or -sconvert=dup on a
>> multi-device filesystem using one of the RAID profiles for metadata
>> fails with a statement to look at the kernel log, which doesn't show
>> anything at all about the failure.
> ^ If this is the case then it is definitely a bug. Can you provide some
> version info? Specifically kernel, btrfs-tools, and Distro.
>> <snip> it appears
>> that the kernel stops you from converting to a dup profile for metadata
>> in this case because it thinks that such a profile doesn't work on
>> multiple devices, despite the fact that you can take a single device
>> filesystem, and a device, and it will still work fine even without
>> converting the metadata/system profiles.
> I believe dup used to work on multiple devices but the facility was
> removed. In the standard case it doesn't make sense to use dup with
> multiple devices: It uses the same amount of diskspace but is more
> vulnerable than the RAID1 alternative.
>> <snip> Ideally, this
>> should be changed to allow converting to dup so that when converting a
>> multi-device filesystem to single-device, you never have to have
>> metadata or system chunks use a single profile.
> This is a good use-case for having the facility. I'm thinking that, if
> it is brought back in, the only caveat is that appropriate warnings
> should be put in place to indicate that it is inappropriate.
> 
> My guess on how you'd like to migrate from raid1/raid1 to single/dup,
> assuming sda and sdb:
> btrfs balance start -dconvert=single -mconvert=dup /
> btrfs device delete /dev/sdb /
> 
Do you happen to know which git repository and branch is preferred to
base patches on?  I'm getting ready to write one to fix this, and would
like to make it as easy as possible for the developers to merge.

Re: Issue with btrfs balance

[Brendan Hide]

On 14/02/14 05:42, Austin S. Hemmelgarn wrote:
>> On 2014/02/10 04:33 AM, Austin S Hemmelgarn wrote:
> Do you happen to know which git repository and branch is preferred to
> base patches on?  I'm getting ready to write one to fix this, and would
> like to make it as easy as possible for the developers to merge.
A list of the "main" repositories is maintained at 
https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories

I'd suggest David Sterba's branch as he maintains it for userspace-tools 
integration.

-- 
__________
Brendan Hide
http://swiftspirit.co.za/
http://www.webafrica.co.za/?AFF1E97

Re: Issue with btrfs balance

[Austin S Hemmelgarn]

On 02/14/2014 02:56 AM, Brendan Hide wrote:
> On 14/02/14 05:42, Austin S. Hemmelgarn wrote:
>>> On 2014/02/10 04:33 AM, Austin S Hemmelgarn wrote:
>> Do you happen to know which git repository and branch is
>> preferred to base patches on?  I'm getting ready to write one to
>> fix this, and would like to make it as easy as possible for the
>> developers to merge.
> A list of the "main" repositories is maintained at 
> https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories
> 
> I'd suggest David Sterba's branch as he maintains it for
> userspace-tools integration.
> 
In this case, it will need to be patched both in the userspace tools,
and in the kernel, it's the kernel itself that prevents the balance,
cause it thinks that you can't do dup profiles with multiple devices.

[PATCH] Allow forced conversion of metadata to dup profile on multiple devices

[Austin S Hemmelgarn]

Currently, btrfs balance start fails when trying to convert metadata or
system chunks to dup profile on filesystems with multiple devices.  This
requires that a conversion from a multi-device filesystem to a single
device filesystem use the following methodology:
    1. btrfs balance start -dconvert=single -mconvert=single \
       -sconvert=single -f /
    2. btrfs device delete / /dev/sdx
    3. btrfs balance start -mconvert=dup -sconvert=dup /
This results in a period of time (possibly very long if the devices are
big) where you don't have the protection guarantees of multiple copies
of metadata chunks.

After applying this patch, one can instead use the following methodology
for conversion from a multi-device filesystem to a single device
filesystem:
    1. btrfs balance start -dconvert=single -mconvert=dup \
       -sconvert=dup -f /
    2. btrfs device delete / /dev/sdx
This greatly reduces the chances of the operation causing data loss due
to a read error during the device delete.

Signed-off-by: Austin S. Hemmelgarn <ahferroin7@gmail.com>
---
 fs/btrfs/volumes.c | 21 +++++++++++++++++----
 1 file changed, 17 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 07629e9..38a9522 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -3152,10 +3152,8 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
 		num_devices--;
 	}
 	btrfs_dev_replace_unlock(&fs_info->dev_replace);
-	allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE;
-	if (num_devices == 1)
-		allowed |= BTRFS_BLOCK_GROUP_DUP;
-	else if (num_devices > 1)
+	allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE | BTRFS_BLOCK_GROUP_DUP;
+	if (num_devices > 1)
 		allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1);
 	if (num_devices > 2)
 		allowed |= BTRFS_BLOCK_GROUP_RAID5;
@@ -3221,6 +3219,21 @@ int btrfs_balance(struct btrfs_balance_control *bctl,
 				goto out;
 			}
 		}
+		if (((bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) &&
+		    (bctl->sys.target & ~BTRFS_BLOCK_GROUP_DUP) ||
+		    (bctl->meta.flags & BTRFS_BALANCE_ARGS_CONVERT) &&
+		    (bctl->meta.target & ~BTRFS_BLOCK_GROUP_DUP)) &&
+		    (num_devs > 1)) {
+			if (bctl->flags & BTRFS_BALANCE_FORCE) {
+				btrfs_info(fs_info, "force conversion of metadata "
+					   "to dup profile on multiple devices");
+			} else {
+				btrfs_err(fs_info, "balance will reduce metadata "
+					  "integrity, use force if you want this");
+				ret = -EINVAL;
+				goto out;
+			}
+		}
 	} while (read_seqretry(&fs_info->profiles_lock, seq));
 
 	if (bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) {
-- 
1.8.5.4