On Wed, Oct 14, 2015 at 05:08:17AM +0000, Duncan wrote:
> Carmine Paolino posted on Tue, 13 Oct 2015 23:21:49 +0200 as excerpted:
> 
> > I have an home server with 3 hard drives that I added to the same btrfs
> > filesystem. Several hours ago I run `btrfs balance start -dconvert=raid0
> > /` and as soon as I run `btrfs fi show /` I lost my ssh connection to
> > the machine. The machine is still on, but it doesn’t even respond to
> > ping[. ...]
> > 
> > (I have a 250gb internal hard drive, a 120gb usb 2.0 one and a 2TB usb
> > 2.0 one so the transfer speeds are pretty low)
> 
> I won't attempt to answer the primary question[1] directly, but can point 
> out that in many cases, USB-connected devices simply don't have a stable 
> enough connection to work reliably in a multi-device btrfs.  There's 
> several possibilities for failure, including flaky connections (sometimes 
> assisted by cats or kids), unstable USB host port drivers, and unstable 
> USB/ATA translators.  A number of folks have reported problems with such 
> filesystems with devices connected over USB, that simply disappear if 
> they direct-connect the exact same devices to a proper SATA port.  The 
> problem seems to be /dramatically/ worse with USB connected devices, than 
> it is with, for instance, PCIE-based SATA expansion cards.
> 
> Single-device btrfs with USB-attached devices seem to work rather better, 
> because at least in that case, if the connection is flaky, the entire 
> filesystem appears and disappears at once, and btrfs' COW, atomic-commit 
> and data-integrity features, kick in to help deal with the connection's 
> instability.
> 
> Arguably, a two-device raid1 (both data/metadata, with metadata including 
> system) should work reasonably well too, as long as scrubs are done after 
> reconnection when there's trouble with one of the pair, because in that 
> case, all data appears on both devices, but single and raid0 modes are 
> likely to have severe issues in that sort of environment, because even 
> temporary disconnection of a single device means loss of access to some 
> data/metadata on the filesystem.  Raid10, 3+-device-raid1, and raid5/6, 
> are more complex situations.  They should survive loss of at least one 
> device, but keeping the filesystem healthy in the presence of unstable 
> connections is... complex enough I'd hate to be the one having to deal 
> with it, which means I can't recommend it to others, either.

   Note also that RAID-0 is a poor choice for this configuration,
because you'll only get 640 GB usable space out of it. With single,
you'll get the full sum of 2370 GB usable. With RAID-1, you'll have
320 GB usable. The low figures for the RAID-0 and -1 come from the
fact that you've got two small devices, and that both RAID-0 and
RAID-1 have a minimum of two devices per block group. You can play
around with the configurations at http://carfax.org.uk/btrfs-usage

   But I second Duncan's advice about not using USB. It's really not a
reliable configuration with btrfs.

   Hugo.

> So I'd recommend either connecting all devices internally if possible, or 
> setting up the USB-connected devices with separate filesystems, if 
> internal direct-connection isn't possible.
> 
> ---
> [1] Sysadmin's rule of backups.  If the data isn't backed up, by 
> definition it is of less value than the resource and hassle cost of 
> backup.  No exceptions -- post-loss claims to the contrary simply put the 
> lie to the claims, as actions spoke louder than words and they defined 
> the cost of the backup as more expensive than the data that would have 
> been backed up.  Worst-case is then loss of data that was by definition 
> of less value than the cost of backup, and the more valuable resource and 
> hassle cost of the backup was avoided, so the comparatively lower value 
> data loss is no big deal.
> 
> So in a case like this, I'd simply power down and take my chances of 
> filesystem loss, strictly limiting the time and resources I'd devote to 
> any further attempt at recovery, because the data is by definition either 
> backed up, or of such low value that a backup was considered too 
> expensive to do, meaning there's a very real possibility of spending more 
> time in a recovery attempt that's iffy at best, than the data on the 
> filesystem is actually worth, either because there are backups, or 
> because it's throw-away data in the first place.
> 

-- 
Hugo Mills             | There's an infinite number of monkeys outside who
hugo@... carfax.org.uk | want to talk to us about this new script for Hamlet
http://carfax.org.uk/  | they've worked out!
PGP: E2AB1DE4          |                                           Arthur Dent