RAID1/RAID0, online replace

RAID1/RAID0, online replace

[Steve Keller]

A few questions on RAID1 on BTRFS:

1. With RAID1, can I replace a failed disk online, i.e. without
   unmounting the filesystem?  If yes, what happens with files that
   have data blocks which have only one copy?  Are they deleted?  Will
   the FS still be in a clean state?

2. Can I have some files with data blocks in RAID1 and other files in
   RAID0 or single mode?  Can I set this per directory?  E.g. I have
   directories with large multimedia files which change seldom or
   never, which I have in the backup, and for which I don't need high
   availability by RAID1.  Other directories, e.g. source code, change
   often which I'd like to have RAID1 for.

   If that's not possible, can RAID1 for data vs. RAID0 be configured
   per sub-volume?

Steve

Re: RAID1/RAID0, online replace

[Chris Murphy]

On Mon, Sep 5, 2022, at 7:16 PM, Steve Keller wrote:
> A few questions on RAID1 on BTRFS:
>
> 1. With RAID1, can I replace a failed disk online, i.e. without
>    unmounting the filesystem?  If yes, what happens with files that
>    have data blocks which have only one copy?  Are they deleted?  Will
>    the FS still be in a clean state?
>
> 2. Can I have some files with data blocks in RAID1 and other files in
>    RAID0 or single mode?  Can I set this per directory?  E.g. I have
>    directories with large multimedia files which change seldom or
>    never, which I have in the backup, and for which I don't need high
>    availability by RAID1.  Other directories, e.g. source code, change
>    often which I'd like to have RAID1 for.
>
>    If that's not possible, can RAID1 for data vs. RAID0 be configured
>    per sub-volume?

Ostensibly it's all one profile per block group type. But if the conversion was interrupted or cancelled, yeah you'd have mixed profile block groups, thus some files could be one or the other. But this is not really controllable.

`btrfs replace` is an online command.


-- 
Chris Murphy

Re: RAID1/RAID0, online replace

[Steve Keller]

Chris Murphy <lists@colorremedies.com> writes:

> Ostensibly it's all one profile per block group type. But if the
> conversion was interrupted or cancelled, yeah you'd have mixed
> profile block groups, thus some files could be one or the other. But
> this is not really controllable.

So, you couldn't even have different RAID config per subvolume, right?

> `btrfs replace` is an online command.

I read that the default is RAID1 for metadata and RAID0 for data.
But that would mean that in case of a disk failure almost all files,
i.e. those which have at least one data block on the failed drive,
are damaged. What does 'btrfs replace' do with these files? Isn't
the purpose of RAID defeated, when in case of disk failure most files
are damaged and you need to restore from backup?

AFAIU, RAID should allow you to continue with no need to unmount or
downtime, no restore from backup, if a disk fails. You would get
that with mkfs.btrfs -m raid1 -d raid1. The default
mkfs.btrfs -m raid1 -d raid0 doesn't make sense, IMO. But useful
would be a mixture of RAID0 and RAID1 for data, where I can configure
which files or directories or subvolumes use RAID0, e.g. directories
of large files, which change seldomly and aren't essential for normal
work, so that restore from the backup storage wouldn't mean loss of
work since last backup and interruption of work.

Steve

Re: RAID1/RAID0, online replace

[Chris Murphy]

On Tue, Sep 6, 2022, at 1:33 AM, Steve Keller wrote:
> Chris Murphy <lists@colorremedies.com> writes:
>
>> Ostensibly it's all one profile per block group type. But if the
>> conversion was interrupted or cancelled, yeah you'd have mixed
>> profile block groups, thus some files could be one or the other. But
>> this is not really controllable.
>
> So, you couldn't even have different RAID config per subvolume, right?

No.


>
>> `btrfs replace` is an online command.
>
> I read that the default is RAID1 for metadata and RAID0 for data.

Default profile for data is single, since btrfs-progs 5.8 or 5.9.

> But that would mean that in case of a disk failure almost all files,
> i.e. those which have at least one data block on the failed drive,
> are damaged. What does 'btrfs replace' do with these files? Isn't
> the purpose of RAID defeated, when in case of disk failure most files
> are damaged and you need to restore from backup?

If a drive fails with raid0 for any profile, it can't be mounted rw anymore and thus you can't use btrfs replace.  You should recreate and restore from backup. Sub optimal fall back is copy off the good data to some other file system, the rest is lost.



-- 
Chris