btrfs raid

btrfs raid

[Carlos Ortega]

I'd like to confirm that btrfs raid actually works.  My filesystem
looks like it's a simple concatenation judging from its size in df -k
output.  btrfs filesystem df says it's a raid10, I just don't
completely trust it.  Also I'm stuck at version 3.19.1, I can't go
higher.

Can someone confirm that raid works in this version?  If so which
levels of raid?  I'd prefer raid5 or raid6.

Thanks

Re: btrfs raid

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]

On Tue, Mar 01, 2016 at 11:19:34AM -0500, Carlos Ortega wrote:
> I'd like to confirm that btrfs raid actually works.  My filesystem
> looks like it's a simple concatenation judging from its size in df -k
> output.  btrfs filesystem df says it's a raid10, I just don't
> completely trust it.  Also I'm stuck at version 3.19.1, I can't go
> higher.

   Don't trust plain df. It's likely to be lying to some degree. btrfs
fi df will tell you how much data is under RAID, and btrfs fi show
will tell you how much space is not yet allocated. There's a couple of
FAQ entries on how to interpret that output correctly.

> Can someone confirm that raid works in this version?  If so which
> levels of raid?  I'd prefer raid5 or raid6.

   Definitely don't use parity RAID on 3.19. It's not really something
I'd trust, personally, even on 4.4, except for testing purposes.

   TBH, I wouldn't really want to be running something as old as 3.19
either. The actual problems of running older kernels are, IME,
considerably worse than the perceived problems of upgrading. If you're
restricted to that version by some upstream supplier, then they should
be giving you support and recommending what's usable in that kernel.

   Hugo.

-- 
Hugo Mills             | There are three mistaikes in this sentance.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs raid

[Rich Freeman]

On Tue, Mar 1, 2016 at 11:27 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
>
>    Definitely don't use parity RAID on 3.19. It's not really something
> I'd trust, personally, even on 4.4, except for testing purposes.

++ - raid 5/6 are fairly unstable at this point.  Raid 1 should be just fine.

>    TBH, I wouldn't really want to be running something as old as 3.19
> either. The actual problems of running older kernels are, IME,
> considerably worse than the perceived problems of upgrading.

I think it depends on how you define "old."  I think that 3.18.28
would be fine as it is a supported longterm.  I've just upgraded to
the 4.1 series which I plan to track until a new longterm has been out
for a few months and things lok quiet.

3.19 is very problematic though, as it is no longer supported.  I'd
sooner "downgrade" to 3.18.28 (which likely has more btrfs backports
unless your distro handles them).  Or, upgrade to 4.1.19.

If you are using highly experimental features like raid5 support on
btrfs then bleeding-edge is probably better, but I've found I've had
the fewest issues sticking with the previous longterm.  I've been
bitten by a few btrfs regressions over the years and I think 3.19 was
actually around the time I got hit by one of them.  Since I've
switched to just staying on a longterm once it hits the x.x.15 version
or so I've found things to be much more reliable.

-- 
Rich

Re: btrfs raid

[Duncan]

Rich Freeman posted on Sun, 06 Mar 2016 07:01:00 -0500 as excerpted:

> On Tue, Mar 1, 2016 at 11:27 AM, Hugo Mills <hugo@carfax.org.uk> wrote:
>>
>>    Definitely don't use parity RAID on 3.19. It's not really something
>> I'd trust, personally, even on 4.4, except for testing purposes.
> 
> ++ - raid 5/6 are fairly unstable at this point.  Raid 1 should be just
> fine.
> 
>>    TBH, I wouldn't really want to be running something as old as 3.19
>> either. The actual problems of running older kernels are, IME,
>> considerably worse than the perceived problems of upgrading.
> 
> I think it depends on how you define "old."  I think that 3.18.28 would
> be fine as it is a supported longterm.  I've just upgraded to the 4.1
> series which I plan to track until a new longterm has been out for a few
> months and things lok quiet.
> 
> 3.19 is very problematic though, as it is no longer supported.  I'd
> sooner "downgrade" to 3.18.28 (which likely has more btrfs backports
> unless your distro handles them).  Or, upgrade to 4.1.19.
> 
> If you are using highly experimental features like raid5 support on
> btrfs then bleeding-edge is probably better, but I've found I've had the
> fewest issues sticking with the previous longterm.  I've been bitten by
> a few btrfs regressions over the years and I think 3.19 was actually
> around the time I got hit by one of them.  Since I've switched to just
> staying on a longterm once it hits the x.x.15 version or so I've found
> things to be much more reliable.

Agreed.

The two generally recommended kernel tracks are current and long-term 
stable.  If you choose current, staying within the last two releases is 
recommended.  That's 4.3 or 4.4 at this point, tho 4.3 is getting a bit 
long in the tooth and 4.5 is close to release (I upgraded to it between 
rc5 and rc6).

For some time, the LTS track was also the last couple releases, which 
with 4.4 being LTS, would be it or 4.1, but the previous LTS 3.18 has 
been fairly stable as well, and as long as you're not trying to run 
parity raid, there hasn't been a hugely pressing reason to recommend 
upgrading for those who prefer to play things conservative.

But older than 3.18 LTS is definitely not recommended, and 3.19 isn't LTS 
and is long out of standard support so isn't recommended either.  Neither 
are 4.0 and 4.2 as they too are out of support.

And parity raid is still unstable enough that you really need to be on 
latest current for it, for another couple kernel series anyway.  Once it 
stabilizes, it's likely 4.4 will be the first LTS series considered 
stable for parity raid, and hopefully 4.6 will do it, but as of now, 
there's at least one more bug that hasn't been traced, whereby restriping 
to cover changes in the number of devices takes about 10 times longer 
than it should, and if that change in the number of devices is due to a 
device failure, that means a window during which additional device 
failures isn't covered also 10 times longer than it should be.  Not good 
for the life of your data, for sure!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs raid

[Chris Murphy]

On Sun, Mar 6, 2016 at 5:01 AM, Rich Freeman <rich0@gentoo.org> wrote:

> I think it depends on how you define "old."  I think that 3.18.28
> would be fine as it is a supported longterm.

For raid56? I disagree. There were substantial raid56 code changes in
3.19 that were not backported to 3.18.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/diff/fs/btrfs/raid56.c?id=v3.19&id2=v3.18.28

There's just no way I'd recommend it, not worth it at all.

Anyone using Btrfs raid56 is still a test subject. I'd only use
current longterm, with the high expectation I'd use current stable or
even mainline if a problem arises before going to the list. If you
can't do any of this, then it's a waste of your time, and you're
better off looking at ZFS on Linux.

Look at the number of changes between 4.1.19 and 4.4.4, just for
extent-tree.c - there's over 1200 changes.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/diff/fs/btrfs/extent-tree.c?id=v4.4.4&id2=v4.1.19

And 4.1.19 doesn't have the dev replace code for raid56. While that's
more feature than bug fix, there are piles of bug fixes between even
the current 4.1.19 and 4.4.4 kernels.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/diff/fs/btrfs/raid56.c?id=v4.4.4&id2=v4.1.19

Things are certainly getting more stable enough that if your chances
of hitting an edge case are remote, you can get away with using older
kernels. But that's the thing, you don't know if you're hitting an
edge case in advance most of the time. If it's just a file server
using Samba, that's a docile use case than as root fs, let alone if
snapshots are being taken regularly. Every feature of Btrfs being used
adds on to the unknown factor.


-- 
Chris Murphy

Re: btrfs raid

[Rich Freeman]

On Sun, Mar 6, 2016 at 4:07 PM, Chris Murphy <lists@colorremedies.com> wrote:
> On Sun, Mar 6, 2016 at 5:01 AM, Rich Freeman <rich0@gentoo.org> wrote:
>
>> I think it depends on how you define "old."  I think that 3.18.28
>> would be fine as it is a supported longterm.
>
> For raid56? I disagree. There were substantial raid56 code changes in
> 3.19 that were not backported to 3.18.

Of course.  I was referring to raid1.  I wouldn't run raid56 without
an expectation of occasionally losing everything on any version of
linux.  :)  If I were just testing it or I could tolerate losing
everything occasionally I'd probably track the current stable, if not
mainline, depending on my goals.

-- 
Rich

Re: btrfs raid

[Chris Murphy]

On Sun, Mar 6, 2016 at 3:27 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Sun, Mar 6, 2016 at 4:07 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> On Sun, Mar 6, 2016 at 5:01 AM, Rich Freeman <rich0@gentoo.org> wrote:
>>
>>> I think it depends on how you define "old."  I think that 3.18.28
>>> would be fine as it is a supported longterm.
>>
>> For raid56? I disagree. There were substantial raid56 code changes in
>> 3.19 that were not backported to 3.18.
>
> Of course.  I was referring to raid1.

Oops. Sorry. Yeah it should be safe. But still there's thousands of
bug fixes that don't get backported even to longterm releases. I
personally wouldn't risk it since there's another option. I guess it
is sort of weighing the bugs you know with the older one, versus the
bugs you don't know with the newer one.


 I wouldn't run raid56 without
> an expectation of occasionally losing everything on any version of
> linux.  :)  If I were just testing it or I could tolerate losing
> everything occasionally I'd probably track the current stable, if not
> mainline, depending on my goals.

Yeah exactly.


-- 
Chris Murphy

Re: BTRFS RAID

[Tomasz Torcz]

On Fri, May 04, 2012 at 09:46:15AM -0500, Michael Bailey wrote:
> Greetings,
> 
> I have a few questions pertaining to BTRFS RAID.  I know it's been rumored a lot recently that kernel 3.4 will have RAID5/6 support, is this still the case.

  It seem to be wrong. Chris Mason hinted that RAID5/6 will hit kernel 3.5.  It is way to late to have it in 3.4.


>  Also, is it possible to change from a single drive system to a raid system or even a multi drive system without raid to a raid system while there is data on the drives an successfully convert them without loosing data?  Thank you for any help you can provide.

  Restriping had landed, so it should be doable.

-- 
Tomasz Torcz                 "God, root, what's the difference?"
xmpp: zdzichubg@chrome.pl         "God is more forgiving."

BTRFS RAID

[Michael Bailey]

Greetings,

I have a few questions pertaining to BTRFS RAID.  I know it's been rumored a lot recently that kernel 3.4 will have RAID5/6 support, is this still the case.  Also, is it possible to change from a single drive system to a raid system or even a multi drive system without raid to a raid system while there is data on the drives an successfully convert them without loosing data?  Thank you for any help you can provide.

Sent from my iPhone