SATA/SAS mixed pool

SATA/SAS mixed pool

[Nathan Dehnel]

Is it possible/safe to replace a SATA drive in a btrfs RAID10 pool
with an SAS drive?

Re: SATA/SAS mixed pool

[Zia Nayamuth]

I've done it with ZFS and hardraid controllers. I see no reason why 
btrfs should care.

On 2018-12-13 07:31, Nathan Dehnel wrote:
> Is it possible/safe to replace a SATA drive in a btrfs RAID10 pool
> with an SAS drive?

Re: SATA/SAS mixed pool

[Adam Borowski]

On Wed, Dec 12, 2018 at 09:31:02PM -0600, Nathan Dehnel wrote:
> Is it possible/safe to replace a SATA drive in a btrfs RAID10 pool
> with an SAS drive?

For btrfs, a block device is a block device, it's not "racist".
You can freely mix and/or replace.  If you want to, say, extend a SD
card with NBD to remote spinning rust, it works well -- tested :p


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄⠀⠀⠀⠀ to the city of his birth to die.

Re: SATA/SAS mixed pool

[Remi Gauvin]

[-- Attachment #1.1.1: Type: text/plain, Size: 718 bytes --]

On 2018-12-13 02:29 AM, Adam Borowski wrote:

> 
> For btrfs, a block device is a block device, it's not "racist".
> You can freely mix and/or replace.  If you want to, say, extend a SD
> card with NBD to remote spinning rust, it works well -- tested :p
> 

The possibility ff NBD certainly intrigues me, but I would be concerned
about the write barriers,, (needed to make sure cached writes are
committed to spinning rust in the correct order, so as to avoid
corruption in the case of interruption.

Assuming the underlaying block device on the remote device has a working
implementation, (as we know, not always a safe assumption...) does NBD
properly pass along the barriers commands/support?




[-- Attachment #1.1.2: remi.vcf --]
[-- Type: text/x-vcard, Size: 203 bytes --]

begin:vcard
fn:Remi Gauvin
n:Gauvin;Remi
org:Georgian Infotech
adr:;;3-51 Sykes St. N.;Meaford;ON;N4L 1X3;Canada
email;internet:remi@georgianit.com
tel;work:226-256-1545
version:2.1
end:vcard


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: SATA/SAS mixed pool

[Austin S. Hemmelgarn]

On 2018-12-13 05:39, Remi Gauvin wrote:
> On 2018-12-13 02:29 AM, Adam Borowski wrote:
> 
>>
>> For btrfs, a block device is a block device, it's not "racist".
>> You can freely mix and/or replace.  If you want to, say, extend a SD
>> card with NBD to remote spinning rust, it works well -- tested :p
>>
> 
> The possibility ff NBD certainly intrigues me, but I would be concerned
> about the write barriers,, (needed to make sure cached writes are
> committed to spinning rust in the correct order, so as to avoid
> corruption in the case of interruption.
> 
> Assuming the underlaying block device on the remote device has a working
> implementation, (as we know, not always a safe assumption...) does NBD
> properly pass along the barriers commands/support?
I don't think there's any auto-detection, but newer versions of the NBD 
server software provide two options in the export configuration, 
`flush`, and `fua`, which will let you enable support for those two 
commands, though they only get used if the client supports them. 
Unless I'm mistaken, they both need support from the block device being 
exported if it's actually a block device, though they do not if it's a 
regular file (`flush` translates to `fdatasync()` or `fsync()`, while 
`fua` translates to `sync_file_range()`).

Re: SATA/SAS mixed pool

[Duncan]

Adam Borowski posted on Thu, 13 Dec 2018 08:29:05 +0100 as excerpted:

> On Wed, Dec 12, 2018 at 09:31:02PM -0600, Nathan Dehnel wrote:
>> Is it possible/safe to replace a SATA drive in a btrfs RAID10 pool with
>> an SAS drive?
> 
> For btrfs, a block device is a block device, it's not "racist".
> You can freely mix and/or replace.  If you want to, say, extend a SD
> card with NBD to remote spinning rust, it works well -- tested :p

FWIW (mostly for other readers not so much this particular case) the 
known exception/caveat to that is USB block devices, which do tend to 
have problems, tho some hardware is fine.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: SATA/SAS mixed pool

[Adam Borowski]

On Fri, Dec 14, 2018 at 05:14:37AM +0000, Duncan wrote:
> Adam Borowski posted on Thu, 13 Dec 2018 08:29:05 +0100 as excerpted:
> > On Wed, Dec 12, 2018 at 09:31:02PM -0600, Nathan Dehnel wrote:
> >> Is it possible/safe to replace a SATA drive in a btrfs RAID10 pool with
> >> an SAS drive?
> > 
> > For btrfs, a block device is a block device, it's not "racist".
> > You can freely mix and/or replace.  If you want to, say, extend a SD
> > card with NBD to remote spinning rust, it works well -- tested :p
> 
> FWIW (mostly for other readers not so much this particular case) the 
> known exception/caveat to that is USB block devices, which do tend to 
> have problems, tho some hardware is fine.

Yeah, but the problem doesn't come from btrfs not supporting or ill
supporting USB, just from the devices themselves being flaky.  If they
supported the spec correctly, all would be ok.

An example might be NBD from one of my machines that has an incredibly bad
network driver -- it drops packets whenever there's even a bit of memory
pressure.  That's ok on RX (no different from packet being dropped on the
wire, the sender will retransmit) but unacceptable on TX -- it should have
slept instead; NBD (reasonably) can't handle this and destroys the block
device.  Yet btrfs can handle such an unexpected but clean disconnect just
fine -- not even a reboot is needed, I need to unmount, restart NBD then
remount.  From the filesystem's point of view this is exactly equivalent
to a power loss -- that the in-memory copy tried to do some writes
afterwards doesn't matter.

So it's not just whether the device fails, it's about _how_ it fails.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ Ivan was a worldly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄⠀⠀⠀⠀ to the city of his birth to die.