striping 2x500G to mirror 1x1T

striping 2x500G to mirror 1x1T

[David T-G]

Hi, all --

I have a 1T data drive (currently in use with data) that I'd like to
mirror with a pair of 500G drives striped together.  I'll be mirroring
two partitions, and I'll be striping partitions to ensure the correct
size, and my understanding is that I'll have to create the mirror on the
two new drives with half missing, mount it up, copy over the data, dump
the original disk, and add it as the other half of the mirror to sync.
If I've missed anything there, please let me know, but all of that
matches my Googling and I don't think I have any questions.

What I do wonder, and what I don't see in any searches since apparently
nobody talks about striping up half of a mirror, is if I should do
anything special with my two-disk RAID0 stripe.  I was gobsmacked at
the simplicity of RAID10 on only two drives by splitting each in half
and "flipping" one to maximize head movement performance.  Awesome! :-)
Are there any brilliant hacks for simple striping?  If I'm just putting
together two [not terribly large] disks, will I benefit from any other
funny stuff, or should I just stripe together two partitions -- each
half the size of my other drive, of course -- to make a "boring basic"
stripe and run with that?


TIA & HANW

:-D
-- 
David T-G
See http://justpickone.org/davidtg/email/
See http://justpickone.org/davidtg/tofu.txt

Re: striping 2x500G to mirror 1x1T

[Natanji]

Hello David,
what you propose here should work, but be aware that you are essentially 
first writing all data to the 2 smaller drives and then reading from 
them only when the mirror 1 TB disk gets added. If there are bad sectors 
that only gets dicovered while reading (not writing), you might lose 
data. A much cleaner way would be to make a backup of your data disk and 
then first crate the array with all disks, then write the data back from 
your backup once the disks are in sync. You should definitely make a 
backup or you could potentially lose data when creating this array.

I don't see the benefit of striping in your setup, by the way. For the 
two smaller disks it might seem that this is beneficial for performance, 
but since every write also needs to be made on the mirror, this will 
essentially lead to your read/write head moving back and forth a lot, 
no? Because if you write four consecutive chunks C1-C4 on the striped 
side, two of these would be written to each smaller disk. And on the 1 
TB disk side this would mean writing the sectors with only one 
read/write head, each around 500GB apart from each other, and in a back 
and worth fashion (although that might be optimized by queuing). So your 
write performance could actually be *worse* than on a single drive, and 
definitely not gain the 2x speed benefit of striping. For read 
performance you might be okay, but nevertheless put unneccessary 
mechanical wear on your 1 TB drive.

[C1 C3 ....] 500GB-1      [C2 C4 ....] 500GB-2
[C1 C3 ... (500G space)  C2 C4 ....] 1TB

At least that's my understanding, someone correct me if I'm wrong and 
mdadm handles such setups more intelligently (i.e. by laying out the 
chunks completely linearly on the 1TB disk instead of 1-to-1 "mirroring" 
the chunks on the 2x500G RAID0). If you think I'm correct, then instead 
of striping I would just use linear mode on the 500GB disks to pool them 
into one large volume, and thus essentially get Raid1 speed benefits of 
double read performance compared to a single drive. This has the 
downside of putting very different wear on the two 500GB volumes, of course.

Best regards,
Natanji

Am 05.03.2022 um 05:47 schrieb David T-G:
> Hi, all --
>
> I have a 1T data drive (currently in use with data) that I'd like to
> mirror with a pair of 500G drives striped together.  I'll be mirroring
> two partitions, and I'll be striping partitions to ensure the correct
> size, and my understanding is that I'll have to create the mirror on the
> two new drives with half missing, mount it up, copy over the data, dump
> the original disk, and add it as the other half of the mirror to sync.
> If I've missed anything there, please let me know, but all of that
> matches my Googling and I don't think I have any questions.
>
> What I do wonder, and what I don't see in any searches since apparently
> nobody talks about striping up half of a mirror, is if I should do
> anything special with my two-disk RAID0 stripe.  I was gobsmacked at
> the simplicity of RAID10 on only two drives by splitting each in half
> and "flipping" one to maximize head movement performance.  Awesome! :-)
> Are there any brilliant hacks for simple striping?  If I'm just putting
> together two [not terribly large] disks, will I benefit from any other
> funny stuff, or should I just stripe together two partitions -- each
> half the size of my other drive, of course -- to make a "boring basic"
> stripe and run with that?
>
>
> TIA & HANW
>
> :-D

Re: striping 2x500G to mirror 1x1T

[Wols Lists]

On 07/03/2022 14:19, Natanji wrote:
> Hello David,
> what you propose here should work, but be aware that you are essentially 
> first writing all data to the 2 smaller drives and then reading from 
> them only when the mirror 1 TB disk gets added. If there are bad sectors 
> that only gets dicovered while reading (not writing), you might lose 
> data. A much cleaner way would be to make a backup of your data disk and 
> then first crate the array with all disks, then write the data back from 
> your backup once the disks are in sync. You should definitely make a 
> backup or you could potentially lose data when creating this array.

But then David is in an equal pickle - what happens if the backup 
contains errors that only get discovered when reading, not writing ... 
no, David's plan is just the same as yours, his "half a mirror" is your 
backup, they suffer the same advantages and disadvantages.
> 
> I don't see the benefit of striping in your setup, by the way. For the 
> two smaller disks it might seem that this is beneficial for performance, 
> but since every write also needs to be made on the mirror, this will 
> essentially lead to your read/write head moving back and forth a lot, 
> no? Because if you write four consecutive chunks C1-C4 on the striped 
> side, two of these would be written to each smaller disk. And on the 1 
> TB disk side this would mean writing the sectors with only one 
> read/write head, each around 500GB apart from each other, and in a back 
> and worth fashion (although that might be optimized by queuing). So your 
> write performance could actually be *worse* than on a single drive, and 
> definitely not gain the 2x speed benefit of striping. For read 
> performance you might be okay, but nevertheless put unneccessary 
> mechanical wear on your 1 TB drive.
> 
> [C1 C3 ....] 500GB-1      [C2 C4 ....] 500GB-2
> [C1 C3 ... (500G space)  C2 C4 ....] 1TB
> 
> At least that's my understanding, someone correct me if I'm wrong and 
> mdadm handles such setups more intelligently (i.e. by laying out the 
> chunks completely linearly on the 1TB disk instead of 1-to-1 "mirroring" 
> the chunks on the 2x500G RAID0).

mdadm doesn't have a clue that the second device in your mirror is a 
raid0. As far as it is concerned you have two completely SEPARATE raids, 
a raid0-stripe, and a raid1-mirror.

So when reading from the mirror it should preferentially read from the 
raid0 because it's the faster device. Which means it WON'T put 
unnecessary mechanical wear on the 1TB device because it preferentially 
will only use it for writing. And because the layer above sees it as a 
1TB device, it won't be scattering writes all over the place. All three 
drives would see a "stream write to the mirror" as a stream write to the 
drive. A1,B1,A2,C1,A3,B2,A4,C2, ...

Cheers,
Wol

Re: striping 2x500G to mirror 1x1T

[Jani Partanen]

Hello, I don't think you gain really any benefit from that raid-0, just 
go linear array, IMHO.


// JiiPee


David T-G kirjoitti 05/03/2022 klo 6.47:
> Hi, all --
>
> I have a 1T data drive (currently in use with data) that I'd like to
> mirror with a pair of 500G drives striped together.  I'll be mirroring
> two partitions, and I'll be striping partitions to ensure the correct
> size, and my understanding is that I'll have to create the mirror on the
> two new drives with half missing, mount it up, copy over the data, dump
> the original disk, and add it as the other half of the mirror to sync.
> If I've missed anything there, please let me know, but all of that
> matches my Googling and I don't think I have any questions.
>
> What I do wonder, and what I don't see in any searches since apparently
> nobody talks about striping up half of a mirror, is if I should do
> anything special with my two-disk RAID0 stripe.  I was gobsmacked at
> the simplicity of RAID10 on only two drives by splitting each in half
> and "flipping" one to maximize head movement performance.  Awesome! :-)
> Are there any brilliant hacks for simple striping?  If I'm just putting
> together two [not terribly large] disks, will I benefit from any other
> funny stuff, or should I just stripe together two partitions -- each
> half the size of my other drive, of course -- to make a "boring basic"
> stripe and run with that?
>
>
> TIA & HANW
>
> :-D

Re: striping 2x500G to mirror 1x1T

[Roman Mamedov]

On Wed, 9 Mar 2022 01:08:47 +0200
Jani Partanen <jiipee@sotapeli.fi> wrote:

> Hello, I don't think you gain really any benefit from that raid-0, just 
> go linear array, IMHO.

There is no gain from the linear array either. In none of the performance,
reliability or even simplicity, since managing a linear md array is not any
simpler than managing an md RAID0.

Smaller disks tend to be slower in linear speeds than larger and more densely
written ones. If that is the case with the particular models used here, then
having the 2x500 side in a RAID0 will help them match up in linear speeds to
the 1TB side.

And a very good point that Wols mentioned, the 1TB-disk member can be set as
--write-mostly, since the 2x500GB RAID0 is likely to overshoot the performance
of a single 1TB disk, even if its individual disks were slower. Not to mention
it has two independent head sets for the same amount of data. (If we're still
talking rotational here...)

-- 
With respect,
Roman

Re: striping 2x500G to mirror 1x1T

[David T-G]

Hi again, all --

...and then Roman Mamedov said...
% 
...
% And a very good point that Wols mentioned, the 1TB-disk member can be set as
% --write-mostly, since the 2x500GB RAID0 is likely to overshoot the performance
% of a single 1TB disk, even if its individual disks were slower. Not to mention
% it has two independent head sets for the same amount of data. (If we're still
% talking rotational here...)

Thanks so much for all of the input!  This sounds like the right way to
go.

I had completely forgotten the "strange magic" ;-) of RAID10 on just
two drives by reordering the two partitions on the second drive, but I
don't think that that applies here and I don't imagine it's worth the
complexity of striping two mirrors, half of each of which are partitions
on the 1T drive, so I'll just stripe [partions on] the two 500s.


Thanks again & have a great day!

:-D
-- 
David T-G
See http://justpickone.org/davidtg/email/
See http://justpickone.org/davidtg/tofu.txt

Re: backups and losses (was "Re: striping 2x500G to mirror 1x1T")

[David T-G]

Natanji, et al --

...and then Natanji said...
% 
% Hello David,
% what you propose here should work, but be aware that you are essentially
% first writing all data to the 2 smaller drives and then reading from them
% only when the mirror 1 TB disk gets added. If there are bad sectors that
% only gets dicovered while reading (not writing), you might lose data. A much
[snip]

Thanks for the input!

While I'd love to have the luxury of redundant backups and more devices,
that's out of the cards on this little server :-/  However, after I rsync
across all content I then use diff -r to check it, so I have to read each
byte on each side anyway.  That gives me at least a little more confidence!


Have a great day

:-D
-- 
David T-G
See http://justpickone.org/davidtg/email/
See http://justpickone.org/davidtg/tofu.txt

to partition or not to partition (was "Re: striping 2x500G to mirror 1x1T")

[David T-G]

Hi again, all --

...and then davidtg-robot@justpickone.org said...
% 
% I have a 1T data drive (currently in use with data) that I'd like to
% mirror with a pair of 500G drives striped together.  I'll be mirroring
% two partitions, and I'll be striping partitions to ensure the correct
[snip]

Is it OK to mirror a partition and an entire device?  Are there any pros
or cons?

I now have my 2ea 500G drives partitioned

  jpo:~ # parted /dev/sdc unit MiB print
  Model: ATA WDC WD5000AAVS-0 (scsi)
  Disk /dev/sdc: 476940MiB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags: 

  Number  Start      End        Size       File system  Name                    Flags
   1      1.00MiB    472171MiB  472170MiB               raid0-WD-WMASU0686160   raid
   4      472171MiB  476940MiB  4769MiB    reiserfs     wwn-0x50014ee055b724ad

  jpo:~ # parted /dev/sdd unit MiB print
  Model: ATA WDC WD5000AAVS-0 (scsi)
  Disk /dev/sdd: 476940MiB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags: 

  Number  Start      End        Size       File system  Name                    Flags
   1      1.00MiB    472171MiB  472170MiB               raid0-WD-WMASU0769665   raid
   4      472171MiB  476940MiB  4769MiB    ext3         wwn-0x50014ee055b72bcd

(I always reserve a small slice at the end of the disk in which to keep
array structure data and other helpful info) and striped 

  jpo:~ # mdadm --detail /dev/md0
  /dev/md0:
             Version : 1.2
       Creation Time : Wed Mar  9 05:48:41 2022
          Raid Level : raid0
          Array Size : 966739968 (921.96 GiB 989.94 GB)
        Raid Devices : 2
       Total Devices : 2
         Persistence : Superblock is persistent

         Update Time : Wed Mar  9 05:48:41 2022
               State : clean 
      Active Devices : 2
     Working Devices : 2
      Failed Devices : 0
       Spare Devices : 0

              Layout : -unknown-
          Chunk Size : 512K

  Consistency Policy : none

                Name : jpo:500Graid0md  (local to host jpo)
                UUID : fc674dbd:577b7d48:5cbe936f:6954eb62
              Events : 0

      Number   Major   Minor   RaidDevice State
         0       8       33        0      active sync   /dev/sdc1
         1       8       49        1      active sync   /dev/sdd1

into a 1T-ish device ready to be half a mirror.  I had originally done
lots of partition size math equal exactly half of my 1T drive partition

  jpo:~ # parted /dev/sdb unit MiB print
  Model: ATA ST31000524AS (scsi)
  Disk /dev/sdb: 953870MiB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags: 

  Number  Start      End        Size       File system  Name  Flags
   1      1.00MiB    858307MiB  858306MiB  xfs          pri
   4      950055MiB  953869MiB  3814MiB                 pri

with existing data (and having to fudge around, not surprisingly, the
striping metadata overhead), and so I've been playing

  jpo:~ # parted /dev/md0 unit MiB print
  Model: Linux Software RAID Array (md)
  Disk /dev/md0: 944082MiB
  Sector size (logical/physical): 512B/512B
  Partition Table: gpt
  Disk Flags: 

  Number  Start      End        Size       File system  Name         Flags
   1      1.00MiB    934641MiB  934640MiB               500Graid0md  raid
   4      934641MiB  944081MiB  9440MiB                 pri          msftdata

with how to slice up the stripe device to size it to match the existing
sdb1 partition with the mirroring metadata overhead, when I wondered ...
do I care?  I don't need the s4 slice on this virtual device; that's just
leftover space.  I hate to think of it going to waste, but it's really
trivial in the grand scheme of things.  Soooooo ... if it's just as happy
to mirror against a whole device as it is a slice of one, then maybe I
can skip ahead.

Thoughts?


Thanks again! :-)

:-D
-- 
David T-G
See http://justpickone.org/davidtg/email/
See http://justpickone.org/davidtg/tofu.txt

Re: to partition or not to partition (was "Re: striping 2x500G to mirror 1x1T")

[Roman Mamedov]

On Wed, 9 Mar 2022 11:04:03 -0500
David T-G <davidtg-robot@justpickone.org> wrote:

> Is it OK to mirror a partition and an entire device?

Yes it is OK.

>   jpo:~ # parted /dev/sdb unit MiB print
>   Model: ATA ST31000524AS (scsi)
>   Disk /dev/sdb: 953870MiB
>   Sector size (logical/physical): 512B/512B
>   Partition Table: gpt
>   Disk Flags: 
> 
>   Number  Start      End        Size       File system  Name  Flags
>    1      1.00MiB    858307MiB  858306MiB  xfs          pri
>    4      950055MiB  953869MiB  3814MiB                 pri
> 

Here it seems weird that your 1TB drive has a 858GB partition, then a hole of
free space about 92 GB in size, and only then another partition of 4 GB. In
preparation for that partition to become a RAID member (instead of xfs data
directly), you could resize it to match the 944082MiB size of md0.

> with existing data (and having to fudge around, not surprisingly, the
> striping metadata overhead), and so I've been playing
> 
>   jpo:~ # parted /dev/md0 unit MiB print
>   Model: Linux Software RAID Array (md)
>   Disk /dev/md0: 944082MiB
>   Sector size (logical/physical): 512B/512B
>   Partition Table: gpt
>   Disk Flags: 
> 
>   Number  Start      End        Size       File system  Name         Flags
>    1      1.00MiB    934641MiB  934640MiB               500Graid0md  raid
>    4      934641MiB  944081MiB  9440MiB                 pri          msftdata
> 
> with how to slice up the stripe device to size it to match the existing
> sdb1 partition with the mirroring metadata overhead, when I wondered ...
> do I care?  I don't need the s4 slice on this virtual device; that's just
> leftover space.  I hate to think of it going to waste, but it's really
> trivial in the grand scheme of things.  Soooooo ... if it's just as happy
> to mirror against a whole device as it is a slice of one, then maybe I
> can skip ahead.

It is definitely an overkill to partition the md0 device. Better to just use
it entirely. If you fix up the partition size on sdb as mentioned above, there
will not be any wasted space on md0.

-- 
With respect,
Roman