* Best strategy to incrementally replace smaller HDDs
@ 2011-09-09 10:48 Michał Sawicz
2011-09-09 12:34 ` David Brown
2011-09-09 13:13 ` Robin Hill
0 siblings, 2 replies; 6+ messages in thread
From: Michał Sawicz @ 2011-09-09 10:48 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: Type: text/plain, Size: 1281 bytes --]
Hi all, given the configuration below:
* 8 x 1TB HDDs
* 2 x 2TB HDDs
On which I currently have:
* (10 x 1TB) RAID6 - persistent storage
* (2 x 1TB) system / temporary storage etc.
I want to replace the 1TB drives for 2TB ones on an as-needed basis,
what strategy would you recommend?
1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
I would need to replace two of the drives just to match current
capacity. Each next 2TB drive would get me 1TB additional
capacity (but actually I'd need to replace two to gain
anything). That sounds to be most future-proof, but most
expensive.
2. If I moved to 2TB RAID5 members, that would reduce redundancy
but replacing just two would gain me 2TB capacity. Most of the
above still applies.
3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
the redundancy to just one drive if it was the 2TB drive that
failed, but each new drive would gain me 1TB capacity and I only
need to replace one-by-one. This sounds like the cheapest, but
worst possible approach.
So, am I missing something? What do you think?
--
Michał (Saviq) Sawicz <michal@sawicz.net>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Best strategy to incrementally replace smaller HDDs
2011-09-09 10:48 Best strategy to incrementally replace smaller HDDs Michał Sawicz
@ 2011-09-09 12:34 ` David Brown
2011-09-09 14:12 ` Michał Sawicz
2011-09-09 13:13 ` Robin Hill
1 sibling, 1 reply; 6+ messages in thread
From: David Brown @ 2011-09-09 12:34 UTC (permalink / raw)
To: linux-raid
On 09/09/2011 12:48, Michał Sawicz wrote:
> Hi all, given the configuration below:
> * 8 x 1TB HDDs
> * 2 x 2TB HDDs
>
> On which I currently have:
> * (10 x 1TB) RAID6 - persistent storage
> * (2 x 1TB) system / temporary storage etc.
>
By this do you mean that you have 8 x 1TB drives with a 1 TB partition
on each, and 2 x 2T drives with 2 x 1TB partition on each? So that the
two big disks are shared with both raids?
> I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> what strategy would you recommend?
This sounds like you are thinking that you can replace a single disk in
your RAID6 array and get more storage - changing 10 x 1 TB raid6 = 8TB
into 9 x 1TB + 1 x 2TB raid6 = 9 TB. It doesn't work like that. You
will have to replace /all/ your 1 TB devices with 2 TB devices (and move
the second raid off the two existing 2TB devices) - all members of the
raid6 must be the same size.
To help you plan your upgrades, it is also useful to know your
partitioning scheme (for example, do you use LVM?), whether you have the
space to put lots more drives in the system or must do it one drive at a
time, whether you can take the system off-line during the process, and
whether you need to do the upgrade quickly or can spend a week or so at
it (some of these are conflicting requirements).
Before you think about upgrading, however, make sure you have a solid
backup. Then make sure you have a backup of that backup - and a plan
for how to restore everything if something goes horribly wrong.
>
> 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
> I would need to replace two of the drives just to match current
> capacity. Each next 2TB drive would get me 1TB additional
> capacity (but actually I'd need to replace two to gain
> anything). That sounds to be most future-proof, but most
> expensive.
> 2. If I moved to 2TB RAID5 members, that would reduce redundancy
> but replacing just two would gain me 2TB capacity. Most of the
> above still applies.
> 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
> the redundancy to just one drive if it was the 2TB drive that
> failed, but each new drive would gain me 1TB capacity and I only
> need to replace one-by-one. This sounds like the cheapest, but
> worst possible approach.
>
> So, am I missing something? What do you think?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Best strategy to incrementally replace smaller HDDs
2011-09-09 10:48 Best strategy to incrementally replace smaller HDDs Michał Sawicz
2011-09-09 12:34 ` David Brown
@ 2011-09-09 13:13 ` Robin Hill
2011-09-09 14:20 ` Michał Sawicz
1 sibling, 1 reply; 6+ messages in thread
From: Robin Hill @ 2011-09-09 13:13 UTC (permalink / raw)
To: Michał Sawicz; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 3152 bytes --]
On Fri Sep 09, 2011 at 12:48:01PM +0200, Michał Sawicz wrote:
> Hi all, given the configuration below:
> * 8 x 1TB HDDs
> * 2 x 2TB HDDs
>
From what you cover below, it sounds like you're limited to 10 drives in
total. Is this the case?
> On which I currently have:
> * (10 x 1TB) RAID6 - persistent storage
> * (2 x 1TB) system / temporary storage etc.
>
> I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> what strategy would you recommend?
>
> 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
> I would need to replace two of the drives just to match current
> capacity. Each next 2TB drive would get me 1TB additional
> capacity (but actually I'd need to replace two to gain
> anything). That sounds to be most future-proof, but most
> expensive.
> 2. If I moved to 2TB RAID5 members, that would reduce redundancy
> but replacing just two would gain me 2TB capacity. Most of the
> above still applies.
So, for both of these, you're planning on changing to:
* (2 x 1TB) system
* (2 x 2TB) + (3 x 1+1TB) persistent storage
This gives you 6TB persistent storage in RAID6, or 8TB in RAID5. As you
say, replacing 2 1TB drives for 2TB drives (or adding a 2TB drive, if
possible) would give you 8TB in RAID6 again.
You'll also need to do a backup, rebuild & restore for either of these
solutions.
In either case, replacing a single drive (assuming you're limited in
total drive count) would not gain any capacity, whereas a pair of drive
replacements would give you a 2TB increase.
I'd steer clear of RAID5 with 2TB drives - rebuilding a 2TB drive will
present a very large window for total array failure.
> 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
> the redundancy to just one drive if it was the 2TB drive that
> failed, but each new drive would gain me 1TB capacity and I only
> need to replace one-by-one. This sounds like the cheapest, but
> worst possible approach.
>
Losing a single 2TB drive would lose 2 partitions, wiping out all
redundancy. You're only needing to rebuild 1TB before you're back to
partial redundancy though, so you're less prone to read failures than
with RAID5 above. A second 2TB drive failure during recovery of either
partition will cause total array failure though.
Replacing a single 1TB drive would gain you 1TB in space.
There's also no need to do a complete wipe & restore - you can just
replace the drive and recover onto the new one.
Performance will be worse (particularly during recovery) as you're
always seeking between the two partitions.
Personally, I'd go with RAID6 on 2TB partitions. It is more expensive,
but I think the performance and redundancy benefits outweigh the costs.
Cheers,
Robin
--
___
( ' } | Robin Hill <robin@robinhill.me.uk> |
/ / ) | Little Jim says .... |
// !! | "He fallen in de water !!" |
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Best strategy to incrementally replace smaller HDDs
2011-09-09 12:34 ` David Brown
@ 2011-09-09 14:12 ` Michał Sawicz
0 siblings, 0 replies; 6+ messages in thread
From: Michał Sawicz @ 2011-09-09 14:12 UTC (permalink / raw)
To: David Brown; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 2630 bytes --]
Dnia 2011-09-09, pią o godzinie 14:34 +0200, David Brown pisze:
> On 09/09/2011 12:48, Michał Sawicz wrote:
> > Hi all, given the configuration below:
> > * 8 x 1TB HDDs
> > * 2 x 2TB HDDs
> >
> > On which I currently have:
> > * (10 x 1TB) RAID6 - persistent storage
> > * (2 x 1TB) system / temporary storage etc.
> >
>
> By this do you mean that you have 8 x 1TB drives with a 1 TB partition
> on each, and 2 x 2T drives with 2 x 1TB partition on each? So that the
> two big disks are shared with both raids?
Yes, that wasn't like that before, but that's a result of exchanging two
1TB drives to 2TB ones.
> > I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> > what strategy would you recommend?
>
> This sounds like you are thinking that you can replace a single disk in
> your RAID6 array and get more storage - changing 10 x 1 TB raid6 = 8TB
> into 9 x 1TB + 1 x 2TB raid6 = 9 TB. It doesn't work like that. You
> will have to replace /all/ your 1 TB devices with 2 TB devices (and move
> the second raid off the two existing 2TB devices) - all members of the
> raid6 must be the same size.
Actually no, I know how RAIDs work, so the three schemes I described
below account for that.
> To help you plan your upgrades, it is also useful to know your
> partitioning scheme (for example, do you use LVM?), whether you have the
> space to put lots more drives in the system or must do it one drive at a
> time, whether you can take the system off-line during the process, and
> whether you need to do the upgrade quickly or can spend a week or so at
> it (some of these are conflicting requirements).
No LVM here. The most important - big RAID6 - is a single partition. The
remaining 2 x 1TB on the big drives are split into RAID1 for boot,
RAID10 for root, RAID0 for temporary storage and swap space. I can
temporarily put more drives in there. I can take it off-line for some
periods and I can do the upgrade over an extended period of time.
> Before you think about upgrading, however, make sure you have a solid
> backup. Then make sure you have a backup of that backup - and a plan
> for how to restore everything if something goes horribly wrong.
Yeah... I plan to rely on RAID's redundancy by moving data around,
growing / shrinking the filesystems and then shuffling the HDDs. I know
that's asking for trouble, but I haven't yet found a place to rent 8TBs
of storage for a week... And that's worked for me in the past, btw.
Twice, even, IIRC.
--
Michał (Saviq) Sawicz <michal@sawicz.net>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Best strategy to incrementally replace smaller HDDs
2011-09-09 13:13 ` Robin Hill
@ 2011-09-09 14:20 ` Michał Sawicz
2011-09-27 8:37 ` Best strategy to incrementally replace smaller HDDs [success story] Michał Sawicz
0 siblings, 1 reply; 6+ messages in thread
From: Michał Sawicz @ 2011-09-09 14:20 UTC (permalink / raw)
To: Robin Hill; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 3760 bytes --]
Dnia 2011-09-09, pią o godzinie 14:13 +0100, Robin Hill pisze:
> On Fri Sep 09, 2011 at 12:48:01PM +0200, Michał Sawicz wrote:
>
> > Hi all, given the configuration below:
> > * 8 x 1TB HDDs
> > * 2 x 2TB HDDs
> >
> From what you cover below, it sounds like you're limited to 10 drives in
> total. Is this the case?
In the long run - yes, but I can have more temporarily. It's just a
physical issue, not controller issue.
> > On which I currently have:
> > * (10 x 1TB) RAID6 - persistent storage
> > * (2 x 1TB) system / temporary storage etc.
> >
> > I want to replace the 1TB drives for 2TB ones on an as-needed basis,
> > what strategy would you recommend?
> >
> > 1. If I moved to 2TB RAID6 members (using RAID0 on the 1TB drives),
> > I would need to replace two of the drives just to match current
> > capacity. Each next 2TB drive would get me 1TB additional
> > capacity (but actually I'd need to replace two to gain
> > anything). That sounds to be most future-proof, but most
> > expensive.
> > 2. If I moved to 2TB RAID5 members, that would reduce redundancy
> > but replacing just two would gain me 2TB capacity. Most of the
> > above still applies.
>
> So, for both of these, you're planning on changing to:
> * (2 x 1TB) system
> * (2 x 2TB) + (3 x 1+1TB) persistent storage
Actually no, currently I'm planning to go:
* (2 x 1TB) system,
* (4 x 2TB) + (2 x 1+1TB)
So that's 8TB in RAID6, 10TB in RAID5. I already have 8TB and the only
way is up. With RAID6 I'd have to replace at least 4x1TB for 3x2TB
initially (one less drive total).
> This gives you 6TB persistent storage in RAID6, or 8TB in RAID5. As you
> say, replacing 2 1TB drives for 2TB drives (or adding a 2TB drive, if
> possible) would give you 8TB in RAID6 again.
>
> You'll also need to do a backup, rebuild & restore for either of these
> solutions.
As I mentioned in my previous reply to David, I'm planning to
shrink/grow as needed.
> In either case, replacing a single drive (assuming you're limited in
> total drive count) would not gain any capacity, whereas a pair of drive
> replacements would give you a 2TB increase.
>
> I'd steer clear of RAID5 with 2TB drives - rebuilding a 2TB drive will
> present a very large window for total array failure.
Yeah, that's why I switched to RAID6 in the first place...
> > 3. If I kept to 1TB RAID6 (two on the 2TB drives), I would reduce
> > the redundancy to just one drive if it was the 2TB drive that
> > failed, but each new drive would gain me 1TB capacity and I only
> > need to replace one-by-one. This sounds like the cheapest, but
> > worst possible approach.
> >
> Losing a single 2TB drive would lose 2 partitions, wiping out all
> redundancy. You're only needing to rebuild 1TB before you're back to
> partial redundancy though, so you're less prone to read failures than
> with RAID5 above. A second 2TB drive failure during recovery of either
> partition will cause total array failure though.
>
> Replacing a single 1TB drive would gain you 1TB in space.
>
> There's also no need to do a complete wipe & restore - you can just
> replace the drive and recover onto the new one.
>
> Performance will be worse (particularly during recovery) as you're
> always seeking between the two partitions.
>
> Personally, I'd go with RAID6 on 2TB partitions. It is more expensive,
> but I think the performance and redundancy benefits outweigh the costs.
That's the direction I'm leaning to as well.
Thanks a lot for the comments!
--
Michał (Saviq) Sawicz <michal@sawicz.net>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Best strategy to incrementally replace smaller HDDs [success story]
2011-09-09 14:20 ` Michał Sawicz
@ 2011-09-27 8:37 ` Michał Sawicz
0 siblings, 0 replies; 6+ messages in thread
From: Michał Sawicz @ 2011-09-27 8:37 UTC (permalink / raw)
To: Robin Hill; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 3401 bytes --]
Thanks again for all the pointers, I will now describe my workflow that
got me going with the new drives!
I'll try and do some ASCII-art [=== used space, --- space (not drives)
used for parity].
1. initial situation
-----------------
~~1TB~~
sda |===========|
sdb |===========|
sdc |=====|
sdd |=====|
sde |=====|
sdf |=====|
sdg |=====|
sdh |=====|
sdi |-----|
sdj |-----|
md1 |=====| - 2 x 1TB RAID0 == 2TB
md2 |=====| - 10 x 1TB RAID6 == 8TB
2. added 4 x 2TB drives
--------------------
sdk | |
sdl | |
sdm | |
sdn |-----------|
md3 | | - 5 x 2TB RAID6 == 6TB (degraded)
3. moved part of md2 data onto md3
----------------------------------
sda | ======|
sdb | ======|
sdc |=====|
sdd |=====|
sde |= |
sdf | |
sdg | |
sdh | |
sdi |-----|
sdj |-----|
sdk |===========|
sdl |===========|
sdm |===========|
sdn |-----------|
md1 |=====| - 2 x 1TB RAID0 == 2TB
md2 |== | - 10 x 1TB RAID6 == 8TB
md3 |===========| - 5 x 2TB RAID6 == 6TB (degraded)
4. shrunk and removed drives from md2
----------------------------------
sda | ======|
sdb | ======|
sdc |=====|
sdd |=====|
sde |= |
sdf |-----|
sdg | |
sdh | |
sdi | |
sdj | |
sdk |===========|
sdl |===========|
sdm |===========|
sdn |-----------|
md1 |=====| - 2 x 1TB RAID0 == 2TB
md2 |==== | - 5 x 1TB RAID6 == 3TB (degraded)
md3 |===========| - 5 x 2TB RAID6 == 6TB (degraded)
5. new arrays to free the old 2TB drives
and create an additional member for md4
---------------------------------------
sda |===========|
sdb |===========|
sdc |=====|
sdd |=====|
sde |= |
sdf |-----|
sdg |=====|
sdh |=====|
sdi |=====|
sdj |=====|
sdk | |
sdl | |
sdm |-----------|
sdn |-----------|
md4 | | - 4 x 1TB RAID10 == 2TB (degraded)
md1 |=====| - 2 x 1TB RAID0 == 2TB
md2 |==== | - 5 x 1TB RAID6 == 3TB (degraded)
md3 |======= | - 7 x 2TB RAID6 == 10TB
6. Moved rest of the data from md2,
dropped it and replaced a disk in md4
-------------------------------------
sda |===========|
sdb |===========|
sdc |=====|
sdg |=====|
sdh |=====|
sdj |=====|
sdk |===========|
sdl | |
sdm |-----------|
sdn |-----------|
md4 | | - 4 x 1TB RAID10 == 2TB (degraded)
md1 |=====| - 2 x 1TB RAID0 == 2TB
md3 |========= | - 7 x 2TB RAID6 == 10TB
And it only took two weeks! ;)
And again, thanks for all the pointers I got on the list, it all
happened without any data loss.
--
Michał (Saviq) Sawicz <michal@sawicz.net>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-09-27 8:37 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-09-09 10:48 Best strategy to incrementally replace smaller HDDs Michał Sawicz
2011-09-09 12:34 ` David Brown
2011-09-09 14:12 ` Michał Sawicz
2011-09-09 13:13 ` Robin Hill
2011-09-09 14:20 ` Michał Sawicz
2011-09-27 8:37 ` Best strategy to incrementally replace smaller HDDs [success story] Michał Sawicz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.