Shrinking an array

Shrinking an array

[Wakko Warner]

I have a question about shrinking an array.  My current array is 4x 2tb
disks in raid6 (md0).  The array was created on the 2nd partition of each
disk and spans most of the disk.  I would like to replace the 2tb disks with
750gb disks.  md0 is a luks container with lvm underneath.  I have less than
1tb actually in use.  What would the recommended procedure be for shrinking
this?  I've watched this list, but I don't think I've come across anyone
actually wanting to do this before.
I'm thinking of these steps already:
1) Shrink PV.
2) Shrink luks.  I'm aware that there is not size metadata, but the dm
mapping would need to be shrunk.
3) Shrink md0.  I did this once when I changed a 6 drive raid6 into a 5
drive raid6.  Would I use --array-size= or --size= ?  I understand the
difference is the size of md0 vs the individual members.

So for number 4, if md0 is now small enough, will it accept a member that is
smaller?  If so, I should beable to add the member to the array and issue
--replace.

Thanks.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: Shrinking an array

[Adam Goryachev]

On 11/04/17 10:30, Wakko Warner wrote:
> I have a question about shrinking an array.  My current array is 4x 2tb
> disks in raid6 (md0).  The array was created on the 2nd partition of each
> disk and spans most of the disk.  I would like to replace the 2tb disks with
> 750gb disks.  md0 is a luks container with lvm underneath.  I have less than
> 1tb actually in use.  What would the recommended procedure be for shrinking
> this?  I've watched this list, but I don't think I've come across anyone
> actually wanting to do this before.
> I'm thinking of these steps already:
> 1) Shrink PV.
> 2) Shrink luks.  I'm aware that there is not size metadata, but the dm
> mapping would need to be shrunk.
> 3) Shrink md0.  I did this once when I changed a 6 drive raid6 into a 5
> drive raid6.  Would I use --array-size= or --size= ?  I understand the
> difference is the size of md0 vs the individual members.
>
> So for number 4, if md0 is now small enough, will it accept a member that is
> smaller?  If so, I should beable to add the member to the array and issue
> --replace.
>
> Thanks.
>
I think the order is wrong.... or I mis-understood the layering. You 
need to shrink the highest layer first and work down the stack, for LVM 
on luks on RAID it would be something like this:
1) Reduce the filesystem size
2) Reduce the LV size
3) Reduce the PV size
4) Reduce the luks size
5) Reduce the RAID (mdadm) size
6) Replace the physical devices with smaller ones (or reduce the 
partition size/etc as needed)

I generally reduce my each one with a decent margin, and then when I'm 
finished, I increase each one to fill the available space (after the 
physical device size is changed). This avoids issues with accidentally 
trimming too much and then losing data/corrupting data. You should also 
verify that all your data is accessible after each step, most steps are 
reversible if you identify the issue quickly enough (at least with 
simple stacks when changing partition size, LVM and/or luks might 
complicate that).

Regards,
Adam



-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

Re: Shrinking an array

[Wakko Warner]

Adam Goryachev wrote:
> On 11/04/17 10:30, Wakko Warner wrote:
> >I have a question about shrinking an array.  My current array is 4x 2tb
> >disks in raid6 (md0).  The array was created on the 2nd partition of each
> >disk and spans most of the disk.  I would like to replace the 2tb disks with
> >750gb disks.  md0 is a luks container with lvm underneath.  I have less than
> >1tb actually in use.  What would the recommended procedure be for shrinking
> >this?  I've watched this list, but I don't think I've come across anyone
> >actually wanting to do this before.
> >I'm thinking of these steps already:
> >1) Shrink PV.
> >2) Shrink luks.  I'm aware that there is not size metadata, but the dm
> >mapping would need to be shrunk.
> >3) Shrink md0.  I did this once when I changed a 6 drive raid6 into a 5
> >drive raid6.  Would I use --array-size= or --size= ?  I understand the
> >difference is the size of md0 vs the individual members.
> >
> >So for number 4, if md0 is now small enough, will it accept a member that is
> >smaller?  If so, I should beable to add the member to the array and issue
> >--replace.
> >
> >Thanks.
> >
> I think the order is wrong.... or I mis-understood the layering. You
> need to shrink the highest layer first and work down the stack, for
> LVM on luks on RAID it would be something like this:
> 1) Reduce the filesystem size
> 2) Reduce the LV size
> 3) Reduce the PV size
> 4) Reduce the luks size
> 5) Reduce the RAID (mdadm) size
> 6) Replace the physical devices with smaller ones (or reduce the
> partition size/etc as needed)

Thanks, however, 1 and 2 doesn't need to happen.  I deleted several LVs.  My
highest PE in use is 771583 (each extent is 1M) and the last lv could be
moved somewhere else which would leave my highest PE 87982. 

> I generally reduce my each one with a decent margin, and then when
> I'm finished, I increase each one to fill the available space (after
> the physical device size is changed). This avoids issues with
> accidentally trimming too much and then losing data/corrupting data.
> You should also verify that all your data is accessible after each
> step, most steps are reversible if you identify the issue quickly
> enough (at least with simple stacks when changing partition size,
> LVM and/or luks might complicate that).

I'm not that concerned about the LV that is at the highest PE range.  If I
loose it, I loose it.  it's the reduce the raid size I was wondering about. 
Can you give me an example of the command to run for it?

I would be interested in doing this while the system is running (it only has
1180 days uptime =)  However, I don't have the replacement disks yet.  I
wanted ideas before I actually do it.

-- 
 Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
 million bugs.

Re: Shrinking an array

[NeilBrown]

[-- Attachment #1: Type: text/plain, Size: 2102 bytes --]

On Mon, Apr 10 2017, Wakko Warner wrote:

> I have a question about shrinking an array.  My current array is 4x 2tb
> disks in raid6 (md0).  The array was created on the 2nd partition of each
> disk and spans most of the disk.  I would like to replace the 2tb disks with
> 750gb disks.  md0 is a luks container with lvm underneath.  I have less than
> 1tb actually in use.  What would the recommended procedure be for shrinking
> this?  I've watched this list, but I don't think I've come across anyone
> actually wanting to do this before.
> I'm thinking of these steps already:
> 1) Shrink PV.
> 2) Shrink luks.  I'm aware that there is not size metadata, but the dm
> mapping would need to be shrunk.
> 3) Shrink md0.  I did this once when I changed a 6 drive raid6 into a 5
> drive raid6.  Would I use --array-size= or --size= ?  I understand the
> difference is the size of md0 vs the individual members.

You don't need --array-size, as reducing --size is non-destructive.
Reducing the number of devices *is* destructive, so if you were to do
that, you would need to adjust the --array-size first.

So when you have prepared the contents of the array, use
  mdadm /dev/mdXXX --grow --size=750G

or whatever size you think is appropriate.

Then check that all your data is still accessible.  e.g. fsck your
filesystem.
If you are confident that the data is still accessible, you can continue
to --replace devices.
If not, you can
  mdadm /dev/mdXXX --grow --size=max

to restore the previous state, and then try to figure out what went
wrong.

NeilBrown


>
> So for number 4, if md0 is now small enough, will it accept a member that is
> smaller?  If so, I should beable to add the member to the array and issue
> --replace.
>
> Thanks.
>
> -- 
>  Microsoft has beaten Volkswagen's world record.  Volkswagen only created 22
>  million bugs.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Shrinking an array

[Roman Mamedov]

On Mon, 10 Apr 2017 20:30:08 -0400
Wakko Warner <wakko@animx.eu.org> wrote:

> I have less than 1tb actually in use.

I'd say just create a new array from scratch on new drives and copy data over.
Going to great lengths with all the layering to ensure proper and safe
shrinking on each is just wasted effort to babysit what is basically an empty
array.

-- 
With respect,
Roman