[dm-crypt] expanding encrypted volume/growing the volume

[dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

My system uses LVM, with LUKS encryption on top of individual logical
volumes.  The volume group has some free space, and I would like to
extend the volume and then grow the encrypted container and then the
file system on it.

When expanding I'll use free space that I don't believe has ever been
zeroed or random filled.  It's possible it was used for a file system
at some point.

Is that much of a concern?  The FAQ advises wipeing it, though the
only explicit reasons seem not much of a concern for space in a volume
group (but later there are references to attacks available if the
attacker can determine which sectors are unused).  As far as I know
there is no way to access the unused area of the volume group (well,
except for mapping all physical device sectors in use and operating on
the remainder after somehow figuring out where metadata is kept), and
attempting to do so seems hazardous.  It seems particularly hazardous
because there are active snapshots, and it seems possible they grab
freespace dynamically.

Is
cryptsetup resize /dev/VG/LV
the right way to expand the container once the LV is extended?  Are
there any things I should look out for in the whole process?  Do I
need to reboot or remount anywhere along the way for changes to take
effect?  The filesystems are ext3 and reiser.

The software on the system is quite old, Debian Lenny + some
backports.  cryptsetup is at 1.0.6 (Debian version 2:1.0.6-7) and the
kernel is 2.6.32 (which is a backport).

Thanks.
Ross Boylan

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Arno Wagner]

On Tue, Sep 09, 2014 at 23:52:03 CEST, Ross Boylan wrote:
> My system uses LVM, with LUKS encryption on top of individual logical
> volumes.  The volume group has some free space, and I would like to
> extend the volume and then grow the encrypted container and then the
> file system on it.

You cannot "grow the encrypted container". A LUKS mapping has no fixed
size associated with it. It merely takes the size of the underlying
raw container as its size. So if you resize the raw container, the
LUKS container changes size automatically on re-mapping.
 
> When expanding I'll use free space that I don't believe has ever been
> zeroed or random filled.  It's possible it was used for a file system
> at some point.
> 
> Is that much of a concern?  The FAQ advises wipeing it, though the
> only explicit reasons seem not much of a concern for space in a volume
> group (but later there are references to attacks available if the
> attacker can determine which sectors are unused).  As far as I know
> there is no way to access the unused area of the volume group (well,
> except for mapping all physical device sectors in use and operating on
> the remainder after somehow figuring out where metadata is kept), and
> attempting to do so seems hazardous.  It seems particularly hazardous
> because there are active snapshots, and it seems possible they grab
> freespace dynamically.

Youa re overlooking one thing: If you resize the raw container and
re-map LUKS, you will still have the old filesystem size in the
LUKS container. Filesystems do nto grow by themselves. You could
just zero the space behinf the filesystem. If you get it wrong,
you may kill the filesystem though. But resizing a partition or
filesystem is not something you should do without a full, verified
backup anyways.

> Is
> cryptsetup resize /dev/VG/LV
> the right way to expand the container once the LV is extended?  

No. cryptsetup resize resizes the mapping temporarily to something
else than the underlying raw container size.

> Are there any things I should look out for in the whole process?  Do I
> need to reboot or remount anywhere along the way for changes to take
> effect?  The filesystems are ext3 and reiser.

Aehm, you need to umount and unmap everything before doing this?
You need a full, verified backup of _all_ data?

Seriously, I do not think you understand what you want to do 
enough to do it safely. Better make that full backup and 
recreate the raw container in question with the new size,
make a new LUKS container out of it and put a new filesystem
in it after zero-wiping it. 

> The software on the system is quite old, Debian Lenny + some
> backports.  cryptsetup is at 1.0.6 (Debian version 2:1.0.6-7) and the
> kernel is 2.6.32 (which is a backport).

That should not matter.

Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

Thanks for your response.  Questions/comments below.
On Wed, Sep 10, 2014 at 01:50:38AM +0200, Arno Wagner wrote:
> On Tue, Sep 09, 2014 at 23:52:03 CEST, Ross Boylan wrote:
> > My system uses LVM, with LUKS encryption on top of individual logical
> > volumes.  The volume group has some free space, and I would like to
> > extend the volume and then grow the encrypted container and then the
> > file system on it.
> 
> You cannot "grow the encrypted container". A LUKS mapping has no fixed
> size associated with it. It merely takes the size of the underlying
> raw container as its size. So if you resize the raw container, the
> LUKS container changes size automatically on re-mapping.

I was wondering if "encrypted container" was the right phrase; it
seems not.  I definitely did not mean the LUKS container.

I'll use this:
/dev/VG/LV  = raw device
/dev/mapper/LV_crypt = decrypted device (cryptsetup luksOpen /dev/VG/LV LV_crypt)

After lvextending LV  it would then be
cryptsetup resize LV_crypt
with resize by default using the size of the underlying raw device.

Are you saying that there is no need to cryptsetup resize?  And that I
must bring down (in the cryptsetup luksClose sense) and up (cryptsetup
luksOpen) LV_crypt to make it full size?

>  
> > When expanding I'll use free space that I don't believe has ever been
> > zeroed or random filled.  It's possible it was used for a file system
> > at some point.
> > 
> > Is that much of a concern?  The FAQ advises wipeing it, though the
> > only explicit reasons seem not much of a concern for space in a volume
> > group (but later there are references to attacks available if the
> > attacker can determine which sectors are unused).  As far as I know
> > there is no way to access the unused area of the volume group (well,
> > except for mapping all physical device sectors in use and operating on
> > the remainder after somehow figuring out where metadata is kept), and
> > attempting to do so seems hazardous.  It seems particularly hazardous
> > because there are active snapshots, and it seems possible they grab
> > freespace dynamically.
> 
> Youa re overlooking one thing: If you resize the raw container and
> re-map LUKS, you will still have the old filesystem size in the
> LUKS container. 

You seem to be overlooking my initial statement that I planned to
"grow the encrypted container [bad phrasing] and then the file system
on it".  I'm aware that the file system must be resized.

> Filesystems do nto grow by themselves. You could
> just zero the space behind the filesystem. If you get it wrong,
> you may kill the filesystem though. But resizing a partition or
> filesystem is not something you should do without a full, verified
> backup anyways.

I have backups, though not completely current--they happen outside of
my control.  I do not have enough space to create a whole new volume
of the desired size.  And a copy would be quite slow.

The most delicate operation involves a mail spool.  I suppose I should
first shut down the server, backup the key files, and backup
everything that has changed since the previous snapshot (I control the
snapshots).

I'm not sure what you mean by "the space behind the file system".  If
I've extended the LV, I know where the new space on the LV is,
approximately.  But I doubt I can determine it exactly.

> 
> > Is
> > cryptsetup resize /dev/VG/LV
> > the right way to expand the container once the LV is extended?  
> 
> No. cryptsetup resize resizes the mapping temporarily to something
> else than the underlying raw container size.

I see now that should have been LV_crypt, not /dev/VG/LV.
> 
> > Are there any things I should look out for in the whole process?  Do I
> > need to reboot or remount anywhere along the way for changes to take
> > effect?  The filesystems are ext3 and reiser.
> 
> Aehm, you need to umount and unmap everything before doing this?
Even if the filesystem supports online resize?
> You need a full, verified backup of _all_ data?
> 
> Seriously, I do not think you understand what you want to do 
> enough to do it safely. 

That's why I'm asking.

I think I was misled by an experience with raid sofware, which has
metadata at or near the end of the device (mdadm with 0.90 superblock
format).  I thought there was some metadata the crypto system put in
its containers after the LUKS container;
http://wiki.cryptsetup.googlecode.com/git/LUKS-standard/on-disk-format.pdf
says not.  And you just said even the metadata in the LUKS container
does not record the device size.


So here's what I'm thinking of doing, all while the filesystem is live.:
#backup
# I was thinking of zeroing here, without knowing how to do it safely.
lvextend -L +20G VG/LV
# zero here?  how to do it safely?
cryptsetup resize LV_crypt  # unnecessary?  insufficient?
resize_reiserfs /dev/mapper/LV_crypt

If crypsetup resize does not do what I need, instead luksClose,
lvextend, luksOpen, resize_reiserfs.  Obviously the filesystem would
not be live during the operation, and surrounding umount and mout
would be needed.

Ross

> Better make that full backup and 
> recreate the raw container in question with the new size,
> make a new LUKS container out of it and put a new filesystem
> in it after zero-wiping it. 

I have some 20G LV's, and so would prefer to avoid the recreation; the
backup is clearly essential.

> 
> > The software on the system is quite old, Debian Lenny + some
> > backports.  cryptsetup is at 1.0.6 (Debian version 2:1.0.6-7) and the
> > kernel is 2.6.32 (which is a backport).
> 
> That should not matter.
> 
> Arno

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

A little more on resizing on the bottom, with related excerpts above it.
On Tue, Sep 09, 2014 at 08:23:37PM -0700, Ross Boylan wrote:
> Thanks for your response.  Questions/comments below.
> On Wed, Sep 10, 2014 at 01:50:38AM +0200, Arno Wagner wrote:
> > On Tue, Sep 09, 2014 at 23:52:03 CEST, Ross Boylan wrote:
> > > My system uses LVM, with LUKS encryption on top of individual logical
> > > volumes.  The volume group has some free space, and I would like to
> > > extend the volume and then grow the encrypted container and then the
> > > file system on it.
> > 
> > You cannot "grow the encrypted container". A LUKS mapping has no fixed
> > size associated with it. It merely takes the size of the underlying
> > raw container as its size. So if you resize the raw container, the
> > LUKS container changes size automatically on re-mapping.
> 
> I was wondering if "encrypted container" was the right phrase; it
> seems not.  I definitely did not mean the LUKS container.
> 
> I'll use this:
> /dev/VG/LV  = raw device
> /dev/mapper/LV_crypt = decrypted device (cryptsetup luksOpen /dev/VG/LV LV_crypt)
> 
> After lvextending LV  it would then be
> cryptsetup resize LV_crypt
> with resize by default using the size of the underlying raw device.
> 
> Are you saying that there is no need to cryptsetup resize?  And that I
> must bring down (in the cryptsetup luksClose sense) and up (cryptsetup
> luksOpen) LV_crypt to make it full size?
....
> > > Is
> > > cryptsetup resize /dev/VG/LV
> > > the right way to expand the container once the LV is extended?  
> > 
> > No. cryptsetup resize resizes the mapping temporarily to something
> > else than the underlying raw container size.
> 
> I see now that should have been LV_crypt, not /dev/VG/LV.
> > 
> > > Are there any things I should look out for in the whole process?  Do I
> > > need to reboot or remount anywhere along the way for changes to take
> > > effect?  The filesystems are ext3 and reiser.
> > 
> > Aehm, you need to umount and unmap everything before doing this?
> Even if the filesystem supports online resize?
...
> 
> So here's what I'm thinking of doing, all while the filesystem is live.:
> #backup
> # I was thinking of zeroing here, without knowing how to do it safely.
> lvextend -L +20G VG/LV
> # zero here?  how to do it safely?
> cryptsetup resize LV_crypt  # unnecessary?  insufficient?
> resize_reiserfs /dev/mapper/LV_crypt
> 
> If crypsetup resize does not do what I need, instead luksClose,
> lvextend, luksOpen, resize_reiserfs.  Obviously the filesystem would
> not be live during the operation, and surrounding umount and mout
> would be needed.
> 

At least one piece of advice on the internet does luksClose, luksOpen 
AND cryptsetup resize:
http://www.xbsd.nl/2012/03/resize-an-encrypted-lvm-logical-volume.html

Ross

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Arno Wagner]

On Wed, Sep 10, 2014 at 06:30:24 CEST, Ross Boylan wrote:
> A little more on resizing on the bottom, with related excerpts above it.
[...]
> At least one piece of advice on the internet does luksClose, luksOpen 
> AND cryptsetup resize:
> http://www.xbsd.nl/2012/03/resize-an-encrypted-lvm-logical-volume.html

That one is broken. Or rather the "resize" does nothing to
the LUKS container. It may just be there for the --verbose.
Really, there is no "partition size" field anywhere in the 
LUKS header and it is not needed.

A brief look into the man-page shows what "resize" does:

 resize <name>

    Resizes an active mapping <name>.
    If --size (in sectors) is not specified, the size of the under‐
    lying  block  device is used. Note that this does not change the
    raw device geometry, it just changes how many sectors of the raw
    device are represented in the mapped device.


As to my other comments, I see now that you left out one very 
critical piece of information: You want to do this _online_.
(Or I read over it. If so, sorry.) That is generally not a 
good idea, but that is indeed one of the scenarios where you 
would need "cryptsetup resize". (But not after a luksOpen.
luksOpen reads the device-size from the kernel anyways.)

You would need to extend the partition first, make the kernel 
aware of the new size (I gather lvextend does that, personally I 
stay away from LVM, far too complicated...), call 
"cryptsetup resize <device>" and then extend the filesystem 
in the LUKS container. If all of that works, good. If anything 
goes wrong, I hope you refreshed that backup and have the time 
to restore from it. 

Generally, in a situation where you have low downtime needs, 
you should have a second identical machine with automatic 
fail-over anyways. There you can take down one machine,
make the changes offline, test them, bring it back up
and then fail-over to it. Repeat with the second one.

If you  do not have low downdime needs, do this offline,
as the risk of doing some real damage is lower.

Arno

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

On Wed, Sep 10, 2014 at 07:16:10AM +0200, Arno Wagner wrote:
> On Wed, Sep 10, 2014 at 06:30:24 CEST, Ross Boylan wrote:
> > A little more on resizing on the bottom, with related excerpts above it.
> [...]
> > At least one piece of advice on the internet does luksClose, luksOpen 
> > AND cryptsetup resize:
> > http://www.xbsd.nl/2012/03/resize-an-encrypted-lvm-logical-volume.html
> 
> That one is broken. Or rather the "resize" does nothing to
> the LUKS container. It may just be there for the --verbose.
> Really, there is no "partition size" field anywhere in the 
> LUKS header and it is not needed.
> 
> A brief look into the man-page shows what "resize" does:
> 
>  resize <name>
> 
>     Resizes an active mapping <name>.
>     If --size (in sectors) is not specified, the size of the under‐
>     lying  block  device is used. Note that this does not change the
>     raw device geometry, it just changes how many sectors of the raw
>     device are represented in the mapped device.
> 
> 
> As to my other comments, I see now that you left out one very 
> critical piece of information: You want to do this _online_.
> (Or I read over it. If so, sorry.) That is generally not a 
> good idea, but that is indeed one of the scenarios where you 
> would need "cryptsetup resize". (But not after a luksOpen.
> luksOpen reads the device-size from the kernel anyways.)

The parenthetical remark sounds as if it's saying not to use
cryptsetup resize after luksOpen, but I don't think that's the
intended meaning since without luksOpen (or cryptsetup create if not
using LUKS) there is nothing to resize.  So is it a restatement of the
fact that if the sequence is lvextend and then luksOpen, there's no
need to resize (unless one is trying to shrink after having shrunk the
filesystem)?

> 
> You would need to extend the partition first, make the kernel 
> aware of the new size (I gather lvextend does that, personally I 
> stay away from LVM, far too complicated...), call 

LVM certainly adds another layer, but it's really handy for growing
volumes.

> "cryptsetup resize <device>" and then extend the filesystem 
> in the LUKS container. If all of that works, good. If anything 
> goes wrong, I hope you refreshed that backup and have the time 
> to restore from it. 
> 
> Generally, in a situation where you have low downtime needs, 
> you should have a second identical machine with automatic 
> fail-over anyways. There you can take down one machine,
> make the changes offline, test them, bring it back up
> and then fail-over to it. Repeat with the second one.
> 

The whole thing is sitting on top of RAID-1 and so I could take one
part of it offline.  But that raises a bunch of other issues.  When I
have more time, I will eventually want to grow the RAID.

> If you  do not have low downdime needs, do this offline,
> as the risk of doing some real damage is lower.

By offline do you mean shutting down the whole OS and using, e.g.,
Knoppix to fiddle with the disks, or just  umounting the decrypted
device and then doing a luksClose?  Obviously in some cases umounting
the device is only possible if the OS is down.

Ross

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Arno Wagner]

On Wed, Sep 10, 2014 at 10:16:39 CEST, Ross Boylan wrote:
> On Wed, Sep 10, 2014 at 07:16:10AM +0200, Arno Wagner wrote:
> > On Wed, Sep 10, 2014 at 06:30:24 CEST, Ross Boylan wrote:
> > > A little more on resizing on the bottom, with related excerpts above it.
> > [...]
> > > At least one piece of advice on the internet does luksClose, luksOpen 
> > > AND cryptsetup resize:
> > > http://www.xbsd.nl/2012/03/resize-an-encrypted-lvm-logical-volume.html
> > 
> > That one is broken. Or rather the "resize" does nothing to
> > the LUKS container. It may just be there for the --verbose.
> > Really, there is no "partition size" field anywhere in the 
> > LUKS header and it is not needed.
> > 
> > A brief look into the man-page shows what "resize" does:
> > 
> >  resize <name>
> > 
> >     Resizes an active mapping <name>.
> >     If --size (in sectors) is not specified, the size of the under‐
> >     lying  block  device is used. Note that this does not change the
> >     raw device geometry, it just changes how many sectors of the raw
> >     device are represented in the mapped device.
> > 
> > 
> > As to my other comments, I see now that you left out one very 
> > critical piece of information: You want to do this _online_.
> > (Or I read over it. If so, sorry.) That is generally not a 
> > good idea, but that is indeed one of the scenarios where you 
> > would need "cryptsetup resize". (But not after a luksOpen.
> > luksOpen reads the device-size from the kernel anyways.)
> 
> The parenthetical remark sounds as if it's saying not to use
> cryptsetup resize after luksOpen, but I don't think that's the
> intended meaning since without luksOpen (or cryptsetup create if not
> using LUKS) there is nothing to resize.  

Using resize _without_ parameter directly after luksOpen is
meaningless, as it will not change anything. That is what the
referenced example does. As I said, the resize may only be there
for what it says because of --verbose.

> So is it a restatement of the
> fact that if the sequence is lvextend and then luksOpen, there's no
> need to resize (unless one is trying to shrink after having shrunk the
> filesystem)?

There is no need to resize to the default size. If, for any 
reason, you want another size, then there may be, but that 
would be calling resize _with_ size parameter.

> > 
> > You would need to extend the partition first, make the kernel 
> > aware of the new size (I gather lvextend does that, personally I 
> > stay away from LVM, far too complicated...), call 
> 
> LVM certainly adds another layer, but it's really handy for growing
> volumes.

I still have trouble with that. For resizing simple volumes,
parted works just fine. I think it adds significant complexity,
while makeing a rare operation a bit easier. That is not in
line with KISS. But opinions on this issue differ. I am an 
old-school CS person, I want KISS and reliability over
everything else except in special situations. The younger
crowd disagrees, possibly because computers have gotten
so much more reliable and most of them are missing a real
disaster-experience.

> > "cryptsetup resize <device>" and then extend the filesystem 
> > in the LUKS container. If all of that works, good. If anything 
> > goes wrong, I hope you refreshed that backup and have the time 
> > to restore from it. 
> > 
> > Generally, in a situation where you have low downtime needs, 
> > you should have a second identical machine with automatic 
> > fail-over anyways. There you can take down one machine,
> > make the changes offline, test them, bring it back up
> > and then fail-over to it. Repeat with the second one.
> > 
> 
> The whole thing is sitting on top of RAID-1 and so I could take one
> part of it offline.  But that raises a bunch of other issues.  When I
> have more time, I will eventually want to grow the RAID.

Personally, I grow RAID1 by simply kicking a component, resizing 
that by re-creation and then making a degraded volume on that.
Copy data, and repeat with other RAID components. (As I have
important stuff on 3-way RAID1, this is far less rispy than it
may sound. With backup it is also fine.) And Yes, I do RAID
on paritions, after all that is one of the major advantages
of software-RAID.

> > If you  do not have low downdime needs, do this offline,
> > as the risk of doing some real damage is lower.
> 
> By offline do you mean shutting down the whole OS and using, e.g.,
> Knoppix to fiddle with the disks, or just  umounting the decrypted
> device and then doing a luksClose?  

The second. The first option is usually overkill. "Donwntime to the
storage=unit being worked on."

> Obviously in some cases umounting
> the device is only possible if the OS is down.

One reason to have a relatively small root partition and to
not have anything except the system on it. 

But anyways, we seem to be getting somewhere. Please make that
current backup nonetheless and make a header backup of the LUKS 
container in addition, it is far to easy to break the header.

Arno
-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Robert Nichols]

On 09/09/2014 04:52 PM, Ross Boylan wrote:
>   The FAQ advises wipeing it, though the
> only explicit reasons seem not much of a concern for space in a volume
> group (but later there are references to attacks available if the
> attacker can determine which sectors are unused).  As far as I know
> there is no way to access the unused area of the volume group ...

Easy.  Create a new LV in that VG and use "--extents 100%FREE" as
its size.  Fill that LV with whatever variety of random data you
choose, then delete that LV and use the space to expand your active
LV.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

On Tue, Sep 09, 2014 at 08:59:03PM -0500, Robert Nichols wrote:
> On 09/09/2014 04:52 PM, Ross Boylan wrote:
>>   The FAQ advises wipeing it, though the
>> only explicit reasons seem not much of a concern for space in a volume
>> group (but later there are references to attacks available if the
>> attacker can determine which sectors are unused).  As far as I know
>> there is no way to access the unused area of the volume group ...
>
> Easy.  Create a new LV in that VG and use "--extents 100%FREE" as
> its size.  Fill that LV with whatever variety of random data you
> choose, then delete that LV and use the space to expand your active
> LV.

Thanks; I wasn't aware of that syntax.

But do the snapshots make that hazardous?  If the maximum space I
specified for them is pre-allocated it should be fine, but I thought
the implementation grabbed blocks as needed.  If that's the case, a
snapshot could fail while I have grabbed all the "free" space.

I suppose worst case I could do 90%Free and be good enough.

Ross

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Robert Nichols]

On 09/09/2014 10:31 PM, Ross Boylan wrote:
> On Tue, Sep 09, 2014 at 08:59:03PM -0500, Robert Nichols wrote:
>> Easy.  Create a new LV in that VG and use "--extents 100%FREE" as
>> its size.  Fill that LV with whatever variety of random data you
>> choose, then delete that LV and use the space to expand your active
>> LV.
>
> Thanks; I wasn't aware of that syntax.
>
> But do the snapshots make that hazardous?  If the maximum space I
> specified for them is pre-allocated it should be fine, but I thought
> the implementation grabbed blocks as needed.  If that's the case, a
> snapshot could fail while I have grabbed all the "free" space.
>
> I suppose worst case I could do 90%Free and be good enough.

The snapshot LV can increase its size only if you created it as thinly
provisioned, and even then it can grab extents only within the LV that
you set up as a "thin pool" LV.  The space within that thin pool LV is
not "free" for the purposes of creating a new LV.  Creating a new LV
with "--extents 100%FREE" will not affect expansion of your snapshot
LVs.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

On Wed, Sep 10, 2014 at 08:25:07AM -0500, Robert Nichols wrote:
> On 09/09/2014 10:31 PM, Ross Boylan wrote:
>> On Tue, Sep 09, 2014 at 08:59:03PM -0500, Robert Nichols wrote:
>>> Easy.  Create a new LV in that VG and use "--extents 100%FREE" as
>>> its size.  Fill that LV with whatever variety of random data you
>>> choose, then delete that LV and use the space to expand your active
>>> LV.
>>
>> Thanks; I wasn't aware of that syntax.
>>
>> But do the snapshots make that hazardous?  If the maximum space I
>> specified for them is pre-allocated it should be fine, but I thought
>> the implementation grabbed blocks as needed.  If that's the case, a
>> snapshot could fail while I have grabbed all the "free" space.
>>
>> I suppose worst case I could do 90%Free and be good enough.
>
> The snapshot LV can increase its size only if you created it as thinly
> provisioned, and even then it can grab extents only within the LV that
> you set up as a "thin pool" LV.  The space within that thin pool LV is
> not "free" for the purposes of creating a new LV.  Creating a new LV
> with "--extents 100%FREE" will not affect expansion of your snapshot
> LVs.
>

Good; I didn't do thin provisioning.

Game Plan (for IMAP server and its spool):
## Verify that backups are OK

## randomize free space in LVM volume group turtle
lvcreate -l 100%FREE -n tozero turtle
cryptsetup open --type plain -d /dev/urandom /dev/turtle/tozero zero_crypt
dd_rescue -w /dev/zero /dev/mapper/zero_crypt
# free space ~ 100G--likely to take awhile. Hours? days?
# if it's really slow I could allocate 2 LVs, one of which is the size
# I need ~50G, and the other of which is filler.  randomize the 50G;
# free the space, and extend my volume.
cryptsetup remove zero_crypt
lvremove turtle/zero

## make encrypted spot to backup recent files
# /usr/local/backup is not encrypted.  Assume I create a small encrypted volume
# and mount it at /usr/local/backup/crypt

## shutdown server

## backup current server state
# do manual backup of selected server state
# If possible make a backup of the directories the usual way.
# Otherwise, as root
cd /usr/local/backup/crypt
# snapshots at 00:10 daily.  Assume current day has not 
# been backed up yet.
tar cjf cyrspool-recent.tar.bz2 --after-date 'Sep 9 00:09 -0800' /var/spool/cyrus

## take directory offline
umount /var/spool/cyrus
cryptsetup luksClose cyrspool_crypt
# maybe close snapshot of turtle/cyrspool before extending?

## Actually grow things
lvextend -L +20G turtle/cyrspool
cryptsetup --key-file xxx luksOpen  /dev/turtle/cyrspool cyrspool_crypt
resize_reiserfs /dev/mapper/cryspool_crypt
reiserfsck /dev/mapper/cyrspool_crypt
mount /var/spool/cyrus

## restart IMAP server

I took Arno's advice and did things offline.
If anyone sees a problem, I'd love to know.

Thanks.
Ross

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Robert Nichols]

On 09/10/2014 03:36 PM, Ross Boylan wrote:
> On Wed, Sep 10, 2014 at 08:25:07AM -0500, Robert Nichols wrote:
>> The snapshot LV can increase its size only if you created it as thinly
>> provisioned, and even then it can grab extents only within the LV that
>> you set up as a "thin pool" LV.  The space within that thin pool LV is
>> not "free" for the purposes of creating a new LV.  Creating a new LV
>> with "--extents 100%FREE" will not affect expansion of your snapshot
>> LVs.
>>
>
> Good; I didn't do thin provisioning.
>
> Game Plan (for IMAP server and its spool):
> ## Verify that backups are OK
>
> ## randomize free space in LVM volume group turtle
> lvcreate -l 100%FREE -n tozero turtle
> cryptsetup open --type plain -d /dev/urandom /dev/turtle/tozero zero_crypt
> dd_rescue -w /dev/zero /dev/mapper/zero_crypt
> # free space ~ 100G--likely to take awhile. Hours? days?
> # if it's really slow I could allocate 2 LVs, one of which is the size
> # I need ~50G, and the other of which is filler.  randomize the 50G;
> # free the space, and extend my volume.
> cryptsetup remove zero_crypt
> lvremove turtle/zero
>
> ## make encrypted spot to backup recent files
> # /usr/local/backup is not encrypted.  Assume I create a small encrypted volume
> # and mount it at /usr/local/backup/crypt
>
> ## shutdown server
>
> ## backup current server state
> # do manual backup of selected server state
> # If possible make a backup of the directories the usual way.
> # Otherwise, as root
> cd /usr/local/backup/crypt
> # snapshots at 00:10 daily.  Assume current day has not
> # been backed up yet.
> tar cjf cyrspool-recent.tar.bz2 --after-date 'Sep 9 00:09 -0800' /var/spool/cyrus
>
> ## take directory offline
> umount /var/spool/cyrus
> cryptsetup luksClose cyrspool_crypt
> # maybe close snapshot of turtle/cyrspool before extending?
>
> ## Actually grow things
> lvextend -L +20G turtle/cyrspool
> cryptsetup --key-file xxx luksOpen  /dev/turtle/cyrspool cyrspool_crypt
> resize_reiserfs /dev/mapper/cryspool_crypt
> reiserfsck /dev/mapper/cyrspool_crypt
> mount /var/spool/cyrus
>
> ## restart IMAP server

I haven't gone over that with the proverbial fine-tooth comb, but it looks
reasonable.  I would expect the initializing of 100GB to take on the order
of 20 minutes (100GB at ~85MB/sec -- a decent CPU and internal disk should
sustain that easily).

Yes, I would merge the snapshot back into the base volume and close the
snapshot before extending the filesystem.  Otherwise, the enlarged
filesystem structure will exist only in the snapshot.  I suppose an
alternative would be to merge the current snapshot, start a new snapshot,
and the enlarge the filesystem.  If anything went wrong during the resizing,
you could just discard the snapshot and be right back where you were with
the filesystem. Merge that snapshot once you've determined that all went
well.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Robert Nichols]

On 09/10/2014 05:44 PM, Robert Nichols wrote:
> Yes, I would merge the snapshot back into the base volume and close the
> snapshot before extending the filesystem.  Otherwise, the enlarged
> filesystem structure will exist only in the snapshot.  I suppose an
> alternative would be to merge the current snapshot, start a new snapshot,
> and the enlarge the filesystem.  If anything went wrong during the resizing,
> you could just discard the snapshot and be right back where you were with
> the filesystem. Merge that snapshot once you've determined that all went
> well.

ALERT:  I'm thinking of that snapshot backwards, because that's how I most
often use them (create the snapshot, mount the snapshot instead of the base
volume, perform the experiment, then throw the snapshot away and mount the
base volume again to forget the experiment.)

Since I don't know the age or purpose of that snapshot, I don't know whether
you would want to close it.  It would be a snapshot of the pre-expansion
filesystem.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Ross Boylan]

Thanks for looking things over.
Snapshot info below.
On Wed, Sep 10, 2014 at 05:52:13PM -0500, Robert Nichols wrote:
> On 09/10/2014 05:44 PM, Robert Nichols wrote:
>> Yes, I would merge the snapshot back into the base volume and close the
>> snapshot before extending the filesystem.  Otherwise, the enlarged
>> filesystem structure will exist only in the snapshot.  I suppose an
>> alternative would be to merge the current snapshot, start a new snapshot,
>> and the enlarge the filesystem.  If anything went wrong during the resizing,
>> you could just discard the snapshot and be right back where you were with
>> the filesystem. Merge that snapshot once you've determined that all went
>> well.
>
> ALERT:  I'm thinking of that snapshot backwards, because that's how I most
> often use them (create the snapshot, mount the snapshot instead of the base
> volume, perform the experiment, then throw the snapshot away and mount the
> base volume again to forget the experiment.)
>
> Since I don't know the age or purpose of that snapshot, I don't know whether
> you would want to close it.  It would be a snapshot of the pre-expansion
> filesystem.


These are read-only snapshots created for the backup program.  The
backup program backs up the file system as seen through the snapshot..

It does seem as if it would be easy for something to go wrong if the
snapshot were active while the underlying volume grew.  At a minimum,
growing the file system would presumably use up a lot of space in the
snapshot.

Ross

Re: [dm-crypt] expanding encrypted volume/growing the volume

[Robert Nichols]

On 09/10/2014 06:47 PM, Ross Boylan wrote:
> These are read-only snapshots created for the backup program.  The
> backup program backs up the file system as seen through the snapshot..
>
> It does seem as if it would be easy for something to go wrong if the
> snapshot were active while the underlying volume grew.  At a minimum,
> growing the file system would presumably use up a lot of space in the
> snapshot.

No reason to keep it, then.  It's essentially an online copy of whatever
last went into the backup.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.