[dm-crypt] crypetsetup and GPT partitions

[dm-crypt] crypetsetup and GPT partitions

[Houtchen, Steven]

[-- Attachment #1: Type: text/plain, Size: 2481 bytes --]

Hello,

I am trying to use "crypsetup" setup ant "parted" together.
I want to use "cryptsetup" to encrypt a whole solid state disk,
and then use "parted" to create partitions on it with a GPT partition
table.  I have be able to do the first task, but not the second.

Or vice versa. Create a few partitions, and then optionally
encrypt each one individually. I have be able to do the first
task, but not the second.

So my question is, is "cryptsetup" compatible with parted and
GPT partition table? Or do  need to use something like "lvm2"
to accomplish what I am trying to do?

I am using CentOS7 with

[root@dts1 ~]# cryptsetup --version
cryptsetup 1.6.7
[root@dts1 ~]#

[root@dts1 ~]# parted --version
parted (GNU parted) 3.1
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
[root@dts1 ~]#


Thanks for any help you can give me..

Steve Houtchen
Senior Software Engineer

Curtiss-Wright
2600 Paramount Place, Suite 200, Fairborn, OH 45324
T: 937.610.5420 | F: 937.252.1465
shoutchen@curtisswright.com<mailto:shoutchen@curtisswright.com> | www.curtisswrightds.com<http://www.curtisswrightds.com/>


_______________________________________________________________________
This e-mail and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed.  If you have reason to believe that you have received this e-mail in error, please notify the sender and destroy this e-mail and any attached files.  Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the Curtiss-Wright Corporation or any of its subsidiaries.  Documents attached hereto may contain technology subject to government export regulations.  Recipient is solely responsible for ensuring that any re-export, transfer or disclosure of this information is in accordance with applicable government export regulations.  The recipient should check this e-mail and any attachments for the presence of viruses.  Curtiss-Wright Corporation and its subsidiaries accept no liability for any damage caused by any virus transmitted by this e-mail.

[-- Attachment #2: Type: text/html, Size: 5869 bytes --]

Re: [dm-crypt] crypetsetup and GPT partitions

[David Christensen]

On 02/08/17 04:33, Houtchen, Steven wrote:
> I am trying to use "crypsetup" setup ant "parted" together.
> I want to use "cryptsetup" to encrypt a whole solid state disk,
> and then use "parted" to create partitions on it with a GPT partition
> table.  I have be able to do the first task, but not the second.

I've never done that.


> Or vice versa. Create a few partitions, and then optionally
> encrypt each one individually. I have be able to do the first
> task, but not the second.

That's how I've done it.


> So my question is, is "cryptsetup" compatible with parted and
> GPT partition table? Or do  need to use something like "lvm2"
> to accomplish what I am trying to do?

I would suggest:

1.  Use the manufacturer tool to do a secure erase of the SSD (this 
could involve using a Microsoft Windows machine).

2.  Use parted to create a MBR partition table.

3.  Use parted to create one primary partition.  Consider 
under-provisioning.

4.  Use cryptsetup luksFormat to put a LUKS container into the partition.

5.  Use cryptsetup luksOpen to open the LUKS container.  Add entry to 
/etc/crypttab (Debian).

6.  Either create a filesystem on the mapped device and add entry to 
/etc/fstab (Debian), or feed the mapped device to LVM (it's been a while 
for me; you'll have to figure that out).


> I am using CentOS7 with
>
> [root@dts1 ~]# cryptsetup --version
> cryptsetup 1.6.7
> [root@dts1 ~]#
>
> [root@dts1 ~]# parted --version
> parted (GNU parted) 3.1
> Copyright (C) 2012 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
>
> Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
> [root@dts1 ~]#
>
>

On 02/08/17 05:44, Houtchen, Steven wrote:
> I did get the scenario to seemingly work where I encrypt the whole block device, and use
> parted to creates a partition on the device mapper device. I attached a file showing the
> command sequence..
>
> My questions here are:
>
> Is this a valid use case,

I dont' know.  If I did put a LUKS container on the raw disk and then 
partitioned the mapped deviced, I don't know how I would specify such in 
/etc/crypttab and /etc/fstab.  I'd have to hack up scripts to set it up 
on boot and tear it down on shutdown.


> and also, can I set the starting block on my partition
> On the device mapper device to be at 1 MB, or would that conflict
> with any of the Luks header info on the actual drive?

The available space of the LUKS mapped device is going to be smaller 
than the partition size.  On one of my 3 TB drives, it's about ~44 GB 
smaller (~1.6%).  The LUKS meta-data is going to be in there, including 
the header.


You should be able use all of the blocks in the mapped device however 
you please; if LUKS breaks, then your LUKS is broken.


David

Re: [dm-crypt] crypetsetup and GPT partitions

[Michael Kjörling]

On 10 Feb 2017 00:15 -0800, from dpchrist@holgerdanske.com (David Christensen):
> The available space of the LUKS mapped device is going to be smaller
> than the partition size.  On one of my 3 TB drives, it's about ~44
> GB smaller (~1.6%).  The LUKS meta-data is going to be in there,
> including the header.

That doesn't make sense. The LUKS header is a shade over 1 MiB,
depending on the specific options (the FAQ has details). The size of
the header isn't related to the size of the container. Something else
is going on in your case.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] crypetsetup and GPT partitions

[Michael Kjörling]

On 8 Feb 2017 12:33 +0000, from shoutchen@curtisswright.com (Houtchen, Steven):
> So my question is, is "cryptsetup" compatible with parted and
> GPT partition table? Or do  need to use something like "lvm2"
> to accomplish what I am trying to do?

LUKS doesn't care what the backing device is. You can have an
encrypted container backed by a file, or by a block device (which in
turn can be either a whole disk, a partition, or a loopback device
exposing a part of a different block device). All it cares about is
that it's big enough to hold the LUKS header, and that writes are
persisted in some manner.

As far as LUKS is concerned, the container doesn't even need to be
large enough to hold any file system.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] crypetsetup and GPT partitions

[Sven Eschenberg]

Hi,

Am 08.02.2017 um 13:33 schrieb Houtchen, Steven:
> Hello,
>
>
>
> I am trying to use “crypsetup” setup ant “parted” together.
>
> I want to use “cryptsetup” to encrypt a whole solid state disk,
>
> and then use “parted” to create partitions on it with a GPT partition
>
> table.  I have be able to do the first task, but not the second.

Usually partitions are handled directly in the kernel which creates the 
inodes (well depending on the system there's udev in the equation 
aswell). Anyhow the kernel only looks at physical devices for that (not 
pseudoblockdevices like device-mapper targets etc.)

That being said:
This mode of operation is possible, but the kernel won't create 
blockdevice inodes for the partitions after the dm-crypt blockdevice 
comes up. It is possible to used specific tools like kpartx IIRC, to get 
additional dm targets for each of the partitions. Be reminded however, 
that gpt does have a secondary partition table in the end of the device, 
which will give you extra fun, when you scale a container holding you 
partitions.

If you want to save time and trouble you don't want to go this route.
>
>
>
> Or vice versa. Create a few partitions, and then optionally
>
> encrypt each one individually. I have be able to do the first
>
> task, but not the second.

This is the typical mode of operation as this removes some critical 
stuff from the equation. Usually you will create your GPT partitions and 
then create the cryptomapping and then the filesystems. You are free 
however to throw in any additional block layer as you wish.

>
>
>
> So my question is, is “cryptsetup” compatible with parted and
>
> GPT partition table? Or do  need to use something like “lvm2”
>
> to accomplish what I am trying to do?

Yes they are compatible, you can use lvm2 if you wish, but there's no 
need if you don't use large scale storage with dynamic needs.
>
>
>
> I am using CentOS7 with
>
>
>
> [root@dts1 ~]# cryptsetup --version
>
> cryptsetup 1.6.7
>
> [root@dts1 ~]#
>
>
>
> [root@dts1 ~]# parted --version
>
> parted (GNU parted) 3.1
>
> Copyright (C) 2012 Free Software Foundation, Inc.
>
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>.
>
> This is free software: you are free to change and redistribute it.
>
> There is NO WARRANTY, to the extent permitted by law.
>
>
>
> Written by
> <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
>
> [root@dts1 ~]#
>
>

Both versions are a little outdated but should work as expected.

>
>
>
> Thanks for any help you can give me..
>
>
>
> *Steve Houtchen
> *Senior Software Engineer
>
> *Curtiss-Wright
> *2600 Paramount Place, Suite 200, Fairborn, OH 45324
> T: 937.610.5420 | F: 937.252.1465
> shoutchen@curtisswright.com <mailto:shoutchen@curtisswright.com> |
> www.curtisswrightds.com <http://www.curtisswrightds.com/>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Regards

-Sven

Re: [dm-crypt] crypetsetup and GPT partitions

[Houtchen, Steven]

Thanks for the replies, but I am still a little unsure here...
If I create the LUKS container on a whole block device, I understand it
puts the LUKS header the beginning of the block device.  I am using GPT
partition tables. I read that parted puts the partition at the beginning of the
block device also 

I tried that scenario, and it seem sto work,,, but here are my questions here.

Once I use luksFormat and luksOpen to "create and open the container", I suppose I need to 
run "parted" on the container , which the  dev/mapper device .  Is that the correct way
to do that?  If I do that , why doesn't the GPT partition table collide with the LUKs header info
on the block device? Does the /dev/mapper layer prevent that? Or parted?

SteveH



-----Original Message-----
From: dm-crypt [mailto:dm-crypt-bounces@saout.de] On Behalf Of Sven Eschenberg
Sent: Friday, February 10, 2017 7:35 AM
To: dm-crypt@saout.de
Subject: Re: [dm-crypt] crypetsetup and GPT partitions

Hi,

Am 08.02.2017 um 13:33 schrieb Houtchen, Steven:
> Hello,
>
>
>
> I am trying to use "crypsetup" setup ant "parted" together.
>
> I want to use "cryptsetup" to encrypt a whole solid state disk,
>
> and then use "parted" to create partitions on it with a GPT partition
>
> table.  I have be able to do the first task, but not the second.

Usually partitions are handled directly in the kernel which creates the inodes (well depending on the system there's udev in the equation aswell). Anyhow the kernel only looks at physical devices for that (not pseudoblockdevices like device-mapper targets etc.)

That being said:
This mode of operation is possible, but the kernel won't create blockdevice inodes for the partitions after the dm-crypt blockdevice comes up. It is possible to used specific tools like kpartx IIRC, to get additional dm targets for each of the partitions. Be reminded however, that gpt does have a secondary partition table in the end of the device, which will give you extra fun, when you scale a container holding you partitions.

If you want to save time and trouble you don't want to go this route.
>
>
>
> Or vice versa. Create a few partitions, and then optionally
>
> encrypt each one individually. I have be able to do the first
>
> task, but not the second.

This is the typical mode of operation as this removes some critical stuff from the equation. Usually you will create your GPT partitions and then create the cryptomapping and then the filesystems. You are free however to throw in any additional block layer as you wish.

>
>
>
> So my question is, is "cryptsetup" compatible with parted and
>
> GPT partition table? Or do  need to use something like "lvm2"
>
> to accomplish what I am trying to do?

Yes they are compatible, you can use lvm2 if you wish, but there's no need if you don't use large scale storage with dynamic needs.
>
>
>
> I am using CentOS7 with
>
>
>
> [root@dts1 ~]# cryptsetup --version
>
> cryptsetup 1.6.7
>
> [root@dts1 ~]#
>
>
>
> [root@dts1 ~]# parted --version
>
> parted (GNU parted) 3.1
>
> Copyright (C) 2012 Free Software Foundation, Inc.
>
> License GPLv3+: GNU GPL version 3 or later 
> <http://gnu.org/licenses/gpl.html>.
>
> This is free software: you are free to change and redistribute it.
>
> There is NO WARRANTY, to the extent permitted by law.
>
>
>
> Written by
> <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
>
> [root@dts1 ~]#
>
>

Both versions are a little outdated but should work as expected.

>
>
>
> Thanks for any help you can give me..
>
>
>
> *Steve Houtchen
> *Senior Software Engineer
>
> *Curtiss-Wright
> *2600 Paramount Place, Suite 200, Fairborn, OH 45324
> T: 937.610.5420 | F: 937.252.1465
> shoutchen@curtisswright.com <mailto:shoutchen@curtisswright.com> | 
> www.curtisswrightds.com <http://www.curtisswrightds.com/>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Regards

-Sven
_______________________________________________
dm-crypt mailing list
dm-crypt@saout.de
http://www.saout.de/mailman/listinfo/dm-crypt

_______________________________________________________________________
This e-mail and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed.  If you have reason to believe that you have received this e-mail in error, please notify the sender and destroy this e-mail and any attached files.  Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the Curtiss-Wright Corporation or any of its subsidiaries.  Documents attached hereto may contain technology subject to government export regulations.  Recipient is solely responsible for ensuring that any re-export, transfer or disclosure of this information is in accordance with applicable government export regulations.  The recipient should check this e-mail and any attachments for the presence of viruses.  Curtiss-Wright Corporation and its subsidiaries accept no liability for any damage caused by any virus transmitted by this e-mail.

Re: [dm-crypt] crypetsetup and GPT partitions

[Sven Eschenberg]

<----- disk ----->

cryptsetup without any GPT or Partition likewise:

<-- cryptsetup header -- data area  --->

data area will be exported as /dev/mapper/yourname and /dev/dm-???

<--- data area --->

using parted to create a GPT on the data area (cleartext mapped device)

<--- primary GPT area ---- cleartext data ---- secondary GPT area --->

looking at the whole disk you'll have:

<--- cryptsetup header (cryptbegin)<--- primary GPT --- 'cleartext data' 
--- secondary GPT>(cryptend)>

1) As noted the kernel does not handle the GPT inside the crypto area, 
as the blockdevice is not physical (See dmesg, the kernel will read the 
GPTs on physical disks and tell you so)
2) GPTs always come with a shadow copy at the end of device with inverse 
offsetting, no matter if you use fdisk, gptfdisk or parted.

Since partitions ontop of pseudo-blockdevices are not kernel handled and 
are harder to manage, this is not recommended in general. There are very 
specific exceptions where this makes sense, esp. when you do not use 
partitions on pseudoblockdevices but volumemanagers or the like. Still, 
harder to manage, esp. when something goes utterly wrong. No fun if you 
don't know all the tools components and physical stuff very well.

Recommended and normale mode of operations will be:

<--- primary GPT --- part1<luks header --- fs> --- part2<luks header 
---- fs> --- <...> --- partN<luks header --- fs> secondary GPT --->

Advantage, once the kernel sees the physical disk, it will automagically 
create device inodes (/dev/[sh]dN). You'd then luksOpen all the N 
partitions and then mount all the filesystems. I assume CentOS uses 
crypttab, that's where the info about your crypto mappings go. Only 
disadvantage is:
You'll have to enter one passphrase per partition. If you don't like 
this, check your distribution, some of them have startup code that i.e. 
let's you use a single passphrase for a keyring which holds all the 
passwords.

As an alternative you can create a single partition create a crypto 
mapping and use lvm2 ontop of that to slice. This is however harder to 
manage, esp if you don't have the experience and using lvm2 to just 
slice a jist is a little bit like using a jackhammer on a needle.

This would roughly look like this:

<--- primary GPT --- crypto header --- <LVM header area --- LV1 <fs> --- 
LV2 <fs> ... > --- secondary GPT --->

This is quite similiar to using partition on top of the crypto 
container. Except that the tools to read a partiton table on a 
pseudoblockdevie which create the necessary mapping are usually not part 
of the core system, whereas LVM2 usually comes as part of the core 
system. The harder managability and recovery when problems arise are 
however important to consider.


Regards

-Sven

P.S.: Sorry for the limited ASCII graphics.


Am 10.02.2017 um 13:58 schrieb Houtchen, Steven:
> Thanks for the replies, but I am still a little unsure here...
> If I create the LUKS container on a whole block device, I understand it
> puts the LUKS header the beginning of the block device.  I am using GPT
> partition tables. I read that parted puts the partition at the beginning of the
> block device also
>
> I tried that scenario, and it seem sto work,,, but here are my questions here.
>
> Once I use luksFormat and luksOpen to "create and open the container", I suppose I need to
> run "parted" on the container , which the  dev/mapper device .  Is that the correct way
> to do that?  If I do that , why doesn't the GPT partition table collide with the LUKs header info
> on the block device? Does the /dev/mapper layer prevent that? Or parted?
>
> SteveH
>
>
>
> -----Original Message-----
> From: dm-crypt [mailto:dm-crypt-bounces@saout.de] On Behalf Of Sven Eschenberg
> Sent: Friday, February 10, 2017 7:35 AM
> To: dm-crypt@saout.de
> Subject: Re: [dm-crypt] crypetsetup and GPT partitions
>
> Hi,
>
> Am 08.02.2017 um 13:33 schrieb Houtchen, Steven:
>> Hello,
>>
>>
>>
>> I am trying to use "crypsetup" setup ant "parted" together.
>>
>> I want to use "cryptsetup" to encrypt a whole solid state disk,
>>
>> and then use "parted" to create partitions on it with a GPT partition
>>
>> table.  I have be able to do the first task, but not the second.
>
> Usually partitions are handled directly in the kernel which creates the inodes (well depending on the system there's udev in the equation aswell). Anyhow the kernel only looks at physical devices for that (not pseudoblockdevices like device-mapper targets etc.)
>
> That being said:
> This mode of operation is possible, but the kernel won't create blockdevice inodes for the partitions after the dm-crypt blockdevice comes up. It is possible to used specific tools like kpartx IIRC, to get additional dm targets for each of the partitions. Be reminded however, that gpt does have a secondary partition table in the end of the device, which will give you extra fun, when you scale a container holding you partitions.
>
> If you want to save time and trouble you don't want to go this route.
>>
>>
>>
>> Or vice versa. Create a few partitions, and then optionally
>>
>> encrypt each one individually. I have be able to do the first
>>
>> task, but not the second.
>
> This is the typical mode of operation as this removes some critical stuff from the equation. Usually you will create your GPT partitions and then create the cryptomapping and then the filesystems. You are free however to throw in any additional block layer as you wish.
>
>>
>>
>>
>> So my question is, is "cryptsetup" compatible with parted and
>>
>> GPT partition table? Or do  need to use something like "lvm2"
>>
>> to accomplish what I am trying to do?
>
> Yes they are compatible, you can use lvm2 if you wish, but there's no need if you don't use large scale storage with dynamic needs.
>>
>>
>>
>> I am using CentOS7 with
>>
>>
>>
>> [root@dts1 ~]# cryptsetup --version
>>
>> cryptsetup 1.6.7
>>
>> [root@dts1 ~]#
>>
>>
>>
>> [root@dts1 ~]# parted --version
>>
>> parted (GNU parted) 3.1
>>
>> Copyright (C) 2012 Free Software Foundation, Inc.
>>
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>.
>>
>> This is free software: you are free to change and redistribute it.
>>
>> There is NO WARRANTY, to the extent permitted by law.
>>
>>
>>
>> Written by
>> <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
>>
>> [root@dts1 ~]#
>>
>>
>
> Both versions are a little outdated but should work as expected.
>
>>
>>
>>
>> Thanks for any help you can give me..
>>
>>
>>
>> *Steve Houtchen
>> *Senior Software Engineer
>>
>> *Curtiss-Wright
>> *2600 Paramount Place, Suite 200, Fairborn, OH 45324
>> T: 937.610.5420 | F: 937.252.1465
>> shoutchen@curtisswright.com <mailto:shoutchen@curtisswright.com> |
>> www.curtisswrightds.com <http://www.curtisswrightds.com/>
>>
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>
>
> Regards
>
> -Sven
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
> _______________________________________________________________________
> This e-mail and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed.  If you have reason to believe that you have received this e-mail in error, please notify the sender and destroy this e-mail and any attached files.  Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the Curtiss-Wright Corporation or any of its subsidiaries.  Documents attached hereto may contain technology subject to government export regulations.  Recipient is solely responsible for ensuring that any re-export, transfer or disclosure of this information is in accordance with applicable government export regulations.  The recipient should check this e-mail and any attachments for the presence of viruses.  Curtiss-Wright Corporation and its subsidiaries accept no liability for any damage caused by any virus transmitted by this e-mail.
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Re: [dm-crypt] crypetsetup and GPT partitions

[David Christensen]

On 02/10/17 01:07, Michael Kjörling wrote:
> On 10 Feb 2017 00:15 -0800, from dpchrist@holgerdanske.com (David Christensen):
>> The available space of the LUKS mapped device is going to be smaller
>> than the partition size.  On one of my 3 TB drives, it's about ~44
>> GB smaller (~1.6%).  The LUKS meta-data is going to be in there,
>> including the header.
>
> That doesn't make sense. The LUKS header is a shade over 1 MiB,
> depending on the specific options (the FAQ has details). The size of
> the header isn't related to the size of the container. Something else
> is going on in your case.

Here's the data:

2017-02-10 07:38:40 root@cd2533 ~
# parted /dev/sda u s p
Model: ATA ST3000DM001-1ER1 (scsi)
Disk /dev/sda: 5860533168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start  End          Size         File system  Name     Flags
  1      2048s  5860532223s  5860530176s               primary

2017-02-10 07:40:57 root@cd2533 ~
# df | egrep 'File|mnt'
Filesystem                1K-blocks      Used  Available Use% Mounted on
/dev/mapper/i3000d_crypt 2884281560 848596104 1889172304  31% /mnt/i3000d


Here's the math:

    5860530176 s / (2 s/kB) - 2884281560 kB
  = 2930265088 kB           - 2884281560 kB
  = 45983528 kB
~= 44905.8 MB


David

Re: [dm-crypt] crypetsetup and GPT partitions

[Sven Eschenberg]

That's the size of the filesystem afterall.

Lookt at blockdev --report to see the blockdev sizes (i.e. physical 
disk, partition, crypt device).
Check against dmsetup --table

Those sectors certainly did not just disappear.


Regards

-Sven



Am 10.02.2017 um 17:05 schrieb David Christensen:
> On 02/10/17 01:07, Michael Kjörling wrote:
>> On 10 Feb 2017 00:15 -0800, from dpchrist@holgerdanske.com (David
>> Christensen):
>>> The available space of the LUKS mapped device is going to be smaller
>>> than the partition size.  On one of my 3 TB drives, it's about ~44
>>> GB smaller (~1.6%).  The LUKS meta-data is going to be in there,
>>> including the header.
>>
>> That doesn't make sense. The LUKS header is a shade over 1 MiB,
>> depending on the specific options (the FAQ has details). The size of
>> the header isn't related to the size of the container. Something else
>> is going on in your case.
>
> Here's the data:
>
> 2017-02-10 07:38:40 root@cd2533 ~
> # parted /dev/sda u s p
> Model: ATA ST3000DM001-1ER1 (scsi)
> Disk /dev/sda: 5860533168s
> Sector size (logical/physical): 512B/4096B
> Partition Table: gpt
>
> Number  Start  End          Size         File system  Name     Flags
>  1      2048s  5860532223s  5860530176s               primary
>
> 2017-02-10 07:40:57 root@cd2533 ~
> # df | egrep 'File|mnt'
> Filesystem                1K-blocks      Used  Available Use% Mounted on
> /dev/mapper/i3000d_crypt 2884281560 848596104 1889172304  31% /mnt/i3000d
>
>
> Here's the math:
>
>    5860530176 s / (2 s/kB) - 2884281560 kB
>  = 2930265088 kB           - 2884281560 kB
>  = 45983528 kB
> ~= 44905.8 MB
>
>
> David
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Re: [dm-crypt] crypetsetup and GPT partitions

[David Christensen]

On 02/10/17 08:22, Sven Eschenberg wrote:
> Am 10.02.2017 um 17:05 schrieb David Christensen:
>> On 02/10/17 01:07, Michael Kjörling wrote:
>>> On 10 Feb 2017 00:15 -0800, from dpchrist@holgerdanske.com (David
>>> Christensen):
>>>> The available space of the LUKS mapped device is going to be smaller
>>>> than the partition size.  On one of my 3 TB drives, it's about ~44
>>>> GB smaller (~1.6%).  The LUKS meta-data is going to be in there,
>>>> including the header.
>>>
>>> That doesn't make sense. The LUKS header is a shade over 1 MiB,
>>> depending on the specific options (the FAQ has details). The size of
>>> the header isn't related to the size of the container. Something else
>>> is going on in your case.
>>
>> Here's the data:
>>
>> 2017-02-10 07:38:40 root@cd2533 ~
>> # parted /dev/sda u s p
>> Model: ATA ST3000DM001-1ER1 (scsi)
>> Disk /dev/sda: 5860533168s
>> Sector size (logical/physical): 512B/4096B
>> Partition Table: gpt
>>
>> Number  Start  End          Size         File system  Name     Flags
>>  1      2048s  5860532223s  5860530176s               primary
>>
>> 2017-02-10 07:40:57 root@cd2533 ~
>> # df | egrep 'File|mnt'
>> Filesystem                1K-blocks      Used  Available Use% Mounted on
>> /dev/mapper/i3000d_crypt 2884281560 848596104 1889172304  31% /mnt/i3000d
>>
>>
>> Here's the math:
>>
>>    5860530176 s / (2 s/kB) - 2884281560 kB
>>  = 2930265088 kB           - 2884281560 kB
>>  = 45983528 kB
>> ~= 44905.8 MB
> That's the size of the filesystem afterall.

D'oh!


> Lookt at blockdev --report to see the blockdev sizes (i.e. physical
> disk, partition, crypt device).


RTFM blockdev(8):

2017-02-10 10:01:08 root@cd2533 ~
# blockdev --report /dev/sda /dev/sda1 /dev/mapper/i3000d_crypt
RO    RA   SSZ   BSZ   StartSec            Size   Device
rw   256   512  4096          0   3000592982016   /dev/sda
rw   256   512  4096       2048   3000591450112   /dev/sda1
rw   256   512  4096          0   3000589352960   /dev/mapper/i3000d_crypt


So, it looks like LUKS consumes:

3000591450112 - 3000589352960 = 2097152 = 2 MB


Thanks for the correction.  :-)


> Check against dmsetup

RTFM dmsetup(8):

2017-02-10 10:14:06 root@cd2533 ~
# dmsetup info /dev/mapper/i3000d_crypt
Name:              i3000d_crypt
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      1
Major, minor:      254, 2
Number of targets: 1
UUID: CRYPT-LUKS1-<redacted>-i3000d_crypt_unformatted


> --table

RTFM dmsetup(8), the --table option requires a <table> value.  WAG:

2017-02-10 10:14:17 root@cd2533 ~
# dmsetup info --table LIVE /dev/mapper/i3000d_crypt
Name:              i3000d_crypt
State:             ACTIVE
Read Ahead:        256
Tables present:    LIVE
Open count:        1
Event number:      1
Major, minor:      254, 2
Number of targets: 1
UUID: CRYPT-LUKS1-<redacted>-i3000d_crypt_unformatted


I'm not sure what I am checking...


David

Re: [dm-crypt] crypetsetup and GPT partitions

[Sven Eschenberg]

Am 10.02.2017 um 19:23 schrieb David Christensen:
> On 02/10/17 08:22, Sven Eschenberg wrote:
>> Am 10.02.2017 um 17:05 schrieb David Christensen:
>>> On 02/10/17 01:07, Michael Kjörling wrote:
>>>> On 10 Feb 2017 00:15 -0800, from dpchrist@holgerdanske.com (David
>>>> Christensen):
>>>>> The available space of the LUKS mapped device is going to be smaller
>>>>> than the partition size.  On one of my 3 TB drives, it's about ~44
>>>>> GB smaller (~1.6%).  The LUKS meta-data is going to be in there,
>>>>> including the header.
>>>>
>>>> That doesn't make sense. The LUKS header is a shade over 1 MiB,
>>>> depending on the specific options (the FAQ has details). The size of
>>>> the header isn't related to the size of the container. Something else
>>>> is going on in your case.
>>>
>>> Here's the data:
>>>
>>> 2017-02-10 07:38:40 root@cd2533 ~
>>> # parted /dev/sda u s p
>>> Model: ATA ST3000DM001-1ER1 (scsi)
>>> Disk /dev/sda: 5860533168s
>>> Sector size (logical/physical): 512B/4096B
>>> Partition Table: gpt
>>>
>>> Number  Start  End          Size         File system  Name     Flags
>>>  1      2048s  5860532223s  5860530176s               primary
>>>
>>> 2017-02-10 07:40:57 root@cd2533 ~
>>> # df | egrep 'File|mnt'
>>> Filesystem                1K-blocks      Used  Available Use% Mounted on
>>> /dev/mapper/i3000d_crypt 2884281560 848596104 1889172304  31%
>>> /mnt/i3000d
>>>
>>>
>>> Here's the math:
>>>
>>>    5860530176 s / (2 s/kB) - 2884281560 kB
>>>  = 2930265088 kB           - 2884281560 kB
>>>  = 45983528 kB
>>> ~= 44905.8 MB
>> That's the size of the filesystem afterall.
>
> D'oh!
>
>
>> Lookt at blockdev --report to see the blockdev sizes (i.e. physical
>> disk, partition, crypt device).
>
>
> RTFM blockdev(8):
>
> 2017-02-10 10:01:08 root@cd2533 ~
> # blockdev --report /dev/sda /dev/sda1 /dev/mapper/i3000d_crypt
> RO    RA   SSZ   BSZ   StartSec            Size   Device
> rw   256   512  4096          0   3000592982016   /dev/sda
> rw   256   512  4096       2048   3000591450112   /dev/sda1
> rw   256   512  4096          0   3000589352960   /dev/mapper/i3000d_crypt
>
>
> So, it looks like LUKS consumes:
>
> 3000591450112 - 3000589352960 = 2097152 = 2 MB
>
>
> Thanks for the correction.  :-)
>
>
>> Check against dmsetup
>
> RTFM dmsetup(8):
>
> 2017-02-10 10:14:06 root@cd2533 ~
> # dmsetup info /dev/mapper/i3000d_crypt
> Name:              i3000d_crypt
> State:             ACTIVE
> Read Ahead:        256
> Tables present:    LIVE
> Open count:        1
> Event number:      1
> Major, minor:      254, 2
> Number of targets: 1
> UUID: CRYPT-LUKS1-<redacted>-i3000d_crypt_unformatted
>
>
>> --table
>
> RTFM dmsetup(8), the --table option requires a <table> value.  WAG:
>
> 2017-02-10 10:14:17 root@cd2533 ~
> # dmsetup info --table LIVE /dev/mapper/i3000d_crypt
> Name:              i3000d_crypt
> State:             ACTIVE
> Read Ahead:        256
> Tables present:    LIVE
> Open count:        1
> Event number:      1
> Major, minor:      254, 2
> Number of targets: 1
> UUID: CRYPT-LUKS1-<redacted>-i3000d_crypt_unformatted
>
>
> I'm not sure what I am checking...

My bad, it is just dmsetup table - this dumps all (active) devicemapper 
targets with all essential data ... you can basically take a look at 
where in the drive a mapping starts, how long it is etc. . The output 
includes the key for crypt targets and whatever additional data each 
type of target offers. It sometimes helps to understand the exact layout 
better (You'll need to know/understand those numbers however). Just in 
case you think you hit some inconsistency or the like. No need however 
to post it here.
>
>
> David
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

As you know by now, the size of the crypto target (the device size 
available to the filesystem) is just 2MBytes less than the partition 
size. So maybe check again on the filesystem size (use the corresponding 
filesystem tools if necessary). If you did not increase the device size 
at any time after fs creation, then somehow your mkfs did something 
wrong. Depending on the fs, if you have a backup of the data, you could 
try growing the fs to the full size of the container.

Regards

-Sven

P.S.: Usually df reports the available space to files AFAIK, this should 
be roughly devicesize-metadatasize, where metadata is internal 
filesystem structures but not filenames and the like (inode space). But 
I cannot tell for 100%. I.E.: for a 16GB filesystem I have a difference 
of roughly 10 MBytes. (Depends certainly on the filesystem used ...)

Re: [dm-crypt] crypetsetup and GPT partitions

[Houtchen, Steven]

[-- Attachment #1.1: Type: text/plain, Size: 3454 bytes --]

dm-crypt,

I did get the scenario to seemingly work where I encrypt the whole block device, and use
parted to creates a partition on the device mapper device. I attached a file showing the
command sequence..

My questions here are:

Is this a valid use case, and also, can I set the starting block on my partition
On the device mapper device to be at 1 MB, or would that conflict
with any of the Luks header info on the actual drive?


Steve Houtchen
Senior Software Engineer

Curtiss-Wright
2600 Paramount Place, Suite 200, Fairborn, OH 45324
T: 937.610.5420 | F: 937.252.1465
shoutchen@curtisswright.com<mailto:shoutchen@curtisswright.com> | www.curtisswrightds.com<http://www.curtisswrightds.com/>



From: Houtchen, Steven
Sent: Wednesday, February 8, 2017 7:34 AM
To: 'dm-crypt@saout.de' <dm-crypt@saout.de>
Cc: Langley, Rich <rlangley@curtisswright.com>; Ramos, Gerardo <gramos@curtisswright.com>
Subject: crypetsetup and GPT partitions

Hello,

I am trying to use "crypsetup" setup ant "parted" together.
I want to use "cryptsetup" to encrypt a whole solid state disk,
and then use "parted" to create partitions on it with a GPT partition
table.  I have be able to do the first task, but not the second.

Or vice versa. Create a few partitions, and then optionally
encrypt each one individually. I have be able to do the first
task, but not the second.

So my question is, is "cryptsetup" compatible with parted and
GPT partition table? Or do  need to use something like "lvm2"
to accomplish what I am trying to do?

I am using CentOS7 with

[root@dts1 ~]# cryptsetup --version
cryptsetup 1.6.7
[root@dts1 ~]#

[root@dts1 ~]# parted --version
parted (GNU parted) 3.1
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.
[root@dts1 ~]#


Thanks for any help you can give me..

Steve Houtchen
Senior Software Engineer

Curtiss-Wright
2600 Paramount Place, Suite 200, Fairborn, OH 45324
T: 937.610.5420 | F: 937.252.1465
shoutchen@curtisswright.com<mailto:shoutchen@curtisswright.com> | www.curtisswrightds.com<http://www.curtisswrightds.com/>


_______________________________________________________________________
This e-mail and any files transmitted with it are proprietary and intended solely for the use of the individual or entity to whom they are addressed.  If you have reason to believe that you have received this e-mail in error, please notify the sender and destroy this e-mail and any attached files.  Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the Curtiss-Wright Corporation or any of its subsidiaries.  Documents attached hereto may contain technology subject to government export regulations.  Recipient is solely responsible for ensuring that any re-export, transfer or disclosure of this information is in accordance with applicable government export regulations.  The recipient should check this e-mail and any attachments for the presence of viruses.  Curtiss-Wright Corporation and its subsidiaries accept no liability for any damage caused by any virus transmitted by this e-mail.

[-- Attachment #1.2: Type: text/html, Size: 9267 bytes --]

[-- Attachment #2: PARTEDCRYPT.TXT --]
[-- Type: text/plain, Size: 6122 bytes --]

[root@dts1 ~]# wipefs -a /dev/rmc0 --force
/dev/rmc0: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/rmc0: 8 bytes were erased at offset 0x7470c05e00 (gpt): 45 46 49 20 50 41 52 54
/dev/rmc0: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
/dev/rmc0: calling ioclt to re-read partition table: Success

***

[root@dts1 ~]# dd if=/dev/zero of=/dev/rmc0  bs=512 count=1024
1024+0 records in
1024+0 records out
524288 bytes (524 kB) copied, 0.0241128 s, 21.7 MB/s

***

[root@dts1 ~]#  cryptsetup luksFormat  /dev/rmc0

WARNING!
========
This will overwrite data on /dev/rmc0 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter passphrase:
Verify passphrase:

[root@dts1 ~]#  cryptsetup luksOpen  /dev/rmc0
Command requires device and mapped name as arguments.
[root@dts1 ~]#  cryptsetup luksOpen  /dev/rmc0 devrmc0
Enter passphrase for /dev/rmc0:
[root@dts1 ~]#

***

[root@dts1 ~]# lsblk
NAME      MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda         8:0    0   7.6G  0 disk
+-sda1      8:1    0   200M  0 part  /boot
+-sda2      8:2    0   3.4G  0 part  /
+-sda3      8:3    0   200M  0 part  /home
sdb         8:16   1  58.4G  0 disk
+-sdb1      8:17   1  58.4G  0 part
sdc         8:32   0 465.8G  0 disk
+-devrmc0 253:0    0 465.8G  0 crypt

***

[root@dts1 ~]# parted /dev/mapper/devrmc0
GNU Parted 3.1
Using /dev/mapper/devrmc0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel
New disk label type? gpt
(parted) mkpart
Partition name?  []? iSCSI0:rmc0p1
File system type?  [ext2]?
Start? 8000
End? 16000
(parted) print
Model: Linux device-mapper (crypt) (dm)
Disk /dev/mapper/devrmc0: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name           Flags
 1      8000MB  16.0GB  8001MB               iSCSI0:rmc0p1

(parted) print
Model: Linux device-mapper (crypt) (dm)
Disk /dev/mapper/devrmc0: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name           Flags
 1      8000MB  16.0GB  8001MB               iSCSI0:rmc0p1

(parted) quit
Information: You may need to update /etc/fstab.

***

[root@dts1 ~]# lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda             8:0    0   7.6G  0 disk
+-sda1          8:1    0   200M  0 part  /boot
+-sda2          8:2    0   3.4G  0 part  /
+-sda3          8:3    0   200M  0 part  /home
sdb             8:16   1  58.4G  0 disk
+-sdb1          8:17   1  58.4G  0 part
sdc             8:32   0 465.8G  0 disk
+-devrmc0     253:0    0 465.8G  0 crypt
  +-devrmc0p1 253:1    0   7.5G  0 part

***

[root@dts1 ~]# cryptsetup luksDump /dev/rmc0
LUKS header information for /dev/rmc0

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha1
Payload offset: 4096
MK bits:        256
MK digest:      7a 17 c2 70 f1 fa 33 11 a7 ec ff bf e0 c4 f9 e2 38 ab 99 a3
MK salt:        34 14 70 c2 22 ea 11 4c 70 a8 9f 4d 06 ba c2 4f
                1c 89 04 e6 b9 17 7a b9 e1 03 8e 8d 78 a1 22 56
MK iterations:  6500
UUID:           78c5666f-ca4e-4127-982e-d04e85dc1329

Key Slot 0: ENABLED
        Iterations:             26100
        Salt:                   ad 42 10 c7 33 f5 0a fc c6 19 85 ff 2f ec 7a 33
                                60 80 4c dd 36 59 8e fa 2d ed 08 cf 55 98 61 98
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED
[root@dts1 ~]# cryptsetup status devrmc0
/dev/mapper/devrmc0 is active and is in use.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sdc
  offset:  4096 sectors
  size:    976769072 sectors
  mode:    read/write

***

[root@dts1 ~]# lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda             8:0    0   7.6G  0 disk
+-sda1          8:1    0   200M  0 part  /boot
+-sda2          8:2    0   3.4G  0 part  /
+-sda3          8:3    0   200M  0 part  /home
sdb             8:16   1  58.4G  0 disk
+-sdb1          8:17   1  58.4G  0 part
sdc             8:32   0 465.8G  0 disk
+-devrmc0     253:0    0 465.8G  0 crypt
  +-devrmc0p1 253:1    0   7.5G  0 part

***

[root@dts1 ~]# mkfs.ext4 /dev/mapper/devrmc0p1
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
488640 inodes, 1953280 blocks
97664 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2000683008
60 block groups
32768 blocks per group, 32768 fragments per group
8144 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

***

[root@dts1 ~]# mount /dev/mapper/devrmc0p1 /rmc_shares/rmc0p1

***

[root@dts1 ~]# ls -al  /rmc_shares/rmc0p1
total 24
drwxr-xr-x. 3 root root  4096 Jan  5 18:08 .
drwxr-xr-x. 6 root root  4096 Jan  7  2012 ..
drwx------. 2 root root 16384 Jan  5 18:08 lost+found

***

[root@dts1 ~]# lsblk
NAME          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda             8:0    0   7.6G  0 disk
+-sda1          8:1    0   200M  0 part  /boot
+-sda2          8:2    0   3.4G  0 part  /
+-sda3          8:3    0   200M  0 part  /home
sdb             8:16   1  58.4G  0 disk
+-sdb1          8:17   1  58.4G  0 part
sdc             8:32   0 465.8G  0 disk
+-devrmc0     253:0    0 465.8G  0 crypt
  +-devrmc0p1 253:1    0   7.5G  0 part  /rmc_shares/rmc0p1

***

[root@dts1 ~]# parted /dev/mapper/devrmc0 print
Model: Linux device-mapper (crypt) (dm)
Disk /dev/mapper/devrmc0: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name           Flags
 1      8000MB  16.0GB  8001MB  ext4         iSCSI0:rmc0p1


***

[root@dts1 ~]#