Re: [dm-crypt] 10 M Luks2 header size?

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 2558 bytes --]

Hi,

Sorry one typo -
See in red below.
Thank you very much for the help!
_____________________________________________
From: Hualing Yu
Sent: Friday, October 18, 2019 3:02 PM
To: 'dm-crypt@saout.de' <dm-crypt@saout.de>
Subject: 10 M Luks2 header size?


Hello,

I have a question on Luks2 header size.  I created luck2 partition with only one passphrase slot enabled.  But it seems to take really 10 M space.  Here is the luks dump:

sh-4.4# cryptsetup luksDump /dev/mmcblk2gp0p2
LUKS header information
Version: 2
Epoch: 3
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes] <<<<<<<<<<<<<<<<<<<<<< why keyslots take so much space?
UUID: 9037890e-0f2b-4d73-b93b-e2bb53579492
Label: (no label)
Subsystem: (no subsystem)
Flags: (no flags)
      Data segments:
      0: crypt
      offset: 16777216 [bytes] <<<<<<<<<<<<<<<<<<<<<<< so this means the space available to user data is after keylots
      length: (whole device)
      cipher: aes-xts-plain64
      sector: 512 [bytes]
I check in the internet and found all luks2 header dumps show the same values for those two commented entries.
I actually also looked into my device content using dd command, and see indeed the space before 16777216 bytes (10 M) is all scatted filled with something, only after that point, it is all '0'. I zeroed out entire device before doing cryptsetup luksFormat.
Also checked the mapped device size from /dev/mapper/<mapped dev>, and from dev/<device> :
sh-4.4# fdisk -l /dev/mmcblk2gp0p2
Disk /dev/mmcblk2gp0p2: 392 MB, 411041792 bytes, 802816 sectors
12544 cylinders, 4 heads, 16 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/mmcblk2gp0p2 doesn't contain a valid partition table
sh-4.4#
sh-4.4# fdisk -l /dev/mapper/gp0p2
Disk /dev/mapper/gp0p2: 376 MB, 394264576 bytes, 770048 sectors
47 cylinders, 255 heads, 63 sectors/track
Units: sectors of 1 * 512 = 512 bytes

411041792 - 394264576 = 16777216 (10M)

Is there anything wrong?  Should luks has so much overhead?
I appreciate it greatly if you could share you thinking on this.

Thank you,


Hualing

_____________________________________________
From: Hualing Yu
Sent: Friday, October 18, 2019 10:22 AM
To: dm-crypt@saout.de<mailto:dm-crypt@saout.de>
Subject: question on LUKS2


Hello,

Is this mailing list still active?
May I still ask questions here?

Thanks,


Hualing
Yu

Firmware Engineering
Security Products
Johnson Controls
6 Technology Park Drive
Westford, MA 01886
USA
+1 978 577 4171 direct




[-- Attachment #2: Type: text/html, Size: 7304 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Milan Broz]

On 18/10/2019 21:24, Hualing Yu wrote:
> Sorry one typo –
> See in red below.
> Thank you very much for the help!

Hi,

Please, could you send your question without using HTML
in the mail next time?

I am usually replaying to the HTML emails, but your mail
is almost unreadable in a text mail client.

For the question, I was able to decode:

Yes, the default LUKS2 header size is 16M, it allocates
much more area for a possible online operation later
(online reencryption).

But it is configurable, and you can decrease pre-allocated areas,
even to the absolute minimum.
It only applies if 1 keyslot is ok for you and you do not want
to use any extensions in the future, more explanation here
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932437#10

For the generic area description read design doc
https://gitlab.com/cryptsetup/LUKS2-docs

For the generic user, if you can, please do not change
the default, 16MB is today really not a big amount of disk storage.
(With the exceptions of embedded systems.)

Milan


> _____________________________________________
> *From:* Hualing Yu
> *Sent:* Friday, October 18, 2019 3:02 PM
> *To:* 'dm-crypt@saout.de' <dm-crypt@saout.de>
> *Subject:* 10 M Luks2 header size?
>  
>  
> Hello,
>  
> I have a question on Luks2 header size.  I created luck2 partition with only one passphrase slot enabled.  But it seems to take really 10 M space.  Here is the luks dump:
> 
> sh-4.4# cryptsetup luksDump /dev/mmcblk2gp0p2
> LUKS header information
> Version: 2
> Epoch: 3
> Metadata area: 16384 [bytes]
> Keyslots area: 16744448 [bytes] <<<<<<<<<<<<<<<<<<<<<< why keyslots take so much space?
> UUID: 9037890e-0f2b-4d73-b93b-e2bb53579492
> Label: (no label)
> Subsystem: (no subsystem)
> Flags: (no flags)
> Data segments:
> 0: crypt
> offset: 16777216 [bytes] <<<<<<<<<<<<<<<<<<<<<<< so this means the space available to user data is after keylots
> length: (whole device)
> cipher: aes-xts-plain64
> sector: 512 [bytes]
> I check in the internet and found all luks2 header dumps show the same values for those two commented entries.
> I actually also looked into my device content using dd command, and see indeed the space before 16777216 bytes (10 M) is all scatted filled with something, only after that point, it is all '0'. I zeroed out entire device before doing cryptsetup luksFormat.
> Also checked the mapped device size from /dev/mapper/<mapped dev>, and from dev/<device> :
> sh-4.4# fdisk -l /dev/mmcblk2gp0p2
> Disk /dev/mmcblk2gp0p2: 392 MB, 411041792 bytes, 802816 sectors
> 12544 cylinders, 4 heads, 16 sectors/track
> Units: sectors of 1 * 512 = 512 bytes
>  
> Disk /dev/mmcblk2gp0p2 doesn't contain a valid partition table
> sh-4.4#
> sh-4.4# fdisk -l /dev/mapper/gp0p2
> Disk /dev/mapper/gp0p2: 376 MB, 394264576 bytes, 770048 sectors
> 47 cylinders, 255 heads, 63 sectors/track
> Units: sectors of 1 * 512 = 512 bytes
>  
> 411041792 – 394264576 = 16777216 (10M)
>  
> Is there anything wrong?  Should luks has so much overhead?
> I appreciate it greatly if you could share you thinking on this.
>  
> Thank you,
>  
>  
> Hualing
>  
> _____________________________________________
> *From:* Hualing Yu
> *Sent:* Friday, October 18, 2019 10:22 AM
> *To:* _dm-crypt@saout.de_ <mailto:dm-crypt@saout.de>
> *Subject:* question on LUKS2
>  
>  
> Hello,
>  
> Is this mailing list still active?
> May I still ask questions here?
>  
> Thanks,
>  
>  
> Hualing
> Yu
>  
> Firmware Engineering
> Security Products
> Johnson Controls
> 6 Technology Park Drive
> Westford, MA 01886
> USA
> +1 978 577 4171 direct
>  
>  
>  
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

Hi Milan,

Finally I found the right person for this!  Thank you very much!!
I will try the way to make a smaller header, but also will share your suggestion with our team about keeping the default settings.  We work on embedded system but not very tiny one.  We may be able to survive with 16 M default header.  

The important part is to understand this is correct.  I was worried if I did something wrong 8-)

Again, thank you very much!!

Hualing

-----Original Message-----
From: Milan Broz [mailto:gmazyland@gmail.com] 
Sent: Saturday, October 19, 2019 3:08 AM
To: Hualing Yu <hualing.yu@jci.com>; dm-crypt@saout.de
Subject: Re: [dm-crypt] 10 M Luks2 header size?

On 18/10/2019 21:24, Hualing Yu wrote:
> Sorry one typo -
> See in red below.
> Thank you very much for the help!

Hi,

Please, could you send your question without using HTML in the mail next time?

I am usually replaying to the HTML emails, but your mail is almost unreadable in a text mail client.

For the question, I was able to decode:

Yes, the default LUKS2 header size is 16M, it allocates much more area for a possible online operation later (online reencryption).

But it is configurable, and you can decrease pre-allocated areas, even to the absolute minimum.
It only applies if 1 keyslot is ok for you and you do not want to use any extensions in the future, more explanation here
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D932437%2310&amp;data=02%7C01%7Chualing.yu%40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=ZqInWp0IIFwUT2tG5HQ1YviL2Bc9UcM1yevFT8bn66w%3D&amp;reserved=0

For the generic area description read design doc
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fcryptsetup%2FLUKS2-docs&amp;data=02%7C01%7Chualing.yu%40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=zv66AtTvvXU6jJHbzRfQkJ2zG5aKENRLuiid41KBp6I%3D&amp;reserved=0

For the generic user, if you can, please do not change the default, 16MB is today really not a big amount of disk storage.
(With the exceptions of embedded systems.)

Milan


> _____________________________________________
> *From:* Hualing Yu
> *Sent:* Friday, October 18, 2019 3:02 PM
> *To:* 'dm-crypt@saout.de' <dm-crypt@saout.de>
> *Subject:* 10 M Luks2 header size?
>  
>  
> Hello,
>  
> I have a question on Luks2 header size.  I created luck2 partition with only one passphrase slot enabled.  But it seems to take really 10 M space.  Here is the luks dump:
> 
> sh-4.4# cryptsetup luksDump /dev/mmcblk2gp0p2 LUKS header information
> Version: 2
> Epoch: 3
> Metadata area: 16384 [bytes]
> Keyslots area: 16744448 [bytes] <<<<<<<<<<<<<<<<<<<<<< why keyslots take so much space?
> UUID: 9037890e-0f2b-4d73-b93b-e2bb53579492
> Label: (no label)
> Subsystem: (no subsystem)
> Flags: (no flags)
> Data segments:
> 0: crypt
> offset: 16777216 [bytes] <<<<<<<<<<<<<<<<<<<<<<< so this means the 
> space available to user data is after keylots
> length: (whole device)
> cipher: aes-xts-plain64
> sector: 512 [bytes]
> I check in the internet and found all luks2 header dumps show the same values for those two commented entries.
> I actually also looked into my device content using dd command, and see indeed the space before 16777216 bytes (10 M) is all scatted filled with something, only after that point, it is all '0'. I zeroed out entire device before doing cryptsetup luksFormat.
> Also checked the mapped device size from /dev/mapper/<mapped dev>, and from dev/<device> :
> sh-4.4# fdisk -l /dev/mmcblk2gp0p2
> Disk /dev/mmcblk2gp0p2: 392 MB, 411041792 bytes, 802816 sectors
> 12544 cylinders, 4 heads, 16 sectors/track
> Units: sectors of 1 * 512 = 512 bytes
>  
> Disk /dev/mmcblk2gp0p2 doesn't contain a valid partition table sh-4.4# 
> sh-4.4# fdisk -l /dev/mapper/gp0p2 Disk /dev/mapper/gp0p2: 376 MB, 
> 394264576 bytes, 770048 sectors
> 47 cylinders, 255 heads, 63 sectors/track
> Units: sectors of 1 * 512 = 512 bytes
>  
> 411041792 - 394264576 = 16777216 (10M)
>  
> Is there anything wrong?  Should luks has so much overhead?
> I appreciate it greatly if you could share you thinking on this.
>  
> Thank you,
>  
>  
> Hualing
>  
> _____________________________________________
> *From:* Hualing Yu
> *Sent:* Friday, October 18, 2019 10:22 AM
> *To:* _dm-crypt@saout.de_ <mailto:dm-crypt@saout.de>
> *Subject:* question on LUKS2
>  
>  
> Hello,
>  
> Is this mailing list still active?
> May I still ask questions here?
>  
> Thanks,
>  
>  
> Hualing
> Yu
>  
> Firmware Engineering
> Security Products
> Johnson Controls
> 6 Technology Park Drive
> Westford, MA 01886
> USA
> +1 978 577 4171 direct
>  
>  
>  
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> saout.de%2Fmailman%2Flistinfo%2Fdm-crypt&amp;data=02%7C01%7Chualing.yu
> %40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e
> 8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=FXr5jwrKa5oVnlMC0svl
> VAk3k55qNL0lUaYkl9NHkvo%3D&amp;reserved=0
> 

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 6559 bytes --]

Hi Milan,



I just setup my outlook email text format to HTML.  Please let me know if this is readable to you.



May I ask a couple of additional questions about this so that we know how to trade off.



1.      What the reencryption can do for us?  Could you explain very briefly as I'm not sure if we need it?

2.      We need only one or at most two keyslots but we do want them to be scattered as much as needed just as if for the default case, what we can do? Use  -luks2-keyslots-size=1 M (or whatever size that will give two key enough space to scatter)?

3.      What the size of metadata size for default configuration?  What's the downside of using 16 K?



I thank you very much for your help on this!



Hualing



-----Original Message-----

From: Hualing Yu

Sent: Saturday, October 19, 2019 2:47 PM

To: Milan Broz <gmazyland@gmail.com>; dm-crypt@saout.de

Subject: RE: [dm-crypt] 10 M Luks2 header size?



Hi Milan,



Finally I found the right person for this!  Thank you very much!!

I will try the way to make a smaller header, but also will share your suggestion with our team about keeping the default settings.  We work on embedded system but not very tiny one.  We may be able to survive with 16 M default header.



The important part is to understand this is correct.  I was worried if I did something wrong 8-)



Again, thank you very much!!



Hualing



-----Original Message-----

From: Milan Broz [mailto:gmazyland@gmail.com]

Sent: Saturday, October 19, 2019 3:08 AM

To: Hualing Yu <hualing.yu@jci.com>; dm-crypt@saout.de

Subject: Re: [dm-crypt] 10 M Luks2 header size?



On 18/10/2019 21:24, Hualing Yu wrote:

> Sorry one typo -

> See in red below.

> Thank you very much for the help!



Hi,



Please, could you send your question without using HTML in the mail next time?



I am usually replaying to the HTML emails, but your mail is almost unreadable in a text mail client.



For the question, I was able to decode:



Yes, the default LUKS2 header size is 16M, it allocates much more area for a possible online operation later (online reencryption).



But it is configurable, and you can decrease pre-allocated areas, even to the absolute minimum.

It only applies if 1 keyslot is ok for you and you do not want to use any extensions in the future, more explanation here

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D932437%2310&amp;data=02%7C01%7Chualing.yu%40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=ZqInWp0IIFwUT2tG5HQ1YviL2Bc9UcM1yevFT8bn66w%3D&amp;reserved=0



For the generic area description read design doc

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fcryptsetup%2FLUKS2-docs&amp;data=02%7C01%7Chualing.yu%40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=zv66AtTvvXU6jJHbzRfQkJ2zG5aKENRLuiid41KBp6I%3D&amp;reserved=0



For the generic user, if you can, please do not change the default, 16MB is today really not a big amount of disk storage.

(With the exceptions of embedded systems.)



Milan





> _____________________________________________

> *From:* Hualing Yu

> *Sent:* Friday, October 18, 2019 3:02 PM

> *To:* 'dm-crypt@saout.de' <dm-crypt@saout.de>

> *Subject:* 10 M Luks2 header size?

>

>

> Hello,

>

> I have a question on Luks2 header size.  I created luck2 partition with only one passphrase slot enabled.  But it seems to take really 10 M space.  Here is the luks dump:

>

> sh-4.4# cryptsetup luksDump /dev/mmcblk2gp0p2 LUKS header information

> Version: 2

> Epoch: 3

> Metadata area: 16384 [bytes]

> Keyslots area: 16744448 [bytes] <<<<<<<<<<<<<<<<<<<<<< why keyslots take so much space?

> UUID: 9037890e-0f2b-4d73-b93b-e2bb53579492

> Label: (no label)

> Subsystem: (no subsystem)

> Flags: (no flags)

> Data segments:

> 0: crypt

> offset: 16777216 [bytes] <<<<<<<<<<<<<<<<<<<<<<< so this means the

> space available to user data is after keylots

> length: (whole device)

> cipher: aes-xts-plain64

> sector: 512 [bytes]

> I check in the internet and found all luks2 header dumps show the same values for those two commented entries.

> I actually also looked into my device content using dd command, and see indeed the space before 16777216 bytes (10 M) is all scatted filled with something, only after that point, it is all '0'. I zeroed out entire device before doing cryptsetup luksFormat.

> Also checked the mapped device size from /dev/mapper/<mapped dev>, and from dev/<device> :

> sh-4.4# fdisk -l /dev/mmcblk2gp0p2

> Disk /dev/mmcblk2gp0p2: 392 MB, 411041792 bytes, 802816 sectors

> 12544 cylinders, 4 heads, 16 sectors/track

> Units: sectors of 1 * 512 = 512 bytes

>

> Disk /dev/mmcblk2gp0p2 doesn't contain a valid partition table sh-4.4#

> sh-4.4# fdisk -l /dev/mapper/gp0p2 Disk /dev/mapper/gp0p2: 376 MB,

> 394264576 bytes, 770048 sectors

> 47 cylinders, 255 heads, 63 sectors/track

> Units: sectors of 1 * 512 = 512 bytes

>

> 411041792 - 394264576 = 16777216 (10M)

>

> Is there anything wrong?  Should luks has so much overhead?

> I appreciate it greatly if you could share you thinking on this.

>

> Thank you,

>

>

> Hualing

>

> _____________________________________________

> *From:* Hualing Yu

> *Sent:* Friday, October 18, 2019 10:22 AM

> *To:* _dm-crypt@saout.de_ <mailto:dm-crypt@saout.de>

> *Subject:* question on LUKS2

>

>

> Hello,

>

> Is this mailing list still active?

> May I still ask questions here?

>

> Thanks,

>

>

> Hualing

> Yu

>

> Firmware Engineering

> Security Products

> Johnson Controls

> 6 Technology Park Drive

> Westford, MA 01886

> USA

> +1 978 577 4171 direct

>

>

>

>

> _______________________________________________

> dm-crypt mailing list

> dm-crypt@saout.de

> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.

> saout.de%2Fmailman%2Flistinfo%2Fdm-crypt&amp;data=02%7C01%7Chualing.yu

> %40jci.com%7Cbddc0c78fcb24d90ca1308d754630972%7Ca1f1e2147ded45b681a19e

> 8ae3459641%7C0%7C0%7C637070656655052689&amp;sdata=FXr5jwrKa5oVnlMC0svl

> VAk3k55qNL0lUaYkl9NHkvo%3D&amp;reserved=0

>

[-- Attachment #2: Type: text/html, Size: 17047 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Milan Broz]

Hi,

this information should be later in FAQ, so I try to explain it here.

Anyway, stay with defaults, if you can.

On 19/10/2019 21:59, Hualing Yu wrote:
> 
> May I ask a couple of additional questions about this so that we know how to trade off.
>  
> 
> 1.      What the reencryption can do for us?  Could you explain very
> briefly as I’m not sure if we need it?

In principle it can perform changes that requires full-device rewrite (change of the volume key).
See man cryptsetup-reencrypt - just for LUKS2 it is more reliable and mainly online
(you can use device while it is in reencryption process).

See slides from Ondra
  https://okozina.fedorapeople.org/online-disk-reencryption-with-luks2-compact.pdf

There should be also some online demos
  Reencryption demo: https://asciinema.org/a/268573
  Encryption demo: https://asciinema.org/a/268574

For this we require some reserved area for storing temporary encryption data.

> 2.      We need only one or at most two keyslots but we do want them
> to be scattered as much as needed just as if for the default case,
> what we can do? Use  –luks2-keyslots-size=1 M (or whatever size that
> will give two key enough space to scatter)?

There are two areas (see LUKS2 docs) - JSON area for metadata and binary area.

JSON has small binary header, than JSON data (it is 16k currently, stored twice).

For the binary area, it depends what you need, exact size depends on the stored
key size (here the binary keyslot data are stored, exactly the same as in LUKS1).

I would expect you are using current default for disk encryption, AES256-XTS.

Then you need to store 512bit (2x256bit) key in each binary keyslot.

With the LUKS AF filter and 4k alignment it should be 256KiB of binary data per keyslot.

So for 1M and 512bit key it allows 4 LUKS keyslots here.

> 3.      What the size of metadata size for default configuration?
> What’s the downside of using 16 K?
The whole LUKS2 default header takes 16MiB.

For JSON area it is 16k, stored twice (we will increase it later, this is for compatibility reasons),
for binary area - it is "16M - 2x16k" (16M minus JSON areas).

There is only several possible sizes of JSON area you can use (see LUKS2 docs),
binary area is basically arbitrary with maximum 128M, it must be aligned to 4k sectors.

JSON areas allows to store user token metadata, so if you do not need it, no need to enlarge it.

Thanks,
Milan

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 5126 bytes --]

Hi Milan,



Thank you very much for the detailed explanation!  This is tremendous help to us!



I had already brought this up in our group meeting.  We will re-arrange out partitions to ensure all have enough space for default configurations.  Thank you very much on that!



May I ask further - (sorry more questions, I just want to do it right and make the best out from your original design.)

1.      I'm using linux kernel keyring as token for passphrase.  Do I need to enlarge JSON?  (BTW, Why JSON area is stored twice, for backup only that area?)

2.      Do we still need to use luksHeaderBackup <device> and luksHeaderRestore <device> are for entire 16 M header backup?  This means each luks partition needs 32 M for its header!



Now here is our story : We have storage redundancy on our board, that is, for each component (for example linux rootfs) we have two partitions to save two copies of the component.  I think with that, we may not need luks header backup.  When we detect anything wrong with current active partition, include luks header, we can switch to use the standby partition for rootfs for example, and then repair, or simply wipe everything and redo luks format and copy the data into it.

Should this work?  Can you suggest some ways, or check points, for our background task to periodically checking to ensure all luks's are good, in case you have something on top of your head?  8-)



Thank you so much!





Hualing





-----Original Message-----
From: Milan Broz [mailto:gmazyland@gmail.com]
Sent: Sunday, October 20, 2019 6:08 AM
To: Hualing Yu <hualing.yu@jci.com>; dm-crypt@saout.de
Subject: Re: [dm-crypt] 10 M Luks2 header size?



Hi,



this information should be later in FAQ, so I try to explain it here.



Anyway, stay with defaults, if you can.



On 19/10/2019 21:59, Hualing Yu wrote:

>

> May I ask a couple of additional questions about this so that we know how to trade off.

>

>

> 1.      What the reencryption can do for us?  Could you explain very

> briefly as I'm not sure if we need it?



In principle it can perform changes that requires full-device rewrite (change of the volume key).

See man cryptsetup-reencrypt - just for LUKS2 it is more reliable and mainly online (you can use device while it is in reencryption process).



See slides from Ondra

  https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fokozina.fedorapeople.org%2Fonline-disk-reencryption-with-luks2-compact.pdf&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=Zn13uT%2B7wsLKex3r6u3LWAC7xFobCn4PLs10ywQYxeU%3D&amp;reserved=0



There should be also some online demos

  Reencryption demo: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fasciinema.org%2Fa%2F268573&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=6DkH8Bwz699zeGzk25vf8gh4%2FKuImVaMeGEu34qHkCA%3D&amp;reserved=0

  Encryption demo: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fasciinema.org%2Fa%2F268574&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=8nuvhvj5fBB%2FeH0pu0%2F0qRNd7l47dVMQwzDrNFoeeMA%3D&amp;reserved=0



For this we require some reserved area for storing temporary encryption data.



> 2.      We need only one or at most two keyslots but we do want them

> to be scattered as much as needed just as if for the default case,

> what we can do? Use  -luks2-keyslots-size=1 M (or whatever size that

> will give two key enough space to scatter)?



There are two areas (see LUKS2 docs) - JSON area for metadata and binary area.



JSON has small binary header, than JSON data (it is 16k currently, stored twice).



For the binary area, it depends what you need, exact size depends on the stored key size (here the binary keyslot data are stored, exactly the same as in LUKS1).



I would expect you are using current default for disk encryption, AES256-XTS.



Then you need to store 512bit (2x256bit) key in each binary keyslot.



With the LUKS AF filter and 4k alignment it should be 256KiB of binary data per keyslot.



So for 1M and 512bit key it allows 4 LUKS keyslots here.



> 3.      What the size of metadata size for default configuration?

> What's the downside of using 16 K?

The whole LUKS2 default header takes 16MiB.



For JSON area it is 16k, stored twice (we will increase it later, this is for compatibility reasons), for binary area - it is "16M - 2x16k" (16M minus JSON areas).



There is only several possible sizes of JSON area you can use (see LUKS2 docs), binary area is basically arbitrary with maximum 128M, it must be aligned to 4k sectors.



JSON areas allows to store user token metadata, so if you do not need it, no need to enlarge it.



Thanks,

Milan

[-- Attachment #2: Type: text/html, Size: 13930 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 6028 bytes --]

Hello Milan,

How are you?
We truly appreciate your help with your expertise on this LUKS configuration!
Could you please answer my last two questions below?

1.      I'm using linux kernel keyring as token for passphrase (likely one passphrase per LUKS partition).  Do I need to enlarge JSON?  (BTW, Why JSON area is stored twice, for backup only that area?)

2.      If we have full filesystem redundancy, do we still need to use luksHeaderBackup <device> and luksHeaderRestore <device> are for entire 16 M header backup?  Any suggestion on what to check to ensure that the standby (inactivate) luks partition is in good condition?

Thank you very much!

Hualing

From: Hualing Yu
Sent: Monday, October 21, 2019 12:13 PM
To: Milan Broz <gmazyland@gmail.com>; dm-crypt@saout.de
Subject: RE: [dm-crypt] 10 M Luks2 header size?


Hi Milan,



Thank you very much for the detailed explanation!  This is tremendous help to us!



I had already brought this up in our group meeting.  We will re-arrange out partitions to ensure all have enough space for default configurations.  Thank you very much on that!



May I ask further - (sorry more questions, I just want to do it right and make the best out from your original design.)

1.      I'm using linux kernel keyring as token for passphrase.  Do I need to enlarge JSON?  (BTW, Why JSON area is stored twice, for backup only that area?)

2.      Do we still need to use luksHeaderBackup <device> and luksHeaderRestore <device> are for entire 16 M header backup?  This means each luks partition needs 32 M for its header!



Now here is our story : We have storage redundancy on our board, that is, for each component (for example linux rootfs) we have two partitions to save two copies of the component.  I think with that, we may not need luks header backup.  When we detect anything wrong with current active partition, include luks header, we can switch to use the standby partition for rootfs for example, and then repair, or simply wipe everything and redo luks format and copy the data into it.

Should this work?  Can you suggest some ways, or check points, for our background task to periodically checking to ensure all luks's are good, in case you have something on top of your head?  8-)



Thank you so much!





Hualing





-----Original Message-----
From: Milan Broz [mailto:gmazyland@gmail.com]
Sent: Sunday, October 20, 2019 6:08 AM
To: Hualing Yu <hualing.yu@jci.com<mailto:hualing.yu@jci.com>>; dm-crypt@saout.de<mailto:dm-crypt@saout.de>
Subject: Re: [dm-crypt] 10 M Luks2 header size?



Hi,



this information should be later in FAQ, so I try to explain it here.



Anyway, stay with defaults, if you can.



On 19/10/2019 21:59, Hualing Yu wrote:

>

> May I ask a couple of additional questions about this so that we know how to trade off.

>

>

> 1.      What the reencryption can do for us?  Could you explain very

> briefly as I'm not sure if we need it?



In principle it can perform changes that requires full-device rewrite (change of the volume key).

See man cryptsetup-reencrypt - just for LUKS2 it is more reliable and mainly online (you can use device while it is in reencryption process).



See slides from Ondra

  https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fokozina.fedorapeople.org%2Fonline-disk-reencryption-with-luks2-compact.pdf&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=Zn13uT%2B7wsLKex3r6u3LWAC7xFobCn4PLs10ywQYxeU%3D&amp;reserved=0



There should be also some online demos

  Reencryption demo: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fasciinema.org%2Fa%2F268573&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=6DkH8Bwz699zeGzk25vf8gh4%2FKuImVaMeGEu34qHkCA%3D&amp;reserved=0

  Encryption demo: https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fasciinema.org%2Fa%2F268574&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&amp;sdata=8nuvhvj5fBB%2FeH0pu0%2F0qRNd7l47dVMQwzDrNFoeeMA%3D&amp;reserved=0



For this we require some reserved area for storing temporary encryption data.



> 2.      We need only one or at most two keyslots but we do want them

> to be scattered as much as needed just as if for the default case,

> what we can do? Use  -luks2-keyslots-size=1 M (or whatever size that

> will give two key enough space to scatter)?



There are two areas (see LUKS2 docs) - JSON area for metadata and binary area.



JSON has small binary header, than JSON data (it is 16k currently, stored twice).



For the binary area, it depends what you need, exact size depends on the stored key size (here the binary keyslot data are stored, exactly the same as in LUKS1).



I would expect you are using current default for disk encryption, AES256-XTS.



Then you need to store 512bit (2x256bit) key in each binary keyslot.



With the LUKS AF filter and 4k alignment it should be 256KiB of binary data per keyslot.



So for 1M and 512bit key it allows 4 LUKS keyslots here.



> 3.      What the size of metadata size for default configuration?

> What's the downside of using 16 K?

The whole LUKS2 default header takes 16MiB.



For JSON area it is 16k, stored twice (we will increase it later, this is for compatibility reasons), for binary area - it is "16M - 2x16k" (16M minus JSON areas).



There is only several possible sizes of JSON area you can use (see LUKS2 docs), binary area is basically arbitrary with maximum 128M, it must be aligned to 4k sectors.



JSON areas allows to store user token metadata, so if you do not need it, no need to enlarge it.



Thanks,

Milan

[-- Attachment #2: Type: text/html, Size: 18088 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Arno Wagner]

I can answer 2.:

Redundancy (of any kind) is not backup. You need that backup.
A backup is independent from the original data storage mechanism,
redundancy is not.

Regards,
Arno

On Sun, Oct 27, 2019 at 14:15:12 CET, Hualing Yu wrote:
>    Hello Milan,
> 
> 
>    How are you?
> 
>    We truly appreciate your help with your expertise on this LUKS
>    configuration!
>    Could you please answer my last two questions below?
> 
>    1.      I’m using linux kernel keyring as token for passphrase (likely
>    one passphrase per LUKS partition).  Do I need to enlarge JSON?  (BTW,
>    Why JSON area is stored twice, for backup only that area?)
> 
>    2.      If we have full filesystem redundancy, do we still need to use
>    luksHeaderBackup <device> and luksHeaderRestore <device> are for entire
>    16 M header backup?  Any suggestion on what to check to ensure that the
>    standby (inactivate) luks partition is in good condition?
> 
> 
>    Thank you very much!
> 
> 
>    Hualing
> 
> 
>    From: Hualing Yu
>    Sent: Monday, October 21, 2019 12:13 PM
>    To: Milan Broz <gmazyland@gmail.com>; dm-crypt@saout.de
>    Subject: RE: [dm-crypt] 10 M Luks2 header size?
> 
> 
>    Hi Milan,
> 
> 
>    Thank you very much for the detailed explanation!  This is tremendous
>    help to us!
> 
> 
>    I had already brought this up in our group meeting.  We will re-arrange
>    out partitions to ensure all have enough space for default
>    configurations.  Thank you very much on that!
> 
> 
>    May I ask further – (sorry more questions, I just want to do it right
>    and make the best out from your original design.)
> 
>    1.      I’m using linux kernel keyring as token for passphrase.  Do I
>    need to enlarge JSON?  (BTW, Why JSON area is stored twice, for backup
>    only that area?)
> 
>    2.      Do we still need to use luksHeaderBackup <device> and
>    luksHeaderRestore <device> are for entire 16 M header backup?  This
>    means each luks partition needs 32 M for its header!
> 
> 
>    Now here is our story : We have storage redundancy on our board, that
>    is, for each component (for example linux rootfs) we have two
>    partitions to save two copies of the component.  I think with that, we
>    may not need luks header backup.  When we detect anything wrong with
>    current active partition, include luks header, we can switch to use the
>    standby partition for rootfs for example, and then repair, or simply
>    wipe everything and redo luks format and copy the data into it.
> 
>    Should this work?  Can you suggest some ways, or check points, for our
>    background task to periodically checking to ensure all luks’s are good,
>    in case you have something on top of your head?  8-)
> 
> 
>    Thank you so much!
> 
> 
> 
>    Hualing
> 
> 
> 
>    -----Original Message-----
>    From: Milan Broz [[1]mailto:gmazyland@gmail.com]
>    Sent: Sunday, October 20, 2019 6:08 AM
>    To: Hualing Yu <[2]hualing.yu@jci.com>; [3]dm-crypt@saout.de
>    Subject: Re: [dm-crypt] 10 M Luks2 header size?
> 
> 
>    Hi,
> 
> 
>    this information should be later in FAQ, so I try to explain it here.
> 
> 
>    Anyway, stay with defaults, if you can.
> 
> 
>    On 19/10/2019 21:59, Hualing Yu wrote:
> 
>    >
> 
>    > May I ask a couple of additional questions about this so that we know
>    how to trade off.
> 
>    >
> 
>    >
> 
>    > 1.      What the reencryption can do for us?  Could you explain very
> 
>    > briefly as I’m not sure if we need it?
> 
> 
>    In principle it can perform changes that requires full-device rewrite
>    (change of the volume key).
> 
>    See man cryptsetup-reencrypt - just for LUKS2 it is more reliable and
>    mainly online (you can use device while it is in reencryption process).
> 
> 
>    See slides from Ondra
> 
> 
>    [4]https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fok
>    ozina.fedorapeople.org%2Fonline-disk-reencryption-with-luks2-compact.pd
>    f&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096abcf38e8483e599808d7554
>    555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1%7C637071628596824108&a
>    mp;sdata=Zn13uT%2B7wsLKex3r6u3LWAC7xFobCn4PLs10ywQYxeU%3D&amp;reserved=
>    0
> 
> 
>    There should be also some online demos
> 
>      Reencryption demo:
>    [5]https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fas
>    ciinema.org%2Fa%2F268573&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096
>    abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1
>    %7C637071628596824108&amp;sdata=6DkH8Bwz699zeGzk25vf8gh4%2FKuImVaMeGEu3
>    4qHkCA%3D&amp;reserved=0
> 
>      Encryption demo:
>    [6]https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fas
>    ciinema.org%2Fa%2F268574&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca096
>    abcf38e8483e599808d7554555fc%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C1
>    %7C637071628596824108&amp;sdata=8nuvhvj5fBB%2FeH0pu0%2F0qRNd7l47dVMQwzD
>    rNFoeeMA%3D&amp;reserved=0
> 
> 
>    For this we require some reserved area for storing temporary encryption
>    data.
> 
> 
>    > 2.      We need only one or at most two keyslots but we do want them
> 
>    > to be scattered as much as needed just as if for the default case,
> 
>    > what we can do? Use  –luks2-keyslots-size=1 M (or whatever size that
> 
>    > will give two key enough space to scatter)?
> 
> 
>    There are two areas (see LUKS2 docs) - JSON area for metadata and
>    binary area.
> 
> 
>    JSON has small binary header, than JSON data (it is 16k currently,
>    stored twice).
> 
> 
>    For the binary area, it depends what you need, exact size depends on
>    the stored key size (here the binary keyslot data are stored, exactly
>    the same as in LUKS1).
> 
> 
>    I would expect you are using current default for disk encryption,
>    AES256-XTS.
> 
> 
>    Then you need to store 512bit (2x256bit) key in each binary keyslot.
> 
> 
>    With the LUKS AF filter and 4k alignment it should be 256KiB of binary
>    data per keyslot.
> 
> 
>    So for 1M and 512bit key it allows 4 LUKS keyslots here.
> 
> 
>    > 3.      What the size of metadata size for default configuration?
> 
>    > What’s the downside of using 16 K?
> 
>    The whole LUKS2 default header takes 16MiB.
> 
> 
>    For JSON area it is 16k, stored twice (we will increase it later, this
>    is for compatibility reasons), for binary area - it is "16M - 2x16k"
>    (16M minus JSON areas).
> 
> 
>    There is only several possible sizes of JSON area you can use (see
>    LUKS2 docs), binary area is basically arbitrary with maximum 128M, it
>    must be aligned to 4k sectors.
> 
> 
>    JSON areas allows to store user token metadata, so if you do not need
>    it, no need to enlarge it.
> 
> 
>    Thanks,
> 
>    Milan
> 
> References
> 
>    1. mailto:gmazyland@gmail.com
>    2. mailto:hualing.yu@jci.com
>    3. mailto:dm-crypt@saout.de
>    4. https://nam02.safelinks.protection.outlook.com/?url=https://okozina.fedorapeople.org/online-disk-reencryption-with-luks2-compact.pdf&amp;data=02|01|hualing.yu@jci.com|a096abcf38e8483e599808d7554555fc|a1f1e2147ded45b681a19e8ae3459641|0|1|637071628596824108&amp;sdata=Zn13uT+7wsLKex3r6u3LWAC7xFobCn4PLs10ywQYxeU=&amp;reserved=0
>    5. https://nam02.safelinks.protection.outlook.com/?url=https://asciinema.org/a/268573&amp;data=02|01|hualing.yu@jci.com|a096abcf38e8483e599808d7554555fc|a1f1e2147ded45b681a19e8ae3459641|0|1|637071628596824108&amp;sdata=6DkH8Bwz699zeGzk25vf8gh4/KuImVaMeGEu34qHkCA=&amp;reserved=0
>    6. https://nam02.safelinks.protection.outlook.com/?url=https://asciinema.org/a/268574&amp;data=02|01|hualing.yu@jci.com|a096abcf38e8483e599808d7554555fc|a1f1e2147ded45b681a19e8ae3459641|0|1|637071628596824108&amp;sdata=8nuvhvj5fBB/eH0pu0/0qRNd7l47dVMQwzDrNFoeeMA=&amp;reserved=0

> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] 10 M Luks2 header size?

[Milan Broz]

Hi,

sorry for the late reply, I have a lot of personal issues these days,
so this mail was just stuck in the queue - thanks for the reminder.

On 27/10/2019 14:15, Hualing Yu wrote:> Could you please answer my last two questions below?
> 
> 1. I'm using linux kernel keyring as token for passphrase
> (likely one passphrase per LUKS partition).  Do I need to enlarge
> JSON?  (BTW, Why JSON area is stored twice, for backup only that
> area?)

Token (JSON) configuration takes some extra space, but if you
are limited to a few keyslots and few keyring tokens, default
JSON area size should be enough.

You can, of course, increase it during format, but I do not suggest
to use big JSON areas (>1MB).
IOW increasing to 64k should be more than enough if you have no extra user
JSON data stored there.)

The best is if you create the most complicated configuration for your
setup (used all slots + all tokens) and try to configure it.
If cryptsetup doesn't complain that there is not enough space, it is enough :)


For the storing JSON twice - long description is in LUKS2 doc
https://gitlab.com/cryptsetup/LUKS2-docs

It is not backup! We store it twice because LUKS2 expects more frequent
modification of metadata, so we can recover in the case one area is
corrupted (power fail or so).

Also, it increases reliability in the case one area is corrupted
by some other tool (it automatically updates from the second copy).

Note that BINARY area (binary stored keyslot content) is stored
only once (duplication would undermine anti-forensic filter).

> 
> 2.      If we have full filesystem redundancy, do we still need to
> use /luksHeaderBackup/<device> and /luksHeaderRestore/<device> are
> for entire 16 M header backup?  Any suggestion on what to check to
> ensure that the standby (inactivate) luks partition is in good
> condition?

RAID is not a backup. The same applies to redundant LUKS headers.

It increases availability (you can quickly recover if possible), but
you should have offline/offsite backup,

The luksHeaderBackup is in principle just copy of the area with
whole LUKS header (so it contains even unused reserved space,
this space is filled with random data - if it is not conversion
of old header).


I hope this helps.

Thanks,
Milan

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 3631 bytes --]

Hi Milan,



Thank you very much for the detailed reply.  This is again very helpful to us!  It's not a problem at all.  I appreciate it very very much for you using your family time helping us!



I am sorry that I cannot resist asking more questions as your answer indicate more opportunities 8-)  If they are very simple to you to answer, thank you for just drop a few lines; if not, you can simply ignore.

1.      You said "...if you have no extra user JSON data stored there".  Can we use that area to store additional user data?  How?

2.      To check if a LUKS is in good condition, can we just use isLuks command?  Does this cmd trigger some internal sanitary checking?



Thank you,



Hualing



-----Original Message-----
From: Milan Broz [mailto:gmazyland@gmail.com]
Sent: Monday, October 28, 2019 6:37 AM
To: Hualing Yu <hualing.yu@jci.com>; dm-crypt@saout.de
Subject: Re: [dm-crypt] 10 M Luks2 header size?



Hi,



sorry for the late reply, I have a lot of personal issues these days, so this mail was just stuck in the queue - thanks for the reminder.



On 27/10/2019 14:15, Hualing Yu wrote:> Could you please answer my last two questions below?

>

> 1. I'm using linux kernel keyring as token for passphrase (likely one

> passphrase per LUKS partition).  Do I need to enlarge JSON?  (BTW, Why

> JSON area is stored twice, for backup only that

> area?)



Token (JSON) configuration takes some extra space, but if you are limited to a few keyslots and few keyring tokens, default JSON area size should be enough.



You can, of course, increase it during format, but I do not suggest to use big JSON areas (>1MB).

IOW increasing to 64k should be more than enough if you have no extra user JSON data stored there.)



The best is if you create the most complicated configuration for your setup (used all slots + all tokens) and try to configure it.

If cryptsetup doesn't complain that there is not enough space, it is enough :)





For the storing JSON twice - long description is in LUKS2 doc

https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fcryptsetup%2FLUKS2-docs&amp;data=02%7C01%7Chualing.yu%40jci.com%7Ca0a4359785fc4219625508d75b92b889%7Ca1f1e2147ded45b681a19e8ae3459641%7C0%7C0%7C637078558030607535&amp;sdata=DH0mLc05InQUjUqMfNUp%2Fzck%2F4esepLUtMSI9p%2ByZig%3D&amp;reserved=0



It is not backup! We store it twice because LUKS2 expects more frequent modification of metadata, so we can recover in the case one area is corrupted (power fail or so).



Also, it increases reliability in the case one area is corrupted by some other tool (it automatically updates from the second copy).



Note that BINARY area (binary stored keyslot content) is stored only once (duplication would undermine anti-forensic filter).



>

> 2.      If we have full filesystem redundancy, do we still need to

> use /luksHeaderBackup/<device> and /luksHeaderRestore/<device> are for

> entire 16 M header backup?  Any suggestion on what to check to ensure

> that the standby (inactivate) luks partition is in good condition?



RAID is not a backup. The same applies to redundant LUKS headers.



It increases availability (you can quickly recover if possible), but you should have offline/offsite backup,



The luksHeaderBackup is in principle just copy of the area with whole LUKS header (so it contains even unused reserved space, this space is filled with random data - if it is not conversion of old header).





I hope this helps.



Thanks,

Milan



[-- Attachment #2: Type: text/html, Size: 10338 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Milan Broz]

On 28/10/2019 14:50, Hualing Yu wrote:
> I am sorry that I cannot resist asking more questions as your answer
> indicate more opportunities 8-)  If they are very simple to you to
> answer, thank you for just drop a few lines; if not, you can simply
> ignore.
> 
> 1.      You said “…if you have no extra user JSON data stored there”.
> Can we use that area to store additional user data?  How?

You can create custom token type and import directly JSON through cryptsetup
token import command. There is just mandatory type object, otherwise it can be
arbitrary valid JSON.

(I plan to write some article about it but I promised this months ago ... :-)
 
> 2.      To check if a LUKS is in good condition, can we just use
> isLuks command?  Does this cmd trigger some internal sanitary
> checking?

Hm, in theory yes (it calls functions that will do autorecovery),
so side effect is that it validates JSON areas.

But note that any LUKS load will recover invalid JSON area if there
is one area valid (it requires access to cryptsetup locks directory and
of course write access rights to the image/device).

IOW it should "autorepair". If not possible, commands starts to fail
as there is no LUKS2 header.

Milan

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

Hi Milan,

Great!  This helps a lot!  I'm going to design our redundant file systems around this.
I'm looking forward for your article on better using JSON area.

No more question now, at this moment 8-)

We definitely will encounter somethings later when implementing this.  Will bother you again then.  And I'll let you know how we make it here no matter what.

Again, very appreciate all you helps, your time and expertise!


Best regard,

Hualing



-----Original Message-----
From: Milan Broz [mailto:gmazyland@gmail.com] 
Sent: Tuesday, October 29, 2019 9:08 AM
To: Hualing Yu <hualing.yu@jci.com>; dm-crypt@saout.de
Subject: Re: [dm-crypt] 10 M Luks2 header size?

On 28/10/2019 14:50, Hualing Yu wrote:
> I am sorry that I cannot resist asking more questions as your answer 
> indicate more opportunities 8-)  If they are very simple to you to 
> answer, thank you for just drop a few lines; if not, you can simply 
> ignore.
> 
> 1.      You said "...if you have no extra user JSON data stored there".
> Can we use that area to store additional user data?  How?

You can create custom token type and import directly JSON through cryptsetup token import command. There is just mandatory type object, otherwise it can be arbitrary valid JSON.

(I plan to write some article about it but I promised this months ago ... :-)
 
> 2.      To check if a LUKS is in good condition, can we just use
> isLuks command?  Does this cmd trigger some internal sanitary 
> checking?

Hm, in theory yes (it calls functions that will do autorecovery), so side effect is that it validates JSON areas.

But note that any LUKS load will recover invalid JSON area if there is one area valid (it requires access to cryptsetup locks directory and of course write access rights to the image/device).

IOW it should "autorepair". If not possible, commands starts to fail as there is no LUKS2 header.

Milan

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 3619 bytes --]

Hi Milan



We have problem now 8-)



I did 'cryptsetup format' at initramfs, where I also 'add token' to luks passphrase slot 0.

It seems to work as expected in later luksOpen (without asking me passphrase) when still in initramfs.  Even next run after power cycle reboot.  However after it runs to normal rootfs, then when I try to do luksOpen still as root user, it ask for passphrase.



I can see my passphrases are both in @u and @us keyring both at initramfs time and when run as root in normal linux.  However, in initramfs, my passphrasses are also in @s, which probably is why in initramfs time, I can auto activate (open) my luks partitions.



Cryptsetup man page says:

token <add|remove> <device>



              Adds a new keyring token to enable auto-activation of the device.   For  the  auto-

              activation,   the   passphrase  must  be  stored  in  keyring  with  the  specified

              description. Usually, the passphrase should  be  stored  in  user  or  user-session

              keyring.  The token command is supported only for LUKS2.



My passphrases are in both user and user-session keyrings, maybe I just ran into some unusual case where passphrases also need to be in session keyring.  Do you know what's the reason?



Thank you!



Hualing



-----Original Message-----
From: Hualing Yu
Sent: Tuesday, October 29, 2019 11:03 AM
To: Milan Broz <gmazyland@gmail.com>; dm-crypt@saout.de
Subject: RE: [dm-crypt] 10 M Luks2 header size?



Hi Milan,



Great!  This helps a lot!  I'm going to design our redundant file systems around this.

I'm looking forward for your article on better using JSON area.



No more question now, at this moment 8-)



We definitely will encounter somethings later when implementing this.  Will bother you again then.  And I'll let you know how we make it here no matter what.



Again, very appreciate all you helps, your time and expertise!





Best regard,



Hualing







-----Original Message-----

From: Milan Broz [mailto:gmazyland@gmail.com]

Sent: Tuesday, October 29, 2019 9:08 AM

To: Hualing Yu <hualing.yu@jci.com<mailto:hualing.yu@jci.com>>; dm-crypt@saout.de<mailto:dm-crypt@saout.de>

Subject: Re: [dm-crypt] 10 M Luks2 header size?



On 28/10/2019 14:50, Hualing Yu wrote:

> I am sorry that I cannot resist asking more questions as your answer

> indicate more opportunities 8-)  If they are very simple to you to

> answer, thank you for just drop a few lines; if not, you can simply

> ignore.

>

> 1.      You said "...if you have no extra user JSON data stored there".

> Can we use that area to store additional user data?  How?



You can create custom token type and import directly JSON through cryptsetup token import command. There is just mandatory type object, otherwise it can be arbitrary valid JSON.



(I plan to write some article about it but I promised this months ago ... :-)

> 2.      To check if a LUKS is in good condition, can we just use

> isLuks command?  Does this cmd trigger some internal sanitary

> checking?



Hm, in theory yes (it calls functions that will do autorecovery), so side effect is that it validates JSON areas.



But note that any LUKS load will recover invalid JSON area if there is one area valid (it requires access to cryptsetup locks directory and of course write access rights to the image/device).



IOW it should "autorepair". If not possible, commands starts to fail as there is no LUKS2 header.



Milan

[-- Attachment #2: Type: text/html, Size: 9301 bytes --]

Re: [dm-crypt] 10 M Luks2 header size?

[Ondrej Kozina]

On 11/3/19 4:33 AM, Hualing Yu wrote:
> Hi Milan
> 
> We have problem now 8-)
> 
> I did 'cryptsetup format' at initramfs, where I also 'add token' to luks 
> passphrase slot 0.
> 
> It seems to work as expected in later luksOpen (without asking me 
> passphrase) when still in initramfs.  Even next run after power cycle 
> reboot.  However after it runs to normal rootfs, then when I try to do 
> luksOpen still as root user, it ask for passphrase.
> 
> I can see my passphrases are both in @u and @us keyring both at 
> initramfs time and when run as root in normal linux.  However, in 
> initramfs, my passphrasses are also in @s, which probably is why in 
> initramfs time, I can auto activate (open) my luks partitions.
> 
> Cryptsetup man page says:
> 
> token <add|remove> <device>
> 
>                Adds a new keyring token to enable auto-activation of the 
> device.   For  the  auto-
> 
>                activation,   the   passphrase  must  be  stored  in  
> keyring  with  the  specified
> 
>                description. Usually, the passphrase should  be  stored  
> in  user  or  user-session
> 
>                keyring.  The token command is supported only for LUKS2.
> 
> My passphrases are in both user and user-session keyrings, maybe I just 
> ran into some unusual case where passphrases also need to be in session 
> keyring.  Do you know what’s the reason?

Maybe the key is unreachable from your current session after switching 
out from initramfs. Can you read the key payload with "keyctl read 
<your_key>" command?

Regards O.

Re: [dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]

Hi Ondrej,



Yes, I can read the key content when it seems not automatically used to activate (open) the luks partition it assigned to.



However,  after I did 'keyctl link @us @s'



Then 'cryptsetup luksOpen' didn't prompt for passphrase but directly activated the partition (show up under /dev/mapper/)



It seem the auto-activation need to have the key in session keyring, not just user session keyring, while as the man page said it only need to be in either @u or @us.



I can add this keyring link command every time try to open luks, but I want to know if we are supposed to do so or this indicated something wrong.



Thanks,



Hualing



-----Original Message-----
From: Ondrej Kozina [mailto:okozina@redhat.com]
Sent: Monday, November 04, 2019 5:34 AM
To: dm-crypt@saout.de
Cc: Hualing Yu <hualing.yu@jci.com>
Subject: Re: [dm-crypt] 10 M Luks2 header size?



On 11/3/19 4:33 AM, Hualing Yu wrote:

> Hi Milan

>

> We have problem now 8-)

>

> I did 'cryptsetup format' at initramfs, where I also 'add token' to

> luks passphrase slot 0.

>

> It seems to work as expected in later luksOpen (without asking me

> passphrase) when still in initramfs.  Even next run after power cycle

> reboot.  However after it runs to normal rootfs, then when I try to do

> luksOpen still as root user, it ask for passphrase.

>

> I can see my passphrases are both in @u and @us keyring both at

> initramfs time and when run as root in normal linux.  However, in

> initramfs, my passphrasses are also in @s, which probably is why in

> initramfs time, I can auto activate (open) my luks partitions.

>

> Cryptsetup man page says:

>

> token <add|remove> <device>

>

>                Adds a new keyring token to enable auto-activation of

> the device.   For  the  auto-

>

>                activation,   the   passphrase  must  be  stored  in

> keyring  with  the  specified

>

>                description. Usually, the passphrase should  be  stored

> in  user  or  user-session

>

>                keyring.  The token command is supported only for LUKS2.

>

> My passphrases are in both user and user-session keyrings, maybe I

> just ran into some unusual case where passphrases also need to be in

> session keyring.  Do you know what's the reason?



Maybe the key is unreachable from your current session after switching out from initramfs. Can you read the key payload with "keyctl read <your_key>" command?



Regards O.



[-- Attachment #2: Type: text/html, Size: 7317 bytes --]

[dm-crypt] 10 M Luks2 header size?

[Hualing Yu]

[-- Attachment #1: Type: text/plain, Size: 2264 bytes --]

Hello,

I have a question on Luks2 header size.  I created luck2 partition with out one passphrase slot enabled.  But it seems to take really 10 M space.  Here is the luks dump:

sh-4.4# cryptsetup luksDump /dev/mmcblk2gp0p2
LUKS header information
Version: 2
Epoch: 3
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes] <<<<<<<<<<<<<<<<<<<<<< why keyslots take so much space?
UUID: 9037890e-0f2b-4d73-b93b-e2bb53579492
Label: (no label)
Subsystem: (no subsystem)
Flags: (no flags)
      Data segments:
      0: crypt
      offset: 16777216 [bytes] <<<<<<<<<<<<<<<<<<<<<<< so this means the space available to user data is after keylots
      length: (whole device)
      cipher: aes-xts-plain64
      sector: 512 [bytes]
I check in the internet and found all luks2 header dumps show the same values for those two commented entries.
I actually also looked into my device content using dd command, and see indeed the space before 16777216 bytes (10 M) is all scatted filled with something, only after that point, it is all '0'. I zeroed out entire device before doing cryptsetup luksFormat.
Also checked the mapped device size from /dev/mapper/<mapped dev>, and from dev/<device> :
sh-4.4# fdisk -l /dev/mmcblk2gp0p2
Disk /dev/mmcblk2gp0p2: 392 MB, 411041792 bytes, 802816 sectors
12544 cylinders, 4 heads, 16 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/mmcblk2gp0p2 doesn't contain a valid partition table
sh-4.4#
sh-4.4# fdisk -l /dev/mapper/gp0p2
Disk /dev/mapper/gp0p2: 376 MB, 394264576 bytes, 770048 sectors
47 cylinders, 255 heads, 63 sectors/track
Units: sectors of 1 * 512 = 512 bytes

411041792 - 394264576 = 16777216 (10M)

Is there anything wrong?  Should luks has so much overhead?
I appreciate it greatly if you could share you thinking on this.

Thank you,


Hualing

_____________________________________________
From: Hualing Yu
Sent: Friday, October 18, 2019 10:22 AM
To: dm-crypt@saout.de
Subject: question on LUKS2


Hello,

Is this mailing list still active?
May I still ask questions here?

Thanks,


Hualing
Yu

Firmware Engineering
Security Products
Johnson Controls
6 Technology Park Drive
Westford, MA 01886
USA
+1 978 577 4171 direct




[-- Attachment #2: Type: text/html, Size: 6221 bytes --]