[dm-crypt] bug in cryptsetup --header option when using luksFormat?

[dm-crypt] bug in cryptsetup --header option when using luksFormat?

[aral]

Dear Christophe,

I am familiarizing myself with cryptsetup, and I have found no way to use the --header option while also providing an --align-payload parameter with the intended effect.
The version I am using is cryptsetup 2.1.0 as live in the debian buster repositories at the moment.

Every way I try, the command:
  cryptsetup --cipher=aes-xts-plain64 --key-size=512 --hash=sha512 --align-payload=40960 luksFormat containerfile --header headerfile

creates a correct header in "headerfile", but enlarges the headerfile to the total size indicated by --align-payload - in this case the headerfile ends up being 20MB in size, and with higher "offsets", it will become huge. All of it seems to be cryptographic data, because the luksBackupHeader command will just duplicate the big header file, and using only the first 16M of it will fail.

The options --offset and --align-payload seem to both trigger the behavior, but only --align-payload seems to store the luks container at an offset as intended.

A workaround that I can do is:
  cryptsetup --cipher=aes-xts-plain64 --key-size=512 --hash=sha512 --align-payload=40960 luksFormat containerfile

# extract the headerfile from the containerfile
dd if=containerfile of=headerfile bs=1M count=16

# overwrite the original header in the container with urandom data
dd conv=notrunc if=/dev/urandom of=containerfile bs=1M count=16

cryptsetup --header headerfile open containerfile devicemappername

After this, the device under /dev/mapper/devicemappername behaves as expected.


However, even in the workaround, the --align-payload option seems to trigger a looooooooooooooooooong write operation - e.g. when trying to align a payload somewhere at +250 Gigabytes, the cryptsetup takes ages to complete and is hammering the disk, while it near instantly completes when not using --align-payload.

I have researched via google etc for days now without finding a solution, so I hope you may be able to do something with the information provided.

Kind regards

   aral

Re: [dm-crypt] bug in cryptsetup --header option when using luksFormat?

[Ondrej Kozina]

Hi,

On 6/14/20 7:03 PM, aral@gmx.de wrote:
> Dear Christophe,
> 
> I am familiarizing myself with cryptsetup, and I have found no way to use the --header option while also providing an --align-payload parameter with the intended effect.
> The version I am using is cryptsetup 2.1.0 as live in the debian buster repositories at the moment

Assume you're trying to achieve detached LUKS2 header 16 MiBs in size 
(default LUKS2 header size) with data offset starting on different data 
device at offset X. Am I right?

Nevertheless you have found some issues (more on that later).

> 
> Every way I try, the command:
>    cryptsetup --cipher=aes-xts-plain64 --key-size=512 --hash=sha512 --align-payload=40960 luksFormat containerfile --header headerfile
> 
> creates a correct header in "headerfile", but enlarges the headerfile to the total size indicated by --align-payload - in this case the headerfile ends up being 20MB in size, and with higher "offsets", it will become huge. All of it seems to be cryptographic data, because the luksBackupHeader command will just duplicate the big header file, and using only the first 16M of it will fail.
> 
> The options --offset and --align-payload seem to both trigger the behavior, but only --align-payload seems to store the luks container at an offset as intended.

Yes, because each parameter does something different. With --offset you 
enforce exact start of encrypted data on a data device. With 
--align-payload you say that data offset should be aligned to some 
value, but the offset calculation itself is left on cryptsetup to do (in 
case of LUKS2 it'll be based on selected metadata size).

Moreover, by default, LUKS2 is trying to use all available space between 
head of device and selected data offset. Thus, when you use large 
--offset we create much larger (even detached) luks header due to this.

> 
> A workaround that I can do is:
>    cryptsetup --cipher=aes-xts-plain64 --key-size=512 --hash=sha512 --align-payload=40960 luksFormat containerfile

If my assumption on your use case in the beginning was correct, the 
proper workaround for you would be as follows (note you should stick 
with --offset in case you want data at specific offset):

You need to enforce --luks2-keyslots-size so that we don't use all space 
available for us. Let's say you want to have data offset at 1 GiB, but 
you'd like to have regular LUKS2 header, which is 16MiBs in total:

cryptsetup luksFormat --offset 2097152 --luks2-metadata-size 16352k 
--luks-metadata-size 16k --header headerfile containerfile

(16352k is in fact 16MiB - 2*16KiB for two binary headers and two json 
metadata copies)

You can later truncate resulting header to 16MiBs because one of bugs 
you've actually discovered is that we allocate much larger file than 
needed. Even though the rest of file is now zeroed.

(In general you can always safely truncate luks2 *detached* header to 
2*metadata_size + keyslots_size)

The second bug is that --align-payload together with --header has 
different impact when compared to same parameters set except the 
--header parameter. This is probably bug as well. Let me dig deeper on 
it. You can create issue for it on 
https://gitlab.com/cryptsetup/cryptsetup/ if you want.

Thank you for the report
O.


> 
> # extract the headerfile from the containerfile
> dd if=containerfile of=headerfile bs=1M count=16
> 
> # overwrite the original header in the container with urandom data
> dd conv=notrunc if=/dev/urandom of=containerfile bs=1M count=16
> 
> cryptsetup --header headerfile open containerfile devicemappername
> 
> After this, the device under /dev/mapper/devicemappername behaves as expected.
> 
> 
> However, even in the workaround, the --align-payload option seems to trigger a looooooooooooooooooong write operation - e.g. when trying to align a payload somewhere at +250 Gigabytes, the cryptsetup takes ages to complete and is hammering the disk, while it near instantly completes when not using --align-payload.
> 
> I have researched via google etc for days now without finding a solution, so I hope you may be able to do something with the information provided.
> 
> Kind regards
> 
>     aral
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] bug in cryptsetup --header option when using luksFormat?

[aral]

Hi Ondrej,

On 2020-06-15 17:33, Ondrej Kozina wrote:
> Assume you're trying to achieve detached LUKS2 header 16 MiBs in size
> (default LUKS2 header size) with data offset starting on different data
> device at offset X. Am I right?

Yes, that is correct - I tried it with a detached header and a
containerfile first on the same device because that way I could just use
a temp folder instead of messing with /dev/sdX while testing. But the
end aim was to create a LUKS container with a detached header on an
external hard drive which never gets to see the header.

> Nevertheless you have found some issues (more on that later).

Well - that was an interesting read - it was really not clear to me that
the header size could be expanded to gigabytes (by default) to increase
the difficulty to extract it by analysis (that is how I think it is
intended?).

Thank you very much for the detailed explanation - I think it might be
worth to add a description / example for that scenario to the manpage /
--help output: The reason I never thought about messing with the header
size was because I didn't realize it was auto-growing.

My initial thought was that this would also explain the long write
operation with my workaround & the --align-payload - but if, in the
process, you were able to find another bug... all the better ;)

I would help you with a bug report, but I am preparing for a move to
another country (hence the need for an encrypted external drive) and
need to arrange stuff still - no time for a bugreport right now... If I
get around to it later, I'll do it.

Cheers & thanks for the good software :)

   /aral

Re: [dm-crypt] bug in cryptsetup --header option when using luksFormat?

[Ondrej Kozina]

On 6/15/20 5:33 PM, Ondrej Kozina wrote:
> 
> You need to enforce --luks2-keyslots-size so that we don't use all space
> available for us. Let's say you want to have data offset at 1 GiB, but
> you'd like to have regular LUKS2 header, which is 16MiBs in total:
> 
> cryptsetup luksFormat --offset 2097152 --luks2-metadata-size 16352k
> --luks-metadata-size 16k --header headerfile containerfile

There's typo. Example should have been:

cryptsetup luksFormat --offset 2097152 --luks2-keyslots-size 16352k 
--luks2-metadata-size 16k --header headerfile containerfile

Just for the record.

> 
> (16352k is in fact 16MiB - 2*16KiB for two binary headers and two json
> metadata copies)
>