[dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ingo Franzki]

Hi,

I have converted an existing LUKS1 volume to LUKS2 via 'cryptsetup convert --type luks2 <device>'.
That worked well. 

After that I am trying to use cryptsetup-reencrypt ro reencrypt the volume using a different volume key.
This fails with 'Cannot format device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new.'

The debug output shows the following:

...
# keyslots_size is too large 4161536 (bytes). Data offset: 2097152, keyslots offset: 32768
Cannot format device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new.
# Releasing crypt device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new context.
# Releasing crypt device /dev/mapper/disk5 context.
# Releasing device-mapper backend.
Creation of LUKS backup headers failed.
...

So the reason certainly is that the header area is too small, because that volume was converted over from LUKS1 which uses a smaller header than LUKS2.
luksDump shows that the offset of the data segment is less on the converted volume than on a volume that was formatted as LUKS2 right away.
Nevertheless, 'cryptsetup convert' seems to be able to produce an (obviously smaller) LUKS2 header for that device. Other commands like luksAddKey are also able to work with that smaller LUKS2 header. 

Is there a way to enlarge the header area of a (converted) LUKS2 volume to the standard header area size? 
I guess not, but I thought I'll ask anyway....

Any other ideas? 
Any way to enhance cryptsetup-reencrypt to be able to work with a smaller header area? 

This would be a perfect solution for converting an existing LUKS1 volume to use a secure volume key with the PAES cipher that is supported by cryptsetup since version 2.0.3. 

Kind regards, Ingo

-- 

IBM Deutschland Research & Development GmbH / Vorsitzender des Aufsichtsrats: Martina Koederitz 
Geschäftsführung: Dirk Wittkopp 
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

Hi Ingo,

thanks for the report! Could you please provide me with more information 
here on the list or on gitlab issue tracker? We're very close to 2.0.4 
release and I'd like to have this fixed if I could reproduce it.

On 08/02/2018 10:16 AM, Ingo Franzki wrote:
> Hi,
> 
> I have converted an existing LUKS1 volume to LUKS2 via 'cryptsetup convert --type luks2 <device>'.
> That worked well.

How did you create original LUKS1 header? Please provide me with either 
exact command or debug output.

> 
> After that I am trying to use cryptsetup-reencrypt ro reencrypt the volume using a different volume key.
> This fails with 'Cannot format device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new.'
> 
> The debug output shows the following:
> 
> ...
> # keyslots_size is too large 4161536 (bytes). Data offset: 2097152, keyslots offset: 32768
> Cannot format device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new.
> # Releasing crypt device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new context.
> # Releasing crypt device /dev/mapper/disk5 context.
> # Releasing device-mapper backend.
> Creation of LUKS backup headers failed.
> ...
> 
> So the reason certainly is that the header area is too small, because that volume was converted over from LUKS1 which uses a smaller header than LUKS2.
> luksDump shows that the offset of the data segment is less on the converted volume than on a volume that was formatted as LUKS2 right away.

Ouch, this sounds like really ugly bug in conversion code. If we really 
changed data offset during it, it's basically data corruption we're 
speaking about. Could you reproduce it and provide me with full debug 
output of cryptsetup convert action? In the meantime I'll try to 
reproduce it myself...

> Nevertheless, 'cryptsetup convert' seems to be able to produce an (obviously smaller) LUKS2 header for that device. Other commands like luksAddKey are also able to work with that smaller LUKS2 header.
> 
> Is there a way to enlarge the header area of a (converted) LUKS2 volume to the standard header area size?
> I guess not, but I thought I'll ask anyway....
> 
> Any other ideas?
> Any way to enhance cryptsetup-reencrypt to be able to work with a smaller header area?
> 
> This would be a perfect solution for converting an existing LUKS1 volume to use a secure volume key with the PAES cipher that is supported by cryptsetup since version 2.0.3.
> 
> Kind regards, Ingo
> 

Kind regards
Ondrej

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

On 08/02/2018 10:16 AM, Ingo Franzki wrote:
> Hi,
> 
> I have converted an existing LUKS1 volume to LUKS2 via 'cryptsetup convert --type luks2 <device>'.
> That worked well.
> 
> After that I am trying to use cryptsetup-reencrypt ro reencrypt the volume using a different volume key.
> This fails with 'Cannot format device LUKS-5d6495ba-b6f9-43c5-883f-dff56f10c72a.new.'

Ok, I know what's wrong. The convert action works as expected and 
there's nothing wrong with data offset. The issue is 
cryptsetup-reencrypt utility currently can't handle setup where existing 
LUKS2 header, on a device you're about to reencrypt is different size 
from default LUKS2 header size which is 4MiBs currently. The converted 
header is as you wrote 2MiBs.

Ondrej

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Michael Kjörling]

On 2 Aug 2018 11:20 +0200, from okozina@redhat.com (Ondrej Kozina):
> Ok, I know what's wrong. The convert action works as expected and
> there's nothing wrong with data offset. The issue is
> cryptsetup-reencrypt utility currently can't handle setup where
> existing LUKS2 header, on a device you're about to reencrypt is
> different size from default LUKS2 header size which is 4MiBs
> currently. The converted header is as you wrote 2MiBs.

Sounds to me like that should be easy enough to add an early check and
specific error message for. Even if the error is just something like
"this container cannot be converted to LUKS2 because of header size
mismatch, no changes made", it's far better than erroring out with a
scary error message. cryptsetup-reencrypt is scary enough as it is.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
  “The most dangerous thought that you can have as a creative person
              is to think you know what you’re doing.” (Bret Victor)

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

On 08/02/2018 11:28 AM, Michael Kjörling wrote:
> On 2 Aug 2018 11:20 +0200, from okozina@redhat.com (Ondrej Kozina):
>> Ok, I know what's wrong. The convert action works as expected and
>> there's nothing wrong with data offset. The issue is
>> cryptsetup-reencrypt utility currently can't handle setup where
>> existing LUKS2 header, on a device you're about to reencrypt is
>> different size from default LUKS2 header size which is 4MiBs
>> currently. The converted header is as you wrote 2MiBs.
> 
> Sounds to me like that should be easy enough to add an early check and
> specific error message for. Even if the error is just something like
> "this container cannot be converted to LUKS2 because of header size
> mismatch, no changes made", it's far better than erroring out with a
> scary error message. cryptsetup-reencrypt is scary enough as it is.
> 

In my reproducer, the data were not damaged and I think neither were in 
Ingo's case (but can't speak for him). In fact, cryptsetup library 
behaved correctly and identified the mismatch. It's exactly just missing 
error message in cryptsetup-reencrypt as you pointed out. The 
reencryption stopped while creating header backups so no harm done (in 
my case). But let me think about it for some time yet.

O.

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ingo Franzki]

On 02.08.2018 11:44, Ondrej Kozina wrote:
> On 08/02/2018 11:28 AM, Michael Kjörling wrote:
>> On 2 Aug 2018 11:20 +0200, from okozina@redhat.com (Ondrej Kozina):
>>> Ok, I know what's wrong. The convert action works as expected and
>>> there's nothing wrong with data offset. The issue is
>>> cryptsetup-reencrypt utility currently can't handle setup where
>>> existing LUKS2 header, on a device you're about to reencrypt is
>>> different size from default LUKS2 header size which is 4MiBs
>>> currently. The converted header is as you wrote 2MiBs.
>>
>> Sounds to me like that should be easy enough to add an early check and
>> specific error message for. Even if the error is just something like
>> "this container cannot be converted to LUKS2 because of header size
>> mismatch, no changes made", it's far better than erroring out with a
>> scary error message. cryptsetup-reencrypt is scary enough as it is.
>>
> 
> In my reproducer, the data were not damaged and I think neither were in Ingo's case (but can't speak for him). In fact, cryptsetup library behaved correctly and identified the mismatch. It's exactly just missing error message in cryptsetup-reencrypt as you pointed out. The reencryption stopped while creating header backups so no harm done (in my case). But let me think about it for some time yet.

Correct, the data was not currupted. So the LUKS1 to LUKS2 coversion works fine. Just cryptsetup-reencrypt does not.

A better error message is fine, but I would rather like to see a fix that makes cryptsetup-reencrypt work with non-default LUKS2 header sizes 

I guess you don't need any debug output anymore, since you can reproduce it on your own?
> 
> O.
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt


-- 
Ingo Franzki
eMail: ifranzki@linux.ibm.com  
Tel: ++49 (0)7031-16-4648
Fax: ++49 (0)7031-16-3456
Linux on z Systems Development, IBM z Systems, Schoenaicher Str. 220, 71032 Boeblingen, Germany

IBM Deutschland Research & Development GmbH / Vorsitzender des Aufsichtsrats: Martina Koederitz 
Geschäftsführung: Dirk Wittkopp 
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

On 08/02/2018 12:38 PM, Ingo Franzki wrote:
> 
> A better error message is fine, but I would rather like to see a fix that makes cryptsetup-reencrypt work with non-default LUKS2 header sizes

I'm trying to find a reasonable fix. But no guaranties it'll land in 
2.0.4 yet.

> 
> I guess you don't need any debug output anymore, since you can reproduce it on your own?

Provided it was default LUKS1 original device up-converted to default 
LUKS2 device with 2.0.3 release, I'm fine and I have all data:) Thanks 
for the report again!

O.

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

On 08/02/2018 12:38 PM, Ingo Franzki wrote:

> A better error message is fine, but I would rather like to see a fix that makes cryptsetup-reencrypt work with non-default LUKS2 header sizes

This commit in wip-luks2 branch should fix it 1f36e33a. I need to add 
some tests yet, but hope this is it. Provided it passes tests I think 
this may get in 2.0.4 since it's one-liner.

Regards
O.

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ingo Franzki]

On 02.08.2018 15:05, Ondrej Kozina wrote:
> On 08/02/2018 12:38 PM, Ingo Franzki wrote:
> 
>> A better error message is fine, but I would rather like to see a fix that makes cryptsetup-reencrypt work with non-default LUKS2 header sizes
> 
> This commit in wip-luks2 branch should fix it 1f36e33a. I need to add some tests yet, but hope this is it. Provided it passes tests I think this may get in 2.0.4 since it's one-liner.
Wow that was quick! 

I checked out wip-luks2, built it, and tested it out on my system. Works great! Even with the PAES cipher.

Will the smaller header area have any negative impact on LUKS operations later on? 
Like does it allow less keyslots due to the reduced header space? 
> 
> Regards
> O.
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt
> 
> 


-- 
Ingo Franzki
eMail: ifranzki@linux.ibm.com  
Tel: ++49 (0)7031-16-4648
Fax: ++49 (0)7031-16-3456
Linux on z Systems Development, IBM z Systems, Schoenaicher Str. 220, 71032 Boeblingen, Germany

IBM Deutschland Research & Development GmbH / Vorsitzender des Aufsichtsrats: Martina Koederitz 
Geschäftsführung: Dirk Wittkopp 
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294

Re: [dm-crypt] cryptsetup-reencrypt fails after converting a LUKS1 volume to LUKS2

[Ondrej Kozina]

On 08/02/2018 03:42 PM, Ingo Franzki wrote:
> On 02.08.2018 15:05, Ondrej Kozina wrote:
>> On 08/02/2018 12:38 PM, Ingo Franzki wrote:
>>
>>> A better error message is fine, but I would rather like to see a fix that makes cryptsetup-reencrypt work with non-default LUKS2 header sizes
>>
>> This commit in wip-luks2 branch should fix it 1f36e33a. I need to add some tests yet, but hope this is it. Provided it passes tests I think this may get in 2.0.4 since it's one-liner.
> Wow that was quick!
> 
> I checked out wip-luks2, built it, and tested it out on my system. Works great! Even with the PAES cipher.
> 
> Will the smaller header area have any negative impact on LUKS operations later on?
> Like does it allow less keyslots due to the reduced header space?

If you grow volume key size after conversion to LUKS2, yes, you may end 
with header where you fit less than 8 active keyslots.

New features that would benefit from larger metadata area in LUKS2 
header may also not work optimally (i.e future online reencryption).

Regards
Ondrej