* [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? @ 2019-09-25 19:40 .. ink .. 2019-09-26 7:41 ` Milan Broz 0 siblings, 1 reply; 9+ messages in thread From: .. ink .. @ 2019-09-25 19:40 UTC (permalink / raw) To: dm-crypt I just added an ability to create an authenticated luks2 device in zuluCrypt[1] and i am wondering why these volumes need to be wiped when created. I made it work by looking at how cryptsetup does it but i don't understand why because i have so far failed to find any documentation about it. [1] https://github.com/mhogomchungu/zuluCrypt/issues/134 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-25 19:40 [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? .. ink .. @ 2019-09-26 7:41 ` Milan Broz 2019-09-26 9:36 ` Christoph Anton Mitterer ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Milan Broz @ 2019-09-26 7:41 UTC (permalink / raw) To: .. ink .., dm-crypt On 25/09/2019 21:40, .. ink .. wrote: > I just added an ability to create an authenticated luks2 device in > zuluCrypt[1] and i am > wondering why these volumes need to be wiped when created. I made it work by > looking at how cryptsetup does it but i don't understand why because i > have so far > failed to find any documentation about it. I think it is explained in the referenced paper, we should add a FAQ about it. Initial wipe recalculates integrity tags - so you can read the device afterward. If you skip initialization (wipe), integrity tags for all sectors is incorrect and read will return integrity failure (EILSEQ errno). In theory, it is not a problem ("do not read what you did not write"). But it reality it cases many programs to fail because it can access device through page cache. If the *write* is not aligned to a page, page cache tries to first read content, then update content, and write it back to the device. But as said above, all read fails because integrity tags are not initialized, thus even page-unaligned writes can fail. (I have seen this problem even in programs like mkfs, where it is apparent bug.) Milan > > [1] https://github.com/mhogomchungu/zuluCrypt/issues/134 > _______________________________________________ > dm-crypt mailing list > dm-crypt@saout.de > https://www.saout.de/mailman/listinfo/dm-crypt > ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 7:41 ` Milan Broz @ 2019-09-26 9:36 ` Christoph Anton Mitterer 2019-09-26 13:38 ` Robert Nichols 2019-09-26 14:23 ` Arno Wagner 2 siblings, 0 replies; 9+ messages in thread From: Christoph Anton Mitterer @ 2019-09-26 9:36 UTC (permalink / raw) To: dm-crypt On Thu, 2019-09-26 at 09:41 +0200, Milan Broz wrote: > Initial wipe recalculates integrity tags - so you can read the device > afterward. Maybe it shouldn't be called "wipe" (which people often relate to secure deletion methods) but "initialise"? :-) Cheers, Chris ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 7:41 ` Milan Broz 2019-09-26 9:36 ` Christoph Anton Mitterer @ 2019-09-26 13:38 ` Robert Nichols 2019-09-26 14:29 ` Ondrej Kozina 2019-09-26 14:23 ` Arno Wagner 2 siblings, 1 reply; 9+ messages in thread From: Robert Nichols @ 2019-09-26 13:38 UTC (permalink / raw) To: dm-crypt On 9/26/19 2:41 AM, Milan Broz wrote: > On 25/09/2019 21:40, .. ink .. wrote: >> I just added an ability to create an authenticated luks2 device in >> zuluCrypt[1] and i am >> wondering why these volumes need to be wiped when created. I made it work by >> looking at how cryptsetup does it but i don't understand why because i >> have so far >> failed to find any documentation about it. > > I think it is explained in the referenced paper, we should add a FAQ about it. > > Initial wipe recalculates integrity tags - so you can read the device afterward. > > If you skip initialization (wipe), integrity tags for all sectors is incorrect > and read will return integrity failure (EILSEQ errno). > > In theory, it is not a problem ("do not read what you did not write"). > > But it reality it cases many programs to fail because it can access device > through page cache. If the *write* is not aligned to a page, page cache tries > to first read content, then update content, and write it back to the device. > > But as said above, all read fails because integrity tags are not initialized, > thus even page-unaligned writes can fail. > (I have seen this problem even in programs like mkfs, where it is apparent bug.) So, LUKS2 is incompatible with LVM thin provisioning or other sparse storage formats like QCOW2. Good to know. -- Bob Nichols "NOSPAM" is really part of my email address. Do NOT delete it. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 13:38 ` Robert Nichols @ 2019-09-26 14:29 ` Ondrej Kozina 2019-09-26 21:27 ` Robert Nichols 0 siblings, 1 reply; 9+ messages in thread From: Ondrej Kozina @ 2019-09-26 14:29 UTC (permalink / raw) To: dm-crypt; +Cc: Robert Nichols On 9/26/19 3:38 PM, Robert Nichols wrote: > > On 9/26/19 2:41 AM, Milan Broz wrote: >> On 25/09/2019 21:40, .. ink .. wrote: >>> I just added an ability to create an authenticated luks2 device in >>> zuluCrypt[1] and i am >>> wondering why these volumes need to be wiped when created. I made it work by >>> looking at how cryptsetup does it but i don't understand why because i >>> have so far >>> failed to find any documentation about it. >> >> I think it is explained in the referenced paper, we should add a FAQ about it. >> >> Initial wipe recalculates integrity tags - so you can read the device afterward. >> >> If you skip initialization (wipe), integrity tags for all sectors is incorrect >> and read will return integrity failure (EILSEQ errno). >> >> In theory, it is not a problem ("do not read what you did not write"). >> >> But it reality it cases many programs to fail because it can access device >> through page cache. If the *write* is not aligned to a page, page cache tries >> to first read content, then update content, and write it back to the device. >> >> But as said above, all read fails because integrity tags are not initialized, >> thus even page-unaligned writes can fail. >> (I have seen this problem even in programs like mkfs, where it is apparent bug.) > > So, LUKS2 is incompatible with LVM thin provisioning or other sparse storage > formats like QCOW2. Good to know. > That is not correct. First, LUKS2 does not enforce use of authenticated encryption in any way. It's optional choice and definitely not default (it's still considered experimental feature, unlike LUKS2 format that is new upstream default). Contrary, if you prefer discards (trim) pass-through via dm-crypt you may use --persistent --allow-discards option so that it's automatically enabled during every following LUKS2 device activation. So it's more thin provisioning friendly if that is important for you (for others it may not be so). Yes, authenticated encryption with dm-integrity in interleave mode does require device initialization for reason Milan described and moreover dm-integrity does not allow discards pass-through so thin provisioning does not make sense with it because you won't be able to reclaim free space from it. O. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 14:29 ` Ondrej Kozina @ 2019-09-26 21:27 ` Robert Nichols 2019-09-26 22:20 ` Arno Wagner 2019-09-27 8:52 ` Milan Broz 0 siblings, 2 replies; 9+ messages in thread From: Robert Nichols @ 2019-09-26 21:27 UTC (permalink / raw) To: dm-crypt On 9/26/19 9:29 AM, Ondrej Kozina wrote: > On 9/26/19 3:38 PM, Robert Nichols wrote: >> >> On 9/26/19 2:41 AM, Milan Broz wrote: >>> On 25/09/2019 21:40, .. ink .. wrote: >>>> I just added an ability to create an authenticated luks2 device in >>>> zuluCrypt[1] and i am >>>> wondering why these volumes need to be wiped when created. I made it work by >>>> looking at how cryptsetup does it but i don't understand why because i >>>> have so far >>>> failed to find any documentation about it. >>> >>> I think it is explained in the referenced paper, we should add a FAQ about it. >>> >>> Initial wipe recalculates integrity tags - so you can read the device afterward. >>> >>> If you skip initialization (wipe), integrity tags for all sectors is incorrect >>> and read will return integrity failure (EILSEQ errno). >>> >>> In theory, it is not a problem ("do not read what you did not write"). >>> >>> But it reality it cases many programs to fail because it can access device >>> through page cache. If the *write* is not aligned to a page, page cache tries >>> to first read content, then update content, and write it back to the device. >>> >>> But as said above, all read fails because integrity tags are not initialized, >>> thus even page-unaligned writes can fail. >>> (I have seen this problem even in programs like mkfs, where it is apparent bug.) >> >> So, LUKS2 is incompatible with LVM thin provisioning or other sparse storage >> formats like QCOW2. Good to know. >> > > That is not correct. > > First, LUKS2 does not enforce use of authenticated encryption in any way. It's optional choice and definitely not default (it's still considered experimental feature, unlike LUKS2 format that is new upstream default). > > Contrary, if you prefer discards (trim) pass-through via dm-crypt you may use --persistent --allow-discards option so that it's automatically enabled during every following LUKS2 device activation. So it's more thin provisioning friendly if that is important for you (for others it may not be so). > > Yes, authenticated encryption with dm-integrity in interleave mode does require device initialization for reason Milan described and moreover dm-integrity does not allow discards pass-through so thin provisioning does not make sense with it because you won't be able to reclaim free space from it. I wasn't even thinking of trim pass-through. I routinely set up VMs with space allocations that, in total, exceed the space available on the host. Such VMs cannot use authenticated encryption since they would have to immediately use all of their allocated space. I also routinely save system images with partclone or clonezilla, specifically to avoid saving the content of the free space in the filesystems. Such an image would fail authentication when restored. These are fairly common practices. LUKS2 authenticated encryption should come with huge warnings. -- Bob Nichols "NOSPAM" is really part of my email address. Do NOT delete it. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 21:27 ` Robert Nichols @ 2019-09-26 22:20 ` Arno Wagner 2019-09-27 8:52 ` Milan Broz 1 sibling, 0 replies; 9+ messages in thread From: Arno Wagner @ 2019-09-26 22:20 UTC (permalink / raw) To: dm-crypt On Thu, Sep 26, 2019 at 23:27:27 CEST, Robert Nichols wrote: > I wasn't even thinking of trim pass-through. I routinely set up VMs with > space allocations that, in total, exceed the space available on the host. > Such VMs cannot use authenticated encryption since they would have to > immediately use all of their allocated space. I also routinely save > system images with partclone or clonezilla, specifically to avoid saving > the content of the free space in the filesystems. Such an image would > fail authentication when restored. These are fairly common practices. > LUKS2 authenticated encryption should come with huge warnings. I think a regular explanation what authenticated encryption does (to those that do not know it) should be entirely enough. The behavior is not in any way surprising, after all. Add to that that it is not the default and there is no need for any "huge warnings". People that do not knwo what they are doing will shoot themselves in the foot anyways. Regards, Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- A good decision is based on knowledge and not on numbers. -- Plato If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 21:27 ` Robert Nichols 2019-09-26 22:20 ` Arno Wagner @ 2019-09-27 8:52 ` Milan Broz 1 sibling, 0 replies; 9+ messages in thread From: Milan Broz @ 2019-09-27 8:52 UTC (permalink / raw) To: Robert Nichols, dm-crypt On 26/09/2019 23:27, Robert Nichols wrote: > I wasn't even thinking of trim pass-through. I routinely set up VMs > with space allocations that, in total, exceed the space available on > the host. Such VMs cannot use authenticated encryption since they > would have to immediately use all of their allocated space. I also > routinely save system images with partclone or clonezilla, > specifically to avoid saving the content of the free space in the > filesystems. Such an image would fail authentication when restored. > These are fairly common practices. LUKS2 authenticated encryption > should come with huge warnings. I do not quite well understand your complaint. LUKS2 is just on-disk format, it is much more flexible, and allows some new features like integrity protection, but nobody forces you to use that. Default options are precisely the same as in LUKS1, and you can still use LUKS1 if you prefer it. Actually, even the authenticated encryption does not need to wipe the space in principle. It is a workaround to avoid bugs (to avoid reading areas that were never written). You can skip it if your environment is doing IO operations correctly. TRIM passthrough is in some situations complicated, so it is not yet supported for authenticated encryption (dm-integrity backed devices). It is just missing a feature, that can be implemented later. A full device wipe is present in many other systems - VeraCrypt by default does it; Debian installer wipes LUKS devices by writing random data to it. And these systems use just length preserving encryption. A common practice is to TRIM the device to free unused space. We are missing better documentation of common use cases and limitations. I am well aware of that, but sometimes things take longer than expected, sorry for that. Milan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? 2019-09-26 7:41 ` Milan Broz 2019-09-26 9:36 ` Christoph Anton Mitterer 2019-09-26 13:38 ` Robert Nichols @ 2019-09-26 14:23 ` Arno Wagner 2 siblings, 0 replies; 9+ messages in thread From: Arno Wagner @ 2019-09-26 14:23 UTC (permalink / raw) To: dm-crypt On Thu, Sep 26, 2019 at 09:41:39 CEST, Milan Broz wrote: > On 25/09/2019 21:40, .. ink .. wrote: > > I just added an ability to create an authenticated luks2 device in > > zuluCrypt[1] and i am > > wondering why these volumes need to be wiped when created. I made it work by > > looking at how cryptsetup does it but i don't understand why because i > > have so far > > failed to find any documentation about it. > > I think it is explained in the referenced paper, we should add a FAQ about > it. > > Initial wipe recalculates integrity tags - so you can read the device afterward. > > If you skip initialization (wipe), integrity tags for all sectors is > incorrect and read will return integrity failure (EILSEQ errno). > > In theory, it is not a problem ("do not read what you did not write"). > > But it reality it cases many programs to fail because it can access device > through page cache. If the *write* is not aligned to a page, page cache tries > to first read content, then update content, and write it back to the device. > > But as said above, all read fails because integrity tags are not > initialized, thus even page-unaligned writes can fail. (I have seen this > problem even in programs like mkfs, where it is apparent bug.) This is a specific problem with anything authenticated: Even non-data needs to be authenticated, because there is no way to distinguish it from data on that level. Hence expecting to have to do a full "initialization" pass at the start on authenticated storage is perfectly reasonable. I can add an FAQ secion on authenticated encryption fpr this. Do you have a link to the reference paper? Regards, Arno -- Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718 ---- A good decision is based on knowledge and not on numbers. -- Plato If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-09-27 8:52 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-09-25 19:40 [dm-crypt] Why is it necessary to "wipe" an authenticated luks2 device when creating it? .. ink .. 2019-09-26 7:41 ` Milan Broz 2019-09-26 9:36 ` Christoph Anton Mitterer 2019-09-26 13:38 ` Robert Nichols 2019-09-26 14:29 ` Ondrej Kozina 2019-09-26 21:27 ` Robert Nichols 2019-09-26 22:20 ` Arno Wagner 2019-09-27 8:52 ` Milan Broz 2019-09-26 14:23 ` Arno Wagner
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.