[dm-crypt] Debian installer formatting LUKS2 devices by default?

[dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

Hi there,

Debian Buster will freeze at the beginning of next year and we have
people asking for the installer to format devices with `--type luks2`.

(FWIW, I think these requests to default to `--type luks2` are mostly
motivated by a better PBKDF, so nothing impossible to obtain by
conversion from an existing LUKS1 device.)

Personally I'd rather *not* have such custom defaults in the installer.
Do you have any plan to have `luksFormat` default to LUKS2 at some
point?  If so, any idea, when that would happen? ;-)  Given the warning
in the latest Release Notes [0] I assume LUKS2 is not mature enough for
our installer yet.  Not sure what other distros are doing, but for
Debian we're waiting for that scary warning to disappear (or
alternatively, an explicit blessing from upstream) before promoting
LUKS2 (and latter authenticated encryption — once a better AEAD
algorithm is available) in our installer and documentation :-)

Cheers,
-- 
Guilhem.

[0] https://kernel.org/pub/linux/utils/cryptsetup/v2.0/v2.0.3-ReleaseNotes

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

On 27/07/18 10:16, Guilhem Moulin wrote:
> Debian Buster will freeze at the beginning of next year and we have
> people asking for the installer to format devices with `--type luks2`.
> 
> (FWIW, I think these requests to default to `--type luks2` are mostly
> motivated by a better PBKDF, so nothing impossible to obtain by
> conversion from an existing LUKS1 device.)

Hi,

(btw another important LUKS2 option is the sector size - 4k sectors are
much faster if used over 4k sector devices)

We already added configure switch for default format, so now it is up to
the distro maintainer.

I would like to change upstream default in some next version (see below),
but we still have some issues to solve.

One important issue is with Argon2 (or with memory-hard functions in general):

If the Argon2 required amount of memory is too high (existing device from different
systems etc), out-of-memory killer can kill cryptsetup.
This is because of (stupid) Linux memory over-committing strategy.
(And that's why we have hard upper limit to 1GB but it is still a lot.)

We will need to tune some switches to work it reliably and perhaps
fork process during key-derivation to have control over possible unconditional
process termination to clean memory with sensitive data.
(I hate it, but seems nothing else is reliable.)

You can workaround it by limiting memory yourself, cryptsetup limits it as
well according to physical memory, but OOM can still happen, it just depends
how other processes allocates memory.

Just be prepared for these reports. Actually, it was Fedora installer where
we saw this when tried to use LUKS2 :-)

Another thing is that I would like to see some independent analysis of Argon2
costs - do we really provide better security margin than PBKDF2 now?


Anyway, the rough plan is:

- later this week there will be stable 2.0.4 release.

The major LUKS2 fix is that libcryptsetup is now linked to libblkid
and will not automatically recover from secondary header if it detects
another (fs) signature.
(Because recovery could revert intentional change... We were too aggressive here.)
Libblkid is already used in dm/udev rules, so it should not increase
any image sizes. (And it can be disabled in configure if needed.)

As a bonus we now warn user before luksFormat if some signature is found.

Related patch for libblkid that can detect secondary LUKS2 header and properly
wipe it in wipefs is in util-linux upstream, not sure if it planned for stable,
but is should be in distro that use LUKS2 as default.
(Now only primary header is wiped and some commands,  unfortunately including isLuks,
will quietly recover from secondary header :-)


- there will be cryptsetup 2.1 release, I hope not later than end of this year (i.e. 2018 :),
where some bigger changes are planned, including OOM "fixes" mentioned above
(but no changes in LUKS2 on-disk format, maybe we just increase default header size,
but that's already supported).

Perhaps I will need to bump library version because of some dm-integrity
parameters struct change (my mistake). Only recompilation will be needed though.

I guess we can make LUKS2 default in 2.1 upstream.

For the authenticated encryption - it will remain experimental for some time still.

There are AEGIS and MORUS AEADs in 4.18 kernel that have 128bit nonces, so are
usable for our scenarios, but some other things are missing in cryptsetup code
(resize, recovery on dm-integrity metadata/journal corruption etc).

Milan
 
> Personally I'd rather *not* have such custom defaults in the installer.
> Do you have any plan to have `luksFormat` default to LUKS2 at some
> point?  If so, any idea, when that would happen? ;-)  Given the warning
> in the latest Release Notes [0] I assume LUKS2 is not mature enough for
> our installer yet.  Not sure what other distros are doing, but for
> Debian we're waiting for that scary warning to disappear (or
> alternatively, an explicit blessing from upstream) before promoting
> LUKS2 (and latter authenticated encryption — once a better AEAD
> algorithm is available) in our installer and documentation :-)
> 
> Cheers,
> 

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

Hi Milan,

On Mon, 30 Jul 2018 at 12:51:11 +0200, Milan Broz wrote:
> We already added configure switch for default format, so now it is up
> to the distro maintainer.

Right, but Debian is quite conservative in that regard, hence I
preferred to asked you to clarify your future plans ;-)
 
> I would like to change upstream default in some next version (see below),
> but we still have some issues to solve.
> […] 

Thank you very much for the explanation and the very detailed roadmap!

> The major LUKS2 fix is that libcryptsetup is now linked to libblkid
> and will not automatically recover from secondary header if it detects
> another (fs) signature.
> (Because recovery could revert intentional change... We were too aggressive here.)
> Libblkid is already used in dm/udev rules, so it should not increase
> any image sizes. (And it can be disabled in configure if needed.)

I guess to you mean 8bee1a2?  I hope 2.33 will be released upstream and
uploaded to Debian before the freeze.

Cheers,
-- 
Guilhem.

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Michael Kjörling]

On 31 Jul 2018 04:47 +0800, from guilhem@fripost.org (Guilhem Moulin):
> On Mon, 30 Jul 2018 at 12:51:11 +0200, Milan Broz wrote:
>> We already added configure switch for default format, so now it is up
>> to the distro maintainer.
> 
> Right, but Debian is quite conservative in that regard, hence I
> preferred to asked you to clarify your future plans ;-)

FWIW, I personally like Debian's conservativeness; my experience is
that it usually leads to less breakage. (It's _really_ nice to be able
to run `apt-get dist-upgrade` and generally trust that the system will
come up just fine on the next reboot.)

Obviously, there's nothing stopping whoever is installing the system
from dropping to a shell and setting up a container themselves, _even
if_ the installer _only_ does one version of on-disk format for LUKS.
So even if the installer _only_ does LUKS1, and the tools are built
with LUKS1 as default, it's not like that will _prevent_ people from
using the LUKS2 format if they really want to.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
  “The most dangerous thought that you can have as a creative person
              is to think you know what you’re doing.” (Bret Victor)

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

On Tue, 31 Jul 2018 at 06:00:30 +0000, Michael Kjörling wrote:
> Obviously, there's nothing stopping whoever is installing the system
> from dropping to a shell and setting up a container themselves, _even
> if_ the installer _only_ does one version of on-disk format for LUKS.
> So even if the installer _only_ does LUKS1, and the tools are built
> with LUKS1 as default, it's not like that will _prevent_ people from
> using the LUKS2 format if they really want to.

Sure, but now we can tell people wanting the installer to default to
LUKS2 that it'll be the new upstream default in the future, and also
give a rough ETA.  It's more efficient at appeasing them than replying
they need to drop to a shell and manually format & unlock the volume :-)

-- 
Guilhem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

On 31/07/2018 09:53, Guilhem Moulin wrote:
> On Tue, 31 Jul 2018 at 06:00:30 +0000, Michael Kjörling wrote:
>> Obviously, there's nothing stopping whoever is installing the system
>> from dropping to a shell and setting up a container themselves, _even
>> if_ the installer _only_ does one version of on-disk format for LUKS.
>> So even if the installer _only_ does LUKS1, and the tools are built
>> with LUKS1 as default, it's not like that will _prevent_ people from
>> using the LUKS2 format if they really want to.
> 
> Sure, but now we can tell people wanting the installer to default to
> LUKS2 that it'll be the new upstream default in the future, and also
> give a rough ETA.  It's more efficient at appeasing them than replying
> they need to drop to a shell and manually format & unlock the volume :-)

Hi,

just an update to LUKS2 as a default:

I had to postpone a plan to release 2.1 with LUKS2 as default format
(to January/February 2019), and we will release very soon 2.0.6 with some fixes
of LUKS2 format validation that need to be in place before we switch the default.

And the reason (long story):

The LUKS2 format supports variable sizes of metadata and keyslot areas,
and documentation clearly defines the supported sizes.

Cryptsetup uses validation functions that should stop reading/writing invalid header
from disk (to hit not only coding mistakes but also intentional header corruptions).

Unfortunately, we kept too strict validation in code by mistake so only default
LUKS2 header size is recognized as a valid header now.

Currently only these default headers are present (both conversion and format
create only the default size), but in 2.1 we will provide an interface to
use different LUKS2 header sizes.
And these headers will be not usable with cryptsetup older than 2.0.6.

(Larger metadata areas are requested by some other projects that plan to use
LUKS2 header for storing own metadata used for unlocking LUKS2 devices.)

IOW the format is ok. We just messed up tests and validation code. Sorry about that.

Milan

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

Hi Milan,

On Fri, 23 Nov 2018 at 10:21:40 +0100, Milan Broz wrote:
> just an update to LUKS2 as a default:

Thanks for the update!  Was about to poke this thread :-)

> I had to postpone a plan to release 2.1 with LUKS2 as default format
> (to January/February 2019), and we will release very soon 2.0.6 with some fixes
> of LUKS2 format validation that need to be in place before we switch the default.

I see, for Debian Buster another blocker right now is the fixed libblkid
from util-linux 2.33, although I've been told it's likely to make it for
Buster (it's in experimental right now).

FWIW, the release dates for Buster are the following [0]:

    2019-01-12  Transition freeze
    2019-02-12  Soft freeze (no new packages, no re-entry, 10-day migrations)
    2019-03-12  Full freeze

So if 2.1 doesn't trigger a transition due to a SONAME bump, releasing
it in February gives enough time to ship it to Buster.  (Strictly
speaking the deadline is March 02, assuming no RC bug is filed at the
last minute.)  I guess I'll poke this thread in early February to see
where we stand.

Cheers,
-- 
Guilhem.

[0] https://release.debian.org/

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

On 23/11/2018 19:26, Guilhem Moulin wrote:
> So if 2.1 doesn't trigger a transition due to a SONAME bump, releasing
> it in February gives enough time to ship it to Buster.

ok, I forget to say this: we are trying to go the way without SONAME bump now.
(IOW only adding new symbols but stay backward compatible.)

It will keep some more old API functions/symbols in place, but it allows update
without full rebuild.

Thanks,
Milan

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

Hi Milan,

On Fri, 23 Nov 2018 at 10:21:40 +0100, Milan Broz wrote:
> I had to postpone a plan to release 2.1 with LUKS2 as default format
> (to January/February 2019), and we will release very soon 2.0.6 with
> some fixes of LUKS2 format validation that need to be in place before
> we switch the default.

With the full Freeze approaching we (Debian package maintainers) are
often asked about LUKS2 status; so I wonder if the plan is still to
release 2.1 before the end of February? :-)

Thanks,
cheers,
-- 
Guilhem.

PS. Also, if you or another cryptsetup upstream will be back to FOSDEM
    this year, I'd love to say hi! ;-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

Hi,

On 25/01/2019 14:54, Guilhem Moulin wrote:
> On Fri, 23 Nov 2018 at 10:21:40 +0100, Milan Broz wrote:
>> I had to postpone a plan to release 2.1 with LUKS2 as default format
>> (to January/February 2019), and we will release very soon 2.0.6 with
>> some fixes of LUKS2 format validation that need to be in place before
>> we switch the default.
> 
> With the full Freeze approaching we (Debian package maintainers) are
> often asked about LUKS2 status; so I wonder if the plan is still to
> release 2.1 before the end of February? :-)

I planned to send some heads-up mail this week. It is a busy week here :)
Sorry.

In short, this is the plan and major changes (I'll write more details later)

- 2.1 should be released 5.2.2019

(Definitely not later, some slight change that it happens earlier.
I am traveling days before it, and we need some time for translators.
But git will be almost ready for 2.1 next week.)

- no soname bump, just compatible version increase (only added symbols)

- LUKS2 becomes the default format

- There are new options to fine-tune LUKS2 header size
(metadata and keyslots area, possible different per-keyslot encryption).

- we switch to OpenSSL as a default cryptographic backend
  (you can still use gcrypt with configure-time option)

- Some small hardening in cipher parameters and PBKDF

- new refresh command to allow parameter changes for the active device
  (for example it allows toggle TRIM support for root device)

- Authenticated encryption stays as an experimental feature
only with a few supported algorithms (with plans to work on it in 2.2).

If you rely on some stable release earlier, let me know, please.

Thanks,
Milan

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

On Fri, 25 Jan 2019 at 15:31:38 +0100, Milan Broz wrote:
> In short, this is the plan and major changes (I'll write more details later)
> 
> - 2.1 should be released 5.2.2019
> 
> (Definitely not later, some slight change that it happens earlier.
> I am traveling days before it, and we need some time for translators.
> But git will be almost ready for 2.1 next week.)
>
> - no soname bump, just compatible version increase (only added symbols)
> […] 
> If you rely on some stable release earlier, let me know, please.

Excellent!  So if we upload to sid later that day that gives us a full 5
weeks transition period for Buster.  Given the significant list of major
changes we'd prefer not to upload at the very last minute, but 5 weeks
is plenty: as far as Debian is concerned there is no need to rush the
upstream release.

Thanks for the update!
-- 
Guilhem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

On 25/01/2019 15:31, Milan Broz wrote:
> 
> - 2.1 should be released 5.2.2019
> 
> (Definitely not later, some slight change that it happens earlier.

And this always happens when an exact release date is announced,
unexpected problems appears :-)

So 2.1 will happen later this week, I see some strange failures with the recent
kernel we need to sort out.

Milan

Re: [dm-crypt] Debian installer formatting LUKS2 devices by default?

[Milan Broz]

On 30/07/18 22:47, Guilhem Moulin wrote:

>> The major LUKS2 fix is that libcryptsetup is now linked to libblkid
>> and will not automatically recover from secondary header if it detects
>> another (fs) signature.
>> (Because recovery could revert intentional change... We were too aggressive here.)
>> Libblkid is already used in dm/udev rules, so it should not increase
>> any image sizes. (And it can be disabled in configure if needed.)
> 
> I guess to you mean 8bee1a2?  I hope 2.33 will be released upstream and
> uploaded to Debian before the freeze.

Yes. It will be part of util-linux 2.33 and perhaps even stable 2.32.2.

Milan