[dm-crypt] [ANNOUNCE] cryptsetup 2.0.4

[dm-crypt] [ANNOUNCE] cryptsetup 2.0.4

[Milan Broz]

[-- Attachment #1.1: Type: text/plain, Size: 5141 bytes --]

Cryptsetup 2.0.4 Release Notes
==============================
Stable bug-fix release with new features.

Cryptsetup 2.x version introduces a new on-disk LUKS2 format.

The legacy LUKS (referenced as LUKS1) will be fully supported
forever as well as a traditional and fully backward compatible format.

Please note that authenticated disk encryption, non-cryptographic
data integrity protection (dm-integrity), use of Argon2 Password-Based
Key Derivation Function and the LUKS2 on-disk format itself are new
features and can contain some bugs.

To provide all security features of authenticated encryption, we need
a better nonce-reuse resistant algorithm in the kernel (see note below).
For now, please use authenticated encryption as an experimental feature.

Please do not use LUKS2 without properly configured backup or in
production systems that need to be compatible with older systems.

Changes since version 2.0.3
~~~~~~~~~~~~~~~~~~~~~~~~~~~

* Use the libblkid (blockid) library to detect foreign signatures
  on a device before LUKS format and LUKS2 auto-recovery.

  This change fixes an unexpected recovery using the secondary
  LUKS2 header after a device was already overwritten with
  another format (filesystem or LVM physical volume).

  LUKS2 will not recreate a primary header if it detects a valid
  foreign signature. In this situation, a user must always
  use cryptsetup repair command for the recovery.

  Note that libcryptsetup and utilities are now linked to libblkid
  as a new dependence.

  To compile code without blockid support (strongly discouraged),
  use --disable-blkid configure switch.

* Add prompt for format and repair actions in cryptsetup and
  integritysetup if foreign signatures are detected on the device
  through the blockid library.

  After the confirmation, all known signatures are then wiped as
  part of the format or repair procedure.

* Print consistent verbose message about keyslot and token numbers.
  For keyslot actions: Key slot <number> unlocked/created/removed.
  For token actions: Token <number> created/removed.

* Print error, if a non-existent token is tried to be removed.

* Add support for LUKS2 token definition export and import.

  The token command now can export/import customized token JSON file
  directly from command line. See the man page for more details.

* Add support for new dm-integrity superblock version 2.

* Add an error message when nothing was read from a key file.

* Update cryptsetup man pages, including --type option usage.

* Add a snapshot of LUKS2 format specification to documentation
  and accordingly fix supported secondary header offsets.

* Add bundled optimized Argon2 SSE (X86_64 platform) code.

  If the bundled Argon2 code is used and the new configure switch
  --enable-internal-sse-argon2 option is present, and compiler flags
  support required optimization, the code will try to use optimized
  and faster variant.

  Always use the shared library (--enable-libargon2) if possible.

  This option was added because an enterprise distribution
  rejected to support the shared Argon2 library and native support
  in generic cryptographic libraries is not ready yet.

* Fix compilation with crypto backend for LibreSSL >= 2.7.0.
  LibreSSL introduced OpenSSL 1.1.x API functions, so compatibility
  wrapper must be commented out.

* Fix on-disk header size calculation for LUKS2 format if a specific
  data alignment is requested. Until now, the code used default size
  that could be wrong for converted devices.

Unfinished things & TODO for next releases
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Authenticated encryption will use new algorithms from CAESAR competition
  https://competitions.cr.yp.to/caesar-submissions.html.
  We plan to use AEGIS and MORUS (in kernel 4.18), as CAESAR finalists.

  NOTE: Currently available authenticated modes (GCM, Chacha20-poly1305)
  in the kernel have too small 96-bit nonces that are problematic with
  randomly generated IVs (the collision probability is not negligible).

  For more info about LUKS2 authenticated encryption, please see our paper
  https://arxiv.org/abs/1807.00309

* Authenticated encryption do not set encryption for a dm-integrity journal.

  While it does not influence data confidentiality or integrity protection,
  an attacker can get some more information from data journal or cause that
  system will corrupt sectors after journal replay. (That corruption will be
  detected though.)

* There are examples of user-defined tokens inside misc/luks2_keyslot_example
  directory (like a simple external program that uses libssh to unlock LUKS2
  using remote keyfile).

* The python binding (pycryptsetup) contains only basic functionality for LUKS1
  (it is not updated for new features) and will be REMOVED in version 2.1
  in favor of python bindings to the libblockdev library.
  See https://github.com/storaged-project/libblockdev/releases that
  already supports LUKS2 and VeraCrypt devices handling through libcryptsetup.




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Test suite is now FS-dependent (Re: cryptsetup 2.0.4)

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 1300 bytes --]

Hi,

FWIW, ‘tests/blockwise-compat’ doesn't work consistently on all file
systems.  With

    BSIZE=$(stat -c "%o" $LOCAL_FILE)  ## 4096

`./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))` fails with ext4
(on read()) and tmpfs (on open(O_DIRECT)), but succeeds with btrfs (and
possibly other file systems).  The test-suite doesn't pass if that test
succeeds.

    ## ext4
    $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
    […]
    openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = 3
    read(3, 0x55914c895000, 4095)           = -1 EINVAL (Invalid argument)
    +++ exited with 1 +++

    ## tmpfs
    $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
    […]
    openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = -1 EINVAL (Invalid argument)
    Failed to open ./blocwise_localfile.
    +++ exited with 1 +++

    ## btrfs
    $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
    […]
    openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = 3
    read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4095) = 4095
    +++ exited with 0 +++

Just so you know :-)
Cheers,
-- 
Guilhem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Test suite is now FS-dependent (Re: cryptsetup 2.0.4)

[Milan Broz]

On 04/08/18 22:40, Guilhem Moulin wrote:
> Hi,
> 
> FWIW, ‘tests/blockwise-compat’ doesn't work consistently on all file
> systems.  With
> 
>     BSIZE=$(stat -c "%o" $LOCAL_FILE)  ## 4096
> 
> `./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))` fails with ext4
> (on read()) and tmpfs (on open(O_DIRECT)), but succeeds with btrfs (and
> possibly other file systems).  The test-suite doesn't pass if that test
> succeeds.
> 
>     ## ext4
>     $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
>     […]
>     openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = 3
>     read(3, 0x55914c895000, 4095)           = -1 EINVAL (Invalid argument)
>     +++ exited with 1 +++
> 
>     ## tmpfs
>     $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
>     […]
>     openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = -1 EINVAL (Invalid argument)
>     Failed to open ./blocwise_localfile.
>     +++ exited with 1 +++
> 
>     ## btrfs
>     $ strace -e trace=openat,read ./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))
>     […]
>     openat(AT_FDCWD, "./blocwise_localfile", O_RDONLY|O_DIRECT) = 3
>     read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4095) = 4095
>     +++ exited with 0 +++

Ah, btrfs. Until now it was nfs nuances that breaks everything :-)

(Direct-io should not be able to read buffer not aligned to block/page boundary,
but btrfs has apparently disagrees.)

Thanks for the report, we will fix it. Can you please send us (or create issue) report
including kernel version? (Just any cryptsetup command with --debug on the same system is ok.)

This is a new unit test, most of corner case scenarios are not used internally.
I guess it is just another special case we did not know about.

Milan

Re: [dm-crypt] Test suite is now FS-dependent (Re: cryptsetup 2.0.4)

[Ondrej Kozina]

On 08/05/2018 10:18 AM, Milan Broz wrote:
> 
> On 04/08/18 22:40, Guilhem Moulin wrote:
>> Hi,
>>
>> FWIW, ‘tests/blockwise-compat’ doesn't work consistently on all file
>> systems.  With
>>
>>      BSIZE=$(stat -c "%o" $LOCAL_FILE)  ## 4096
>>
>> `./unit-utils-io $LOCAL_FILE read_buffer $((BSIZE-1))` fails with ext4
>> (on read()) and tmpfs (on open(O_DIRECT)), but succeeds with btrfs (and
>> possibly other file systems).  The test-suite doesn't pass if that test
>> succeeds.

We'll add some less pedantic mode for tests related to misaligned direct 
I/O on filesystems.

> 
> Ah, btrfs. Until now it was nfs nuances that breaks everything :-)
> 
> (Direct-io should not be able to read buffer not aligned to block/page boundary,
> but btrfs has apparently disagrees.)

Not only misaligned read. It passes even for misaligned direct write! 
I'm getting old for this... :)

Anyway, thank you for the report!
O.

Re: [dm-crypt] Test suite is now FS-dependent

[Guilhem Moulin]

[-- Attachment #1: Type: text/plain, Size: 303 bytes --]

On Sun, 05 Aug 2018 at 10:18:41 +0200, Milan Broz wrote:
> Thanks for the report, we will fix it. Can you please send us (or
> create issue) report including kernel version? (Just any cryptsetup
> command with --debug on the same system is ok.)

Sure, I just filed #403.  Thanks!

-- 
Guilhem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Test suite is now FS-dependent

[Ondrej Kozina]

On 08/07/2018 05:13 PM, Guilhem Moulin wrote:
> On Sun, 05 Aug 2018 at 10:18:41 +0200, Milan Broz wrote:
>> Thanks for the report, we will fix it. Can you please send us (or
>> create issue) report including kernel version? (Just any cryptsetup
>> command with --debug on the same system is ok.)
> 
> Sure, I just filed #403.  Thanks!
> 

Hi,

I've closed the bug since we don't consider it to be a bug in the test 
(or cryptsetup). It's just how btrfs (and possibly some other fs) work. 
So, some basic explanation I failed to deliver yesterday on the list:

When userspace pass misaligned i/o with O_DIRECT flag, btrfs silently 
switches to buffered i/o instead. It happens for both offset and length 
misalignment. Also, operation may be system page size misaligned or 
physical sector size misaligned wrt to underlying block device. This has 
some implications (i.e. write only operation become silently 
read-modify-write and so on...).

Discussing it with fs kernel engineer, it's unfortunately not an easy 
fix for btrfs and they're aware of it.

 From our perspective this is (at least) grey zone and probably it would 
be better to fail such i/o passed from userspace. Userspace would be 
better notified it did something wrong, but well I guess that's a life 
with modern fs:)

O.