[dm-crypt] Missing keyslot or broken header or still some hope?

[dm-crypt] Missing keyslot or broken header or still some hope?

[Zero Tonin]

[-- Attachment #1: Type: text/plain, Size: 4597 bytes --]

Hi all on this list,

after reading the faq, I suppose I am out of luck and "one of those cases", but I will take the liberty to ask for help still, before I format my luks drive...


my fully LUKS encrypted disk is failing to decrypt since two days ago. I am 100% confident the password is entered correctly,  yet I get "no key with this passphrase available".

Previously, Debian (8) was acting up on the last proper boot where I could still decrypt the drive (changed wallpaper, keyboard strokes incorrect, so when I  typed >l< the result was >sl< and such, pressing T would open a new terminal, Q opened up some KDE specific settings …)

I thus restarted the laptop and the issue started directly after that reboot, when using the internal keyboard as well as on multiple USB keyboards on multiple USB ports.

I booted into a live usb from debian 8.6 and try to unlock the disk as follows:

user@debian:~$ sudo apt-get install cryptsetup lvm2
cryptsetup: WARNING: failed to detect canonical device of aufs
cryptsetup: WARNING: could not determine root device from /etc/fstab
Warning: /sbin/fsck.aufs doesn't exist, can't install to initramfs, ignoring.
live-boot: core filesystems devices utils udev wget blockdev.
user@debian:~$ sudo modprobe dm-crypt
user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
Enter passphrase for /dev/sda5:
No key available with this passphrase.

Again, my confidence into the passphrase is 100% (I verified keyboard layout in some random text file, I  thus can rule out typos and layout).

I took a hex dump of the disk (sda5) as suggested in a thread with a failed partition resize (I am not familiar with hexdump at all and only add it in the hope it might prove useful):

00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00  |LUKS....aes.....|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000020  00 00 00 00 00 00 00 00  78 74 73 2d 70 6c 61 69  |........xts-plai|
00000030  6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00  |n64.............|
00000040  00 00 00 00 00 00 00 00  73 68 61 31 00 00 00 00 |........sha1....|
00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000060  00 00 00 00 00 00 00 00  00 00 10 00 00 00 00 40 |...............@|
00000070  2a e2 25 b4 eb ec 89 d5  ff 04 36 17 c4 a6 86 c1  |*.%.......6.....|
00000080  23 14 05 d2 d9 63 b5 17  38 51 c9 f7 e5 bf 87 ea  |#....c..8Q......|
00000090  56 fa a9 93 71 f1 19 0d  fe c6 51 ea d8 64 5a 3e  |V...q.....Q..dZ>|
000000a0  68 97 51 5b 00 01 38 80  34 36 36 39 33 66 38 34  |h.Q[..8.46693f84|
000000b0  2d 65 64 63 66 2d 34 66  66 39 2d 38 39 64 66 2d  |-edcf-4ff9-89df-|
000000c0  37 38 64 36 32 61 39 32  62 36 66 33 00 00 00 00  |78d62a92b6f3....|
000000d0  00 ac 71 f3 00 05 38 e5  72 3c b6 82 b3 33 a7 f6  |..q...8.r<...3..|
000000e0  5a 55 f9 3d 6b f3 8c b8  d9 6a 66 31 9e 03 b1 57  |ZU.=k....jf1...W|
000000f0  b9 bf 00 5d d7 4a dd c9  00 00 00 08 00 00 0f a0  |...].J..........|
00000100  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000120  00 00 00 00 00 00 00 00  00 00 02 00 00 00 0f a0  |................|
00000130  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000150  00 00 00 00 00 00 00 00  00 00 03 f8 00 00 0f a0  |................|
00000160  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
00000180  00 00 00 00 00 00 00 00  00 00 05 f0 00 00 0f a0  |................|
00000190  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001b0  00 00 00 00 00 00 00 00  00 00 07 e8 00 00 0f a0  |................|
000001c0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000001e0  00 00 00 00 00 00 00 00  00 00 09 e0 00 00 0f a0  |................|
000001f0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Does this look like a effed crypt header (of which I, naturally, don't have  a backup - even though I can honestly say that, as far as I know, I did not do anything to the header … ) or is there any hope left (there is no second keyslot in use)

Thanks ever so much for any word of advice,
Zero

[-- Attachment #2: Type: text/html, Size: 15700 bytes --]

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Michael Kjörling]

On 3 Nov 2016 18:30 +0000, from zero.tonin@web.de (Zero Tonin):
> user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
> Enter passphrase for /dev/sda5:
> No key available with this passphrase.

Could you try running this again, but add the `--debug` option to
cryptsetup, then post the resulting log?

Make sure to sanitize the passphrase itself from the log if it's there
(I don't know), but leave everything else intact.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Zero Tonin]

[-- Attachment #1: Type: text/plain, Size: 3831 bytes --]

Hi Michael,

thank you very much for your response, I appreciate your time and willingnes to help a stranger!
 
 Below I will paste the output of --debug a well as, in case it provides usefull information, the output of sfdisk -l for the partitions on the drive.
 
 Again, thank you ever so much, please do let me know if there is any further detail or informaion I could provide to hopefulyl be bale  to recover this.
 
 Kind rgeards,
 Mark 
(I was unaware this mailing list is a "clear name" environemt, sorry for the anonymity in my first mail)
 
 
 
user@debian:~$ sudo /sbin/sfdisk -l

Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
sfdisk: Warning: extended partition does not start at a cylinder boundary.
DOS and Linux will interpret the contents differently.
Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0

   Device Boot Start     End   #cyls    #blocks   Id  System
/dev/sda1   *      0+     31-     31-    248832   83  Linux
/dev/sda2         31+  77825-  77795- 624880641    5  Extended
/dev/sda3          0       -       0          0    0  Empty
/dev/sda4          0       -       0          0    0  Empty
/dev/sda5         31+  77825-  77795- 624880640   83  Linux


user@debian:~$ sudo cryptsetup --debug luksOpen /dev/sda5 crypt1
# cryptsetup 1.6.6 processing "cryptsetup --debug luksOpen /dev/sda5 crypt1"
# Running command open.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.
# Allocating crypt device /dev/sda5 context.
# Trying to open and read device /dev/sda5.
# Initialising device-mapper backend library.
# Trying to load LUKS1 crypt type from device /dev/sda5.
# Crypto backend (gcrypt 1.6.3) initialized.
# Detected kernel Linux 3.16.0-4-amd64 x86_64.
# Reading LUKS header of size 1024 from device /dev/sda5
# Key length 64, device size 1249761280 sectors, header size 4036 sectors.
# Timeout set to 0 miliseconds.
# Password retry count set to 3.
# Password verification disabled.
# Iteration time set to 1000 miliseconds.
# Activating volume crypt1 [keyslot -1] using [none] passphrase.
# dm version   OF   [16384] (*1)
# dm versions   OF   [16384] (*1)
# Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
# Device-mapper backend running with UDEV support enabled.
# dm status crypt1  OF   [16384] (*1)
# Interactive passphrase entry requested.
Enter passphrase for /dev/sda5: 
# Trying to open key slot 0 [ACTIVE_LAST].
# Reading key slot 0 area.
# Using userspace crypto wrapper to access keyslot area.
# Trying to open key slot 1 [INACTIVE].
# Trying to open key slot 2 [INACTIVE].
# Trying to open key slot 3 [INACTIVE].
# Trying to open key slot 4 [INACTIVE].
# Trying to open key slot 5 [INACTIVE].
# Trying to open key slot 6 [INACTIVE].
# Trying to open key slot 7 [INACTIVE].
No key available with this passphrase.

> On 3 Nov 2016, at 19:04, Michael Kjörling <michael@kjorling.se> wrote:
> 
> On 3 Nov 2016 18:30 +0000, from zero.tonin@web.de (Zero Tonin):
>> user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
>> Enter passphrase for /dev/sda5:
>> No key available with this passphrase.
> 
> Could you try running this again, but add the `--debug` option to
> cryptsetup, then post the resulting log?
> 
> Make sure to sanitize the passphrase itself from the log if it's there
> (I don't know), but leave everything else intact.
> 
> -- 
> Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
>                 “People who think they know everything really annoy
>                 those of us who know we don’t.” (Bjarne Stroustrup)
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

[-- Attachment #2: Type: text/html, Size: 13940 bytes --]

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Arno Wagner]

Hi,

first, please do not post HTML-'emails' to this list.
It cuts you off from most people here.

Second, from the 'acting up' I would deduce that you
have some kind of severe hardware problem. It may be that
this prevents the unlock. Can you try this disk in a 
different computer?

There is also the keyslot-checker in misc/keyslot_checker/
of the cryptsetup source distribution, that may tell
you more.

Regards,
Arno


On Thu, Nov 03, 2016 at 21:58:30 CET, Zero Tonin wrote:
>    Hi Michael,
> 
>    thank you very much for your response, I appreciate your time and
>    willingnes to help a stranger!
> 
> 
>     Below I will paste the output of --debug a well as, in case it
>    provides usefull information, the output of sfdisk -l for the
>    partitions on the drive.
> 
> 
>     Again, thank you ever so much, please do let me know if there is any
>    further detail or informaion I could provide to hopefulyl be bale  to
>    recover this.
> 
> 
>     Kind rgeards,
> 
>     Mark
> 
>    (I was unaware this mailing list is a "clear name" environemt, sorry
>    for the anonymity in my first mail)
> 
> 
> 
> 
>    user@debian:~$ sudo /sbin/sfdisk -l
> 
>    Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
> 
>    sfdisk: Warning: extended partition does not start at a cylinder
>    boundary.
> 
>    DOS and Linux will interpret the contents differently.
> 
>    Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from
>    0
> 
>       Device Boot Start     End   #cyls    #blocks   Id  System
> 
>    /dev/sda1   *      0+     31-     31-    248832   83  Linux
> 
>    /dev/sda2         31+  77825-  77795- 624880641    5  Extended
> 
>    /dev/sda3          0       -       0          0    0  Empty
> 
>    /dev/sda4          0       -       0          0    0  Empty
> 
>    /dev/sda5         31+  77825-  77795- 624880640   83  Linux
> 
>    user@debian:~$ sudo cryptsetup --debug luksOpen /dev/sda5 crypt1
> 
>    # cryptsetup 1.6.6 processing "cryptsetup --debug luksOpen /dev/sda5
>    crypt1"
> 
>    # Running command open.
> 
>    # Locking memory.
> 
>    # Installing SIGINT/SIGTERM handler.
> 
>    # Unblocking interruption on signal.
> 
>    # Allocating crypt device /dev/sda5 context.
> 
>    # Trying to open and read device /dev/sda5.
> 
>    # Initialising device-mapper backend library.
> 
>    # Trying to load LUKS1 crypt type from device /dev/sda5.
> 
>    # Crypto backend (gcrypt 1.6.3) initialized.
> 
>    # Detected kernel Linux 3.16.0-4-amd64 x86_64.
> 
>    # Reading LUKS header of size 1024 from device /dev/sda5
> 
>    # Key length 64, device size 1249761280 sectors, header size 4036
>    sectors.
> 
>    # Timeout set to 0 miliseconds.
> 
>    # Password retry count set to 3.
> 
>    # Password verification disabled.
> 
>    # Iteration time set to 1000 miliseconds.
> 
>    # Activating volume crypt1 [keyslot -1] using [none] passphrase.
> 
>    # dm version   OF   [16384] (*1)
> 
>    # dm versions   OF   [16384] (*1)
> 
>    # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
> 
>    # Device-mapper backend running with UDEV support enabled.
> 
>    # dm status crypt1  OF   [16384] (*1)
> 
>    # Interactive passphrase entry requested.
> 
>    Enter passphrase for /dev/sda5:
> 
>    # Trying to open key slot 0 [ACTIVE_LAST].
> 
>    # Reading key slot 0 area.
> 
>    # Using userspace crypto wrapper to access keyslot area.
> 
>    # Trying to open key slot 1 [INACTIVE].
> 
>    # Trying to open key slot 2 [INACTIVE].
> 
>    # Trying to open key slot 3 [INACTIVE].
> 
>    # Trying to open key slot 4 [INACTIVE].
> 
>    # Trying to open key slot 5 [INACTIVE].
> 
>    # Trying to open key slot 6 [INACTIVE].
> 
>    # Trying to open key slot 7 [INACTIVE].
> 
>    No key available with this passphrase.
> 
>    On 3 Nov 2016, at 19:04, Michael Kjörling <[1]michael@kjorling.se>
>    wrote:
> 
>    On 3 Nov 2016 18:30 +0000, from [2]zero.tonin@web.de (Zero Tonin):
> 
>      user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
> 
>      Enter passphrase for /dev/sda5:
> 
>      No key available with this passphrase.
> 
>    Could you try running this again, but add the `--debug` option to
>    cryptsetup, then post the resulting log?
>    Make sure to sanitize the passphrase itself from the log if it's there
>    (I don't know), but leave everything else intact.
>    --
>    Michael Kjörling • [3]https://michael.kjorling.se •
>    [4]michael@kjorling.se
>                    “People who think they know everything really annoy
>                    those of us who know we don’t.” (Bjarne Stroustrup)
>    _______________________________________________
>    dm-crypt mailing list
>    [5]dm-crypt@saout.de
>    [6]http://www.saout.de/mailman/listinfo/dm-crypt
> 
> References
> 
>    1. mailto:michael@kjorling.se
>    2. mailto:zero.tonin@web.de
>    3. https://michael.kjorling.se/
>    4. mailto:michael@kjorling.se
>    5. mailto:dm-crypt@saout.de
>    6. http://www.saout.de/mailman/listinfo/dm-crypt

> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[zero.tonin]

Hi all, and hi Arno,
first of all, sorry the html "emails" - I don't usually do this and usually use plain-text only myself. The last mails were, however, in this emergency situation, sent from my phone, where I cannot change this behavior, unfortunately...

After fighting a little bit with cryptsetup (i must have missed some information which packages are required to compile from source), I did get the keyslot checker to work.
Unfortunately, the output is obscure to me, so I home someone can help me interpret this.

I suspected a hw issue and thus, at least, ran the vendor's diagnostic tools, but no issue could be found, including memory and HDD - would it more likely be something related to the disk itself (bad sectors, broken read-heads et cetera?)

Great idea to test the drive on a different machine - would a dd copy suffice for that, as I am afraid I do not posses the skills to take my laptop apart. not as long as there might be hope to rescue stuff otherwise. I would do this as a last resort, if the hw is broken o a degree anyway, of course.

Thanks again for your time and efforts, everybody,
Mark

user@debian:~/.bin/cryptsetup/misc/keyslot_checker$ sudo ./chk_luks_keyslots -v /dev/sda5

parameters (commandline and LUKS header):
  sector size: 512
  threshold:   0.900000

- processing keyslot 0:  start: 0x001000   end: 0x03f800 
- processing keyslot 1:  keyslot not in use
- processing keyslot 2:  keyslot not in use
- processing keyslot 3:  keyslot not in use
- processing keyslot 4:  keyslot not in use
- processing keyslot 5:  keyslot not in use
- processing keyslot 6:  keyslot not in use
- processing keyslot 7:  keyslot not in use


> Gesendet: Freitag, 04. November 2016 um 11:32 Uhr
> Von: "Arno Wagner" <arno@wagner.name>
> An: dm-crypt@saout.de
> Betreff: Re: [dm-crypt] Missing keyslot or broken header or still some hope?
>
> Hi,
> 
> first, please do not post HTML-'emails' to this list.
> It cuts you off from most people here.
> 
> Second, from the 'acting up' I would deduce that you
> have some kind of severe hardware problem. It may be that
> this prevents the unlock. Can you try this disk in a 
> different computer?
> 
> There is also the keyslot-checker in misc/keyslot_checker/
> of the cryptsetup source distribution, that may tell
> you more.
> 
> Regards,
> Arno
> 
> 
> On Thu, Nov 03, 2016 at 21:58:30 CET, Zero Tonin wrote:
> >    Hi Michael,
> > 
> >    thank you very much for your response, I appreciate your time and
> >    willingnes to help a stranger!
> > 
> > 
> >     Below I will paste the output of --debug a well as, in case it
> >    provides usefull information, the output of sfdisk -l for the
> >    partitions on the drive.
> > 
> > 
> >     Again, thank you ever so much, please do let me know if there is any
> >    further detail or informaion I could provide to hopefulyl be bale  to
> >    recover this.
> > 
> > 
> >     Kind rgeards,
> > 
> >     Mark
> > 
> >    (I was unaware this mailing list is a "clear name" environemt, sorry
> >    for the anonymity in my first mail)
> > 
> > 
> > 
> > 
> >    user@debian:~$ sudo /sbin/sfdisk -l
> > 
> >    Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
> > 
> >    sfdisk: Warning: extended partition does not start at a cylinder
> >    boundary.
> > 
> >    DOS and Linux will interpret the contents differently.
> > 
> >    Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from
> >    0
> > 
> >       Device Boot Start     End   #cyls    #blocks   Id  System
> > 
> >    /dev/sda1   *      0+     31-     31-    248832   83  Linux
> > 
> >    /dev/sda2         31+  77825-  77795- 624880641    5  Extended
> > 
> >    /dev/sda3          0       -       0          0    0  Empty
> > 
> >    /dev/sda4          0       -       0          0    0  Empty
> > 
> >    /dev/sda5         31+  77825-  77795- 624880640   83  Linux
> > 
> >    user@debian:~$ sudo cryptsetup --debug luksOpen /dev/sda5 crypt1
> > 
> >    # cryptsetup 1.6.6 processing "cryptsetup --debug luksOpen /dev/sda5
> >    crypt1"
> > 
> >    # Running command open.
> > 
> >    # Locking memory.
> > 
> >    # Installing SIGINT/SIGTERM handler.
> > 
> >    # Unblocking interruption on signal.
> > 
> >    # Allocating crypt device /dev/sda5 context.
> > 
> >    # Trying to open and read device /dev/sda5.
> > 
> >    # Initialising device-mapper backend library.
> > 
> >    # Trying to load LUKS1 crypt type from device /dev/sda5.
> > 
> >    # Crypto backend (gcrypt 1.6.3) initialized.
> > 
> >    # Detected kernel Linux 3.16.0-4-amd64 x86_64.
> > 
> >    # Reading LUKS header of size 1024 from device /dev/sda5
> > 
> >    # Key length 64, device size 1249761280 sectors, header size 4036
> >    sectors.
> > 
> >    # Timeout set to 0 miliseconds.
> > 
> >    # Password retry count set to 3.
> > 
> >    # Password verification disabled.
> > 
> >    # Iteration time set to 1000 miliseconds.
> > 
> >    # Activating volume crypt1 [keyslot -1] using [none] passphrase.
> > 
> >    # dm version   OF   [16384] (*1)
> > 
> >    # dm versions   OF   [16384] (*1)
> > 
> >    # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
> > 
> >    # Device-mapper backend running with UDEV support enabled.
> > 
> >    # dm status crypt1  OF   [16384] (*1)
> > 
> >    # Interactive passphrase entry requested.
> > 
> >    Enter passphrase for /dev/sda5:
> > 
> >    # Trying to open key slot 0 [ACTIVE_LAST].
> > 
> >    # Reading key slot 0 area.
> > 
> >    # Using userspace crypto wrapper to access keyslot area.
> > 
> >    # Trying to open key slot 1 [INACTIVE].
> > 
> >    # Trying to open key slot 2 [INACTIVE].
> > 
> >    # Trying to open key slot 3 [INACTIVE].
> > 
> >    # Trying to open key slot 4 [INACTIVE].
> > 
> >    # Trying to open key slot 5 [INACTIVE].
> > 
> >    # Trying to open key slot 6 [INACTIVE].
> > 
> >    # Trying to open key slot 7 [INACTIVE].
> > 
> >    No key available with this passphrase.
> > 
> >    On 3 Nov 2016, at 19:04, Michael Kjörling <[1]michael@kjorling.se>
> >    wrote:
> > 
> >    On 3 Nov 2016 18:30 +0000, from [2]zero.tonin@web.de (Zero Tonin):
> > 
> >      user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
> > 
> >      Enter passphrase for /dev/sda5:
> > 
> >      No key available with this passphrase.
> > 
> >    Could you try running this again, but add the `--debug` option to
> >    cryptsetup, then post the resulting log?
> >    Make sure to sanitize the passphrase itself from the log if it's there
> >    (I don't know), but leave everything else intact.
> >    --
> >    Michael Kjörling • [3]https://michael.kjorling.se •
> >    [4]michael@kjorling.se
> >                    “People who think they know everything really annoy
> >                    those of us who know we don’t.” (Bjarne Stroustrup)
> >    _______________________________________________
> >    dm-crypt mailing list
> >    [5]dm-crypt@saout.de
> >    [6]http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> > References
> > 
> >    1. mailto:michael@kjorling.se
> >    2. mailto:zero.tonin@web.de
> >    3. https://michael.kjorling.se/
> >    4. mailto:michael@kjorling.se
> >    5. mailto:dm-crypt@saout.de
> >    6. http://www.saout.de/mailman/listinfo/dm-crypt
> 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> 
> 
> -- 
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -- Plato
> 
> If it's in the news, don't worry about it.  The very definition of 
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Michael Kjörling]

On 4 Nov 2016 20:35 +0100, from zero.tonin@web.de:
> I suspected a hw issue and thus, at least, ran the vendor's
> diagnostic tools, but no issue could be found, including memory and
> HDD - would it more likely be something related to the disk itself
> (bad sectors, broken read-heads et cetera?)

My first assumption would not be that the disk is physically broken
yet still manages to read data in any meaningful way, but silent data
corruption is a real thing, despite HDD manufacturers' attempts at
correcting or at least detecting any failed reads. That said, though,
your LUKS header looks _sane_; I would expect silent corruption to
yield essentially random data for the full sector.


> Great idea to test the drive on a different machine - would a dd
> copy suffice for that, as I am afraid I do not posses the skills to
> take my laptop apart.

A binary copy as made by e.g. dd should absolutely be sufficient. In
fact, it's probably a good idea to make such a copy in any case;
having that copy will allow you to experiment.

If you can spare the disk space, make one copy, and then duplicate it,
then work on one of those copies while making sure to not touch the
other; that way, no matter what you do and no matter what happens to
the physical media from that point onwards, you can always go back to
the original copy and make a new working copy.

I _strongly_ recommend ddrescue over dd; ddrescue is far better suited
for this use case. It also gives you a nice progress indication while
it is working.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Arno Wagner]

Hi Mark,

On Fri, Nov 04, 2016 at 20:35:32 CET, zero.tonin@web.de wrote:
> Hi all, and hi Arno,
>

> first of all, sorry the html "emails" - I don't usually do this and
> usually use plain-text only myself.  The last mails were, however, in this
> emergency situation, sent from my phone, where I cannot change this
> behavior, unfortunately...

Understandable. No harm done.

> 
> After fighting a little bit with cryptsetup (i must have missed some
> information which packages are required to compile from source), I did get
> the keyslot checker to work.  Unfortunately, the output is obscure to me,
> so I home someone can help me interpret this.

It says your key-slots have no larger areas overwritten with other data.
That is by far the most common thing that happens. Not here, it seems.
 
> I suspected a hw issue and thus, at least, ran the vendor's diagnostic
> tools, but no issue could be found, including memory and HDD - would it
> more likely be something related to the disk itself (bad sectors, broken
> read-heads et cetera?)

No idea. Maybe bad buffer-memory on the disk or something like
it.
 
> Great idea to test the drive on a different machine - would a dd copy
> suffice for that, as I am afraid I do not posses the skills to take my
> laptop apart.  not as long as there might be hope to rescue stuff
> otherwise.  I would do this as a last resort, if the hw is broken o a
> degree anyway, of course.

In principle, yes, but if you have a problem with bit-errors on
reading or the like, then you would at least need to also 
do an md5sum or the like of copy and original to make
sure there are no errors. A single bit-error in a 
keyslot makes it unusable.

> Thanks again for your time and efforts, everybody,

No problem.

Regards,
Arno

> Mark
> 
> user@debian:~/.bin/cryptsetup/misc/keyslot_checker$ sudo ./chk_luks_keyslots -v /dev/sda5
> 
> parameters (commandline and LUKS header):
>   sector size: 512
>   threshold:   0.900000
> 
> - processing keyslot 0:  start: 0x001000   end: 0x03f800 
> - processing keyslot 1:  keyslot not in use
> - processing keyslot 2:  keyslot not in use
> - processing keyslot 3:  keyslot not in use
> - processing keyslot 4:  keyslot not in use
> - processing keyslot 5:  keyslot not in use
> - processing keyslot 6:  keyslot not in use
> - processing keyslot 7:  keyslot not in use
> 
> 
> > Gesendet: Freitag, 04. November 2016 um 11:32 Uhr
> > Von: "Arno Wagner" <arno@wagner.name>
> > An: dm-crypt@saout.de
> > Betreff: Re: [dm-crypt] Missing keyslot or broken header or still some hope?
> >
> > Hi,
> > 
> > first, please do not post HTML-'emails' to this list.
> > It cuts you off from most people here.
> > 
> > Second, from the 'acting up' I would deduce that you
> > have some kind of severe hardware problem. It may be that
> > this prevents the unlock. Can you try this disk in a 
> > different computer?
> > 
> > There is also the keyslot-checker in misc/keyslot_checker/
> > of the cryptsetup source distribution, that may tell
> > you more.
> > 
> > Regards,
> > Arno
> > 
> > 
> > On Thu, Nov 03, 2016 at 21:58:30 CET, Zero Tonin wrote:
> > >    Hi Michael,
> > > 
> > >    thank you very much for your response, I appreciate your time and
> > >    willingnes to help a stranger!
> > > 
> > > 
> > >     Below I will paste the output of --debug a well as, in case it
> > >    provides usefull information, the output of sfdisk -l for the
> > >    partitions on the drive.
> > > 
> > > 
> > >     Again, thank you ever so much, please do let me know if there is any
> > >    further detail or informaion I could provide to hopefulyl be bale  to
> > >    recover this.
> > > 
> > > 
> > >     Kind rgeards,
> > > 
> > >     Mark
> > > 
> > >    (I was unaware this mailing list is a "clear name" environemt, sorry
> > >    for the anonymity in my first mail)
> > > 
> > > 
> > > 
> > > 
> > >    user@debian:~$ sudo /sbin/sfdisk -l
> > > 
> > >    Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
> > > 
> > >    sfdisk: Warning: extended partition does not start at a cylinder
> > >    boundary.
> > > 
> > >    DOS and Linux will interpret the contents differently.
> > > 
> > >    Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from
> > >    0
> > > 
> > >       Device Boot Start     End   #cyls    #blocks   Id  System
> > > 
> > >    /dev/sda1   *      0+     31-     31-    248832   83  Linux
> > > 
> > >    /dev/sda2         31+  77825-  77795- 624880641    5  Extended
> > > 
> > >    /dev/sda3          0       -       0          0    0  Empty
> > > 
> > >    /dev/sda4          0       -       0          0    0  Empty
> > > 
> > >    /dev/sda5         31+  77825-  77795- 624880640   83  Linux
> > > 
> > >    user@debian:~$ sudo cryptsetup --debug luksOpen /dev/sda5 crypt1
> > > 
> > >    # cryptsetup 1.6.6 processing "cryptsetup --debug luksOpen /dev/sda5
> > >    crypt1"
> > > 
> > >    # Running command open.
> > > 
> > >    # Locking memory.
> > > 
> > >    # Installing SIGINT/SIGTERM handler.
> > > 
> > >    # Unblocking interruption on signal.
> > > 
> > >    # Allocating crypt device /dev/sda5 context.
> > > 
> > >    # Trying to open and read device /dev/sda5.
> > > 
> > >    # Initialising device-mapper backend library.
> > > 
> > >    # Trying to load LUKS1 crypt type from device /dev/sda5.
> > > 
> > >    # Crypto backend (gcrypt 1.6.3) initialized.
> > > 
> > >    # Detected kernel Linux 3.16.0-4-amd64 x86_64.
> > > 
> > >    # Reading LUKS header of size 1024 from device /dev/sda5
> > > 
> > >    # Key length 64, device size 1249761280 sectors, header size 4036
> > >    sectors.
> > > 
> > >    # Timeout set to 0 miliseconds.
> > > 
> > >    # Password retry count set to 3.
> > > 
> > >    # Password verification disabled.
> > > 
> > >    # Iteration time set to 1000 miliseconds.
> > > 
> > >    # Activating volume crypt1 [keyslot -1] using [none] passphrase.
> > > 
> > >    # dm version   OF   [16384] (*1)
> > > 
> > >    # dm versions   OF   [16384] (*1)
> > > 
> > >    # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
> > > 
> > >    # Device-mapper backend running with UDEV support enabled.
> > > 
> > >    # dm status crypt1  OF   [16384] (*1)
> > > 
> > >    # Interactive passphrase entry requested.
> > > 
> > >    Enter passphrase for /dev/sda5:
> > > 
> > >    # Trying to open key slot 0 [ACTIVE_LAST].
> > > 
> > >    # Reading key slot 0 area.
> > > 
> > >    # Using userspace crypto wrapper to access keyslot area.
> > > 
> > >    # Trying to open key slot 1 [INACTIVE].
> > > 
> > >    # Trying to open key slot 2 [INACTIVE].
> > > 
> > >    # Trying to open key slot 3 [INACTIVE].
> > > 
> > >    # Trying to open key slot 4 [INACTIVE].
> > > 
> > >    # Trying to open key slot 5 [INACTIVE].
> > > 
> > >    # Trying to open key slot 6 [INACTIVE].
> > > 
> > >    # Trying to open key slot 7 [INACTIVE].
> > > 
> > >    No key available with this passphrase.
> > > 
> > >    On 3 Nov 2016, at 19:04, Michael Kjörling <[1]michael@kjorling.se>
> > >    wrote:
> > > 
> > >    On 3 Nov 2016 18:30 +0000, from [2]zero.tonin@web.de (Zero Tonin):
> > > 
> > >      user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
> > > 
> > >      Enter passphrase for /dev/sda5:
> > > 
> > >      No key available with this passphrase.
> > > 
> > >    Could you try running this again, but add the `--debug` option to
> > >    cryptsetup, then post the resulting log?
> > >    Make sure to sanitize the passphrase itself from the log if it's there
> > >    (I don't know), but leave everything else intact.
> > >    --
> > >    Michael Kjörling • [3]https://michael.kjorling.se •
> > >    [4]michael@kjorling.se
> > >                    “People who think they know everything really annoy
> > >                    those of us who know we don’t.” (Bjarne Stroustrup)
> > >    _______________________________________________
> > >    dm-crypt mailing list
> > >    [5]dm-crypt@saout.de
> > >    [6]http://www.saout.de/mailman/listinfo/dm-crypt
> > > 
> > > References
> > > 
> > >    1. mailto:michael@kjorling.se
> > >    2. mailto:zero.tonin@web.de
> > >    3. https://michael.kjorling.se/
> > >    4. mailto:michael@kjorling.se
> > >    5. mailto:dm-crypt@saout.de
> > >    6. http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> > 
> > -- 
> > Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> > GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> > ----
> > A good decision is based on knowledge and not on numbers. -- Plato
> > 
> > If it's in the news, don't worry about it.  The very definition of 
> > "news" is "something that hardly ever happens." -- Bruce Schneier
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[zero.tonin]

Hi all, Arno and Michael,
thanks again for your continued help and advise - a great experience and yet another time I love the linux community (and pity I can't contribute myself much).


> It says your key-slots have no larger areas overwritten with other data.
> That is by far the most common thing that happens. Not here, it seems.

Thanks for the clarification, I think I do understand.

> In principle, yes, but if you have a problem with bit-errors on
> reading or the like, then you would at least need to also 
> do an md5sum or the like of copy and original to make
> sure there are no errors. A single bit-error in a 
> keyslot makes it unusable.

Ok, that might explain why - at the moment - it is not working. I create a ddrescue (thanks, Michael, for reminding me of this!) clone but failed to realise the destination disk was 100GB short (I am looking to do it with a NAS drive now...).
With this "clone" (which might be insufficient) I tried unlocking the disk on a virtual machine running pureOS (on VM Fusion on a macOS Sierra iMac), but I as well cant decrypt the disk with the "No key available with this passphrase" message.
I hope this is due to the insufficient size on the drive.


>My first assumption would not be that the disk is physically broken
>yet still manages to read data in any meaningful way, but silent data
>corruption is a real thing, despite HDD manufacturers' attempts at
>correcting or at least detecting any failed reads. That said, though,
>your LUKS header looks _sane_; I would expect silent corruption to
>yield essentially random data for the full sector.

That, at least, gives some hope to continue working on the drive. Also a great reminder for _regular_ rsyncs (I have another disk which had been encrypted with truecrypt. A firmware update for the drive itself corrupted the truecrypt header. I did have a RAID backup, also encrypted with truecrypt. Smart as I am, the password for it is stored on the unusable disk  and I did not yet pgp -email it to someone I trust... different story, though, but maybe  it contributes to your amusement)


>A binary copy as made by e.g. dd should absolutely be sufficient. In
>fact, it's probably a good idea to make such a copy in any case;
>having that copy will allow you to experiment.

>If you can spare the disk space, make one copy, and then duplicate it,
>then work on one of those copies while making sure to not touch the
>other; that way, no matter what you do and no matter what happens to
>the physical media from that point onwards, you can always go back to
>the original copy and make a new working copy.

Very good plan of action, I wil lsee can I get ddrescue to work onto a NAS drive, which should at least gve me enough storage...

>I _strongly_ recommend ddrescue over dd; ddrescue is far better suited
>for this use case. It also gives you a nice progress indication while
>it is working.

Very true, not sure why I had "dd" saved in my head...

Again, thanks so much, folks, this is really great and I appreciate your words and time a lot!
Mark

> 
> > Mark
> > 
> > user@debian:~/.bin/cryptsetup/misc/keyslot_checker$ sudo ./chk_luks_keyslots -v /dev/sda5
> > 
> > parameters (commandline and LUKS header):
> >   sector size: 512
> >   threshold:   0.900000
> > 
> > - processing keyslot 0:  start: 0x001000   end: 0x03f800 
> > - processing keyslot 1:  keyslot not in use
> > - processing keyslot 2:  keyslot not in use
> > - processing keyslot 3:  keyslot not in use
> > - processing keyslot 4:  keyslot not in use
> > - processing keyslot 5:  keyslot not in use
> > - processing keyslot 6:  keyslot not in use
> > - processing keyslot 7:  keyslot not in use
> > 
> > 
> > > Gesendet: Freitag, 04. November 2016 um 11:32 Uhr
> > > Von: "Arno Wagner" <arno@wagner.name>
> > > An: dm-crypt@saout.de
> > > Betreff: Re: [dm-crypt] Missing keyslot or broken header or still some hope?
> > >
> > > Hi,
> > > 
> > > first, please do not post HTML-'emails' to this list.
> > > It cuts you off from most people here.
> > > 
> > > Second, from the 'acting up' I would deduce that you
> > > have some kind of severe hardware problem. It may be that
> > > this prevents the unlock. Can you try this disk in a 
> > > different computer?
> > > 
> > > There is also the keyslot-checker in misc/keyslot_checker/
> > > of the cryptsetup source distribution, that may tell
> > > you more.
> > > 
> > > Regards,
> > > Arno
> > > 
> > > 
> > > On Thu, Nov 03, 2016 at 21:58:30 CET, Zero Tonin wrote:
> > > >    Hi Michael,
> > > > 
> > > >    thank you very much for your response, I appreciate your time and
> > > >    willingnes to help a stranger!
> > > > 
> > > > 
> > > >     Below I will paste the output of --debug a well as, in case it
> > > >    provides usefull information, the output of sfdisk -l for the
> > > >    partitions on the drive.
> > > > 
> > > > 
> > > >     Again, thank you ever so much, please do let me know if there is any
> > > >    further detail or informaion I could provide to hopefulyl be bale  to
> > > >    recover this.
> > > > 
> > > > 
> > > >     Kind rgeards,
> > > > 
> > > >     Mark
> > > > 
> > > >    (I was unaware this mailing list is a "clear name" environemt, sorry
> > > >    for the anonymity in my first mail)
> > > > 
> > > > 
> > > > 
> > > > 
> > > >    user@debian:~$ sudo /sbin/sfdisk -l
> > > > 
> > > >    Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
> > > > 
> > > >    sfdisk: Warning: extended partition does not start at a cylinder
> > > >    boundary.
> > > > 
> > > >    DOS and Linux will interpret the contents differently.
> > > > 
> > > >    Units: cylinders of 8225280 bytes, blocks of 1024 bytes, counting from
> > > >    0
> > > > 
> > > >       Device Boot Start     End   #cyls    #blocks   Id  System
> > > > 
> > > >    /dev/sda1   *      0+     31-     31-    248832   83  Linux
> > > > 
> > > >    /dev/sda2         31+  77825-  77795- 624880641    5  Extended
> > > > 
> > > >    /dev/sda3          0       -       0          0    0  Empty
> > > > 
> > > >    /dev/sda4          0       -       0          0    0  Empty
> > > > 
> > > >    /dev/sda5         31+  77825-  77795- 624880640   83  Linux
> > > > 
> > > >    user@debian:~$ sudo cryptsetup --debug luksOpen /dev/sda5 crypt1
> > > > 
> > > >    # cryptsetup 1.6.6 processing "cryptsetup --debug luksOpen /dev/sda5
> > > >    crypt1"
> > > > 
> > > >    # Running command open.
> > > > 
> > > >    # Locking memory.
> > > > 
> > > >    # Installing SIGINT/SIGTERM handler.
> > > > 
> > > >    # Unblocking interruption on signal.
> > > > 
> > > >    # Allocating crypt device /dev/sda5 context.
> > > > 
> > > >    # Trying to open and read device /dev/sda5.
> > > > 
> > > >    # Initialising device-mapper backend library.
> > > > 
> > > >    # Trying to load LUKS1 crypt type from device /dev/sda5.
> > > > 
> > > >    # Crypto backend (gcrypt 1.6.3) initialized.
> > > > 
> > > >    # Detected kernel Linux 3.16.0-4-amd64 x86_64.
> > > > 
> > > >    # Reading LUKS header of size 1024 from device /dev/sda5
> > > > 
> > > >    # Key length 64, device size 1249761280 sectors, header size 4036
> > > >    sectors.
> > > > 
> > > >    # Timeout set to 0 miliseconds.
> > > > 
> > > >    # Password retry count set to 3.
> > > > 
> > > >    # Password verification disabled.
> > > > 
> > > >    # Iteration time set to 1000 miliseconds.
> > > > 
> > > >    # Activating volume crypt1 [keyslot -1] using [none] passphrase.
> > > > 
> > > >    # dm version   OF   [16384] (*1)
> > > > 
> > > >    # dm versions   OF   [16384] (*1)
> > > > 
> > > >    # Detected dm-crypt version 1.13.0, dm-ioctl version 4.27.0.
> > > > 
> > > >    # Device-mapper backend running with UDEV support enabled.
> > > > 
> > > >    # dm status crypt1  OF   [16384] (*1)
> > > > 
> > > >    # Interactive passphrase entry requested.
> > > > 
> > > >    Enter passphrase for /dev/sda5:
> > > > 
> > > >    # Trying to open key slot 0 [ACTIVE_LAST].
> > > > 
> > > >    # Reading key slot 0 area.
> > > > 
> > > >    # Using userspace crypto wrapper to access keyslot area.
> > > > 
> > > >    # Trying to open key slot 1 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 2 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 3 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 4 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 5 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 6 [INACTIVE].
> > > > 
> > > >    # Trying to open key slot 7 [INACTIVE].
> > > > 
> > > >    No key available with this passphrase.
> > > > 
> > > >    On 3 Nov 2016, at 19:04, Michael Kjörling <[1]michael@kjorling.se>
> > > >    wrote:
> > > > 
> > > >    On 3 Nov 2016 18:30 +0000, from [2]zero.tonin@web.de (Zero Tonin):
> > > > 
> > > >      user@debian:~$ sudo cryptsetup luksOpen /dev/sda5 crypt1
> > > > 
> > > >      Enter passphrase for /dev/sda5:
> > > > 
> > > >      No key available with this passphrase.
> > > > 
> > > >    Could you try running this again, but add the `--debug` option to
> > > >    cryptsetup, then post the resulting log?
> > > >    Make sure to sanitize the passphrase itself from the log if it's there
> > > >    (I don't know), but leave everything else intact.
> > > >    --
> > > >    Michael Kjörling • [3]https://michael.kjorling.se •
> > > >    [4]michael@kjorling.se
> > > >                    “People who think they know everything really annoy
> > > >                    those of us who know we don’t.” (Bjarne Stroustrup)
> > > >    _______________________________________________
> > > >    dm-crypt mailing list
> > > >    [5]dm-crypt@saout.de
> > > >    [6]http://www.saout.de/mailman/listinfo/dm-crypt
> > > > 
> > > > References
> > > > 
> > > >    1. mailto:michael@kjorling.se
> > > >    2. mailto:zero.tonin@web.de
> > > >    3. https://michael.kjorling.se/
> > > >    4. mailto:michael@kjorling.se
> > > >    5. mailto:dm-crypt@saout.de
> > > >    6. http://www.saout.de/mailman/listinfo/dm-crypt
> > > 
> > > > _______________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt@saout.de
> > > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > 
> > > 
> > > -- 
> > > Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> > > GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> > > ----
> > > A good decision is based on knowledge and not on numbers. -- Plato
> > > 
> > > If it's in the news, don't worry about it.  The very definition of 
> > > "news" is "something that hardly ever happens." -- Bruce Schneier
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > >
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> 
> -- 
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -- Plato
> 
> If it's in the news, don't worry about it.  The very definition of 
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Heinz Diehl]

On 04.11.2016, Michael Kjörling wrote: 

> I _strongly_ recommend ddrescue over dd; ddrescue is far better suited
> for this use case. It also gives you a nice progress indication while
> it is working.

FWIW: You can also get dd to show transfer statistics by using the
"status=progress" flag.

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Michael Kjörling]

On 5 Nov 2016 11:16 +0100, from htd+ml@fritha.org (Heinz Diehl):
>> I _strongly_ recommend ddrescue over dd; ddrescue is far better suited
>> for this use case. It also gives you a nice progress indication while
>> it is working.
> 
> FWIW: You can also get dd to show transfer statistics by using the
> "status=progress" flag.

IMO, that's not worth much compared to dd's greater shortcoming when
dealing with marginal media: Even when running with conv=noerror, as
is often suggested, dd will simply skip over any unreadable parts in
the input, which will cause any relative or absolute offsets which
cross that boundary, and any absolute offsets referring to after the
problematic portion, come after it to be wrong. ddrescue is meant to
be used with potentially marginal media and thus has better handling
of that situation.

Or so I've been told.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Michael Kjörling]

On 5 Nov 2016 08:56 +0100, from zero.tonin@web.de:
> I hope this is due to the insufficient size on the drive.

Unfortunately, that particular explanation is highly unlikely. The
LUKS header is right at the beginning of the drive (with default
settings I believe it occupies the first MiB and change); there is no
LUKS metadata elsewhere. Truncating the data on the drive near the end
would be far more likely to lead to file system driver confusion or
plain corrupted data within the file system than problems with the
LUKS container.

That said, working on a copy while trying to troubleshoot storage
problems is _never_ a bad idea.

Oh, and please do trim your posts. Every list post is archived; we
don't need them repeated in every reply.

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[zero.tonin]

Hi again, everybody, 
and yet another sorry - it is indeed weird to work on an unknown system and I ask ye to please accept my apology for causing any inconvenience with html or TOFU posts. I am slowly getting my debian VM to a workable degree, so I hope less errors occur from now on!

I did another ddrescue today after formatting one of my drives, as Michael suggested the missing 100 or so GB wouldn't cause the "no key with this passphrase" issue. 

Running the keyslotchecker from /misc results in the same as before (start: 0x001000, end: 0x03f800) , which, if I understood correctly, would indicate that the keyslot technically is still there and no bytes have been accidentally overwritten. 

The hexdump also still indicates the LUKS header where, as far as a layman like me can understand in this short period of time, it should be, with a hexdump resulting in
00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00  |LUKS....aes.....|

The drive is also (isLuks) recognized as a LUKS drive, still so - in theory- it al looks well and "I don't understand"


I tried adding a key to keyslot1, hoping that maybe this somehow would work with the original key in slot0, but, alas, no joy, the same, naturally, goes for attempting to luksChangekey, --dump-master-key or crptsetup-reencrypt

I was going through the options fro the man page and treid all those that looked somehow relevant to my situation, I thus created a luksDump, which resolves to this:
Key Slot 0: ENABLED
        Iterations:             342245
        Salt:                   72 3c b6 82 b3 33 a7 f6 5a 55 f9 3d 6b f3 8c b8 
                                d9 6a 66 31 9e 03 b1 57 b9 bf 00 5d d7 4a dd c9 
        Key material offset:    8
        AF stripes:             4000


I see there is a folder /test in the cryptsetup folder, but I could not locate a readme or something like it for them - would there be anything relevant I could try? 

I am also curios what my debian (or my HW, for that matter) could have done t the drive to render this state, as after the decrypt, when I realised the issue, I shut down the laptop immediately without "playing" with the LUKS. The only thing I could imagine would be some evil wizzard genius having somehow gotten luksErase into my cronjobs (which, of course would result in an empty keyslot 0, if I understand correctly...) or something like that. I suppose that's rather unlikely, though. Could the corrupt OS have ... changed the passphrase whilst the drive had been unlocked, without further user input?

I also see there is a "repair" mentioned in man, but I do not understand how to call this one (I have created a header backup in the meantime) or whether it would even make sense, as I am unsure what exactly is broken in the first place...

I also understand that the mailinglist is not a personal support tool, so again my gratitude for the comments and help I receive here!

Is there anything left to try with this drive or, at this stage, is "all lost" and I might as well wipe the drive, reinstall an OS and see (as it seems to be HW related) where I can safe up some money for a new machine?

Kind regards and thanks a mill,
Mark

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Sven Eschenberg]

Hi there,

I did not have the opportunity to read all of the discussion, but 
thought I might add in some bits.

Am 05.11.2016 um 22:58 schrieb zero.tonin@web.de:
> Hi again, everybody,
> and yet another sorry - it is indeed weird to work on an unknown system and I ask ye to please accept my apology for causing any inconvenience with html or TOFU posts. I am slowly getting my debian VM to a workable degree, so I hope less errors occur from now on!
>
> I did another ddrescue today after formatting one of my drives, as Michael suggested the missing 100 or so GB wouldn't cause the "no key with this passphrase" issue.
>

In fact only the header is really relevant to cryptsetup. If the image 
was truncated the filesystem might have been partially damaged (within 
the image that is), but you'd at least be able to unlock() and see the 
fs-signature if you captured enough sectors at the beginning of the LUKS 
container.

> Running the keyslotchecker from /misc results in the same as before (start: 0x001000, end: 0x03f800) , which, if I understood correctly, would indicate that the keyslot technically is still there and no bytes have been accidentally overwritten.

Exactly, the slot itself seems to be intact, as far as analysis can go.

>
> The hexdump also still indicates the LUKS header where, as far as a layman like me can understand in this short period of time, it should be, with a hexdump resulting in
> 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00  |LUKS....aes.....|
>
> The drive is also (isLuks) recognized as a LUKS drive, still so - in theory- it al looks well and "I don't understand"

Well, if the key material was damaged, then even when your password is 
correct, the hash value would not match and even worse, the retrieval of 
the actual disk key would fail. There is no redundancy in the keyslot 
that can compensate for bit-errors.

> I tried adding a key to keyslot1, hoping that maybe this somehow would work with the original key in slot0, but, alas, no joy, the same, naturally, goes for attempting to luksChangekey, --dump-master-key or crptsetup-reencrypt

Adding a key needs the drive key, which would have to be restored from a 
working slot. Well, it could be retrieved from mem, when the container 
is open, but that does not apply in your case.

>
> I was going through the options fro the man page and treid all those that looked somehow relevant to my situation, I thus created a luksDump, which resolves to this:
> Key Slot 0: ENABLED
>         Iterations:             342245
>         Salt:                   72 3c b6 82 b3 33 a7 f6 5a 55 f9 3d 6b f3 8c b8
>                                 d9 6a 66 31 9e 03 b1 57 b9 bf 00 5d d7 4a dd c9
>         Key material offset:    8
>         AF stripes:             4000
>
>
> I see there is a folder /test in the cryptsetup folder, but I could not locate a readme or something like it for them - would there be anything relevant I could try?
>

Did you actaully try to luksDump on the original drive from some live 
system? And maybe dump the first 8MB of the LUKS container, including 
the header and see if the dump is stable?(i.e. do multiple dumps and 
compare hashes or diff) If it changes then you are really having issues 
with unstable read results and would have to have enormous luck to get 
the correct data to unlock the slot.

> I am also curios what my debian (or my HW, for that matter) could have done t the drive to render this state, as after the decrypt, when I realised the issue, I shut down the laptop immediately without "playing" with the LUKS. The only thing I could imagine would be some evil wizzard genius having somehow gotten luksErase into my cronjobs (which, of course would result in an empty keyslot 0, if I understand correctly...) or something like that. I suppose that's rather unlikely, though. Could the corrupt OS have ... changed the passphrase whilst the drive had been unlocked, without further user input?

Well of course a keyslot can be overriden purposefully, cryptsetup, when 
unlocking the container, does however not write to the header area at 
all. The results so far do not show signs of typical errors like 
overwriting with fs-signatures or so. Since the header structure seems 
to be okay, something would have had to overwrite some parts of the 
actual keyslot area. There are a lot of possibilities how this could 
happen if there's some defect.

>
> I also see there is a "repair" mentioned in man, but I do not understand how to call this one (I have created a header backup in the meantime) or whether it would even make sense, as I am unsure what exactly is broken in the first place...

Sorry, never had any need for it so far, better wait for an answer from 
someone else regarding this.

>
> I also understand that the mailinglist is not a personal support tool, so again my gratitude for the comments and help I receive here!
>
> Is there anything left to try with this drive or, at this stage, is "all lost" and I might as well wipe the drive, reinstall an OS and see (as it seems to be HW related) where I can safe up some money for a new machine?
>
> Kind regards and thanks a mill,
> Mark
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>

Final words: You said, you are sure the PW is correct. Are you 100% sure 
that the keyboard layout was correct and no character mapping issues are 
involved? Double checked on the live env?

Regards

-Sven

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[zero.tonin]

Good morning all, good morning Sven,

thank you very much for your message and insight and also for confirming parts of my understanding of the situation.

Now, getting back to my drive, I have to make the most embarrassing confession - for the last three days I actually tried to unlock the drive with my user account password and, in contrast to my initial words (which I eat now), _not_ with  the drive password. I have not the lsightest idea why, apart from the two (password and passphrase) being in use both for the same amount of time, which is roughly 5 years or so. 

The luks header and keyslot are not only intact, they are fully working and I can decrypt the drive no problem. Turns out it was, indeed, a "layer 8 problem". 

I am, while relieved I can recover the data, add a second keyslot and rsync the drive immediately, sincerely sorry for having used (some may say wasted) this list's and some individual's time,, but thanks again for all your help.

On the positive side, this forced me to learn a bit more about luks and what it actually does (instead of just using it) and also was an opportunity to lear to never again claim I am "100% certain about the password"…


Thanks again, all, and a great Sunday to everyone,
Mark

(I will take the liberty to remain on this list for future interesting topics)

Re: [dm-crypt] Missing keyslot or broken header or still some hope? SOLVED

[Michael Kjörling]

On 6 Nov 2016 08:26 +0100, from zero.tonin@web.de:
> Now, getting back to my drive, I have to make the most embarrassing
> confession - for the last three days I actually tried to unlock the
> drive with my user account password and, in contrast to my initial
> words (which I eat now), _not_ with  the drive password. I have not
> the lsightest idea why, apart from the two (password and passphrase)
> being in use both for the same amount of time, which is roughly 5
> years or so. 

Given what we have found out, I was about to suggest the same thing as
Sven: to quadruple-check that the LUKS passphrase really was correct,
because that was about the only thing remaining that could reasonably
explain what you were seeing.


> I am, while relieved I can recover the data, add a second keyslot
> and rsync the drive immediately, sincerely sorry for having used
> (some may say wasted) this list's and some individual's time,, but
> thanks again for all your help.
> 
> On the positive side, this forced me to learn a bit more about luks
> and what it actually does (instead of just using it) and also was an
> opportunity to lear to never again claim I am "100% certain about
> the password"…

It also forced you to consider your restore strategy. Remember, it's
not truly a backup until you have restored from it onto bare metal.

Now, add a second key slot with a different passphrase, then store
that passphrase securely. To avoid key mapping issues, you may want to
use Yubico's Modhex alphabet (cbdefghijklnrtuv), as they selected
those characters specifically because they are as independent of
keyboard layout settings as possible while providing four bits per
character. Apparently in that set, "c" is known to be potentially
ambigous. https://forum.yubico.com/viewtopic.php?f=6&t=96

-- 
Michael Kjörling • https://michael.kjorling.se • michael@kjorling.se
                 “People who think they know everything really annoy
                 those of us who know we don’t.” (Bjarne Stroustrup)

Re: [dm-crypt] Missing keyslot or broken header or still some hope?

[Arno Wagner]

It happens. Don't worry about it.

Regards,
Arno

On Sun, Nov 06, 2016 at 08:26:50 CET, zero.tonin@web.de wrote:
> Good morning all, good morning Sven,
> 
> thank you very much for your message and insight and also for confirming parts of my understanding of the situation.
> 
> Now, getting back to my drive, I have to make the most embarrassing confession - for the last three days I actually tried to unlock the drive with my user account password and, in contrast to my initial words (which I eat now), _not_ with  the drive password. I have not the lsightest idea why, apart from the two (password and passphrase) being in use both for the same amount of time, which is roughly 5 years or so. 
> 
> The luks header and keyslot are not only intact, they are fully working and I can decrypt the drive no problem. Turns out it was, indeed, a "layer 8 problem". 
> 
> I am, while relieved I can recover the data, add a second keyslot and rsync the drive immediately, sincerely sorry for having used (some may say wasted) this list's and some individual's time,, but thanks again for all your help.
> 
> On the positive side, this forced me to learn a bit more about luks and what it actually does (instead of just using it) and also was an opportunity to lear to never again claim I am "100% certain about the password"…
> 
> 
> Thanks again, all, and a great Sunday to everyone,
> Mark
> 
> (I will take the liberty to remain on this list for future interesting topics)
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier