* [dm-crypt] "not a valid LUKS device" after distro change
@ 2014-08-18 20:58 John Wells
2014-08-19 19:51 ` Arno Wagner
0 siblings, 1 reply; 27+ messages in thread
From: John Wells @ 2014-08-18 20:58 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 13801 bytes --]
Guys,
I have an odd problem. I had two luks partitions...both created upon
install on Fedora 20.
Last week, I switched to Ubuntu 14.04. However, anticipating problems, I
*made sure Ubuntu did nothing with these partitions*. No mounting, nothing.
Now, when I boot into Ubuntu, I can mount one manually and my data is
there. However, the other partition tells me "not a valid LUKS device". So,
thinking it might be a problem with Ubuntu, I downloaded the Fedora 20 live
cd, booted up and tried it there. Shockingly, in Fedora 20, *both*
partitions give the "not a valid LUKS device" error, and neither can be
mounted.
The partition I can mount from Ubuntu is MORE_VG-MORE_LV in the output
below. The partition I can't mount from either distro now is
FINALFRONTIER_VG-HOME_LV.
Here's what things like like from Ubuntu (note, /dev/MORE_VG-MORE_LV is
mounted successfully on /home...this is one of the partitions in question):
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 37.3G 0 disk
├─sda1 8:1 0 100M 0 part
└─sda2 8:2 0 37.2G 0 part
sdb 8:16 0 1.8T 0 disk
└─sdb1 8:17 0 1.8T 0 part
└─md0 9:0 0 1.8T 0 raid1
└─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
└─cryptdevice (dm-7) 252:7 0 800G 0 crypt /home
sdc 8:32 0 1.8T 0 disk
└─sdc1 8:33 0 1.8T 0 part
└─md2 9:2 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
└─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
sdd 8:48 0 1.8T 0 disk
└─sdd1 8:49 0 1.8T 0 part
└─md2 9:2 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
└─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
sde 8:64 0 167.7G 0 disk
├─sde1 8:65 0 3G 0 part /boot
├─sde2 8:66 0 12G 0 part
└─sde3 8:67 0 152.8G 0 part
└─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
sdf 8:80 0 1.8T 0 disk
└─sdf1 8:81 0 1.8T 0 part
sr0 11:0 1 1024M 0 rom
# blkid
/dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
/dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs"
/dev/sdb1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
TYPE="linux_raid_member"
/dev/sdc1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
TYPE="linux_raid_member"
/dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
TYPE="linux_raid_member"
/dev/sde1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
/dev/sde3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb" TYPE="LVM2_member"
/dev/sdf1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
TYPE="linux_raid_member"
/dev/md0: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ" TYPE="LVM2_member"
/dev/md2: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ" TYPE="LVM2_member"
/dev/mapper/SSDROOT_VG-ROOT_LV: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
TYPE="ext4"
/dev/mapper/MORE_VG-MORE_LV: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44"
TYPE="crypto_LUKS"
/dev/mapper/FINALFRONTIER_VG-SWAP_LV:
UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
/dev/mapper/FINALFRONTIER_VG-OPT_LV:
UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
/dev/mapper/FINALFRONTIER_VG-VMS_LV:
UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
/dev/mapper/FINALFRONTIER_VG-DATA_LV:
UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
/dev/mapper/cryptdevice: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d"
TYPE="ext4"
Note how FINALFRONTIER_VG-HOME_LV doesn't appear in the output of blkid.
# blkid /dev/dm*
/dev/dm-0: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
/dev/dm-1: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44" TYPE="crypto_LUKS"
/dev/dm-3: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
/dev/dm-4: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
/dev/dm-5: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
/dev/dm-6: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
/dev/dm-7: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d" TYPE="ext4"
# dpkg -l | grep cryptsetup
ii cryptsetup 2:1.6.1-1ubuntu1
amd64 disk encryption support -
startup scripts
ii cryptsetup-bin 2:1.6.1-1ubuntu1
amd64 disk encryption support -
command line tools
ii libcryptsetup4 2:1.6.1-1ubuntu1
amd64 disk encryption support -
shared library
I do note that Ubuntu uses 1.6.1, while Fedora 20 uses 1.6.2 (see below
output).
Did the Ubuntu 14.04 install somehow corrupt my FINALFRONTIER_VG-HOME_LV
partition? Could the version differences be causing this? Is all my data
there lost? I do have backups for the *really* important stuff, but there's
about 72 hours of coding in there I'd really like to get back if possible.
Any help is greatly appreciated. I'm pasting similar output from the Fedora
20 live session below for your reference as well.
Thanks,
John
------------------------------ FROM FEDORA 20 ------------------------------
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 37.3G 0 disk
├─sda1 8:1 0 100M 0 part
└─sda2 8:2 0 37.2G 0 part
sdb 8:16 1 14.6G 0 disk
├─sdb1 8:17 1 953M 0 part
/run/initramfs/live
├─sdb2 8:18 1 4.9M 0 part
└─sdb3 8:19 1 19.7M 0 part
sdc 8:32 0 1.8T 0 disk
└─sdc1 8:33 0 1.8T 0 part
sdd 8:48 0 1.8T 0 disk
└─sdd1 8:49 0 1.8T 0 part
└─md127 9:127 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
└─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
sde 8:64 0 1.8T 0 disk
└─sde1 8:65 0 1.8T 0 part
└─md127 9:127 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
└─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
sdf 8:80 0 167.7G 0 disk
├─sdf1 8:81 0 3G 0 part
├─sdf2 8:82 0 12G 0 part
└─sdf3 8:83 0 152.8G 0 part
└─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
sdg 8:96 0 1.8T 0 disk
└─sdg1 8:97 0 1.8T 0 part
└─md126 9:126 0 1.8T 0 raid1
└─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
sr0 11:0 1 1024M 0 rom
loop0 7:0 0 8K 1 loop
loop1 7:1 0 1.2M 1 loop
└─live-osimg-min 253:2 0 4G 1 dm
loop2 7:2 0 886.8M 1 loop
loop3 7:3 0 4G 1 loop
├─live-rw 253:0 0 4G 0 dm /
├─live-base 253:1 0 4G 1 dm
└─live-osimg-min 253:2 0 4G 1 dm
loop4 7:4 0 512M 0 loop
└─live-rw 253:0 0 4G 0 dm /
[root@localhost]# cryptsetup luksDump /dev/MORE_VG/MORE_LV
Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
[root@localhost]# cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
# blkid
/dev/sdb1: UUID="2013-12-12-00-56-55-00"
LABEL="Fedora-Live-Desktop-x86_64-20-1" TYPE="iso9660" PTUUID="5513338d"
PTTYPE="dos" PARTUUID="5513338d-01"
/dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
PARTUUID="9ff59ff5-01"
/dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs" PARTUUID="9ff59ff5-02"
/dev/sdb2: SEC_TYPE="msdos" LABEL="EFI" UUID="7E31-E62C" TYPE="vfat"
PARTUUID="5513338d-02"
/dev/sdb3: UUID="11c76dff-3163-3083-8cec-c9045139ec1b" LABEL="Fedora Live"
TYPE="hfsplus" PARTUUID="5513338d-03"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="DM_snapshot_cow"
/dev/loop2: TYPE="squashfs"
/dev/loop3: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/loop4: TYPE="DM_snapshot_cow"
/dev/mapper/live-rw: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/mapper/live-base: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/mapper/live-osimg-min: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/sdc1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
TYPE="linux_raid_member" PARTUUID="c8aa8e9b-01"
/dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
TYPE="linux_raid_member" PARTUUID="0004869f-01"
/dev/sde1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
TYPE="linux_raid_member" PARTUUID="000956e0-01"
/dev/sdf1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
PARTUUID="00005669-01"
/dev/sdf2: PARTUUID="00005669-02"
/dev/sdf3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb" TYPE="LVM2_member"
PARTUUID="00005669-03"
/dev/sdg1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
TYPE="linux_raid_member" PARTUUID="bde5c281-01"
/dev/md127: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ"
TYPE="LVM2_member"
/dev/md126: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ"
TYPE="LVM2_member"
/dev/mapper/SSDROOT_VG-ROOT_LV: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
TYPE="ext4"
/dev/mapper/FINALFRONTIER_VG-SWAP_LV:
UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
/dev/mapper/FINALFRONTIER_VG-OPT_LV:
UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
/dev/mapper/FINALFRONTIER_VG-VMS_LV:
UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
/dev/mapper/FINALFRONTIER_VG-DATA_LV:
UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
# blkid /dev/dm*
/dev/dm-0: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-1: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-2: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
/dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
/dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
/dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
/dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
/dev/dm-0: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-1: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-2: LABEL="_Fedora-Live-Des"
UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
/dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
/dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
/dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
/dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
/dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
# rpm -qa | grep cryptsetup
cryptsetup-1.6.2-1.fc20.x86_64
cryptsetup-libs-1.6.2-1.fc20.x86_64
[-- Attachment #2: Type: text/html, Size: 19446 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-18 20:58 [dm-crypt] "not a valid LUKS device" after distro change John Wells
@ 2014-08-19 19:51 ` Arno Wagner
2014-08-19 22:22 ` John Wells
0 siblings, 1 reply; 27+ messages in thread
From: Arno Wagner @ 2014-08-19 19:51 UTC (permalink / raw)
To: dm-crypt
I Have no idea about what the problem may be, but just a
few days ago, Jeff Esquivel was also reporting problems with
a LUKS container that suddenly became unopenable under
Ubuntu 14.04 and we were unable to determine the cause.
He did create the container under Ubuntu and a few days
later it refused to open, just like yours.
I suspect something in Ubuntu 14.04 may indeed corrupt
LUKS key-slots (or salt values). It would not
be the first time that Ubuntu messes up handling LUKS
containers. That possibility is also one reason why a
header-backup is so important.
I find it especially troubling that this affects only
one of your two containers and that the Fedora 20 that
was fine before now fails to open the LIKS containers.
That can basically only happen on some header or key-slot
corruption.
Could you post the output of luksDump for the container
that you cannot open? (This does not compromise your security.)
There may be some similar patterns in the salts to the ones
Jeff posted.
If you have a spare disk and time, could you try the following:
Recreate a partition setup similar to yours using Fedora 20,
make header-backups (see FAQ) and then see whether the switch
to Ubuntu 14.04 kills one or both LUKS containers. Then make
new header backups and compare them. If they are not
bit-identical, we have something we can look at.
Arno
On Mon, Aug 18, 2014 at 22:58:49 CEST, John Wells wrote:
> Guys,
>
> I have an odd problem. I had two luks partitions...both created upon
> install on Fedora 20.
>
> Last week, I switched to Ubuntu 14.04. However, anticipating problems, I
> *made sure Ubuntu did nothing with these partitions*. No mounting, nothing.
>
> Now, when I boot into Ubuntu, I can mount one manually and my data is
> there. However, the other partition tells me "not a valid LUKS device". So,
> thinking it might be a problem with Ubuntu, I downloaded the Fedora 20 live
> cd, booted up and tried it there. Shockingly, in Fedora 20, *both*
> partitions give the "not a valid LUKS device" error, and neither can be
> mounted.
>
> The partition I can mount from Ubuntu is MORE_VG-MORE_LV in the output
> below. The partition I can't mount from either distro now is
> FINALFRONTIER_VG-HOME_LV.
>
> Here's what things like like from Ubuntu (note, /dev/MORE_VG-MORE_LV is
> mounted successfully on /home...this is one of the partitions in question):
>
> # lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 37.3G 0 disk
> ├─sda1 8:1 0 100M 0 part
> └─sda2 8:2 0 37.2G 0 part
> sdb 8:16 0 1.8T 0 disk
> └─sdb1 8:17 0 1.8T 0 part
> └─md0 9:0 0 1.8T 0 raid1
> └─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
> └─cryptdevice (dm-7) 252:7 0 800G 0 crypt /home
> sdc 8:32 0 1.8T 0 disk
> └─sdc1 8:33 0 1.8T 0 part
> └─md2 9:2 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> sdd 8:48 0 1.8T 0 disk
> └─sdd1 8:49 0 1.8T 0 part
> └─md2 9:2 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> sde 8:64 0 167.7G 0 disk
> ├─sde1 8:65 0 3G 0 part /boot
> ├─sde2 8:66 0 12G 0 part
> └─sde3 8:67 0 152.8G 0 part
> └─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
> sdf 8:80 0 1.8T 0 disk
> └─sdf1 8:81 0 1.8T 0 part
> sr0 11:0 1 1024M 0 rom
>
> # blkid
>
> /dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
> /dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs"
> /dev/sdb1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
> TYPE="linux_raid_member"
> /dev/sdc1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
> TYPE="linux_raid_member"
> /dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
> TYPE="linux_raid_member"
> /dev/sde1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
> /dev/sde3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb" TYPE="LVM2_member"
> /dev/sdf1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
> TYPE="linux_raid_member"
> /dev/md0: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ" TYPE="LVM2_member"
> /dev/md2: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ" TYPE="LVM2_member"
> /dev/mapper/SSDROOT_VG-ROOT_LV: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
> TYPE="ext4"
> /dev/mapper/MORE_VG-MORE_LV: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44"
> TYPE="crypto_LUKS"
> /dev/mapper/FINALFRONTIER_VG-SWAP_LV:
> UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> /dev/mapper/FINALFRONTIER_VG-OPT_LV:
> UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> /dev/mapper/FINALFRONTIER_VG-VMS_LV:
> UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> /dev/mapper/FINALFRONTIER_VG-DATA_LV:
> UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> /dev/mapper/cryptdevice: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d"
> TYPE="ext4"
>
> Note how FINALFRONTIER_VG-HOME_LV doesn't appear in the output of blkid.
>
> # blkid /dev/dm*
> /dev/dm-0: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> /dev/dm-1: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44" TYPE="crypto_LUKS"
> /dev/dm-3: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> /dev/dm-4: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> /dev/dm-5: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> /dev/dm-6: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> /dev/dm-7: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d" TYPE="ext4"
>
> # dpkg -l | grep cryptsetup
> ii cryptsetup 2:1.6.1-1ubuntu1
> amd64 disk encryption support -
> startup scripts
> ii cryptsetup-bin 2:1.6.1-1ubuntu1
> amd64 disk encryption support -
> command line tools
> ii libcryptsetup4 2:1.6.1-1ubuntu1
> amd64 disk encryption support -
> shared library
>
> I do note that Ubuntu uses 1.6.1, while Fedora 20 uses 1.6.2 (see below
> output).
>
> Did the Ubuntu 14.04 install somehow corrupt my FINALFRONTIER_VG-HOME_LV
> partition? Could the version differences be causing this? Is all my data
> there lost? I do have backups for the *really* important stuff, but there's
> about 72 hours of coding in there I'd really like to get back if possible.
>
> Any help is greatly appreciated. I'm pasting similar output from the Fedora
> 20 live session below for your reference as well.
>
> Thanks,
> John
>
> ------------------------------ FROM FEDORA 20 ------------------------------
> # lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 37.3G 0 disk
> ├─sda1 8:1 0 100M 0 part
> └─sda2 8:2 0 37.2G 0 part
> sdb 8:16 1 14.6G 0 disk
> ├─sdb1 8:17 1 953M 0 part
> /run/initramfs/live
> ├─sdb2 8:18 1 4.9M 0 part
> └─sdb3 8:19 1 19.7M 0 part
> sdc 8:32 0 1.8T 0 disk
> └─sdc1 8:33 0 1.8T 0 part
> sdd 8:48 0 1.8T 0 disk
> └─sdd1 8:49 0 1.8T 0 part
> └─md127 9:127 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
> ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> sde 8:64 0 1.8T 0 disk
> └─sde1 8:65 0 1.8T 0 part
> └─md127 9:127 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
> ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> sdf 8:80 0 167.7G 0 disk
> ├─sdf1 8:81 0 3G 0 part
> ├─sdf2 8:82 0 12G 0 part
> └─sdf3 8:83 0 152.8G 0 part
> └─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
> sdg 8:96 0 1.8T 0 disk
> └─sdg1 8:97 0 1.8T 0 part
> └─md126 9:126 0 1.8T 0 raid1
> └─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
> sr0 11:0 1 1024M 0 rom
> loop0 7:0 0 8K 1 loop
> loop1 7:1 0 1.2M 1 loop
> └─live-osimg-min 253:2 0 4G 1 dm
> loop2 7:2 0 886.8M 1 loop
> loop3 7:3 0 4G 1 loop
> ├─live-rw 253:0 0 4G 0 dm /
> ├─live-base 253:1 0 4G 1 dm
> └─live-osimg-min 253:2 0 4G 1 dm
> loop4 7:4 0 512M 0 loop
> └─live-rw 253:0 0 4G 0 dm /
> [root@localhost]# cryptsetup luksDump /dev/MORE_VG/MORE_LV
> Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> [root@localhost]# cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
>
> # blkid
>
> /dev/sdb1: UUID="2013-12-12-00-56-55-00"
> LABEL="Fedora-Live-Desktop-x86_64-20-1" TYPE="iso9660" PTUUID="5513338d"
> PTTYPE="dos" PARTUUID="5513338d-01"
> /dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
> PARTUUID="9ff59ff5-01"
> /dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs" PARTUUID="9ff59ff5-02"
> /dev/sdb2: SEC_TYPE="msdos" LABEL="EFI" UUID="7E31-E62C" TYPE="vfat"
> PARTUUID="5513338d-02"
> /dev/sdb3: UUID="11c76dff-3163-3083-8cec-c9045139ec1b" LABEL="Fedora Live"
> TYPE="hfsplus" PARTUUID="5513338d-03"
> /dev/loop0: TYPE="squashfs"
> /dev/loop1: TYPE="DM_snapshot_cow"
> /dev/loop2: TYPE="squashfs"
> /dev/loop3: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/loop4: TYPE="DM_snapshot_cow"
> /dev/mapper/live-rw: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/mapper/live-base: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/mapper/live-osimg-min: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/sdc1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
> TYPE="linux_raid_member" PARTUUID="c8aa8e9b-01"
> /dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
> TYPE="linux_raid_member" PARTUUID="0004869f-01"
> /dev/sde1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
> TYPE="linux_raid_member" PARTUUID="000956e0-01"
> /dev/sdf1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
> PARTUUID="00005669-01"
> /dev/sdf2: PARTUUID="00005669-02"
> /dev/sdf3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb" TYPE="LVM2_member"
> PARTUUID="00005669-03"
> /dev/sdg1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
> TYPE="linux_raid_member" PARTUUID="bde5c281-01"
> /dev/md127: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ"
> TYPE="LVM2_member"
> /dev/md126: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ"
> TYPE="LVM2_member"
> /dev/mapper/SSDROOT_VG-ROOT_LV: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
> TYPE="ext4"
> /dev/mapper/FINALFRONTIER_VG-SWAP_LV:
> UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> /dev/mapper/FINALFRONTIER_VG-OPT_LV:
> UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> /dev/mapper/FINALFRONTIER_VG-VMS_LV:
> UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> /dev/mapper/FINALFRONTIER_VG-DATA_LV:
> UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
>
> # blkid /dev/dm*
> /dev/dm-0: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-1: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-2: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> /dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> /dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> /dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> /dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> /dev/dm-0: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-1: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-2: LABEL="_Fedora-Live-Des"
> UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> /dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> /dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> /dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> /dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> /dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
>
> # rpm -qa | grep cryptsetup
> cryptsetup-1.6.2-1.fc20.x86_64
> cryptsetup-libs-1.6.2-1.fc20.x86_64
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-19 19:51 ` Arno Wagner
@ 2014-08-19 22:22 ` John Wells
2014-08-20 9:09 ` Arno Wagner
0 siblings, 1 reply; 27+ messages in thread
From: John Wells @ 2014-08-19 22:22 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 17277 bytes --]
Thanks for your response. This is the result of luksDump on the container:
# cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
I will try to find the time to recreate the entire scenario. Do you think
the current container I'm able to open is at risk of corruption as well?
On Tue, Aug 19, 2014 at 3:51 PM, Arno Wagner <arno@wagner.name> wrote:
>
> I Have no idea about what the problem may be, but just a
> few days ago, Jeff Esquivel was also reporting problems with
> a LUKS container that suddenly became unopenable under
> Ubuntu 14.04 and we were unable to determine the cause.
> He did create the container under Ubuntu and a few days
> later it refused to open, just like yours.
>
> I suspect something in Ubuntu 14.04 may indeed corrupt
> LUKS key-slots (or salt values). It would not
> be the first time that Ubuntu messes up handling LUKS
> containers. That possibility is also one reason why a
> header-backup is so important.
>
> I find it especially troubling that this affects only
> one of your two containers and that the Fedora 20 that
> was fine before now fails to open the LIKS containers.
> That can basically only happen on some header or key-slot
> corruption.
>
> Could you post the output of luksDump for the container
> that you cannot open? (This does not compromise your security.)
> There may be some similar patterns in the salts to the ones
> Jeff posted.
>
> If you have a spare disk and time, could you try the following:
> Recreate a partition setup similar to yours using Fedora 20,
> make header-backups (see FAQ) and then see whether the switch
> to Ubuntu 14.04 kills one or both LUKS containers. Then make
> new header backups and compare them. If they are not
> bit-identical, we have something we can look at.
>
> Arno
>
>
>
>
> On Mon, Aug 18, 2014 at 22:58:49 CEST, John Wells wrote:
> > Guys,
> >
> > I have an odd problem. I had two luks partitions...both created upon
> > install on Fedora 20.
> >
> > Last week, I switched to Ubuntu 14.04. However, anticipating problems, I
> > *made sure Ubuntu did nothing with these partitions*. No mounting,
> nothing.
> >
> > Now, when I boot into Ubuntu, I can mount one manually and my data is
> > there. However, the other partition tells me "not a valid LUKS device".
> So,
> > thinking it might be a problem with Ubuntu, I downloaded the Fedora 20
> live
> > cd, booted up and tried it there. Shockingly, in Fedora 20, *both*
> > partitions give the "not a valid LUKS device" error, and neither can be
> > mounted.
> >
> > The partition I can mount from Ubuntu is MORE_VG-MORE_LV in the output
> > below. The partition I can't mount from either distro now is
> > FINALFRONTIER_VG-HOME_LV.
> >
> > Here's what things like like from Ubuntu (note, /dev/MORE_VG-MORE_LV is
> > mounted successfully on /home...this is one of the partitions in
> question):
> >
> > # lsblk
> > NAME MAJ:MIN RM SIZE RO TYPE
> MOUNTPOINT
> > sda 8:0 0 37.3G 0 disk
> > ├─sda1 8:1 0 100M 0 part
> > └─sda2 8:2 0 37.2G 0 part
> > sdb 8:16 0 1.8T 0 disk
> > └─sdb1 8:17 0 1.8T 0 part
> > └─md0 9:0 0 1.8T 0 raid1
> > └─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
> > └─cryptdevice (dm-7) 252:7 0 800G 0 crypt /home
> > sdc 8:32 0 1.8T 0 disk
> > └─sdc1 8:33 0 1.8T 0 part
> > └─md2 9:2 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> > ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> > └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> > sdd 8:48 0 1.8T 0 disk
> > └─sdd1 8:49 0 1.8T 0 part
> > └─md2 9:2 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> > ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> > └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> > sde 8:64 0 167.7G 0 disk
> > ├─sde1 8:65 0 3G 0 part /boot
> > ├─sde2 8:66 0 12G 0 part
> > └─sde3 8:67 0 152.8G 0 part
> > └─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
> > sdf 8:80 0 1.8T 0 disk
> > └─sdf1 8:81 0 1.8T 0 part
> > sr0 11:0 1 1024M 0 rom
> >
> > # blkid
> >
> > /dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
> > /dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs"
> > /dev/sdb1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> > UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
> > TYPE="linux_raid_member"
> > /dev/sdc1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> > UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
> > TYPE="linux_raid_member"
> > /dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> > UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
> > TYPE="linux_raid_member"
> > /dev/sde1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
> > /dev/sde3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb"
> TYPE="LVM2_member"
> > /dev/sdf1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> > UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
> > TYPE="linux_raid_member"
> > /dev/md0: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ"
> TYPE="LVM2_member"
> > /dev/md2: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ"
> TYPE="LVM2_member"
> > /dev/mapper/SSDROOT_VG-ROOT_LV:
> UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
> > TYPE="ext4"
> > /dev/mapper/MORE_VG-MORE_LV: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44"
> > TYPE="crypto_LUKS"
> > /dev/mapper/FINALFRONTIER_VG-SWAP_LV:
> > UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> > /dev/mapper/FINALFRONTIER_VG-OPT_LV:
> > UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> > /dev/mapper/FINALFRONTIER_VG-VMS_LV:
> > UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> > /dev/mapper/FINALFRONTIER_VG-DATA_LV:
> > UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> > /dev/mapper/cryptdevice: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d"
> > TYPE="ext4"
> >
> > Note how FINALFRONTIER_VG-HOME_LV doesn't appear in the output of blkid.
> >
> > # blkid /dev/dm*
> > /dev/dm-0: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> > /dev/dm-1: UUID="6cc188db-ccdb-4c8f-97b2-e41198ec6e44" TYPE="crypto_LUKS"
> > /dev/dm-3: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> > /dev/dm-4: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> > /dev/dm-5: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> > /dev/dm-6: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> > /dev/dm-7: UUID="4de2ef32-9894-4bb4-bf64-ced1b312796d" TYPE="ext4"
> >
> > # dpkg -l | grep cryptsetup
> > ii cryptsetup
> 2:1.6.1-1ubuntu1
> > amd64 disk encryption support -
> > startup scripts
> > ii cryptsetup-bin
> 2:1.6.1-1ubuntu1
> > amd64 disk encryption support -
> > command line tools
> > ii libcryptsetup4
> 2:1.6.1-1ubuntu1
> > amd64 disk encryption support -
> > shared library
> >
> > I do note that Ubuntu uses 1.6.1, while Fedora 20 uses 1.6.2 (see below
> > output).
> >
> > Did the Ubuntu 14.04 install somehow corrupt my FINALFRONTIER_VG-HOME_LV
> > partition? Could the version differences be causing this? Is all my data
> > there lost? I do have backups for the *really* important stuff, but
> there's
> > about 72 hours of coding in there I'd really like to get back if
> possible.
> >
> > Any help is greatly appreciated. I'm pasting similar output from the
> Fedora
> > 20 live session below for your reference as well.
> >
> > Thanks,
> > John
> >
> > ------------------------------ FROM FEDORA 20
> ------------------------------
> > # lsblk
> > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> > sda 8:0 0 37.3G 0 disk
> > ├─sda1 8:1 0 100M 0 part
> > └─sda2 8:2 0 37.2G 0 part
> > sdb 8:16 1 14.6G 0 disk
> > ├─sdb1 8:17 1 953M 0 part
> > /run/initramfs/live
> > ├─sdb2 8:18 1 4.9M 0 part
> > └─sdb3 8:19 1 19.7M 0 part
> > sdc 8:32 0 1.8T 0 disk
> > └─sdc1 8:33 0 1.8T 0 part
> > sdd 8:48 0 1.8T 0 disk
> > └─sdd1 8:49 0 1.8T 0 part
> > └─md127 9:127 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> > ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
> > ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> > └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> > sde 8:64 0 1.8T 0 disk
> > └─sde1 8:65 0 1.8T 0 part
> > └─md127 9:127 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> > ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /tmp/opt
> > ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> > └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> > sdf 8:80 0 167.7G 0 disk
> > ├─sdf1 8:81 0 3G 0 part
> > ├─sdf2 8:82 0 12G 0 part
> > └─sdf3 8:83 0 152.8G 0 part
> > └─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
> > sdg 8:96 0 1.8T 0 disk
> > └─sdg1 8:97 0 1.8T 0 part
> > └─md126 9:126 0 1.8T 0 raid1
> > └─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
> > sr0 11:0 1 1024M 0 rom
> > loop0 7:0 0 8K 1 loop
> > loop1 7:1 0 1.2M 1 loop
> > └─live-osimg-min 253:2 0 4G 1 dm
> > loop2 7:2 0 886.8M 1 loop
> > loop3 7:3 0 4G 1 loop
> > ├─live-rw 253:0 0 4G 0 dm /
> > ├─live-base 253:1 0 4G 1 dm
> > └─live-osimg-min 253:2 0 4G 1 dm
> > loop4 7:4 0 512M 0 loop
> > └─live-rw 253:0 0 4G 0 dm /
> > [root@localhost]# cryptsetup luksDump /dev/MORE_VG/MORE_LV
> > Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> > [root@localhost]# cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> > Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
> >
> > # blkid
> >
> > /dev/sdb1: UUID="2013-12-12-00-56-55-00"
> > LABEL="Fedora-Live-Desktop-x86_64-20-1" TYPE="iso9660" PTUUID="5513338d"
> > PTTYPE="dos" PARTUUID="5513338d-01"
> > /dev/sda1: LABEL="System Reserved" UUID="4884488784487988" TYPE="ntfs"
> > PARTUUID="9ff59ff5-01"
> > /dev/sda2: UUID="2CC04A94C04A63E4" TYPE="ntfs" PARTUUID="9ff59ff5-02"
> > /dev/sdb2: SEC_TYPE="msdos" LABEL="EFI" UUID="7E31-E62C" TYPE="vfat"
> > PARTUUID="5513338d-02"
> > /dev/sdb3: UUID="11c76dff-3163-3083-8cec-c9045139ec1b" LABEL="Fedora
> Live"
> > TYPE="hfsplus" PARTUUID="5513338d-03"
> > /dev/loop0: TYPE="squashfs"
> > /dev/loop1: TYPE="DM_snapshot_cow"
> > /dev/loop2: TYPE="squashfs"
> > /dev/loop3: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/loop4: TYPE="DM_snapshot_cow"
> > /dev/mapper/live-rw: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/mapper/live-base: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/mapper/live-osimg-min: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/sdc1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> > UUID_SUB="1f3e63f5-7f25-4139-a708-efd1dfee3d0a" LABEL="pragdesk:0"
> > TYPE="linux_raid_member" PARTUUID="c8aa8e9b-01"
> > /dev/sdd1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> > UUID_SUB="100a43fb-5c9d-045a-2f03-daedfb9a4c6d" LABEL="pragdesk:2"
> > TYPE="linux_raid_member" PARTUUID="0004869f-01"
> > /dev/sde1: UUID="d2d341dc-e3a8-c880-3856-e56702f6ab33"
> > UUID_SUB="fdac08d6-8629-d95f-c18c-a0e3c4b3ef31" LABEL="pragdesk:2"
> > TYPE="linux_raid_member" PARTUUID="000956e0-01"
> > /dev/sdf1: UUID="ffdcd97d-6352-45b1-89ea-ec18d1476097" TYPE="ext2"
> > PARTUUID="00005669-01"
> > /dev/sdf2: PARTUUID="00005669-02"
> > /dev/sdf3: UUID="Rj1Rkx-YUX1-GV38-15ix-upRW-ocbF-0lY1lb"
> TYPE="LVM2_member"
> > PARTUUID="00005669-03"
> > /dev/sdg1: UUID="96b277d2-a5fa-1d13-fc5d-40a17ee26d57"
> > UUID_SUB="45ee2620-9ba3-3ac9-4b1c-fe4d87599db3" LABEL="pragdesk:0"
> > TYPE="linux_raid_member" PARTUUID="bde5c281-01"
> > /dev/md127: UUID="Em7bB4-ijwz-YdmO-dBCi-e3N6-EzaA-hX50vJ"
> > TYPE="LVM2_member"
> > /dev/md126: UUID="mZufjx-oWfV-YuZ0-gHat-7cim-skJV-Mc9elZ"
> > TYPE="LVM2_member"
> > /dev/mapper/SSDROOT_VG-ROOT_LV:
> UUID="7a56249a-63ac-4239-a874-7aa8c2f33195"
> > TYPE="ext4"
> > /dev/mapper/FINALFRONTIER_VG-SWAP_LV:
> > UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> > /dev/mapper/FINALFRONTIER_VG-OPT_LV:
> > UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> > /dev/mapper/FINALFRONTIER_VG-VMS_LV:
> > UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> > /dev/mapper/FINALFRONTIER_VG-DATA_LV:
> > UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> >
> > # blkid /dev/dm*
> > /dev/dm-0: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-1: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-2: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> > /dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> > /dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> > /dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> > /dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> > /dev/dm-0: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-1: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-2: LABEL="_Fedora-Live-Des"
> > UUID="24375583-5faa-4172-8345-5dff1a0e89e5" TYPE="ext4"
> > /dev/dm-3: UUID="7a56249a-63ac-4239-a874-7aa8c2f33195" TYPE="ext4"
> > /dev/dm-6: UUID="55232f85-5753-4de8-bdf5-0df3067f35eb" TYPE="swap"
> > /dev/dm-7: UUID="d9fe324e-abe7-47e0-80c8-7f2e29011249" TYPE="ext4"
> > /dev/dm-8: UUID="e0f5ed2a-6382-43c1-88d9-74da4aa440c0" TYPE="ext4"
> > /dev/dm-9: UUID="23d38b64-99b2-42b8-8064-4468f0f71b1c" TYPE="ext4"
> >
> > # rpm -qa | grep cryptsetup
> > cryptsetup-1.6.2-1.fc20.x86_64
> > cryptsetup-libs-1.6.2-1.fc20.x86_64
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. - Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 22747 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-19 22:22 ` John Wells
@ 2014-08-20 9:09 ` Arno Wagner
2014-08-20 16:18 ` John Wells
0 siblings, 1 reply; 27+ messages in thread
From: Arno Wagner @ 2014-08-20 9:09 UTC (permalink / raw)
To: dm-crypt
On Wed, Aug 20, 2014 at 00:22:29 CEST, John Wells wrote:
> Thanks for your response. This is the result of luksDump on the container:
>
> # cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
Ah, sorry. Did not see that access completely failed.
That means the header was at least partially overwritten.
Can you post or send me a hex-dump of the first 1024 bytes
of this device and the other one?
Command to do so is, e.g.
head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
This will not compromise your security.
> I will try to find the time to recreate the entire scenario. Do you think
> the current container I'm able to open is at risk of corruption as well?
Yes. Something seems to be running amok.
Another queston: After Fedora 20 told you both were not
valid LUKS devices, could you still open the one in Ubuntu
that you could open before?
Arno
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-20 9:09 ` Arno Wagner
@ 2014-08-20 16:18 ` John Wells
2014-08-20 21:15 ` Arno Wagner
0 siblings, 1 reply; 27+ messages in thread
From: John Wells @ 2014-08-20 16:18 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 5188 bytes --]
Thanks Arno.
Something definitely looks amiss:
$ sudo head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
Loopb|
00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
0...........|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000400
$ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hd
��ޭ
00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
|LUKS....aes.....|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
|........xts-plai|
00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
|n64.............|
00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
|........sha1....|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
|...............@|
00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
|N...Q.%.ts...n..|
00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
|o.q....^..[.4:..|
00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
|.y.a.........)..|
000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
|#.Y...w.6cc188db|
000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
|-ccdb-4c8f-97b2-|
000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
|e41198ec6e44....|
000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
|..q.....y...)Y.c|
000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
[;W.....|
000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
|6WE...&.........|
00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
|................|
00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
|................|
00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
|................|
00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
|................|
000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
|................|
000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
|................|
00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
|................|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
Yes, even though Fedora 20 will mount neither volume, Ubuntu 14.04 will
mount the /dev/MORE_VG/MORE_LV partition.
Thanks,
John
On Wed, Aug 20, 2014 at 5:09 AM, Arno Wagner <arno@wagner.name> wrote:
> On Wed, Aug 20, 2014 at 00:22:29 CEST, John Wells wrote:
> > Thanks for your response. This is the result of luksDump on the
> container:
> >
> > # cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> > Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
>
> Ah, sorry. Did not see that access completely failed.
> That means the header was at least partially overwritten.
>
> Can you post or send me a hex-dump of the first 1024 bytes
> of this device and the other one?
>
> Command to do so is, e.g.
>
> head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
>
> This will not compromise your security.
>
> > I will try to find the time to recreate the entire scenario. Do you think
> > the current container I'm able to open is at risk of corruption as well?
>
> Yes. Something seems to be running amok.
>
> Another queston: After Fedora 20 told you both were not
> valid LUKS devices, could you still open the one in Ubuntu
> that you could open before?
>
> Arno
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. - Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 6626 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-20 16:18 ` John Wells
@ 2014-08-20 21:15 ` Arno Wagner
2014-08-21 14:00 ` John Wells
0 siblings, 1 reply; 27+ messages in thread
From: Arno Wagner @ 2014-08-20 21:15 UTC (permalink / raw)
To: dm-crypt
Hi John,
while I have no idea how HOME_LV got in this state, the
hexdump shows what is wrong. I suspect some LVM or Parted
"Magic" on installation caused this.
As the salts in the header are critical for decryption, unless
the LUKS header is somewhere else and the offsets are
wrong (i.e. you are not looking at the place you are think
you are looking at, e.g. due to some LVM problem) the data is
gone.
As to MORE_LV, this should work. I suspect you did the dump
below on Ubuntu, correct? I think Ubuntu may have screwed
up the partitioning so that Fedora 20 does not find MORE_LV
anymore, but Ubuntu finds it.
One more test would show this:
Copy the first 10MB of MORE_LV on *Ubuntu*
head -c 10M /dev/MORE_VG/MORE_LV >> header.dump
do a loopback mount of it it on *Fedora 20*
losetup /dev/loop0 header.dump
and then try to luksOpen /dev/loop0. If that works
on Fedora 20 but MORE_LV does not work on FEDORA 20,
then this is a problem with Fedora 20 having dome issue
accessing the raw MORE_LV partition.
Note that header.dump will contain the key-slots,
so make sure you can secure-erase the header.dump-file again!
(You are still secure, but your passphrase is in there and
if that becomes compromised, changing it will not be enough.)
Arno
On Wed, Aug 20, 2014 at 18:18:33 CEST, John Wells wrote:
> Thanks Arno.
>
> Something definitely looks amiss:
> $ sudo head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> Loopb|
> 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> 0...........|
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> *
> 00000400
>
> $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hd
> ��ޭ
> 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> |LUKS....aes.....|
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> |........xts-plai|
> 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> |n64.............|
> 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> |........sha1....|
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> |...............@|
> 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> |N...Q.%.ts...n..|
> 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> |o.q....^..[.4:..|
> 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> |.y.a.........)..|
> 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> |#.Y...w.6cc188db|
> 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> |-ccdb-4c8f-97b2-|
> 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> |e41198ec6e44....|
> 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> |..q.....y...)Y.c|
> 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> [;W.....|
> 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> |6WE...&.........|
> 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> |................|
> 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> |................|
> 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> |................|
> 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> |................|
> 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> |................|
> 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> |................|
> 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> |................|
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
>
> Yes, even though Fedora 20 will mount neither volume, Ubuntu 14.04 will
> mount the /dev/MORE_VG/MORE_LV partition.
>
> Thanks,
> John
>
>
> On Wed, Aug 20, 2014 at 5:09 AM, Arno Wagner <arno@wagner.name> wrote:
>
> > On Wed, Aug 20, 2014 at 00:22:29 CEST, John Wells wrote:
> > > Thanks for your response. This is the result of luksDump on the
> > container:
> > >
> > > # cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> > > Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
> >
> > Ah, sorry. Did not see that access completely failed.
> > That means the header was at least partially overwritten.
> >
> > Can you post or send me a hex-dump of the first 1024 bytes
> > of this device and the other one?
> >
> > Command to do so is, e.g.
> >
> > head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> >
> > This will not compromise your security.
> >
> > > I will try to find the time to recreate the entire scenario. Do you think
> > > the current container I'm able to open is at risk of corruption as well?
> >
> > Yes. Something seems to be running amok.
> >
> > Another queston: After Fedora 20 told you both were not
> > valid LUKS devices, could you still open the one in Ubuntu
> > that you could open before?
> >
> > Arno
> >
> > --
> > Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> > GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> > ----
> > A good decision is based on knowledge and not on numbers. - Plato
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-20 21:15 ` Arno Wagner
@ 2014-08-21 14:00 ` John Wells
2014-08-21 14:25 ` Robert Nichols
2014-08-21 14:29 ` Arno Wagner
0 siblings, 2 replies; 27+ messages in thread
From: John Wells @ 2014-08-21 14:00 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 8489 bytes --]
I will try what you say. To add to the weirdness, I came into work this
morning with a unresponsive machine. I hard reset, booted Ubuntu, but at
that point *Ubuntu wouldn't recognize either partition, and both had the
same "GNU Parted Loopback 0" in the output of "head -c 1024 /volume | hd".
Neither partition was recognized by luksDump. I panicked.
Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to normal and I
could mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with the
"GNU Parted Loopback 0" output.
This makes no sense to me. How could the leading bits be different each
time I booted up? Could datamapper be assigning the wrong device to the
logical volume in some way? It just makes no sense.
On Wed, Aug 20, 2014 at 5:15 PM, Arno Wagner <arno@wagner.name> wrote:
> Hi John,
>
> while I have no idea how HOME_LV got in this state, the
> hexdump shows what is wrong. I suspect some LVM or Parted
> "Magic" on installation caused this.
> As the salts in the header are critical for decryption, unless
> the LUKS header is somewhere else and the offsets are
> wrong (i.e. you are not looking at the place you are think
> you are looking at, e.g. due to some LVM problem) the data is
> gone.
>
> As to MORE_LV, this should work. I suspect you did the dump
> below on Ubuntu, correct? I think Ubuntu may have screwed
> up the partitioning so that Fedora 20 does not find MORE_LV
> anymore, but Ubuntu finds it.
>
> One more test would show this:
>
> Copy the first 10MB of MORE_LV on *Ubuntu*
>
> head -c 10M /dev/MORE_VG/MORE_LV >> header.dump
>
> do a loopback mount of it it on *Fedora 20*
>
> losetup /dev/loop0 header.dump
>
> and then try to luksOpen /dev/loop0. If that works
> on Fedora 20 but MORE_LV does not work on FEDORA 20,
> then this is a problem with Fedora 20 having dome issue
> accessing the raw MORE_LV partition.
>
> Note that header.dump will contain the key-slots,
> so make sure you can secure-erase the header.dump-file again!
> (You are still secure, but your passphrase is in there and
> if that becomes compromised, changing it will not be enough.)
>
> Arno
>
>
>
> On Wed, Aug 20, 2014 at 18:18:33 CEST, John Wells wrote:
> > Thanks Arno.
> >
> > Something definitely looks amiss:
> > $ sudo head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> > 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> > Loopb|
> > 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> > 0...........|
> > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00000400
> >
> > $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hd
> >
> ��ޭ
> > 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> > |LUKS....aes.....|
> > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> > |........xts-plai|
> > 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |n64.............|
> > 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> > |........sha1....|
> > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> > |...............@|
> > 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> > |N...Q.%.ts...n..|
> > 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> > |o.q....^..[.4:..|
> > 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> > |.y.a.........)..|
> > 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> > |#.Y...w.6cc188db|
> > 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> > |-ccdb-4c8f-97b2-|
> > 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> > |e41198ec6e44....|
> > 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> > |..q.....y...)Y.c|
> > 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> > [;W.....|
> > 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> > |6WE...&.........|
> > 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> > |................|
> > 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> > |................|
> > 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> > |................|
> > 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> > |................|
> > 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> > |................|
> > 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> > |................|
> > 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> > |................|
> > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> >
> > Yes, even though Fedora 20 will mount neither volume, Ubuntu 14.04 will
> > mount the /dev/MORE_VG/MORE_LV partition.
> >
> > Thanks,
> > John
> >
> >
> > On Wed, Aug 20, 2014 at 5:09 AM, Arno Wagner <arno@wagner.name> wrote:
> >
> > > On Wed, Aug 20, 2014 at 00:22:29 CEST, John Wells wrote:
> > > > Thanks for your response. This is the result of luksDump on the
> > > container:
> > > >
> > > > # cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> > > > Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
> > >
> > > Ah, sorry. Did not see that access completely failed.
> > > That means the header was at least partially overwritten.
> > >
> > > Can you post or send me a hex-dump of the first 1024 bytes
> > > of this device and the other one?
> > >
> > > Command to do so is, e.g.
> > >
> > > head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> > >
> > > This will not compromise your security.
> > >
> > > > I will try to find the time to recreate the entire scenario. Do you
> think
> > > > the current container I'm able to open is at risk of corruption as
> well?
> > >
> > > Yes. Something seems to be running amok.
> > >
> > > Another queston: After Fedora 20 told you both were not
> > > valid LUKS devices, could you still open the one in Ubuntu
> > > that you could open before?
> > >
> > > Arno
> > >
> > > --
> > > Arno Wagner, Dr. sc. techn., Dipl. Inform., Email:
> arno@wagner.name
> > > GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D
> 9718
> > > ----
> > > A good decision is based on knowledge and not on numbers. - Plato
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > >
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. - Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 10668 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 14:00 ` John Wells
@ 2014-08-21 14:25 ` Robert Nichols
2014-08-21 15:55 ` John Wells
2014-08-21 14:29 ` Arno Wagner
1 sibling, 1 reply; 27+ messages in thread
From: Robert Nichols @ 2014-08-21 14:25 UTC (permalink / raw)
To: dm-crypt
On 08/21/2014 09:00 AM, John Wells wrote:
> I will try what you say. To add to the weirdness, I came into work this morning
> with a unresponsive machine. I hard reset, booted Ubuntu, but at that point
> *Ubuntu wouldn't recognize either partition, and both had the same "GNU Parted
> Loopback 0" in the output of "head -c 1024 /volume | hd". Neither partition was
> recognized by luksDump. I panicked.
>
> Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to normal and I could
> mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with the "GNU Parted
> Loopback 0" output.
>
> This makes no sense to me. How could the leading bits be different each time I
> booted up? Could datamapper be assigning the wrong device to the logical volume
> in some way? It just makes no sense.
You can run (as root) "dmsetup deps" on each device in /dev/mapper to see
the major and minor device numbers needed for each entry.
--
Bob Nichols "NOSPAM" is really part of my email address.
Do NOT delete it.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 14:00 ` John Wells
2014-08-21 14:25 ` Robert Nichols
@ 2014-08-21 14:29 ` Arno Wagner
1 sibling, 0 replies; 27+ messages in thread
From: Arno Wagner @ 2014-08-21 14:29 UTC (permalink / raw)
To: dm-crypt
On Thu, Aug 21, 2014 at 16:00:21 CEST, John Wells wrote:
> I will try what you say. To add to the weirdness, I came into work this
> morning with a unresponsive machine. I hard reset, booted Ubuntu, but at
> that point *Ubuntu wouldn't recognize either partition, and both had the
> same "GNU Parted Loopback 0" in the output of "head -c 1024 /volume | hd".
Urgh. Pretty bad.
> Neither partition was recognized by luksDump. I panicked.
Naturally, cryptsetup needs to see the header.
> Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to normal and I
> could mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with the
> "GNU Parted Loopback 0" output.
O.k., but that tells us that the LUKS header does _not_
get overwritten by this, but rather that there is some
"probabilistic" (i.e. completely messed up) partition
detection and remapping going on. All these things with
LVM, user-space RAID assembly, udev, systemd, etc. have
not been good for stability at all. It used to be that
Linux was rock-solid, but it seems that those days are past
for mainstream Linux. All this "magic" does more harm
than good, and not least because of mismatch between
developper egos and skill.
> This makes no sense to me. How could the leading bits be different each
> time I booted up? Could datamapper be assigning the wrong device to the
> logical volume in some way? It just makes no sense.
It does only make sense of some developer(s) completely
borked some central boot-up functionality. Unfortunately,
several teams have been hard at work to make that a reality,
see above. One consequence is that, as you found, a disk-
layout done with Fedora can cause random, hard to debug
and non-intuitive failures with Ubuntu.
Some people have lost context and sight of the goal
completely here. This is really a disgrace. KISS is
what distinguishes engineers and amateurs.
Just do the test, and if that works we may find a way to
recover from this mess. It goes without saying that you
should probably avoid Ubuntu in the future or only use
native Ubuntu installatons.
Arno
P.S.: Sorry for the rant, but the more amateurs mess with
it, the more Linux gets to be like Windows. I find that
comletely unacceptable.
>
> On Wed, Aug 20, 2014 at 5:15 PM, Arno Wagner <arno@wagner.name> wrote:
>
> > Hi John,
> >
> > while I have no idea how HOME_LV got in this state, the
> > hexdump shows what is wrong. I suspect some LVM or Parted
> > "Magic" on installation caused this.
> > As the salts in the header are critical for decryption, unless
> > the LUKS header is somewhere else and the offsets are
> > wrong (i.e. you are not looking at the place you are think
> > you are looking at, e.g. due to some LVM problem) the data is
> > gone.
> >
> > As to MORE_LV, this should work. I suspect you did the dump
> > below on Ubuntu, correct? I think Ubuntu may have screwed
> > up the partitioning so that Fedora 20 does not find MORE_LV
> > anymore, but Ubuntu finds it.
> >
> > One more test would show this:
> >
> > Copy the first 10MB of MORE_LV on *Ubuntu*
> >
> > head -c 10M /dev/MORE_VG/MORE_LV >> header.dump
> >
> > do a loopback mount of it it on *Fedora 20*
> >
> > losetup /dev/loop0 header.dump
> >
> > and then try to luksOpen /dev/loop0. If that works
> > on Fedora 20 but MORE_LV does not work on FEDORA 20,
> > then this is a problem with Fedora 20 having dome issue
> > accessing the raw MORE_LV partition.
> >
> > Note that header.dump will contain the key-slots,
> > so make sure you can secure-erase the header.dump-file again!
> > (You are still secure, but your passphrase is in there and
> > if that becomes compromised, changing it will not be enough.)
> >
> > Arno
> >
> >
> >
> > On Wed, Aug 20, 2014 at 18:18:33 CEST, John Wells wrote:
> > > Thanks Arno.
> > >
> > > Something definitely looks amiss:
> > > $ sudo head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> > > 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> > > Loopb|
> > > 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> > > 0...........|
> > > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > *
> > > 00000400
> > >
> > > $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hd
> > >
> > ��ޭ
> > > 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> > > |LUKS....aes.....|
> > > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> > > |........xts-plai|
> > > 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |n64.............|
> > > 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> > > |........sha1....|
> > > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> > > |...............@|
> > > 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> > > |N...Q.%.ts...n..|
> > > 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> > > |o.q....^..[.4:..|
> > > 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> > > |.y.a.........)..|
> > > 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> > > |#.Y...w.6cc188db|
> > > 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> > > |-ccdb-4c8f-97b2-|
> > > 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> > > |e41198ec6e44....|
> > > 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> > > |..q.....y...)Y.c|
> > > 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> > > [;W.....|
> > > 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> > > |6WE...&.........|
> > > 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> > > |................|
> > > 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> > > |................|
> > > 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> > > |................|
> > > 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> > > |................|
> > > 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> > > |................|
> > > 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> > > |................|
> > > 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > > 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> > > |................|
> > > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > > |................|
> > >
> > > Yes, even though Fedora 20 will mount neither volume, Ubuntu 14.04 will
> > > mount the /dev/MORE_VG/MORE_LV partition.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Wed, Aug 20, 2014 at 5:09 AM, Arno Wagner <arno@wagner.name> wrote:
> > >
> > > > On Wed, Aug 20, 2014 at 00:22:29 CEST, John Wells wrote:
> > > > > Thanks for your response. This is the result of luksDump on the
> > > > container:
> > > > >
> > > > > # cryptsetup luksDump /dev/FINALFRONTIER_VG/HOME_LV
> > > > > Device /dev/FINALFRONTIER_VG/HOME_LV is not a valid LUKS device.
> > > >
> > > > Ah, sorry. Did not see that access completely failed.
> > > > That means the header was at least partially overwritten.
> > > >
> > > > Can you post or send me a hex-dump of the first 1024 bytes
> > > > of this device and the other one?
> > > >
> > > > Command to do so is, e.g.
> > > >
> > > > head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hd
> > > >
> > > > This will not compromise your security.
> > > >
> > > > > I will try to find the time to recreate the entire scenario. Do you
> > think
> > > > > the current container I'm able to open is at risk of corruption as
> > well?
> > > >
> > > > Yes. Something seems to be running amok.
> > > >
> > > > Another queston: After Fedora 20 told you both were not
> > > > valid LUKS devices, could you still open the one in Ubuntu
> > > > that you could open before?
> > > >
> > > > Arno
> > > >
> > > > --
> > > > Arno Wagner, Dr. sc. techn., Dipl. Inform., Email:
> > arno@wagner.name
> > > > GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D
> > 9718
> > > > ----
> > > > A good decision is based on knowledge and not on numbers. - Plato
> > > > _______________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt@saout.de
> > > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > >
> >
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> >
> > --
> > Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> > GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> > ----
> > A good decision is based on knowledge and not on numbers. - Plato
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 14:25 ` Robert Nichols
@ 2014-08-21 15:55 ` John Wells
2014-08-21 16:13 ` Jonas Meurer
2014-08-21 17:01 ` Robert Nichols
0 siblings, 2 replies; 27+ messages in thread
From: John Wells @ 2014-08-21 15:55 UTC (permalink / raw)
To: Robert Nichols; +Cc: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 1387 bytes --]
Thanks. How would I use this information to track back to a potential
problem?
On Aug 21, 2014 10:41 AM, "Robert Nichols" <rnicholsNOSPAM@comcast.net>
wrote:
> On 08/21/2014 09:00 AM, John Wells wrote:
>
>> I will try what you say. To add to the weirdness, I came into work this
>> morning
>> with a unresponsive machine. I hard reset, booted Ubuntu, but at that
>> point
>> *Ubuntu wouldn't recognize either partition, and both had the same "GNU
>> Parted
>> Loopback 0" in the output of "head -c 1024 /volume | hd". Neither
>> partition was
>> recognized by luksDump. I panicked.
>>
>> Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to normal and I
>> could
>> mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with the "GNU
>> Parted
>> Loopback 0" output.
>>
>> This makes no sense to me. How could the leading bits be different each
>> time I
>> booted up? Could datamapper be assigning the wrong device to the logical
>> volume
>> in some way? It just makes no sense.
>>
>
> You can run (as root) "dmsetup deps" on each device in /dev/mapper to see
> the major and minor device numbers needed for each entry.
>
> --
> Bob Nichols "NOSPAM" is really part of my email address.
> Do NOT delete it.
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 2010 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 15:55 ` John Wells
@ 2014-08-21 16:13 ` Jonas Meurer
2014-08-21 20:46 ` John Wells
2014-08-21 17:01 ` Robert Nichols
1 sibling, 1 reply; 27+ messages in thread
From: Jonas Meurer @ 2014-08-21 16:13 UTC (permalink / raw)
To: dm-crypt
Hello John,
Am 21.08.2014 um 17:55 schrieb John Wells:
> Thanks. How would I use this information to track back to a potential
> problem?
I'm not convinced yet, that your system screwed up partition and/or
device ordering. First you should try to find out which devices are
recognized both in Fedora and Ubuntu and compare these.
Try to run 'lsblk', 'dmsetup ls' and 'dmsetup deps' on both systems and
compare or post the result here.
Which Ubuntu version do you run? Do you use an installed Ubuntu or
rather a live image? It would help if you could provide some more
information regarding your system.
Kind regards,
jonas
>
> On Aug 21, 2014 10:41 AM, "Robert Nichols" <rnicholsNOSPAM@comcast.net
> <mailto:rnicholsNOSPAM@comcast.net>> wrote:
>
> On 08/21/2014 09:00 AM, John Wells wrote:
>
> I will try what you say. To add to the weirdness, I came into
> work this morning
> with a unresponsive machine. I hard reset, booted Ubuntu, but at
> that point
> *Ubuntu wouldn't recognize either partition, and both had the
> same "GNU Parted
> Loopback 0" in the output of "head -c 1024 /volume | hd".
> Neither partition was
> recognized by luksDump. I panicked.
>
> Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to
> normal and I could
> mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with
> the "GNU Parted
> Loopback 0" output.
>
> This makes no sense to me. How could the leading bits be
> different each time I
> booted up? Could datamapper be assigning the wrong device to the
> logical volume
> in some way? It just makes no sense.
>
>
> You can run (as root) "dmsetup deps" on each device in /dev/mapper
> to see
> the major and minor device numbers needed for each entry.
>
> --
> Bob Nichols "NOSPAM" is really part of my email address.
> Do NOT delete it.
>
> _________________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de <mailto:dm-crypt@saout.de>
> http://www.saout.de/mailman/__listinfo/dm-crypt
> <http://www.saout.de/mailman/listinfo/dm-crypt>
>
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 15:55 ` John Wells
2014-08-21 16:13 ` Jonas Meurer
@ 2014-08-21 17:01 ` Robert Nichols
1 sibling, 0 replies; 27+ messages in thread
From: Robert Nichols @ 2014-08-21 17:01 UTC (permalink / raw)
To: dm-crypt
On 08/21/2014 10:55 AM, John Wells wrote:
> Thanks. How would I use this information to track back to a potential problem?
>
> On Aug 21, 2014 10:41 AM, "Robert Nichols" <rnicholsNOSPAM@comcast.net
> <mailto:rnicholsNOSPAM@comcast.net>> wrote:
>
> You can run (as root) "dmsetup deps" on each device in /dev/mapper to see
> the major and minor device numbers needed for each entry.
>
You could at least see whether the same device numbers show up for the
boots where the volume is working vs. the times when it is not working.
If they are different, that would be an important clue.
--
Bob Nichols "NOSPAM" is really part of my email address.
Do NOT delete it.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 16:13 ` Jonas Meurer
@ 2014-08-21 20:46 ` John Wells
2014-08-21 22:54 ` Arno Wagner
` (2 more replies)
0 siblings, 3 replies; 27+ messages in thread
From: John Wells @ 2014-08-21 20:46 UTC (permalink / raw)
To: Jonas Meurer; +Cc: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 14465 bytes --]
Attached below, I've included the output of running these commands from
first Ubuntu and then Fedora. Note, apparently they map the raid device
numbers differently. Also, you'll note there's a cryptd mounted in Ubuntu.
That's the one i can successfully mount from Ubuntu.
To make things even stranger, I was ***able to luksOpen and then mount
/dev/MORE_VG/MORE_LV this time while booted in the Fedora 20*** live cd,
while as you recall I was unable to before. I'm also including the results
of dumping the head of the dev file from Fedora.
I'm completely at a loss. At this point I feel I should probably backup the
entire machine and re-install from scratch, but I'll hold off a bit longer.
The inconsistent behavior is really disconcerting.
Thanks,
John
####################### FIRST FROM UBUNTU ########################
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 37.3G 0 disk
├─sda1 8:1 0 100M 0 part
└─sda2 8:2 0 37.2G 0 part
sdb 8:16 0 1.8T 0 disk
└─sdb1 8:17 0 1.8T 0 part
└─md0 9:0 0 1.8T 0 raid1
└─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
└─cryptd (dm-7) 252:7 0 800G 0 crypt /home
sdc 8:32 0 1.8T 0 disk
└─sdc1 8:33 0 1.8T 0 part
└─md2 9:2 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
└─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
sdd 8:48 0 1.8T 0 disk
└─sdd1 8:49 0 1.8T 0 part
└─md2 9:2 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
└─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
sde 8:64 0 167.7G 0 disk
├─sde1 8:65 0 3G 0 part /boot
├─sde2 8:66 0 12G 0 part
└─sde3 8:67 0 152.8G 0 part
└─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
sdf 8:80 0 1.8T 0 disk
└─sdf1 8:81 0 1.8T 0 part
sdg 8:96 0 1.8T 0 disk
└─sdg1 8:97 0 1.8T 0 part
└─BACKUPVG-BACKUPLV (dm-8) 252:8 0 1.2T 0 lvm /backup
sr0 11:0 1 1024M 0 rom
# dmsetup ls
FINALFRONTIER_VG-VMS_LV (252:5)
BACKUPVG-BACKUPLV (252:8)
MORE_VG-MORE_LV (252:1)
FINALFRONTIER_VG-OPT_LV (252:4)
FINALFRONTIER_VG-SWAP_LV (252:3)
SSDROOT_VG-ROOT_LV (252:0)
cryptd (252:7)
FINALFRONTIER_VG-DATA_LV (252:6)
FINALFRONTIER_VG-HOME_LV (252:2)
# dmsetup deps
FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 2)
BACKUPVG-BACKUPLV: 1 dependencies : (8, 97)
MORE_VG-MORE_LV: 1 dependencies : (9, 0)
FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 2)
FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 2)
SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 67)
cryptd: 1 dependencies : (252, 1)
FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 2)
FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 2)
########################## THEN FROM FEDORA #########################
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 37.3G 0 disk
├─sda1 8:1 0 100M 0 part
└─sda2 8:2 0 37.2G 0 part
sdb 8:16 1 14.6G 0 disk
├─sdb1 8:17 1 953M 0 part
/run/initramfs/live
├─sdb2 8:18 1 4.9M 0 part
└─sdb3 8:19 1 19.7M 0 part
sdc 8:32 0 1.8T 0 disk
└─sdc1 8:33 0 1.8T 0 part
└─md126 9:126 0 1.8T 0 raid1
└─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
sdd 8:48 0 1.8T 0 disk
└─sdd1 8:49 0 1.8T 0 part
└─md127 9:127 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
└─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
sde 8:64 0 1.8T 0 disk
└─sde1 8:65 0 1.8T 0 part
└─md127 9:127 0 1.8T 0 raid1
├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
└─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
sdf 8:80 0 167.7G 0 disk
├─sdf1 8:81 0 3G 0 part
├─sdf2 8:82 0 12G 0 part
└─sdf3 8:83 0 152.8G 0 part
└─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
sdg 8:96 0 1.8T 0 disk
└─sdg1 8:97 0 1.8T 0 part
sdh 8:112 0 1.8T 0 disk
└─sdh1 8:113 0 1.8T 0 part
└─BACKUPVG-BACKUPLV 253:10 0 1.2T 0 lvm /backup
sr0 11:0 1 1024M 0 rom
loop0 7:0 0 8K 1 loop
loop1 7:1 0 1.2M 1 loop
└─live-osimg-min 253:2 0 4G 1 dm
loop2 7:2 0 886.8M 1 loop
loop3 7:3 0 4G 1 loop
├─live-rw 253:0 0 4G 0 dm /
├─live-base 253:1 0 4G 1 dm
└─live-osimg-min 253:2 0 4G 1 dm
loop4 7:4 0 512M 0 loop
└─live-rw 253:0 0 4G 0 dm /
# dmsetup ls
live-base (253:1)
FINALFRONTIER_VG-VMS_LV (253:8)
BACKUPVG-BACKUPLV (253:10)
MORE_VG-MORE_LV (253:4)
FINALFRONTIER_VG-OPT_LV (253:7)
live-osimg-min (253:2)
FINALFRONTIER_VG-SWAP_LV (253:6)
live-rw (253:0)
SSDROOT_VG-ROOT_LV (253:3)
FINALFRONTIER_VG-DATA_LV (253:9)
FINALFRONTIER_VG-HOME_LV (253:5)
# dmsetup deps
live-base: 1 dependencies : (7, 3)
FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 127)
BACKUPVG-BACKUPLV: 1 dependencies : (8, 113)
MORE_VG-MORE_LV: 1 dependencies : (9, 126)
FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 127)
live-osimg-min: 2 dependencies : (7, 1) (7, 3)
FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 127)
live-rw: 2 dependencies : (7, 4) (7, 3)
SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 83)
FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 127)
FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 127)
###################### AND NOW THE dumps of both devs from FEDORA
############################
# head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hexdump -C
00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
Loopb|
00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
0...........|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000400
# head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
|LUKS....aes.....|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
|........xts-plai|
00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
|n64.............|
00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
|........sha1....|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
|...............@|
00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
|N...Q.%.ts...n..|
00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
|o.q....^..[.4:..|
00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
|.y.a.........)..|
000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
|#.Y...w.6cc188db|
000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
|-ccdb-4c8f-97b2-|
000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
|e41198ec6e44....|
000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
|..q.....y...)Y.c|
000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
[;W.....|
000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
|6WE...&.........|
00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
|................|
00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
|................|
00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
|................|
00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
|................|
000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
|................|
000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
|................|
00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
|................|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
On Thu, Aug 21, 2014 at 12:13 PM, Jonas Meurer <jonas@freesources.org>
wrote:
> Hello John,
>
> Am 21.08.2014 um 17:55 schrieb John Wells:
> > Thanks. How would I use this information to track back to a potential
> > problem?
>
> I'm not convinced yet, that your system screwed up partition and/or
> device ordering. First you should try to find out which devices are
> recognized both in Fedora and Ubuntu and compare these.
>
> Try to run 'lsblk', 'dmsetup ls' and 'dmsetup deps' on both systems and
> compare or post the result here.
>
> Which Ubuntu version do you run? Do you use an installed Ubuntu or
> rather a live image? It would help if you could provide some more
> information regarding your system.
>
> Kind regards,
> jonas
>
> >
> > On Aug 21, 2014 10:41 AM, "Robert Nichols" <rnicholsNOSPAM@comcast.net
> > <mailto:rnicholsNOSPAM@comcast.net>> wrote:
> >
> > On 08/21/2014 09:00 AM, John Wells wrote:
> >
> > I will try what you say. To add to the weirdness, I came into
> > work this morning
> > with a unresponsive machine. I hard reset, booted Ubuntu, but at
> > that point
> > *Ubuntu wouldn't recognize either partition, and both had the
> > same "GNU Parted
> > Loopback 0" in the output of "head -c 1024 /volume | hd".
> > Neither partition was
> > recognized by luksDump. I panicked.
> >
> > Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to
> > normal and I could
> > mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with
> > the "GNU Parted
> > Loopback 0" output.
> >
> > This makes no sense to me. How could the leading bits be
> > different each time I
> > booted up? Could datamapper be assigning the wrong device to the
> > logical volume
> > in some way? It just makes no sense.
> >
> >
> > You can run (as root) "dmsetup deps" on each device in /dev/mapper
> > to see
> > the major and minor device numbers needed for each entry.
> >
> > --
> > Bob Nichols "NOSPAM" is really part of my email address.
> > Do NOT delete it.
> >
> > _________________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de <mailto:dm-crypt@saout.de>
> > http://www.saout.de/mailman/__listinfo/dm-crypt
> > <http://www.saout.de/mailman/listinfo/dm-crypt>
> >
> >
> >
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 21736 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 20:46 ` John Wells
@ 2014-08-21 22:54 ` Arno Wagner
2014-08-22 3:52 ` John Wells
2014-08-22 5:30 ` Heinz Diehl
2014-08-22 9:08 ` Jonas Meurer
2 siblings, 1 reply; 27+ messages in thread
From: Arno Wagner @ 2014-08-21 22:54 UTC (permalink / raw)
To: dm-crypt
This looks to me like some massive issues with user-space tools,
starting with the md-devices, that in Fedora are md0 and md2,
but in Ubuntu are md126 and md127.
This likely means they are of the completely braindead "new"
RAID-superblock formats where the RAID assembly is not done
by the RAID controller (i.e. the kernel) as it should be, but
by something in userspace. Change the userspace, lose proper
RAID assembly. The idiots responsible for that have not
understood what layering is and why you do it and have
implemented a massive step backwards.
The way around this is to use the old superblock format 0.90
with partition type 0xfd. With that and any kernel doing RAID
auto-assembly, you always get the same evices. Of course, some
distros are utterly brain-dead and re-map the 0.90 devices to
new names via user-space re-assembly, which are then persistent.
Had that with one "recovery CD once", that I needed an hour to
recover from booting.
Yes, the 1.x formats have better capabilities, but what use
is that when they are fundamentally broken otherwise? "Better
capabilities" is not a valid reason to break things that worked
before.
My advice around this whole mess would be to make a simple,
flat patitioning and RAID schema yourself manually (superblock
0.90, type 0xfd, no LVM, no partitioning in RAID, just make
more RAID arrays) and install into that. That seems the only
way to get KISS with regard to RAID and partitioning these
days and KISS is the only thing that gives you reliability
and ease of debugging.
I think I will add a warning to the FAQ that changing distros
can make your LUKS containers hard or impossible to access due
to LVM and new RAID superblock userspace differences and that
it may mess things up enough that a move back to the old distro
may not be sufficient to fix things. How these jokers expect
people to deal with multiple distros on the same machine is
beyond me.
Arno
On Thu, Aug 21, 2014 at 22:46:04 CEST, John Wells wrote:
> Attached below, I've included the output of running these commands from
> first Ubuntu and then Fedora. Note, apparently they map the raid device
> numbers differently. Also, you'll note there's a cryptd mounted in Ubuntu.
> That's the one i can successfully mount from Ubuntu.
>
> To make things even stranger, I was ***able to luksOpen and then mount
> /dev/MORE_VG/MORE_LV this time while booted in the Fedora 20*** live cd,
> while as you recall I was unable to before. I'm also including the results
> of dumping the head of the dev file from Fedora.
>
> I'm completely at a loss. At this point I feel I should probably backup the
> entire machine and re-install from scratch, but I'll hold off a bit longer.
> The inconsistent behavior is really disconcerting.
>
> Thanks,
> John
>
> ####################### FIRST FROM UBUNTU ########################
> # lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 37.3G 0 disk
> ├─sda1 8:1 0 100M 0 part
> └─sda2 8:2 0 37.2G 0 part
> sdb 8:16 0 1.8T 0 disk
> └─sdb1 8:17 0 1.8T 0 part
> └─md0 9:0 0 1.8T 0 raid1
> └─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
> └─cryptd (dm-7) 252:7 0 800G 0 crypt /home
> sdc 8:32 0 1.8T 0 disk
> └─sdc1 8:33 0 1.8T 0 part
> └─md2 9:2 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> sdd 8:48 0 1.8T 0 disk
> └─sdd1 8:49 0 1.8T 0 part
> └─md2 9:2 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> sde 8:64 0 167.7G 0 disk
> ├─sde1 8:65 0 3G 0 part /boot
> ├─sde2 8:66 0 12G 0 part
> └─sde3 8:67 0 152.8G 0 part
> └─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
> sdf 8:80 0 1.8T 0 disk
> └─sdf1 8:81 0 1.8T 0 part
> sdg 8:96 0 1.8T 0 disk
> └─sdg1 8:97 0 1.8T 0 part
> └─BACKUPVG-BACKUPLV (dm-8) 252:8 0 1.2T 0 lvm /backup
> sr0 11:0 1 1024M 0 rom
> # dmsetup ls
> FINALFRONTIER_VG-VMS_LV (252:5)
> BACKUPVG-BACKUPLV (252:8)
> MORE_VG-MORE_LV (252:1)
> FINALFRONTIER_VG-OPT_LV (252:4)
> FINALFRONTIER_VG-SWAP_LV (252:3)
> SSDROOT_VG-ROOT_LV (252:0)
> cryptd (252:7)
> FINALFRONTIER_VG-DATA_LV (252:6)
> FINALFRONTIER_VG-HOME_LV (252:2)
> # dmsetup deps
> FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 2)
> BACKUPVG-BACKUPLV: 1 dependencies : (8, 97)
> MORE_VG-MORE_LV: 1 dependencies : (9, 0)
> FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 2)
> FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 2)
> SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 67)
> cryptd: 1 dependencies : (252, 1)
> FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 2)
> FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 2)
>
> ########################## THEN FROM FEDORA #########################
>
> # lsblk
> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda 8:0 0 37.3G 0 disk
> ├─sda1 8:1 0 100M 0 part
> └─sda2 8:2 0 37.2G 0 part
> sdb 8:16 1 14.6G 0 disk
> ├─sdb1 8:17 1 953M 0 part
> /run/initramfs/live
> ├─sdb2 8:18 1 4.9M 0 part
> └─sdb3 8:19 1 19.7M 0 part
> sdc 8:32 0 1.8T 0 disk
> └─sdc1 8:33 0 1.8T 0 part
> └─md126 9:126 0 1.8T 0 raid1
> └─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
> sdd 8:48 0 1.8T 0 disk
> └─sdd1 8:49 0 1.8T 0 part
> └─md127 9:127 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> sde 8:64 0 1.8T 0 disk
> └─sde1 8:65 0 1.8T 0 part
> └─md127 9:127 0 1.8T 0 raid1
> ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
> ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> sdf 8:80 0 167.7G 0 disk
> ├─sdf1 8:81 0 3G 0 part
> ├─sdf2 8:82 0 12G 0 part
> └─sdf3 8:83 0 152.8G 0 part
> └─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
> sdg 8:96 0 1.8T 0 disk
> └─sdg1 8:97 0 1.8T 0 part
> sdh 8:112 0 1.8T 0 disk
> └─sdh1 8:113 0 1.8T 0 part
> └─BACKUPVG-BACKUPLV 253:10 0 1.2T 0 lvm /backup
> sr0 11:0 1 1024M 0 rom
> loop0 7:0 0 8K 1 loop
> loop1 7:1 0 1.2M 1 loop
> └─live-osimg-min 253:2 0 4G 1 dm
> loop2 7:2 0 886.8M 1 loop
> loop3 7:3 0 4G 1 loop
> ├─live-rw 253:0 0 4G 0 dm /
> ├─live-base 253:1 0 4G 1 dm
> └─live-osimg-min 253:2 0 4G 1 dm
> loop4 7:4 0 512M 0 loop
> └─live-rw 253:0 0 4G 0 dm /
> # dmsetup ls
> live-base (253:1)
> FINALFRONTIER_VG-VMS_LV (253:8)
> BACKUPVG-BACKUPLV (253:10)
> MORE_VG-MORE_LV (253:4)
> FINALFRONTIER_VG-OPT_LV (253:7)
> live-osimg-min (253:2)
> FINALFRONTIER_VG-SWAP_LV (253:6)
> live-rw (253:0)
> SSDROOT_VG-ROOT_LV (253:3)
> FINALFRONTIER_VG-DATA_LV (253:9)
> FINALFRONTIER_VG-HOME_LV (253:5)
> # dmsetup deps
> live-base: 1 dependencies : (7, 3)
> FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 127)
> BACKUPVG-BACKUPLV: 1 dependencies : (8, 113)
> MORE_VG-MORE_LV: 1 dependencies : (9, 126)
> FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 127)
> live-osimg-min: 2 dependencies : (7, 1) (7, 3)
> FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 127)
> live-rw: 2 dependencies : (7, 4) (7, 3)
> SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 83)
> FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 127)
> FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 127)
>
> ###################### AND NOW THE dumps of both devs from FEDORA
> ############################
> # head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hexdump -C
> 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> Loopb|
> 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> 0...........|
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> *
> 00000400
>
>
>
> # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> |LUKS....aes.....|
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> |........xts-plai|
> 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> |n64.............|
> 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> |........sha1....|
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> |...............@|
> 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> |N...Q.%.ts...n..|
> 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> |o.q....^..[.4:..|
> 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> |.y.a.........)..|
> 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> |#.Y...w.6cc188db|
> 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> |-ccdb-4c8f-97b2-|
> 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> |e41198ec6e44....|
> 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> |..q.....y...)Y.c|
> 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> [;W.....|
> 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> |6WE...&.........|
> 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> |................|
> 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> |................|
> 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> |................|
> 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> |................|
> 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> |................|
> 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> |................|
> 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> |................|
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
>
>
>
>
> On Thu, Aug 21, 2014 at 12:13 PM, Jonas Meurer <jonas@freesources.org>
> wrote:
>
> > Hello John,
> >
> > Am 21.08.2014 um 17:55 schrieb John Wells:
> > > Thanks. How would I use this information to track back to a potential
> > > problem?
> >
> > I'm not convinced yet, that your system screwed up partition and/or
> > device ordering. First you should try to find out which devices are
> > recognized both in Fedora and Ubuntu and compare these.
> >
> > Try to run 'lsblk', 'dmsetup ls' and 'dmsetup deps' on both systems and
> > compare or post the result here.
> >
> > Which Ubuntu version do you run? Do you use an installed Ubuntu or
> > rather a live image? It would help if you could provide some more
> > information regarding your system.
> >
> > Kind regards,
> > jonas
> >
> > >
> > > On Aug 21, 2014 10:41 AM, "Robert Nichols" <rnicholsNOSPAM@comcast.net
> > > <mailto:rnicholsNOSPAM@comcast.net>> wrote:
> > >
> > > On 08/21/2014 09:00 AM, John Wells wrote:
> > >
> > > I will try what you say. To add to the weirdness, I came into
> > > work this morning
> > > with a unresponsive machine. I hard reset, booted Ubuntu, but at
> > > that point
> > > *Ubuntu wouldn't recognize either partition, and both had the
> > > same "GNU Parted
> > > Loopback 0" in the output of "head -c 1024 /volume | hd".
> > > Neither partition was
> > > recognized by luksDump. I panicked.
> > >
> > > Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to
> > > normal and I could
> > > mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted with
> > > the "GNU Parted
> > > Loopback 0" output.
> > >
> > > This makes no sense to me. How could the leading bits be
> > > different each time I
> > > booted up? Could datamapper be assigning the wrong device to the
> > > logical volume
> > > in some way? It just makes no sense.
> > >
> > >
> > > You can run (as root) "dmsetup deps" on each device in /dev/mapper
> > > to see
> > > the major and minor device numbers needed for each entry.
> > >
> > > --
> > > Bob Nichols "NOSPAM" is really part of my email address.
> > > Do NOT delete it.
> > >
> > > _________________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de <mailto:dm-crypt@saout.de>
> > > http://www.saout.de/mailman/__listinfo/dm-crypt
> > > <http://www.saout.de/mailman/listinfo/dm-crypt>
> > >
> > >
> > >
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > >
> >
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 22:54 ` Arno Wagner
@ 2014-08-22 3:52 ` John Wells
0 siblings, 0 replies; 27+ messages in thread
From: John Wells @ 2014-08-22 3:52 UTC (permalink / raw)
To: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 18761 bytes --]
Thanks Arno. Assuming I don't want to give up LVM just yet, is there a
particular distro which tends to cautiously tie raid, lvm and luks together
reliably? Or is it that they're all doing this, but in incompatible ways?
On Aug 21, 2014 6:54 PM, "Arno Wagner" <arno@wagner.name> wrote:
> This looks to me like some massive issues with user-space tools,
> starting with the md-devices, that in Fedora are md0 and md2,
> but in Ubuntu are md126 and md127.
>
> This likely means they are of the completely braindead "new"
> RAID-superblock formats where the RAID assembly is not done
> by the RAID controller (i.e. the kernel) as it should be, but
> by something in userspace. Change the userspace, lose proper
> RAID assembly. The idiots responsible for that have not
> understood what layering is and why you do it and have
> implemented a massive step backwards.
>
> The way around this is to use the old superblock format 0.90
> with partition type 0xfd. With that and any kernel doing RAID
> auto-assembly, you always get the same evices. Of course, some
> distros are utterly brain-dead and re-map the 0.90 devices to
> new names via user-space re-assembly, which are then persistent.
> Had that with one "recovery CD once", that I needed an hour to
> recover from booting.
>
> Yes, the 1.x formats have better capabilities, but what use
> is that when they are fundamentally broken otherwise? "Better
> capabilities" is not a valid reason to break things that worked
> before.
>
> My advice around this whole mess would be to make a simple,
> flat patitioning and RAID schema yourself manually (superblock
> 0.90, type 0xfd, no LVM, no partitioning in RAID, just make
> more RAID arrays) and install into that. That seems the only
> way to get KISS with regard to RAID and partitioning these
> days and KISS is the only thing that gives you reliability
> and ease of debugging.
>
> I think I will add a warning to the FAQ that changing distros
> can make your LUKS containers hard or impossible to access due
> to LVM and new RAID superblock userspace differences and that
> it may mess things up enough that a move back to the old distro
> may not be sufficient to fix things. How these jokers expect
> people to deal with multiple distros on the same machine is
> beyond me.
>
> Arno
>
>
>
>
>
>
> On Thu, Aug 21, 2014 at 22:46:04 CEST, John Wells wrote:
> > Attached below, I've included the output of running these commands from
> > first Ubuntu and then Fedora. Note, apparently they map the raid device
> > numbers differently. Also, you'll note there's a cryptd mounted in
> Ubuntu.
> > That's the one i can successfully mount from Ubuntu.
> >
> > To make things even stranger, I was ***able to luksOpen and then mount
> > /dev/MORE_VG/MORE_LV this time while booted in the Fedora 20*** live cd,
> > while as you recall I was unable to before. I'm also including the
> results
> > of dumping the head of the dev file from Fedora.
> >
> > I'm completely at a loss. At this point I feel I should probably backup
> the
> > entire machine and re-install from scratch, but I'll hold off a bit
> longer.
> > The inconsistent behavior is really disconcerting.
> >
> > Thanks,
> > John
> >
> > ####################### FIRST FROM UBUNTU ########################
> > # lsblk
> > NAME MAJ:MIN RM SIZE RO TYPE
> MOUNTPOINT
> > sda 8:0 0 37.3G 0 disk
> > ├─sda1 8:1 0 100M 0 part
> > └─sda2 8:2 0 37.2G 0 part
> > sdb 8:16 0 1.8T 0 disk
> > └─sdb1 8:17 0 1.8T 0 part
> > └─md0 9:0 0 1.8T 0 raid1
> > └─MORE_VG-MORE_LV (dm-1) 252:1 0 800G 0 lvm
> > └─cryptd (dm-7) 252:7 0 800G 0 crypt /home
> > sdc 8:32 0 1.8T 0 disk
> > └─sdc1 8:33 0 1.8T 0 part
> > └─md2 9:2 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> > ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> > └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> > sdd 8:48 0 1.8T 0 disk
> > └─sdd1 8:49 0 1.8T 0 part
> > └─md2 9:2 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3 0 22.4G 0 lvm [SWAP]
> > ├─FINALFRONTIER_VG-OPT_LV (dm-4) 252:4 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV (dm-5) 252:5 0 102.5G 0 lvm /vms
> > └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6 0 279.4G 0 lvm /data
> > sde 8:64 0 167.7G 0 disk
> > ├─sde1 8:65 0 3G 0 part /boot
> > ├─sde2 8:66 0 12G 0 part
> > └─sde3 8:67 0 152.8G 0 part
> > └─SSDROOT_VG-ROOT_LV (dm-0) 252:0 0 97.7G 0 lvm /
> > sdf 8:80 0 1.8T 0 disk
> > └─sdf1 8:81 0 1.8T 0 part
> > sdg 8:96 0 1.8T 0 disk
> > └─sdg1 8:97 0 1.8T 0 part
> > └─BACKUPVG-BACKUPLV (dm-8) 252:8 0 1.2T 0 lvm /backup
> > sr0 11:0 1 1024M 0 rom
> > # dmsetup ls
> > FINALFRONTIER_VG-VMS_LV (252:5)
> > BACKUPVG-BACKUPLV (252:8)
> > MORE_VG-MORE_LV (252:1)
> > FINALFRONTIER_VG-OPT_LV (252:4)
> > FINALFRONTIER_VG-SWAP_LV (252:3)
> > SSDROOT_VG-ROOT_LV (252:0)
> > cryptd (252:7)
> > FINALFRONTIER_VG-DATA_LV (252:6)
> > FINALFRONTIER_VG-HOME_LV (252:2)
> > # dmsetup deps
> > FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 2)
> > BACKUPVG-BACKUPLV: 1 dependencies : (8, 97)
> > MORE_VG-MORE_LV: 1 dependencies : (9, 0)
> > FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 2)
> > FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 2)
> > SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 67)
> > cryptd: 1 dependencies : (252, 1)
> > FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 2)
> > FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 2)
> >
> > ########################## THEN FROM FEDORA #########################
> >
> > # lsblk
> > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> > sda 8:0 0 37.3G 0 disk
> > ├─sda1 8:1 0 100M 0 part
> > └─sda2 8:2 0 37.2G 0 part
> > sdb 8:16 1 14.6G 0 disk
> > ├─sdb1 8:17 1 953M 0 part
> > /run/initramfs/live
> > ├─sdb2 8:18 1 4.9M 0 part
> > └─sdb3 8:19 1 19.7M 0 part
> > sdc 8:32 0 1.8T 0 disk
> > └─sdc1 8:33 0 1.8T 0 part
> > └─md126 9:126 0 1.8T 0 raid1
> > └─MORE_VG-MORE_LV 253:4 0 800G 0 lvm
> > sdd 8:48 0 1.8T 0 disk
> > └─sdd1 8:49 0 1.8T 0 part
> > └─md127 9:127 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> > ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> > └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> > sde 8:64 0 1.8T 0 disk
> > └─sde1 8:65 0 1.8T 0 part
> > └─md127 9:127 0 1.8T 0 raid1
> > ├─FINALFRONTIER_VG-HOME_LV 253:5 0 698.5G 0 lvm
> > ├─FINALFRONTIER_VG-SWAP_LV 253:6 0 22.4G 0 lvm
> > ├─FINALFRONTIER_VG-OPT_LV 253:7 0 139.7G 0 lvm /opt
> > ├─FINALFRONTIER_VG-VMS_LV 253:8 0 102.5G 0 lvm
> > └─FINALFRONTIER_VG-DATA_LV 253:9 0 279.4G 0 lvm
> > sdf 8:80 0 167.7G 0 disk
> > ├─sdf1 8:81 0 3G 0 part
> > ├─sdf2 8:82 0 12G 0 part
> > └─sdf3 8:83 0 152.8G 0 part
> > └─SSDROOT_VG-ROOT_LV 253:3 0 97.7G 0 lvm
> > sdg 8:96 0 1.8T 0 disk
> > └─sdg1 8:97 0 1.8T 0 part
> > sdh 8:112 0 1.8T 0 disk
> > └─sdh1 8:113 0 1.8T 0 part
> > └─BACKUPVG-BACKUPLV 253:10 0 1.2T 0 lvm /backup
> > sr0 11:0 1 1024M 0 rom
> > loop0 7:0 0 8K 1 loop
> > loop1 7:1 0 1.2M 1 loop
> > └─live-osimg-min 253:2 0 4G 1 dm
> > loop2 7:2 0 886.8M 1 loop
> > loop3 7:3 0 4G 1 loop
> > ├─live-rw 253:0 0 4G 0 dm /
> > ├─live-base 253:1 0 4G 1 dm
> > └─live-osimg-min 253:2 0 4G 1 dm
> > loop4 7:4 0 512M 0 loop
> > └─live-rw 253:0 0 4G 0 dm /
> > # dmsetup ls
> > live-base (253:1)
> > FINALFRONTIER_VG-VMS_LV (253:8)
> > BACKUPVG-BACKUPLV (253:10)
> > MORE_VG-MORE_LV (253:4)
> > FINALFRONTIER_VG-OPT_LV (253:7)
> > live-osimg-min (253:2)
> > FINALFRONTIER_VG-SWAP_LV (253:6)
> > live-rw (253:0)
> > SSDROOT_VG-ROOT_LV (253:3)
> > FINALFRONTIER_VG-DATA_LV (253:9)
> > FINALFRONTIER_VG-HOME_LV (253:5)
> > # dmsetup deps
> > live-base: 1 dependencies : (7, 3)
> > FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 127)
> > BACKUPVG-BACKUPLV: 1 dependencies : (8, 113)
> > MORE_VG-MORE_LV: 1 dependencies : (9, 126)
> > FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 127)
> > live-osimg-min: 2 dependencies : (7, 1) (7, 3)
> > FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 127)
> > live-rw: 2 dependencies : (7, 4) (7, 3)
> > SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 83)
> > FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 127)
> > FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 127)
> >
> > ###################### AND NOW THE dumps of both devs from FEDORA
> > ############################
> > # head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hexdump -C
> > 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> > Loopb|
> > 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> > 0...........|
> > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00000400
> >
> >
> >
> > # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> > |LUKS....aes.....|
> > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> > |........xts-plai|
> > 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |n64.............|
> > 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> > |........sha1....|
> > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> > |...............@|
> > 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> > |N...Q.%.ts...n..|
> > 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> > |o.q....^..[.4:..|
> > 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> > |.y.a.........)..|
> > 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> > |#.Y...w.6cc188db|
> > 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> > |-ccdb-4c8f-97b2-|
> > 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> > |e41198ec6e44....|
> > 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> > |..q.....y...)Y.c|
> > 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> > [;W.....|
> > 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> > |6WE...&.........|
> > 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> > |................|
> > 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> > |................|
> > 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> > |................|
> > 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> > |................|
> > 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> > |................|
> > 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> > |................|
> > 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> > |................|
> > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> >
> >
> >
> >
> > On Thu, Aug 21, 2014 at 12:13 PM, Jonas Meurer <jonas@freesources.org>
> > wrote:
> >
> > > Hello John,
> > >
> > > Am 21.08.2014 um 17:55 schrieb John Wells:
> > > > Thanks. How would I use this information to track back to a potential
> > > > problem?
> > >
> > > I'm not convinced yet, that your system screwed up partition and/or
> > > device ordering. First you should try to find out which devices are
> > > recognized both in Fedora and Ubuntu and compare these.
> > >
> > > Try to run 'lsblk', 'dmsetup ls' and 'dmsetup deps' on both systems and
> > > compare or post the result here.
> > >
> > > Which Ubuntu version do you run? Do you use an installed Ubuntu or
> > > rather a live image? It would help if you could provide some more
> > > information regarding your system.
> > >
> > > Kind regards,
> > > jonas
> > >
> > > >
> > > > On Aug 21, 2014 10:41 AM, "Robert Nichols" <
> rnicholsNOSPAM@comcast.net
> > > > <mailto:rnicholsNOSPAM@comcast.net>> wrote:
> > > >
> > > > On 08/21/2014 09:00 AM, John Wells wrote:
> > > >
> > > > I will try what you say. To add to the weirdness, I came into
> > > > work this morning
> > > > with a unresponsive machine. I hard reset, booted Ubuntu,
> but at
> > > > that point
> > > > *Ubuntu wouldn't recognize either partition, and both had the
> > > > same "GNU Parted
> > > > Loopback 0" in the output of "head -c 1024 /volume | hd".
> > > > Neither partition was
> > > > recognized by luksDump. I panicked.
> > > >
> > > > Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to
> > > > normal and I could
> > > > mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted
> with
> > > > the "GNU Parted
> > > > Loopback 0" output.
> > > >
> > > > This makes no sense to me. How could the leading bits be
> > > > different each time I
> > > > booted up? Could datamapper be assigning the wrong device to
> the
> > > > logical volume
> > > > in some way? It just makes no sense.
> > > >
> > > >
> > > > You can run (as root) "dmsetup deps" on each device in
> /dev/mapper
> > > > to see
> > > > the major and minor device numbers needed for each entry.
> > > >
> > > > --
> > > > Bob Nichols "NOSPAM" is really part of my email address.
> > > > Do NOT delete it.
> > > >
> > > > _________________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt@saout.de <mailto:dm-crypt@saout.de>
> > > > http://www.saout.de/mailman/__listinfo/dm-crypt
> > > > <http://www.saout.de/mailman/listinfo/dm-crypt>
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt@saout.de
> > > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > >
> > >
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > >
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
> GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. - Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 23896 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 20:46 ` John Wells
2014-08-21 22:54 ` Arno Wagner
@ 2014-08-22 5:30 ` Heinz Diehl
2014-08-22 11:46 ` Arno Wagner
2014-08-22 9:08 ` Jonas Meurer
2 siblings, 1 reply; 27+ messages in thread
From: Heinz Diehl @ 2014-08-22 5:30 UTC (permalink / raw)
To: dm-crypt
On 21.08.2014, John Wells wrote:
> The inconsistent behavior is really disconcerting.
Another possibility is that the software relevant to your problem is
patched different. Most of the distributions out there doesn't manage
to keep their fingers off the original sourcecode. This could have
introduced the behaviour you're encountering.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-21 20:46 ` John Wells
2014-08-21 22:54 ` Arno Wagner
2014-08-22 5:30 ` Heinz Diehl
@ 2014-08-22 9:08 ` Jonas Meurer
2014-08-22 11:55 ` Arno Wagner
2 siblings, 1 reply; 27+ messages in thread
From: Jonas Meurer @ 2014-08-22 9:08 UTC (permalink / raw)
To: dm-crypt
Hi John,
Am 21.08.2014 um 22:46 schrieb John Wells:
> Attached below, I've included the output of running these commands from
> first Ubuntu and then Fedora. Note, apparently they map the raid device
> numbers differently. Also, you'll note there's a cryptd mounted in
> Ubuntu. That's the one i can successfully mount from Ubuntu.
>
> To make things even stranger, I was ***able to luksOpen and then mount
> /dev/MORE_VG/MORE_LV this time while booted in the Fedora 20*** live cd,
> while as you recall I was unable to before. I'm also including the
> results of dumping the head of the dev file from Fedora.
Actually I don't think that your experiences make it even stranger. Now
you've a consistent behavior of both distros: Ubuntu and Fedora both
recognize your MORE_LV in MORE_VG as a valid LUKS container. For the
moment I recommend to ignore the fact that Fedora once didn't see it.
That might be due to a bug in Fedora, due to a user error or whatever.
Important is, that now both distros see one LUKS container, and don't
see another.
> I'm completely at a loss. At this point I feel I should probably backup
> the entire machine and re-install from scratch, but I'll hold off a bit
> longer. The inconsistent behavior is really disconcerting.
As Arno already wrote, the hexdump from your meant-to-be LUKS container
on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
heard about 'GNU Parted Loopback 0' before, but it sounds like something
caused Parted to overwrite your LUKS header.
Last hope is to find the LUKS header with an offset on the partition.
Try to search for 'LUKS' by grepping the output of 'strings
/dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
followed by something describing your cipher and hash algorithm, I fear
that this second partition is lost.
According to your hexdump it's not astonishing at all that neither of
the distros do recognize the FINALFRONTIER_VG/HOME_LV logical volume as
LUKS container. It simply doesn't start with a LUKS header.
The question rather is: what action resulted in the LUKS header being
overwritten. In the past the partition configuration in the Ubuntu
installer was kind of confusing, and some people trashed their existent
LUKS containers by reformatting them as new LUKS container, but it
doesn't look like that happened to you.
I don't want to join the rant against modern Linux distros, LVM and MD-1
format here. In fact there've been quite some serious bugs in Linux
(vanilla kernel, userspace tools as well as distro implementations) in
the past, and ranting about how fucked up everything is today compared
to the good all days doesn't help to fix them. I prefer to isolate,
identify and fix the bugs and improve the user experience that way :)
Kind regards,
jonas
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-22 5:30 ` Heinz Diehl
@ 2014-08-22 11:46 ` Arno Wagner
0 siblings, 0 replies; 27+ messages in thread
From: Arno Wagner @ 2014-08-22 11:46 UTC (permalink / raw)
To: dm-crypt
On Fri, Aug 22, 2014 at 07:30:32 CEST, Heinz Diehl wrote:
> On 21.08.2014, John Wells wrote:
>
> > The inconsistent behavior is really disconcerting.
>
> Another possibility is that the software relevant to your problem is
> patched different. Most of the distributions out there doesn't manage
> to keep their fingers off the original sourcecode. This could have
> introduced the behaviour you're encountering.
Good point.
Arno
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-22 9:08 ` Jonas Meurer
@ 2014-08-22 11:55 ` Arno Wagner
2014-08-22 12:55 ` Jonas Meurer
0 siblings, 1 reply; 27+ messages in thread
From: Arno Wagner @ 2014-08-22 11:55 UTC (permalink / raw)
To: dm-crypt
On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
> Hi John,
> Am 21.08.2014 um 22:46 schrieb John Wells:
[...]
> As Arno already wrote, the hexdump from your meant-to-be LUKS container
> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
> heard about 'GNU Parted Loopback 0' before, but it sounds like something
> caused Parted to overwrite your LUKS header.
Not necessarily. The other container had that in it the one time as
well, but it fixed itself ...
> Last hope is to find the LUKS header with an offset on the partition.
> Try to search for 'LUKS' by grepping the output of 'strings
> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
> followed by something describing your cipher and hash algorithm, I fear
> that this second partition is lost.
I second searching for that. It may not be the only way though.
Easy way to search:
hd <partition> | grep "LUKS"
That gives you hex offset(s) to examine further.
> According to your hexdump it's not astonishing at all that neither of
> the distros do recognize the FINALFRONTIER_VG/HOME_LV logical volume as
> LUKS container. It simply doesn't start with a LUKS header.
>
> The question rather is: what action resulted in the LUKS header being
> overwritten. In the past the partition configuration in the Ubuntu
> installer was kind of confusing, and some people trashed their existent
> LUKS containers by reformatting them as new LUKS container, but it
> doesn't look like that happened to you.
I think it is configuration mis-detection and with a randomized
component.
> I don't want to join the rant against modern Linux distros, LVM and MD-1
> format here. In fact there've been quite some serious bugs in Linux
> (vanilla kernel, userspace tools as well as distro implementations) in
> the past, and ranting about how fucked up everything is today compared
> to the good all days doesn't help to fix them. I prefer to isolate,
> identify and fix the bugs and improve the user experience that way :)
I am not ranting about bugs, I am ranting about broken architecture
and KISS violations. Having seens some commercial software
source code, I do know that there, it often is even worse. There
are just to many peiople creating software that are not good at
it, and FOSS is really no exception.
Arno
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. - Plato
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-22 11:55 ` Arno Wagner
@ 2014-08-22 12:55 ` Jonas Meurer
2014-10-28 15:34 ` John Wells
0 siblings, 1 reply; 27+ messages in thread
From: Jonas Meurer @ 2014-08-22 12:55 UTC (permalink / raw)
To: dm-crypt
Am 22.08.2014 um 13:55 schrieb Arno Wagner:
> On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
>> Hi John,
>> Am 21.08.2014 um 22:46 schrieb John Wells:
> [...]
>> As Arno already wrote, the hexdump from your meant-to-be LUKS container
>> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
>> heard about 'GNU Parted Loopback 0' before, but it sounds like something
>> caused Parted to overwrite your LUKS header.
>
> Not necessarily. The other container had that in it the one time as
> well, but it fixed itself ...
I'm not sure about that one. Event though John wrote earlier, that »both
had the same "GNU Parted Loopback 0" in the output of "head -c 1024
/volume | hd"«, so maybe you're correct.
>> Last hope is to find the LUKS header with an offset on the partition.
>> Try to search for 'LUKS' by grepping the output of 'strings
>> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
>> followed by something describing your cipher and hash algorithm, I fear
>> that this second partition is lost.
>
> I second searching for that. It may not be the only way though.
> Easy way to search:
>
> hd <partition> | grep "LUKS"
>
> That gives you hex offset(s) to examine further.
Maybe the best is to grep the whole physical partition for "LUKS"
instead of just searching on the LVM logical volume device. If it's
really linux-md or lvm that mix things up here, then one should not
trust in alignment and layering by them.
So, John, best would be to do
# hd /dev/sdc | grep "LUKS"
# hd /dev/sdd | grep "LUKS"
(given that sdc and sdd are the disks with md raid-1 and lvm on top).
If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
LUKS-encrypted lv on these disks. Thus, when you get a result, try to
extract the LUKS header from it using the offset and check its validity.
Kind regards,
jonas
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-08-22 12:55 ` Jonas Meurer
@ 2014-10-28 15:34 ` John Wells
2014-10-28 15:35 ` John Wells
` (2 more replies)
0 siblings, 3 replies; 27+ messages in thread
From: John Wells @ 2014-10-28 15:34 UTC (permalink / raw)
To: Jonas Meurer; +Cc: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 6610 bytes --]
Hi guys, I know I went silent on this. I ended up being on the road a good
bit and just restored from back up and accepted it as an isolated incident.
However, today I received some odd messages and the inability to write to
files.
Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks info
as well.
$ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
$ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
Loopb|
00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
0...........|
00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000400
Recall, before this happened, it looked like this:
# head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
|LUKS....aes.....|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
|........xts-plai|
00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
|n64.............|
00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
|........sha1....|
00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
|...............@|
00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
|N...Q.%.ts...n..|
00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
|o.q....^..[.4:..|
00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
|.y.a.........)..|
000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
|#.Y...w.6cc188db|
000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
|-ccdb-4c8f-97b2-|
000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
|e41198ec6e44....|
000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
|..q.....y...)Y.c|
000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
[;W.....|
000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
|6WE...&.........|
00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
|................|
00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
|................|
00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
|................|
00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
|................|
000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
|................|
000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
|................|
00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
|................|
00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
This has happened even though I haven't changed OSes. This is on Ubuntu
14.04.
And, sadly for me, my backup software was silently malfunctioning, so I
have lost a good bit of data in this case.
Any ideas why a LUKS device would suddenly appear as a loopback device? I'm
totally at a loss and not sure if I can trust luks any more with my data,
which is sad but understand considering what I paid for it. ;-)
Thanks,
John
On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas@freesources.org> wrote:
> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
> >> Hi John,
> >> Am 21.08.2014 um 22:46 schrieb John Wells:
> > [...]
> >> As Arno already wrote, the hexdump from your meant-to-be LUKS container
> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
> >> heard about 'GNU Parted Loopback 0' before, but it sounds like something
> >> caused Parted to overwrite your LUKS header.
> >
> > Not necessarily. The other container had that in it the one time as
> > well, but it fixed itself ...
>
> I'm not sure about that one. Event though John wrote earlier, that »both
> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
> /volume | hd"«, so maybe you're correct.
>
> >> Last hope is to find the LUKS header with an offset on the partition.
> >> Try to search for 'LUKS' by grepping the output of 'strings
> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
> >> followed by something describing your cipher and hash algorithm, I fear
> >> that this second partition is lost.
> >
> > I second searching for that. It may not be the only way though.
> > Easy way to search:
> >
> > hd <partition> | grep "LUKS"
> >
> > That gives you hex offset(s) to examine further.
>
> Maybe the best is to grep the whole physical partition for "LUKS"
> instead of just searching on the LVM logical volume device. If it's
> really linux-md or lvm that mix things up here, then one should not
> trust in alignment and layering by them.
>
> So, John, best would be to do
> # hd /dev/sdc | grep "LUKS"
> # hd /dev/sdd | grep "LUKS"
> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
>
> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
> extract the LUKS header from it using the offset and check its validity.
>
> Kind regards,
> jonas
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 8554 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-28 15:34 ` John Wells
@ 2014-10-28 15:35 ` John Wells
2014-10-28 18:34 ` Arno Wagner
2014-10-28 23:03 ` Jonas Meurer
2014-10-29 23:49 ` Sven Eschenberg
2 siblings, 1 reply; 27+ messages in thread
From: John Wells @ 2014-10-28 15:35 UTC (permalink / raw)
To: Jonas Meurer; +Cc: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 7173 bytes --]
Note, if you read far enough back in the thread, the same thing happened to
the original partition which failed...it suddenly showed up as a loopback
device and wasn't recongized by cryptsetup.
On Tue, Oct 28, 2014 at 11:34 AM, John Wells <jbwellsiv@gmail.com> wrote:
> Hi guys, I know I went silent on this. I ended up being on the road a good
> bit and just restored from back up and accepted it as an isolated incident.
>
> However, today I received some odd messages and the inability to write to
> files.
>
> Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks info
> as well.
>
> $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> Loopb|
> 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> 0...........|
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> *
> 00000400
>
> Recall, before this happened, it looked like this:
>
> # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> |LUKS....aes.....|
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> |........xts-plai|
> 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> |n64.............|
> 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> |........sha1....|
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> |...............@|
> 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> |N...Q.%.ts...n..|
> 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> |o.q....^..[.4:..|
> 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> |.y.a.........)..|
> 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> |#.Y...w.6cc188db|
> 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> |-ccdb-4c8f-97b2-|
> 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> |e41198ec6e44....|
> 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> |..q.....y...)Y.c|
> 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> [;W.....|
> 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> |6WE...&.........|
> 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> |................|
> 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> |................|
> 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> |................|
> 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> |................|
> 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> |................|
> 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> |................|
> 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> |................|
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
>
> This has happened even though I haven't changed OSes. This is on Ubuntu
> 14.04.
>
> And, sadly for me, my backup software was silently malfunctioning, so I
> have lost a good bit of data in this case.
>
> Any ideas why a LUKS device would suddenly appear as a loopback device?
> I'm totally at a loss and not sure if I can trust luks any more with my
> data, which is sad but understand considering what I paid for it. ;-)
>
> Thanks,
> John
>
> On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas@freesources.org>
> wrote:
>
>> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
>> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
>> >> Hi John,
>> >> Am 21.08.2014 um 22:46 schrieb John Wells:
>> > [...]
>> >> As Arno already wrote, the hexdump from your meant-to-be LUKS container
>> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
>> >> heard about 'GNU Parted Loopback 0' before, but it sounds like
>> something
>> >> caused Parted to overwrite your LUKS header.
>> >
>> > Not necessarily. The other container had that in it the one time as
>> > well, but it fixed itself ...
>>
>> I'm not sure about that one. Event though John wrote earlier, that »both
>> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
>> /volume | hd"«, so maybe you're correct.
>>
>> >> Last hope is to find the LUKS header with an offset on the partition.
>> >> Try to search for 'LUKS' by grepping the output of 'strings
>> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
>> >> followed by something describing your cipher and hash algorithm, I fear
>> >> that this second partition is lost.
>> >
>> > I second searching for that. It may not be the only way though.
>> > Easy way to search:
>> >
>> > hd <partition> | grep "LUKS"
>> >
>> > That gives you hex offset(s) to examine further.
>>
>> Maybe the best is to grep the whole physical partition for "LUKS"
>> instead of just searching on the LVM logical volume device. If it's
>> really linux-md or lvm that mix things up here, then one should not
>> trust in alignment and layering by them.
>>
>> So, John, best would be to do
>> # hd /dev/sdc | grep "LUKS"
>> # hd /dev/sdd | grep "LUKS"
>> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
>>
>> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
>> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
>> extract the LUKS header from it using the offset and check its validity.
>>
>> Kind regards,
>> jonas
>>
>>
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>
>
>
[-- Attachment #2: Type: text/html, Size: 9128 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-28 15:35 ` John Wells
@ 2014-10-28 18:34 ` Arno Wagner
0 siblings, 0 replies; 27+ messages in thread
From: Arno Wagner @ 2014-10-28 18:34 UTC (permalink / raw)
To: dm-crypt
This is really catatstrophic, agreed. But it seems to come
from a common cause and it is not a LUKS problem. It is a
problem with something messing with partitions
"auto-magically" and that "something" happens to not
recognize LUKS containers as things to leave intact.
I think I said it before: Ubuntu has a tendency to not
recognize LUKS containers, see for example the installer
that deletes present LUKS containers without clear warning
(AFAIK still not fixed).
My take is this is possible some kind of automtic recognition
of the disk-setup that under some circumstances fails and
compounds ints inability by writing to the misidentified
partition.
It would be pretty good finding out what this is, so I
can write an additional warning for the FAQ and send
the Ubuntu maintainers a "WTF do you thing you are
doing?" email (if it is their fault) and, as importantly,
get this fixed.
So, here is some debugging ideas:
- Go though all the boot-scripts in /etc/init.d and
see whether there are any calls to parted.
- Same with the root cron-jobs.
- Same for "partx" and "kpartx"
And more:
- Look in the logs (/var/log) for any instances of
parted, partx, kpartx and lvm* and see whether it
says anything about loo-mounting.
Hot candidates are the minutes/hours right before
it failed.
- If this failed pretty soon after booting: Also look
at the startup messages for any partition detection
messeges or the like.
On the risk of repeating myself, this also nicely
illustrates that LVM gives you too much flexibility
and not enough safety. It also shows that backup
needs to be tested in order to deserve any trust.
Gr"usse,
Arno
On Tue, Oct 28, 2014 at 16:35:30 CET, John Wells wrote:
> Note, if you read far enough back in the thread, the same thing happened to
> the original partition which failed...it suddenly showed up as a loopback
> device and wasn't recongized by cryptsetup.
>
> On Tue, Oct 28, 2014 at 11:34 AM, John Wells <jbwellsiv@gmail.com> wrote:
>
> > Hi guys, I know I went silent on this. I ended up being on the road a good
> > bit and just restored from back up and accepted it as an isolated incident.
> >
> > However, today I received some odd messages and the inability to write to
> > files.
> >
> > Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks info
> > as well.
> >
> > $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> > Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> > $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> > Loopb|
> > 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> > 0...........|
> > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00000400
> >
> > Recall, before this happened, it looked like this:
> >
> > # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> > |LUKS....aes.....|
> > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> > |........xts-plai|
> > 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |n64.............|
> > 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> > |........sha1....|
> > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> > |...............@|
> > 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> > |N...Q.%.ts...n..|
> > 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> > |o.q....^..[.4:..|
> > 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> > |.y.a.........)..|
> > 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> > |#.Y...w.6cc188db|
> > 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> > |-ccdb-4c8f-97b2-|
> > 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> > |e41198ec6e44....|
> > 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> > |..q.....y...)Y.c|
> > 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> > [;W.....|
> > 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> > |6WE...&.........|
> > 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> > |................|
> > 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> > |................|
> > 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> > |................|
> > 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> > |................|
> > 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> > |................|
> > 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> > |................|
> > 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> > |................|
> > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> >
> > This has happened even though I haven't changed OSes. This is on Ubuntu
> > 14.04.
> >
> > And, sadly for me, my backup software was silently malfunctioning, so I
> > have lost a good bit of data in this case.
> >
> > Any ideas why a LUKS device would suddenly appear as a loopback device?
> > I'm totally at a loss and not sure if I can trust luks any more with my
> > data, which is sad but understand considering what I paid for it. ;-)
> >
> > Thanks,
> > John
> >
> > On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas@freesources.org>
> > wrote:
> >
> >> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
> >> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
> >> >> Hi John,
> >> >> Am 21.08.2014 um 22:46 schrieb John Wells:
> >> > [...]
> >> >> As Arno already wrote, the hexdump from your meant-to-be LUKS container
> >> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
> >> >> heard about 'GNU Parted Loopback 0' before, but it sounds like
> >> something
> >> >> caused Parted to overwrite your LUKS header.
> >> >
> >> > Not necessarily. The other container had that in it the one time as
> >> > well, but it fixed itself ...
> >>
> >> I'm not sure about that one. Event though John wrote earlier, that »both
> >> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
> >> /volume | hd"«, so maybe you're correct.
> >>
> >> >> Last hope is to find the LUKS header with an offset on the partition.
> >> >> Try to search for 'LUKS' by grepping the output of 'strings
> >> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
> >> >> followed by something describing your cipher and hash algorithm, I fear
> >> >> that this second partition is lost.
> >> >
> >> > I second searching for that. It may not be the only way though.
> >> > Easy way to search:
> >> >
> >> > hd <partition> | grep "LUKS"
> >> >
> >> > That gives you hex offset(s) to examine further.
> >>
> >> Maybe the best is to grep the whole physical partition for "LUKS"
> >> instead of just searching on the LVM logical volume device. If it's
> >> really linux-md or lvm that mix things up here, then one should not
> >> trust in alignment and layering by them.
> >>
> >> So, John, best would be to do
> >> # hd /dev/sdc | grep "LUKS"
> >> # hd /dev/sdd | grep "LUKS"
> >> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
> >>
> >> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
> >> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
> >> extract the LUKS header from it using the offset and check its validity.
> >>
> >> Kind regards,
> >> jonas
> >>
> >>
> >> _______________________________________________
> >> dm-crypt mailing list
> >> dm-crypt@saout.de
> >> http://www.saout.de/mailman/listinfo/dm-crypt
> >>
> >
> >
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato
If it's in the news, don't worry about it. The very definition of
"news" is "something that hardly ever happens." -- Bruce Schneier
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-28 15:34 ` John Wells
2014-10-28 15:35 ` John Wells
@ 2014-10-28 23:03 ` Jonas Meurer
2014-10-29 0:57 ` Arno Wagner
2014-10-29 23:49 ` Sven Eschenberg
2 siblings, 1 reply; 27+ messages in thread
From: Jonas Meurer @ 2014-10-28 23:03 UTC (permalink / raw)
To: dm-crypt
Hi John,
Am 28.10.2014 um 16:34 schrieb John Wells:
> Hi guys, I know I went silent on this. I ended up being on the road a
> good bit and just restored from back up and accepted it as an isolated
> incident.
>
> However, today I received some odd messages and the inability to write
> to files.
Too bad, so at least it was not a one-time incident. Instead it follows
a pattern to some degree.
> Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks
> info as well.
>
> $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> Loopb|
> 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> 0...........|
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> *
> 00000400
Did you try to grep for 'LUKS' on /dev/sdc and /dev/sdd in the meantime?
> Maybe the best is to grep the whole physical partition for "LUKS"
> instead of just searching on the LVM logical volume device. If it's
> really linux-md or lvm that mix things up here, then one should not
> trust in alignment and layering by them.
>
> So, John, best would be to do
> # hd /dev/sdc | grep "LUKS"
> # hd /dev/sdd | grep "LUKS"
> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
>
> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
> extract the LUKS header from it using the offset and check its
> validity.
> Recall, before this happened, it looked like this:
>
> # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> [...]
>
> And, sadly for me, my backup software was silently malfunctioning, so I
> have lost a good bit of data in this case.
Maybe you're able to restore the data with the header backup you posted
earlier. If it contains the full header, then you should be able to
restore the header and use it for decrypting the data. Though the
question is, whether the encrypted data is still valid after your
'incident'.
> Any ideas why a LUKS device would suddenly appear as a loopback device?
> I'm totally at a loss and not sure if I can trust luks any more with my
> data, which is sad but understand considering what I paid for it. ;-)
I don't believe that either cryptsetup/LUKS or LVM is the reason for
your troubles.
Do you use the crypttab function of Ubuntu cryptsetup packages? If so,
please paste the content of /etc/crypttab.
Kind regards,
jonas
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-28 23:03 ` Jonas Meurer
@ 2014-10-29 0:57 ` Arno Wagner
0 siblings, 0 replies; 27+ messages in thread
From: Arno Wagner @ 2014-10-29 0:57 UTC (permalink / raw)
To: dm-crypt
On Wed, Oct 29, 2014 at 00:03:01 CET, Jonas Meurer wrote:
> Hi John,
>
[...]
>
> I don't believe that either cryptsetup/LUKS or LVM is the reason for
> your troubles.
I agree on that, at least as to direct reason. Some kind of
automation seems to misidentify the LUKS container and then
triggers some kind of destructive and wrong handling.
Arno
> Do you use the crypttab function of Ubuntu cryptsetup packages? If so,
> please paste the content of /etc/crypttab.
>
> Kind regards,
> jonas
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
--
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno@wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato
If it's in the news, don't worry about it. The very definition of
"news" is "something that hardly ever happens." -- Bruce Schneier
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-28 15:34 ` John Wells
2014-10-28 15:35 ` John Wells
2014-10-28 23:03 ` Jonas Meurer
@ 2014-10-29 23:49 ` Sven Eschenberg
2014-10-30 0:37 ` John Wells
2 siblings, 1 reply; 27+ messages in thread
From: Sven Eschenberg @ 2014-10-29 23:49 UTC (permalink / raw)
To: dm-crypt
Hi,
I wanna jump in with some more hints:
what actually was written to the disk is a loop device signature by
libparted. This will be written, if you'd issue a parted mklabel loop
/dev/MORE_VG/MORE_LV.
If any other tool would write such a signature, it would have to use
libparted most certainly.
If the device given is a (virtual) disk and has any partition table with
partition and fs on it the signature will not be written. (Well that
explains why the LUKS header won't keep write_loop() from libparted from
doing it's work).
Anyway, if any other tool destroys the data it would have to call
write_loop().
---
Referring to Arno's mail:
partx is non destructive and uses the code from util-linux'es libblkid.
Probably not a cadidate.
kpartx uses the device-mapper to create partition nodes on arbitary
devices and is non destructive as far as the symbol table is concerned.
Probably not a candidate either.
I cannot guarantee for Ubuntu though - if they changed the code.. - you
could do an objdump/ldd on these tools just to make sure.
---
Anyhow, back on topic: udisks2 uses parted directly (calling out on it)
and does destructive operations, but no trace of loop labels.
Did you use any tool like parted or gparted between backup restore and
failure?
Regards
-Sven
On Tue, October 28, 2014 16:34, John Wells wrote:
> Hi guys, I know I went silent on this. I ended up being on the road a good
> bit and just restored from back up and accepted it as an isolated
> incident.
>
> However, today I received some odd messages and the inability to write to
> files.
>
> Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks info
> as well.
>
> $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> Loopb|
> 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> 0...........|
> 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> *
> 00000400
>
> Recall, before this happened, it looked like this:
>
> # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> |LUKS....aes.....|
> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> |........xts-plai|
> 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> |n64.............|
> 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> |........sha1....|
> 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> |...............@|
> 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> |N...Q.%.ts...n..|
> 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> |o.q....^..[.4:..|
> 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> |.y.a.........)..|
> 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> |#.Y...w.6cc188db|
> 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> |-ccdb-4c8f-97b2-|
> 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> |e41198ec6e44....|
> 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> |..q.....y...)Y.c|
> 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> [;W.....|
> 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> |6WE...&.........|
> 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> |................|
> 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> |................|
> 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> |................|
> 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> |................|
> 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> |................|
> 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> |................|
> 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
> 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> |................|
> 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> |................|
>
> This has happened even though I haven't changed OSes. This is on Ubuntu
> 14.04.
>
> And, sadly for me, my backup software was silently malfunctioning, so I
> have lost a good bit of data in this case.
>
> Any ideas why a LUKS device would suddenly appear as a loopback device?
> I'm
> totally at a loss and not sure if I can trust luks any more with my data,
> which is sad but understand considering what I paid for it. ;-)
>
> Thanks,
> John
>
> On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas@freesources.org>
> wrote:
>
>> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
>> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
>> >> Hi John,
>> >> Am 21.08.2014 um 22:46 schrieb John Wells:
>> > [...]
>> >> As Arno already wrote, the hexdump from your meant-to-be LUKS
>> container
>> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
>> >> heard about 'GNU Parted Loopback 0' before, but it sounds like
>> something
>> >> caused Parted to overwrite your LUKS header.
>> >
>> > Not necessarily. The other container had that in it the one time as
>> > well, but it fixed itself ...
>>
>> I'm not sure about that one. Event though John wrote earlier, that »both
>> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
>> /volume | hd"«, so maybe you're correct.
>>
>> >> Last hope is to find the LUKS header with an offset on the partition.
>> >> Try to search for 'LUKS' by grepping the output of 'strings
>> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
>> >> followed by something describing your cipher and hash algorithm, I
>> fear
>> >> that this second partition is lost.
>> >
>> > I second searching for that. It may not be the only way though.
>> > Easy way to search:
>> >
>> > hd <partition> | grep "LUKS"
>> >
>> > That gives you hex offset(s) to examine further.
>>
>> Maybe the best is to grep the whole physical partition for "LUKS"
>> instead of just searching on the LVM logical volume device. If it's
>> really linux-md or lvm that mix things up here, then one should not
>> trust in alignment and layering by them.
>>
>> So, John, best would be to do
>> # hd /dev/sdc | grep "LUKS"
>> # hd /dev/sdd | grep "LUKS"
>> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
>>
>> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
>> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
>> extract the LUKS header from it using the offset and check its validity.
>>
>> Kind regards,
>> jonas
>>
>>
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [dm-crypt] "not a valid LUKS device" after distro change
2014-10-29 23:49 ` Sven Eschenberg
@ 2014-10-30 0:37 ` John Wells
0 siblings, 0 replies; 27+ messages in thread
From: John Wells @ 2014-10-30 0:37 UTC (permalink / raw)
To: sven; +Cc: dm-crypt
[-- Attachment #1: Type: text/plain, Size: 9734 bytes --]
Regarding using any tool like parted or gparted, no. Not directly that I
recall.
Regarding the use of /etc/crypttab, I wasn't using it. Since the first
failure, I've been very busy (as I mentioned), so each time I had to
reboot, I'd manually use "cryptsetup luksOpen /dev/MORE_VG/MORE_LV
cryptdev" and then "mount /dev/mapper/cryptdev /home", and then login.
Thanks for all the help. I'm back on the road but will investigate all
recommendations when I return. I'd be very, very interested to know the
culprit so that I know what and what not to trust in the future.
Thanks guys.
John
On Wed, Oct 29, 2014 at 7:49 PM, Sven Eschenberg <sven@whgl.uni-frankfurt.de
> wrote:
> Hi,
>
> I wanna jump in with some more hints:
>
> what actually was written to the disk is a loop device signature by
> libparted. This will be written, if you'd issue a parted mklabel loop
> /dev/MORE_VG/MORE_LV.
>
> If any other tool would write such a signature, it would have to use
> libparted most certainly.
>
> If the device given is a (virtual) disk and has any partition table with
> partition and fs on it the signature will not be written. (Well that
> explains why the LUKS header won't keep write_loop() from libparted from
> doing it's work).
>
> Anyway, if any other tool destroys the data it would have to call
> write_loop().
>
> ---
>
> Referring to Arno's mail:
>
> partx is non destructive and uses the code from util-linux'es libblkid.
> Probably not a cadidate.
>
> kpartx uses the device-mapper to create partition nodes on arbitary
> devices and is non destructive as far as the symbol table is concerned.
> Probably not a candidate either.
>
> I cannot guarantee for Ubuntu though - if they changed the code.. - you
> could do an objdump/ldd on these tools just to make sure.
>
> ---
>
> Anyhow, back on topic: udisks2 uses parted directly (calling out on it)
> and does destructive operations, but no trace of loop labels.
>
> Did you use any tool like parted or gparted between backup restore and
> failure?
>
> Regards
>
> -Sven
>
>
> On Tue, October 28, 2014 16:34, John Wells wrote:
> > Hi guys, I know I went silent on this. I ended up being on the road a
> good
> > bit and just restored from back up and accepted it as an isolated
> > incident.
> >
> > However, today I received some odd messages and the inability to write to
> > files.
> >
> > Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks
> info
> > as well.
> >
> > $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> > Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> > $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000 47 4e 55 20 50 61 72 74 65 64 20 4c 6f 6f 70 62 |GNU Parted
> > Loopb|
> > 00000010 61 63 6b 20 30 00 00 00 00 00 00 00 00 00 00 00 |ack
> > 0...........|
> > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > *
> > 00000400
> >
> > Recall, before this happened, it looked like this:
> >
> > # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000 4c 55 4b 53 ba be 00 01 61 65 73 00 00 00 00 00
> > |LUKS....aes.....|
> > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000020 00 00 00 00 00 00 00 00 78 74 73 2d 70 6c 61 69
> > |........xts-plai|
> > 00000030 6e 36 34 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |n64.............|
> > 00000040 00 00 00 00 00 00 00 00 73 68 61 31 00 00 00 00
> > |........sha1....|
> > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000060 00 00 00 00 00 00 00 00 00 00 10 00 00 00 00 40
> > |...............@|
> > 00000070 4e 94 98 c0 51 a9 25 bb 74 73 88 a4 a7 6e c7 d3
> > |N...Q.%.ts...n..|
> > 00000080 6f 18 71 fa 94 9a f4 5e d1 e8 5b 1a 34 3a 9f 9f
> > |o.q....^..[.4:..|
> > 00000090 1d 79 1f 61 f5 dd 98 09 e1 d6 2e ed c4 29 af 1a
> > |.y.a.........)..|
> > 000000a0 23 c9 59 da 00 00 77 a1 36 63 63 31 38 38 64 62
> > |#.Y...w.6cc188db|
> > 000000b0 2d 63 63 64 62 2d 34 63 38 66 2d 39 37 62 32 2d
> > |-ccdb-4c8f-97b2-|
> > 000000c0 65 34 31 31 39 38 65 63 36 65 34 34 00 00 00 00
> > |e41198ec6e44....|
> > 000000d0 00 ac 71 f3 00 01 df d7 79 85 dd f0 29 59 98 63
> > |..q.....y...)Y.c|
> > 000000e0 0b 69 80 fe 48 61 8c 40 5b 3b 57 0f 82 9c ae 90 |.i..Ha.@
> > [;W.....|
> > 000000f0 36 57 45 e2 03 82 26 c5 00 00 00 08 00 00 0f a0
> > |6WE...&.........|
> > 00000100 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000120 00 00 00 00 00 00 00 00 00 00 02 00 00 00 0f a0
> > |................|
> > 00000130 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000150 00 00 00 00 00 00 00 00 00 00 03 f8 00 00 0f a0
> > |................|
> > 00000160 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000180 00 00 00 00 00 00 00 00 00 00 05 f0 00 00 0f a0
> > |................|
> > 00000190 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001b0 00 00 00 00 00 00 00 00 00 00 07 e8 00 00 0f a0
> > |................|
> > 000001c0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 000001e0 00 00 00 00 00 00 00 00 00 00 09 e0 00 00 0f a0
> > |................|
> > 000001f0 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000210 00 00 00 00 00 00 00 00 00 00 0b d8 00 00 0f a0
> > |................|
> > 00000220 00 00 de ad 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> > 00000240 00 00 00 00 00 00 00 00 00 00 0d d0 00 00 0f a0
> > |................|
> > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > |................|
> >
> > This has happened even though I haven't changed OSes. This is on Ubuntu
> > 14.04.
> >
> > And, sadly for me, my backup software was silently malfunctioning, so I
> > have lost a good bit of data in this case.
> >
> > Any ideas why a LUKS device would suddenly appear as a loopback device?
> > I'm
> > totally at a loss and not sure if I can trust luks any more with my data,
> > which is sad but understand considering what I paid for it. ;-)
> >
> > Thanks,
> > John
> >
> > On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas@freesources.org>
> > wrote:
> >
> >> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
> >> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
> >> >> Hi John,
> >> >> Am 21.08.2014 um 22:46 schrieb John Wells:
> >> > [...]
> >> >> As Arno already wrote, the hexdump from your meant-to-be LUKS
> >> container
> >> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
> >> >> heard about 'GNU Parted Loopback 0' before, but it sounds like
> >> something
> >> >> caused Parted to overwrite your LUKS header.
> >> >
> >> > Not necessarily. The other container had that in it the one time as
> >> > well, but it fixed itself ...
> >>
> >> I'm not sure about that one. Event though John wrote earlier, that »both
> >> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
> >> /volume | hd"«, so maybe you're correct.
> >>
> >> >> Last hope is to find the LUKS header with an offset on the partition.
> >> >> Try to search for 'LUKS' by grepping the output of 'strings
> >> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
> >> >> followed by something describing your cipher and hash algorithm, I
> >> fear
> >> >> that this second partition is lost.
> >> >
> >> > I second searching for that. It may not be the only way though.
> >> > Easy way to search:
> >> >
> >> > hd <partition> | grep "LUKS"
> >> >
> >> > That gives you hex offset(s) to examine further.
> >>
> >> Maybe the best is to grep the whole physical partition for "LUKS"
> >> instead of just searching on the LVM logical volume device. If it's
> >> really linux-md or lvm that mix things up here, then one should not
> >> trust in alignment and layering by them.
> >>
> >> So, John, best would be to do
> >> # hd /dev/sdc | grep "LUKS"
> >> # hd /dev/sdd | grep "LUKS"
> >> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
> >>
> >> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
> >> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
> >> extract the LUKS header from it using the offset and check its validity.
> >>
> >> Kind regards,
> >> jonas
> >>
> >>
> >> _______________________________________________
> >> dm-crypt mailing list
> >> dm-crypt@saout.de
> >> http://www.saout.de/mailman/listinfo/dm-crypt
> >>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> >
>
>
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
[-- Attachment #2: Type: text/html, Size: 12240 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2014-10-30 0:38 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-18 20:58 [dm-crypt] "not a valid LUKS device" after distro change John Wells
2014-08-19 19:51 ` Arno Wagner
2014-08-19 22:22 ` John Wells
2014-08-20 9:09 ` Arno Wagner
2014-08-20 16:18 ` John Wells
2014-08-20 21:15 ` Arno Wagner
2014-08-21 14:00 ` John Wells
2014-08-21 14:25 ` Robert Nichols
2014-08-21 15:55 ` John Wells
2014-08-21 16:13 ` Jonas Meurer
2014-08-21 20:46 ` John Wells
2014-08-21 22:54 ` Arno Wagner
2014-08-22 3:52 ` John Wells
2014-08-22 5:30 ` Heinz Diehl
2014-08-22 11:46 ` Arno Wagner
2014-08-22 9:08 ` Jonas Meurer
2014-08-22 11:55 ` Arno Wagner
2014-08-22 12:55 ` Jonas Meurer
2014-10-28 15:34 ` John Wells
2014-10-28 15:35 ` John Wells
2014-10-28 18:34 ` Arno Wagner
2014-10-28 23:03 ` Jonas Meurer
2014-10-29 0:57 ` Arno Wagner
2014-10-29 23:49 ` Sven Eschenberg
2014-10-30 0:37 ` John Wells
2014-08-21 17:01 ` Robert Nichols
2014-08-21 14:29 ` Arno Wagner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.