Thanks for the help!

I doubt that dm-crypt had anything to do with it - your mention of guid got me thinking that it is almost certainly something in the boot sequence. (Not sure why it decided to hit now, but..) They are spinning disks, so no ssd bugs but that doesn't rule out firmware or bootloader issues.

Unfortunately I don't have 7T to back up the devices. Is there a relatively safe way to test? Worst case, I can make the change, set the entire device ro and try, if dm-crypt can decrypt a readonly device.

I did confirm farther down the header, the keys and empty slots appear to be present.

On Feb 25, 2014 11:11 PM, "Arno Wagner" <arno@wagner.name> wrote:
>
> This is pretty strange. You seem to have lost parts of the
> header, but not the hash spec. And it seems to have happened
> two times.
>
> This should not be happening at all. LUKS does not require a
> "clean shutdown", unless you luksFormat or change
> passphrases, cryptsetup does not write anything at all for
> LUKS and even ripping out the disk directly while it
> runs should not cause any header damage. (It may damage the
> filesystem in the LUKS container though...).
>
> What seems to have happened here is that some application
> read the header, replaced the first 0x2d bytes (leaving
> "ain64" of "xts-plain64". It seem to have put in a GUID
> of sorts. I have no idea what this is. The header was not
> simply overwritten, so this is something deliberately.
> On the other hand, no application should write anything to
> the start of a partition in normal operation.
>
> Maybe somebody else here recognizes the pattern seen?
> Maybe these are SSDs with a serious firmware bug?
> Or maybe you have wrap-around because the USB3 interface
> cannot handle the full disk size?
>
>
> As to recovery:
> 1. Make a sector-wise backup of the whole LUKS containers
>    or whole disk(s) before messing with anything!
> 2. Just putting in what should be there in the first
>    0x2d bytes with a hex editor may be enough to get this
>    working again if there is no other damage. If there
>    is damage in the keyslot areas or to the salt values
>    in the header, no recovery is possible.
>
> Hexediting the header is pretty simple, just read the
> first 1024 bytes (head -c 1024 /dev/disk > hdr.img),
> hexedit it and then write it back (cat hdr.img > /dev/disk).
> The risk of doing more damage is high, so do not try this
> without that backup.
>
> Here is what the first 0x2d bytes should look like (defaul
> parameters):
>
> 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00 |LUKS....aes.....|
> 00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
> 00000020  00 00 00 00 00 00 00 00  78 74 73 2d 70 6c 61 69 |........xts-plai|
>
> If after that cryptsetup works, but does not accept
> your passphrase, you can run the keyslot-checker from
> the source package and you can post the full headers
> (592 bytes long) here for more advice.
>
> Arno
>
>
>
>
>
> On Tue, Feb 25, 2014 at 15:33:20 CET, Dis McCarthy wrote:
> > I've got 2 full-device luks devices (usb3) that somehow got corrupted
> > headers at the same time. They were working, I powered down the system to
> > move some plugs and when I brought it back up, my headers are unrecognized.
> > (See below.) The system is a mac mini running arch linux. It has been
> > rebooted and powercycled repeatedly since being built and never had an
> > issue, so I hesitate to blame refit. The only thing I can see is that there
> > was a recent update of cryptsetup (to 1.6.3-2). I'm not sure if the devices
> > were stopped cleanly before reboot (probably not, since its arch..) but the
> > volumes were unmounted and I wouldn't expect to get corruption.
> >
> > I'm dumb, and do not have a backup of the headers. (Well.. there might be a
> > backup of one of them. Inside the encrypted volume.. yeah, like I said, I'm
> > dumb. If I get this back, that is getting remedied immediately..)
> >
> > Comparing with other users, it looks like some of the entries are swapped
> > around. I started doing a comparison to the on-disk format pdf, but I
> > haven't had a chance to print it out and get intimate with it yet (beyond
> > noting that LUKS 0xba0xbe is missing of course.)
> >
> > Is there any hope? Here is what I've got at the beginning the drives:
> >
> > 00000000  00 00 00 00 48 00 00 00  00 00 00 00 32 32 30 64
> >  |....H.......220d|
> > 00000010  32 30 33 64 2d 39 62 62  34 2d 34 35 00 00 00 00
> >  |203d-9bb4-45....|
> > 00000020  24 00 00 00 12 00 00 00  24 00 00 00 00 00 61 69
> >  |$.......$.....ai|
> > 00000030  6e 36 34 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |n64.............|
> > 00000040  00 00 00 00 00 00 00 00  73 68 61 32 35 36 00 00
> >  |........sha256..|
> > 00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000060  00 00 00 00 00 00 00 00  00 00 10 00 00 00 00 40
> >  |...............@|
> > 00000070  da 22 00 00 00 00 24 01  00 00 00 00 00 00 00 00
> >  |."....$.........|
> >
> > and:
> > 00000000  00 00 00 00 48 00 00 00  00 00 00 00 32 33 63 38
> >  |....H.......23c8|
> > 00000010  62 61 33 31 2d 37 61 63  66 2d 34 32 00 00 00 00
> >  |ba31-7acf-42....|
> > 00000020  24 00 00 00 12 00 00 00  24 00 00 00 00 00 61 69
> >  |$.......$.....ai|
> > 00000030  6e 36 34 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |n64.............|
> > 00000040  00 00 00 00 00 00 00 00  73 68 61 32 35 36 00 00
> >  |........sha256..|
> > 00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000060  00 00 00 00 00 00 00 00  00 00 10 00 00 00 00 40
> >  |...............@|
> > 00000070  de 8b 00 00 00 00 24 01  00 00 00 00 00 00 00 00
> >  |......$.........|
> >
> > Thanks!
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -  Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt