[dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

Hi,


I got a new HDD a few days ago and I put it in an external case and connected
it over USB with my home server. It is a 2000GB WD disk,  WD20EARS.

I already read a few articles how should I format this "special"
drive, and I did it like this:
http://ubuntuforums.org/showthread.php?t=1456251

I set the partition to begin on sector 64 (instead of 63), run
luksFormat, run luksOpen,
created XFS and copied my files on this disk. I umounted it later and
I rebooted
the server today.

First thing I noticed is that disk/paritions are not visible in
/dev/disk/by-uuid/ although
/dev/sdm and /dev/sdm1 are available. fdisk also does not look same as before:

fdisk output:
--------------
root@road:/dev/disk/by-uuid#  fdisk /dev/sdm

The number of cylinders for this disk is set to 243201.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdm1               1      243202  1953514552   83  Linux

--------------

"Start" is 1 instead of 64.... pretty strange because I set it to 64
when I was making this partition.


luksOpen does not work and luksDump/isLuks reported that /dev/sdm1 is
not luks partition.


I have cryptsetup 1.0.5 on my server.

There are two questions for me here:
1. I have luksDump of /dev/sdm1 before this happened. Can I use
luksDump ouput  to
recover/overwrite Luks partition?

fdisk reports "start" on sector 1 instead of 64, which means that
something there went wrong.
Or not? Ideas? If I can use luksDump to recover luks-header should the
partition table maybe be
rewritten beginning with sector 64 before I recreate luks header.


2. How all this could happen?

I would be very happy to experiment with this, to repeat all the steps
to recreat the situation
 if I know that I am able to recover my partition. Any ideas before I
start to play with this?
How much of the partition header should I backup before starting with
all of this?


Thanks a lot for any advice.

M.

p.s. I have all the data but I would like to avoid copying again
because I have to copy everything
(over 500gb) over SSH trough a 10mb connection

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Arno Wagner]

Hi,

On Tue, Sep 14, 2010 at 06:55:36PM +0200, Ma Begaj wrote:
> Hi,
> 
> 
> I got a new HDD a few days ago and I put it in an external case and connected
> it over USB with my home server. It is a 2000GB WD disk,  WD20EARS.

That would be one of the 2kB sector drives...
 
> I already read a few articles how should I format this "special"
> drive, and I did it like this:
> http://ubuntuforums.org/showthread.php?t=1456251
> 
> I set the partition to begin on sector 64 (instead of 63), run
> luksFormat, run luksOpen,
> created XFS and copied my files on this disk. I umounted it later and
> I rebooted
> the server today.
> 
> First thing I noticed is that disk/paritions are not visible in
> /dev/disk/by-uuid/ although
> /dev/sdm and /dev/sdm1 are available. fdisk also does not look same 
> as before:

Not good.

> fdisk output:
> --------------
> root@road:/dev/disk/by-uuid#  fdisk /dev/sdm
> 
> The number of cylinders for this disk is set to 243201.
> There is nothing wrong with that, but this is larger than 1024,
> and could in certain setups cause problems with:
> 1) software that runs at boot time (e.g., old versions of LILO)
> 2) booting and partitioning software from other OSs
>    (e.g., DOS FDISK, OS/2 FDISK)
> 
> Command (m for help): p
> 
> Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
> 255 heads, 63 sectors/track, 243201 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes
> Disk identifier: 0x00000000
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdm1               1      243202  1953514552   83  Linux

You did create one large partition?

What is the output of "fdisk -lu /dev/sdm1"?


> --------------
> 
> "Start" is 1 instead of 64.... pretty strange because I set it to 64
> when I was making this partition.

You set it to 64 sectors. The 1 is a cylinder. You cannto see in this
view where exactly the partition starts. Use the -u option to fdisk.

> 
> luksOpen does not work and luksDump/isLuks reported that /dev/sdm1 is
> not luks partition.

Interesting.
 
> I have cryptsetup 1.0.5 on my server.

Aehm, that is hiostoric. Please try with the current one,
which would be 1.1.3.
 
> There are two questions for me here:
> 1. I have luksDump of /dev/sdm1 before this happened. Can I use
> luksDump ouput  to
> recover/overwrite Luks partition?

I think not, but I am not sure. Before trying that, lets figure
out whether this is simply an offset issue. Can you look 
through the start of the disk and give the offset where the
"LUKS" string is exactly?

Something like this should do the trick:

hd /dev/sdm | grep LUKS

> fdisk reports "start" on sector 1 instead of 64, which means that
> something there went wrong.

No, see above.

> Or not? Ideas? If I can use luksDump to recover luks-header should the
> partition table maybe be
> rewritten beginning with sector 64 before I recreate luks header.
> 
> 
> 2. How all this could happen?
> 
> I would be very happy to experiment with this, to repeat all the steps
> to recreat the situation
>  if I know that I am able to recover my partition. Any ideas before I
> start to play with this?
> How much of the partition header should I backup before starting with
> all of this?

See the FAQ. If you back up the first 100MB, you are on the safe side.
 
 
> Thanks a lot for any advice.
> 
> M.
> 
> p.s. I have all the data but I would like to avoid copying again
> because I have to copy everything
> (over 500gb) over SSH trough a 10mb connection

Half a pound (500g) of 'b' though a 10 milli-bit connection?
Sounds like a pain (yes, capitalization can carry meaning).

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

> Hi,
>
> On Tue, Sep 14, 2010 at 06:55:36PM +0200, Ma Begaj wrote:
>> Hi,
>>
>>
>> I got a new HDD a few days ago and I put it in an external case and connected
>> it over USB with my home server. It is a 2000GB WD disk,  WD20EARS.
>
> That would be one of the 2kB sector drives...

you mean 4kb?

>
>> I already read a few articles how should I format this "special"
>> drive, and I did it like this:
>> http://ubuntuforums.org/showthread.php?t=1456251
>>
>> I set the partition to begin on sector 64 (instead of 63), run
>> luksFormat, run luksOpen,
>> created XFS and copied my files on this disk. I umounted it later and
>> I rebooted
>> the server today.
>>
>> First thing I noticed is that disk/paritions are not visible in
>> /dev/disk/by-uuid/ although
>> /dev/sdm and /dev/sdm1 are available. fdisk also does not look same
>> as before:
>
> Not good.
>
>> fdisk output:
>> --------------
>> root@road:/dev/disk/by-uuid#  fdisk /dev/sdm
>>
>> The number of cylinders for this disk is set to 243201.
>> There is nothing wrong with that, but this is larger than 1024,
>> and could in certain setups cause problems with:
>> 1) software that runs at boot time (e.g., old versions of LILO)
>> 2) booting and partitioning software from other OSs
>>    (e.g., DOS FDISK, OS/2 FDISK)
>>
>> Command (m for help): p
>>
>> Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
>> 255 heads, 63 sectors/track, 243201 cylinders
>> Units = cylinders of 16065 * 512 = 8225280 bytes
>> Disk identifier: 0x00000000
>>
>>    Device Boot      Start         End      Blocks   Id  System
>> /dev/sdm1               1      243202  1953514552   83  Linux
>
> You did create one large partition?

yes just one filling whole disk.


>
> What is the output of "fdisk -lu /dev/sdm1"?
>

root@road:/dev/disk/by-uuid#  fdisk -lu /dev/sdm1

Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdm1              64  3907029167  1953514552   83  Linux

Well, it looks good.



>
>> --------------
>>
>> "Start" is 1 instead of 64.... pretty strange because I set it to 64
>> when I was making this partition.
>
> You set it to 64 sectors. The 1 is a cylinder. You cannto see in this
> view where exactly the partition starts. Use the -u option to fdisk.

Good to know. Thanks.



>
>>
>> luksOpen does not work and luksDump/isLuks reported that /dev/sdm1 is
>> not luks partition.
>
> Interesting.
>
>> I have cryptsetup 1.0.5 on my server.
>
> Aehm, that is hiostoric. Please try with the current one,
> which would be 1.1.3.

Right now before continuing with the rest? I have 3 other luks
partitions on this machine. Should I expect some hiccups?

I am planning to upgrade whole system in the next week and
if possible I will wait until than.



>
>> There are two questions for me here:
>> 1. I have luksDump of /dev/sdm1 before this happened. Can I use
>> luksDump ouput  to
>> recover/overwrite Luks partition?
>
> I think not, but I am not sure. Before trying that, lets figure
> out whether this is simply an offset issue. Can you look
> through the start of the disk and give the offset where the
> "LUKS" string is exactly?
>
> Something like this should do the trick:
>
> hd /dev/sdm | grep LUKS

"hd"? I don't have hd command on my server.

but I found the position with with dd and vi:

dd if=/dev/sdm of=/tmp/sdm bs=32256 count=1 obs=10240 skip=1


this gives me LUKS on the beginning of /tmp/sdm so offset should be 32256:

LUKSº¾^@^Aaes^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@cbc-essiv:sha256^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@sha1^@^@^@^@^@^@^@^@^@^
   @^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^D^H^@^@^@^Pq^W:<93>CEÚñè©äkÛ)ÍQlÉ<8f>¦ç0à<9b>m^Yr
¡^E


and like this:
root@road:/dev/disk/by-uuid#  dd if=/dev/sdm of=/tmp/sdm bs=1024000 count=1
1+0 records in
1+0 records out
1024000 bytes (1.0 MB) copied, 0.0327064 s, 31.3 MB/s
root@road:/dev/disk/by-uuid#  grep -ob --binary-files=text LUKS /tmp/sdm
32256:LUKS

or grep directly on /dev/sdm

[ 32256 ] should be offset.


What should I do with this?

>
>> fdisk reports "start" on sector 1 instead of 64, which means that
>> something there went wrong.
>
> No, see above.
>
>> Or not? Ideas? If I can use luksDump to recover luks-header should the
>> partition table maybe be
>> rewritten beginning with sector 64 before I recreate luks header.
>>
>>
>> 2. How all this could happen?
>>
>> I would be very happy to experiment with this, to repeat all the steps
>> to recreat the situation
>>  if I know that I am able to recover my partition. Any ideas before I
>> start to play with this?
>> How much of the partition header should I backup before starting with
>> all of this?
>
> See the FAQ. If you back up the first 100MB, you are on the safe side.
>

root@road:/dev/disk/by-uuid#  dd if=/dev/sdm of=/root/sdm.iso
bs=1024000 count=100
100+0 records in
100+0 records out
102400000 bytes (102 MB) copied, 3.01704 s, 33.9 MB/s


>>
>> p.s. I have all the data but I would like to avoid copying again
>> because I have to copy everything
>> (over 500gb) over SSH trough a 10mb connection
>
> Half a pound (500g) of 'b' though a 10 milli-bit connection?
> Sounds like a pain (yes, capitalization can carry meaning).

;-). believe me even 10[M]b is pretty slow if you have to copy 500[GB].

Thanks

M.

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Arno Wagner]

On Tue, Sep 14, 2010 at 08:50:19PM +0200, Ma Begaj wrote:
> > Hi,
> >
> > On Tue, Sep 14, 2010 at 06:55:36PM +0200, Ma Begaj wrote:
> >> Hi,
> >>
> >>
> >> I got a new HDD a few days ago and I put it in an external case and connected
> >> it over USB with my home server. It is a 2000GB WD disk, ?WD20EARS.
> >
> > That would be one of the 2kB sector drives...
> 
> you mean 4kb?

Yes ;-)
 
> >
> >> I already read a few articles how should I format this "special"
> >> drive, and I did it like this:
> >> http://ubuntuforums.org/showthread.php?t=1456251
> >>
> >> I set the partition to begin on sector 64 (instead of 63), run
> >> luksFormat, run luksOpen,
> >> created XFS and copied my files on this disk. I umounted it later and
> >> I rebooted
> >> the server today.
> >>
> >> First thing I noticed is that disk/paritions are not visible in
> >> /dev/disk/by-uuid/ although
> >> /dev/sdm and /dev/sdm1 are available. fdisk also does not look same
> >> as before:
> >
> > Not good.
> >
> >> fdisk output:
> >> --------------
> >> root@road:/dev/disk/by-uuid# ?fdisk /dev/sdm
> >>
> >> The number of cylinders for this disk is set to 243201.
> >> There is nothing wrong with that, but this is larger than 1024,
> >> and could in certain setups cause problems with:
> >> 1) software that runs at boot time (e.g., old versions of LILO)
> >> 2) booting and partitioning software from other OSs
> >> ? ?(e.g., DOS FDISK, OS/2 FDISK)
> >>
> >> Command (m for help): p
> >>
> >> Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
> >> 255 heads, 63 sectors/track, 243201 cylinders
> >> Units = cylinders of 16065 * 512 = 8225280 bytes
> >> Disk identifier: 0x00000000
> >>
> >> ? ?Device Boot ? ? ?Start ? ? ? ? End ? ? ?Blocks ? Id ?System
> >> /dev/sdm1 ? ? ? ? ? ? ? 1 ? ? ?243202 ?1953514552 ? 83 ?Linux
> >
> > You did create one large partition?
> 
> yes just one filling whole disk.
> 
> 
> >
> > What is the output of "fdisk -lu /dev/sdm1"?
> >
> 
> root@road:/dev/disk/by-uuid#  fdisk -lu /dev/sdm1
> 
> Disk /dev/sdm: 2000.3 GB, 2000398934016 bytes
> 255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Disk identifier: 0x00000000
> 
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdm1              64  3907029167  1953514552   83  Linux
> 
> Well, it looks good.

Indeed. Now there is the question why cryptsetup fails and
the partition is not detected.

> 
> 
> >
> >> --------------
> >>
> >> "Start" is 1 instead of 64.... pretty strange because I set it to 64
> >> when I was making this partition.
> >
> > You set it to 64 sectors. The 1 is a cylinder. You cannto see in this
> > view where exactly the partition starts. Use the -u option to fdisk.
> 
> Good to know. Thanks.
> 
> 
> 
> >
> >>
> >> luksOpen does not work and luksDump/isLuks reported that /dev/sdm1 is
> >> not luks partition.
> >
> > Interesting.
> >
> >> I have cryptsetup 1.0.5 on my server.
> >
> > Aehm, that is hiostoric. Please try with the current one,
> > which would be 1.1.3.
> 
> Right now before continuing with the rest? I have 3 other luks
> partitions on this machine. Should I expect some hiccups?
> 
> I am planning to upgrade whole system in the next week and
> if possible I will wait until than.

Hmm. Better wait. Therw was some issue that could hit you
if you have other LUKS partitions. I am honestly not sure
whether it applies.


> 
> >
> >> There are two questions for me here:
> >> 1. I have luksDump of /dev/sdm1 before this happened. Can I use
> >> luksDump ouput ?to
> >> recover/overwrite Luks partition?
> >
> > I think not, but I am not sure. Before trying that, lets figure
> > out whether this is simply an offset issue. Can you look
> > through the start of the disk and give the offset where the
> > "LUKS" string is exactly?
> >
> > Something like this should do the trick:
> >
> > hd /dev/sdm | grep LUKS
> 
> "hd"? I don't have hd command on my server.

HexDump.

> but I found the position with with dd and vi:
> 
> dd if=/dev/sdm of=/tmp/sdm bs=32256 count=1 obs=10240 skip=1
> 
> 
> this gives me LUKS on the beginning of /tmp/sdm so offset should be 32256:
> 
> LUKS??^@^Aaes^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@cbc-essiv:sha256^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@sha1^@^@^@^@^@^@^@^@^@^
>    @^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^D^H^@^@^@^Pq^W:<93>CE?????k?)?Ql?<8f>??0?<9b>m^Yr
> ?^E
> 
> 
> and like this:
> root@road:/dev/disk/by-uuid#  dd if=/dev/sdm of=/tmp/sdm bs=1024000 count=1
> 1+0 records in
> 1+0 records out
> 1024000 bytes (1.0 MB) copied, 0.0327064 s, 31.3 MB/s
> root@road:/dev/disk/by-uuid#  grep -ob --binary-files=text LUKS /tmp/sdm
> 32256:LUKS
> 
> or grep directly on /dev/sdm
> 
> [ 32256 ] should be offset.
> 
> 
> What should I do with this?

This confirms that you did create the LUKS container at the
right place, namely the LUKS header is at 64 x 512 B offset. 

This seems to be some bizzarre issue with your system not detecting the
partitions right and LUKS is actually working as expected.


Ok, lets do some triage.

- What Linux, what kernel version?

- Any special set-up like virtuzlization?

- Any special partition management system?


> >
> >> fdisk reports "start" on sector 1 instead of 64, which means that
> >> something there went wrong.
> >
> > No, see above.
> >
> >> Or not? Ideas? If I can use luksDump to recover luks-header should the
> >> partition table maybe be
> >> rewritten beginning with sector 64 before I recreate luks header.
> >>
> >>
> >> 2. How all this could happen?
> >>
> >> I would be very happy to experiment with this, to repeat all the steps
> >> to recreat the situation
> >> ?if I know that I am able to recover my partition. Any ideas before I
> >> start to play with this?
> >> How much of the partition header should I backup before starting with
> >> all of this?
> >
> > See the FAQ. If you back up the first 100MB, you are on the safe side.
> >
> 
> root@road:/dev/disk/by-uuid#  dd if=/dev/sdm of=/root/sdm.iso
> bs=1024000 count=100
> 100+0 records in
> 100+0 records out
> 102400000 bytes (102 MB) copied, 3.01704 s, 33.9 MB/s

Yes, like that.

> 
> >>
> >> p.s. I have all the data but I would like to avoid copying again
> >> because I have to copy everything
> >> (over 500gb) over SSH trough a 10mb connection
> >
> > Half a pound (500g) of 'b' though a 10 milli-bit connection?
> > Sounds like a pain (yes, capitalization can carry meaning).
> 
> ;-). believe me even 10[M]b is pretty slow if you have to copy 500[GB].

Hehe.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

>> root@road:/dev/disk/by-uuid#  grep -ob --binary-files=text LUKS /tmp/sdm
>> 32256:LUKS
>>
>> or grep directly on /dev/sdm
>>
>> [ 32256 ] should be offset.
>>
>> What should I do with this?
>
> This confirms that you did create the LUKS container at the
> right place, namely the LUKS header is at 64 x 512 B offset.

64x512 =32768
32768  !=  32256

512 bytes is partition table? should it be 32780?
I am probably missing something.


I just tested it on other LUKS disks and every disk gives 32256 which means
that this offset is OK.


> This seems to be some bizzarre issue with your system not detecting the
> partitions right and LUKS is actually working as expected.

grep -ob --binary-files=text LUKS  /dev/sdm1

/dev/sdm1 does not return anything although it should return "0:LUKS".

 but grep on /dev/sdm returns "32256:LUKS"


>
> Ok, lets do some triage.
>
> - What Linux, what kernel version?

 2.6.29.4, vanilla

I have a few other HDDs in this machine, some of them with
LUKS,  raid5 with LUKS etc.
This is the only disk with LUKS on USB (others are intern/SATA)
but that should not be the reason.


> - Any special set-up like virtuzlization?
> - Any special partition management system?

No to both.


I think it has something to do with 4Kb sector size and some kind
partition table incompatibilty with the kernel.

or maybe mkfs.xfs did something because I did not use "-b 4096" (block size
in XFS is 4kB by default).


LUKS header starts obviously where it should if I look at /dev/sdm.

I don't know enough about partition tables and headers but it looks like
/dev/sdm1 is starting little bit too far and LUKS header stays before the
beginning of /dev/sdm1 (first partition).

Only obvious reason (at lease for me)  for this could be in this 4Kb
sector size.


Thanks

M.

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Arno Wagner]

On Tue, Sep 14, 2010 at 11:37:03PM +0200, Ma Begaj wrote:
> >> root@road:/dev/disk/by-uuid# ?grep -ob --binary-files=text LUKS /tmp/sdm
> >> 32256:LUKS
> >>
> >> or grep directly on /dev/sdm
> >>
> >> [ 32256 ] should be offset.
> >>
> >> What should I do with this?
> >
> > This confirms that you did create the LUKS container at the
> > right place, namely the LUKS header is at 64 x 512 B offset.
> 
> 64x512 =32768
> 32768  !=  32256
> 
> 512 bytes is partition table? should it be 32780?
> I am probably missing something.

No, you see what I missed. Your LUKS sector is at 63 sectors
offset, not 64. However the partition starts at 64 and therefore
the LUKS sector is not at its start. 

I don't know what went wrong here, but partitions 1-4 are special in
that you can change them without any changes to the disk except 
in the first sector (for partitions >= a chained partition sector
is writte to disk and that can kill a LUKS header and do other dmage).

This smeans you can just delete the partiton with fdisk and create
a new one, which will the start at sector 63 and should have
your LUKS container in it. It will not be 4kB alingned, though.
You should unplug and replug the disk after the partition creation.

If this goes wrong, you still have the first 100MB as backup.

> I just tested it on other LUKS disks and every disk gives 32256 which means
> that this offset is OK.

This is the standard offset when using 512 byte sectors, i.e.
the standard first sector of a partition starting at sector 63.

 
> > This seems to be some bizzarre issue with your system not detecting the
> > partitions right and LUKS is actually working as expected.
> 
> grep -ob --binary-files=text LUKS  /dev/sdm1
> 
> /dev/sdm1 does not return anything although it should return "0:LUKS".
> 
>  but grep on /dev/sdm returns "32256:LUKS"

Yes, because the poartition starts at offset 32768 and the header is
one sector before that.

> >
> > Ok, lets do some triage.
> >
> > - What Linux, what kernel version?
> 
>  2.6.29.4, vanilla
> 
> I have a few other HDDs in this machine, some of them with
> LUKS,  raid5 with LUKS etc.
> This is the only disk with LUKS on USB (others are intern/SATA)
> but that should not be the reason.

I agree.

> 
> > - Any special set-up like virtuzlization?
> > - Any special partition management system?
> 
> No to both.
> 
> 
> I think it has something to do with 4Kb sector size and some kind
> partition table incompatibilty with the kernel.
> 
> or maybe mkfs.xfs did something because I did not use "-b 4096" (block size
> in XFS is 4kB by default).
> 
> 
> LUKS header starts obviously where it should if I look at /dev/sdm.
>
> I don't know enough about partition tables and headers but it looks like
> /dev/sdm1 is starting little bit too far and LUKS header stays before the
> beginning of /dev/sdm1 (first partition).

Indeed.
 
> Only obvious reason (at lease for me)  for this could be in this 4Kb
> sector size.

I think something has gone wrong when you did the alignment.
Did you first create a not-4kB aligned partition and delete 
that again? If so the kernel would could remembered the partition
but not that it got deleted and replaced (that needs a disk unplug).
The reason is that the kernel sees partition changes only under some
circumstances.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

>> > This confirms that you did create the LUKS container at the
>> > right place, namely the LUKS header is at 64 x 512 B offset.
>>
>> 64x512 =32768
>> 32768  !=  32256
>>
>> 512 bytes is partition table? should it be 32780?
>> I am probably missing something.
>
> No, you see what I missed. Your LUKS sector is at 63 sectors
> offset, not 64. However the partition starts at 64 and therefore
> the LUKS sector is not at its start.
>
> I don't know what went wrong here, but partitions 1-4 are special in
> that you can change them without any changes to the disk except
> in the first sector (for partitions >= a chained partition sector
> is writte to disk and that can kill a LUKS header and do other dmage).
>
> This smeans you can just delete the partiton with fdisk and create
> a new one, which will the start at sector 63 and should have
> your LUKS container in it. It will not be 4kB alingned, though.
> You should unplug and replug the disk after the partition creation.
>
> If this goes wrong, you still have the first 100MB as backup.

;-)). you are the man :).

Working like a charm.  Deleted partition, created new starting at 63,
unplug/plug and luks partition is there. Thanks.

I owe you at least a drink.


>

>> I don't know enough about partition tables and headers but it looks like
>> /dev/sdm1 is starting little bit too far and LUKS header stays before the
>> beginning of /dev/sdm1 (first partition).
>
> Indeed.
>
>> Only obvious reason (at lease for me)  for this could be in this 4Kb
>> sector size.
>
> I think something has gone wrong when you did the alignment.
> Did you first create a not-4kB aligned partition and delete
> that again? If so the kernel would could remembered the partition
> but not that it got deleted and replaced (that needs a disk unplug).
> The reason is that the kernel sees partition changes only under some
> circumstances.

Wow great assumption. I did exactly that. I was doing this remotely
(I was not able to unplug) and I forgot first time to set 4kB sector size.
I deleted partition (which already was luksFormated too) and recreated
it again but this time in 4kB format.


Thanks a lot.


One more question because this is going pretty well now :) :

Do you have an idea how could I recreate 4kB aligned partition
without deleting data from the disk? This is more facultative question,
because I know have my data.

I thought about shrinking this partition, creating a second parition
 with 500gb to copy all data on this new partition, creating first paritition
in 4kB aligned format, copying all data back to this partition etc.

That could maybe  even work if XFS would support shrinking. But it
does not as I can remember ;-). Could partition start be moved from
sector 63 to sector 64 and keep working filesystem (of couse at the same
time risking to destroy some of the data)? Or is this (if possible) would
destroy filesystem totally?


M.

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Heinz Diehl]

On 15.09.2010, Ma Begaj wrote: 

> That could maybe  even work if XFS would support shrinking. But it
> does not as I can remember ;-).

Yes, it doesn't, you remember correctly. Besides of the whole thread,
you should mount with "inode64" if you're not already doing that..

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

2010/9/15 Heinz Diehl <htd@fancy-poultry.org>:
> On 15.09.2010, Ma Begaj wrote:
>
>> That could maybe  even work if XFS would support shrinking. But it
>> does not as I can remember ;-).
>
> Yes, it doesn't, you remember correctly. Besides of the whole thread,
> you should mount with "inode64" if you're not already doing that..
>

I started using inode64 with kernel 2.6.29 1-2 years ago just for testing
because it was not available on 32bit system before.

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Arno Wagner]

On Wed, Sep 15, 2010 at 01:02:25AM +0200, Ma Begaj wrote:
> >> > This confirms that you did create the LUKS container at the
> >> > right place, namely the LUKS header is at 64 x 512 B offset.
> >>
> >> 64x512 =32768
> >> 32768 ?!= ?32256
> >>
> >> 512 bytes is partition table? should it be 32780?
> >> I am probably missing something.
> >
> > No, you see what I missed. Your LUKS sector is at 63 sectors
> > offset, not 64. However the partition starts at 64 and therefore
> > the LUKS sector is not at its start.
> >
> > I don't know what went wrong here, but partitions 1-4 are special in
> > that you can change them without any changes to the disk except
> > in the first sector (for partitions >= a chained partition sector
> > is writte to disk and that can kill a LUKS header and do other dmage).
> >
> > This smeans you can just delete the partiton with fdisk and create
> > a new one, which will the start at sector 63 and should have
> > your LUKS container in it. It will not be 4kB alingned, though.
> > You should unplug and replug the disk after the partition creation.
> >
> > If this goes wrong, you still have the first 100MB as backup.
> 
> ;-)). you are the man :).
> 
> Working like a charm.  Deleted partition, created new starting at 63,
> unplug/plug and luks partition is there. Thanks.
> 
> I owe you at least a drink.

You are welcome. Made my now day too ;-)

> >> I don't know enough about partition tables and headers but it looks like
> >> /dev/sdm1 is starting little bit too far and LUKS header stays before the
> >> beginning of /dev/sdm1 (first partition).
> >
> > Indeed.
> >
> >> Only obvious reason (at lease for me) ?for this could be in this 4Kb
> >> sector size.
> >
> > I think something has gone wrong when you did the alignment.
> > Did you first create a not-4kB aligned partition and delete
> > that again? If so the kernel would could remembered the partition
> > but not that it got deleted and replaced (that needs a disk unplug).
> > The reason is that the kernel sees partition changes only under some
> > circumstances.
> 
> Wow great assumption. I did exactly that. I was doing this remotely
> (I was not able to unplug) and I forgot first time to set 4kB sector size.
> I deleted partition (which already was luksFormated too) and recreated
> it again but this time in 4kB format.

Its easy to have the idea when you already messed that up yourself 
once ;-)
  
> Thanks a lot.
> 
> 
> One more question because this is going pretty well now :) :
> 
> Do you have an idea how could I recreate 4kB aligned partition
> without deleting data from the disk? This is more facultative question,
> because I know have my data.

Difficult. You would need to shift the wole partitoon by
one sector.
 
> I thought about shrinking this partition, creating a second parition
>  with 500gb to copy all data on this new partition, creating first paritition
> in 4kB aligned format, copying all data back to this partition etc.

That would work if you can shrink. At least my standard
tool (Gnu parted) cannot do much with XFS.

Best option would be if you could temporary store all the date
somewhere else on the machine and repartition. 

The other option is to live with it, USB2 is very slow anyways,
sector alignment may not even matter that much.

> That could maybe  even work if XFS would support shrinking. But it
> does not as I can remember ;-). Could partition start be moved from
> sector 63 to sector 64 and keep working filesystem (of couse at the same
> time risking to destroy some of the data)? Or is this (if possible) would
> destroy filesystem totally?

Well, you could either use dd_rescue or write a program in C that 
moves the raw partition. From your fdisk -ul output, I conclude
that there is no space left at the end of the disk, so you would 
have to move the start from sector 63 to sector 60. That
should still work. Another option is to do away with partitioning
and move the start from sector 63 to 0. Your filesystem is then
on /dev/sdm.

A possible commandline for dd_rescue would be something like this
(moving the partition to the start of the disk)

  dd_rescue -s 63 -S 0 -b /dev/sdm /dev/sdm

Looks good, but better assume a 99% probability of complete data 
loss. Connection problems, read errors, all can kill the data.
And this will take something like > 10 days for 2TB on USB2.

Arno
-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

Re: [dm-crypt] WD20EARS (4KB sector size) lost Luks header after reboot

[Ma Begaj]

2010/9/15 Arno Wagner <arno@wagner.name>:
> On Wed, Sep 15, 2010 at 01:02:25AM +0200, Ma Begaj wrote:
>> >> > This confirms that you did create the LUKS container at the
>> >> > right place, namely the LUKS header is at 64 x 512 B offset.
>> >>
>> >> 64x512 =32768
>> >> 32768 ?!= ?32256
>> >>
>> >> 512 bytes is partition table? should it be 32780?
>> >> I am probably missing something.
>> >
>> > No, you see what I missed. Your LUKS sector is at 63 sectors
>> > offset, not 64. However the partition starts at 64 and therefore
>> > the LUKS sector is not at its start.
>> >
>> > I don't know what went wrong here, but partitions 1-4 are special in
>> > that you can change them without any changes to the disk except
>> > in the first sector (for partitions >= a chained partition sector
>> > is writte to disk and that can kill a LUKS header and do other dmage).
>> >
>> > This smeans you can just delete the partiton with fdisk and create
>> > a new one, which will the start at sector 63 and should have
>> > your LUKS container in it. It will not be 4kB alingned, though.
>> > You should unplug and replug the disk after the partition creation.
>> >
>> > If this goes wrong, you still have the first 100MB as backup.
>>
>> ;-)). you are the man :).
>>
>> Working like a charm.  Deleted partition, created new starting at 63,
>> unplug/plug and luks partition is there. Thanks.
>>
>> I owe you at least a drink.
>
> You are welcome. Made my now day too ;-)
>
>> >> I don't know enough about partition tables and headers but it looks like
>> >> /dev/sdm1 is starting little bit too far and LUKS header stays before the
>> >> beginning of /dev/sdm1 (first partition).
>> >
>> > Indeed.
>> >
>> >> Only obvious reason (at lease for me) ?for this could be in this 4Kb
>> >> sector size.
>> >
>> > I think something has gone wrong when you did the alignment.
>> > Did you first create a not-4kB aligned partition and delete
>> > that again? If so the kernel would could remembered the partition
>> > but not that it got deleted and replaced (that needs a disk unplug).
>> > The reason is that the kernel sees partition changes only under some
>> > circumstances.
>>
>> Wow great assumption. I did exactly that. I was doing this remotely
>> (I was not able to unplug) and I forgot first time to set 4kB sector size.
>> I deleted partition (which already was luksFormated too) and recreated
>> it again but this time in 4kB format.
>
> Its easy to have the idea when you already messed that up yourself
> once ;-)

Well, I will not forget my experience either ;-).


>
>> Thanks a lot.
>>
>>
>> One more question because this is going pretty well now :) :
>>
>> Do you have an idea how could I recreate 4kB aligned partition
>> without deleting data from the disk? This is more facultative question,
>> because I know have my data.
>
> Difficult. You would need to shift the wole partitoon by
> one sector.
>
>> I thought about shrinking this partition, creating a second parition
>>  with 500gb to copy all data on this new partition, creating first paritition
>> in 4kB aligned format, copying all data back to this partition etc.
>
> That would work if you can shrink. At least my standard
> tool (Gnu parted) cannot do much with XFS.
>
> Best option would be if you could temporary store all the date
> somewhere else on the machine and repartition.
>
> The other option is to live with it, USB2 is very slow anyways,
> sector alignment may not even matter that much.

Well, I was "pre-thinking" (this word sounds nice ;-) ) and I bought a USB3
case for the HDD although my old server still only have USB2 ports.



>> That could maybe  even work if XFS would support shrinking. But it
>> does not as I can remember ;-). Could partition start be moved from
>> sector 63 to sector 64 and keep working filesystem (of couse at the same
>> time risking to destroy some of the data)? Or is this (if possible) would
>> destroy filesystem totally?
>
> Well, you could either use dd_rescue or write a program in C that
> moves the raw partition. From your fdisk -ul output, I conclude
> that there is no space left at the end of the disk, so you would
> have to move the start from sector 63 to sector 60. That
> should still work. Another option is to do away with partitioning
> and move the start from sector 63 to 0. Your filesystem is then
> on /dev/sdm.
>
> A possible commandline for dd_rescue would be something like this
> (moving the partition to the start of the disk)
>
>  dd_rescue -s 63 -S 0 -b /dev/sdm /dev/sdm
>
> Looks good, but better assume a 99% probability of complete data
> loss. Connection problems, read errors, all can kill the data.
> And this will take something like > 10 days for 2TB on USB2.

Good that I was not patient enough ;-). I backed up everything, deleted
and recreated partition again.


Thanks.

M.

p.s.: I should contact you for a drink when I visit which city? ;-)