All of lore.kernel.org
 help / color / mirror / Atom feed
* [dm-crypt] crypsetup segfaulting during luksFormat
@ 2010-07-07 17:19 Sven Eschenberg
  2010-07-07 17:36 ` Arno Wagner
  2010-07-07 18:36 ` Milan Broz
  0 siblings, 2 replies; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-07 17:19 UTC (permalink / raw)
  To: dm-crypt

Hi list,

I was just trying to setup a new luks device, and for some reason (I
cannot determine yet) cryptsetup just segfaults (setting up the old
mappings works as usual).

These are the last couple lines before cryptsetup dies:
2370  read(4, "|\27\211<e\263\36\35j\201j\204e\326 \270\364\277\373
\300LY[\343d[\215\32\17\315N\333", 32) = 32
2370  open("/dev/md125", O_RDONLY)      = 5
2370  ioctl(5, BLKIOMIN, 0x7fff561ea000) = 0
2370  ioctl(5, BLKIOOPT, 0x7fff561e9ff8) = 0
2370  ioctl(5, BLKALIGNOFF, 0x7fff561ea00c) = 0
2370  close(5)                          = 0
2370  read(4, "\242\270\205|\207@\253\227 \321)\f\r/\"R>EN(wo\224\205\7e
\36\313g\232\n\33", 32) = 32
2370  rt_sigaction(SIGVTALRM, {0x40d3f0, [VTALRM], SA_RESTORER|
SA_RESTART, 0xf0000000fc0c748}, {SIG_DFL, [], 0}, 8) = 0
2370  setitimer(ITIMER_VIRTUAL, {it_interval={0, 0}, it_value={1, 0}},
NULL) = 0
2370  --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
2370  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
2370  +++ killed by SIGSEGV +++

where /dev/md125 is the device I am trying to setup via luksFormat.

Anyone with any suggestion what the problem can be?

Oh I forgot:
cryptsetup --version
cryptsetup 1.1.2

Regards

-Sven

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 17:19 [dm-crypt] crypsetup segfaulting during luksFormat Sven Eschenberg
@ 2010-07-07 17:36 ` Arno Wagner
  2010-07-07 17:49   ` Sven Eschenberg
  2010-07-07 18:36 ` Milan Broz
  1 sibling, 1 reply; 25+ messages in thread
From: Arno Wagner @ 2010-07-07 17:36 UTC (permalink / raw)
  To: dm-crypt


Just a guess: Your CPU is too fast, the time
measurement for the PKBF2 iterations did not 
expect that and does nonesense, resutling in 
the segfault.

Tht would be a classic ;-)

Arno

On Wed, Jul 07, 2010 at 07:19:38PM +0200, Sven Eschenberg wrote:
> Hi list,
> 
> I was just trying to setup a new luks device, and for some reason (I
> cannot determine yet) cryptsetup just segfaults (setting up the old
> mappings works as usual).
> 
> These are the last couple lines before cryptsetup dies:
> 2370  read(4, "|\27\211<e\263\36\35j\201j\204e\326 \270\364\277\373
> \300LY[\343d[\215\32\17\315N\333", 32) = 32
> 2370  open("/dev/md125", O_RDONLY)      = 5
> 2370  ioctl(5, BLKIOMIN, 0x7fff561ea000) = 0
> 2370  ioctl(5, BLKIOOPT, 0x7fff561e9ff8) = 0
> 2370  ioctl(5, BLKALIGNOFF, 0x7fff561ea00c) = 0
> 2370  close(5)                          = 0
> 2370  read(4, "\242\270\205|\207@\253\227 \321)\f\r/\"R>EN(wo\224\205\7e
> \36\313g\232\n\33", 32) = 32
> 2370  rt_sigaction(SIGVTALRM, {0x40d3f0, [VTALRM], SA_RESTORER|
> SA_RESTART, 0xf0000000fc0c748}, {SIG_DFL, [], 0}, 8) = 0
> 2370  setitimer(ITIMER_VIRTUAL, {it_interval={0, 0}, it_value={1, 0}},
> NULL) = 0
> 2370  --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> 2370  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> 2370  +++ killed by SIGSEGV +++
> 
> where /dev/md125 is the device I am trying to setup via luksFormat.
> 
> Anyone with any suggestion what the problem can be?
> 
> Oh I forgot:
> cryptsetup --version
> cryptsetup 1.1.2
> 
> Regards
> 
> -Sven
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 17:36 ` Arno Wagner
@ 2010-07-07 17:49   ` Sven Eschenberg
  2010-07-07 20:19     ` Arno Wagner
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-07 17:49 UTC (permalink / raw)
  To: dm-crypt

Hummm,

should I consider this geek humor or are you serious, the CPU is rather
slow (imho)?

Anyway, tweaking the iterationcount didn't chaneg a thing, messing with
the params did not help either.

The Backtrace does not really help a lot, I guess:

(gdb) bt
#0  0x000000000040d3fe in sigvtalarm ()
#1  0x0f0000000fc0c748 in ?? ()
#2  0x0000000000000000 in ?? ()

Well at least we know the crash happens in sigvtalarm, which I assume is
a signal handler of some sort within cryptsetup.

Regards

-Sven

On Wed, 2010-07-07 at 19:36 +0200, Arno Wagner wrote:
> Just a guess: Your CPU is too fast, the time
> measurement for the PKBF2 iterations did not 
> expect that and does nonesense, resutling in 
> the segfault.
> 
> Tht would be a classic ;-)
> 
> Arno
> 
> On Wed, Jul 07, 2010 at 07:19:38PM +0200, Sven Eschenberg wrote:
> > Hi list,
> > 
> > I was just trying to setup a new luks device, and for some reason (I
> > cannot determine yet) cryptsetup just segfaults (setting up the old
> > mappings works as usual).
> > 
> > These are the last couple lines before cryptsetup dies:
> > 2370  read(4, "|\27\211<e\263\36\35j\201j\204e\326 \270\364\277\373
> > \300LY[\343d[\215\32\17\315N\333", 32) = 32
> > 2370  open("/dev/md125", O_RDONLY)      = 5
> > 2370  ioctl(5, BLKIOMIN, 0x7fff561ea000) = 0
> > 2370  ioctl(5, BLKIOOPT, 0x7fff561e9ff8) = 0
> > 2370  ioctl(5, BLKALIGNOFF, 0x7fff561ea00c) = 0
> > 2370  close(5)                          = 0
> > 2370  read(4, "\242\270\205|\207@\253\227 \321)\f\r/\"R>EN(wo\224\205\7e
> > \36\313g\232\n\33", 32) = 32
> > 2370  rt_sigaction(SIGVTALRM, {0x40d3f0, [VTALRM], SA_RESTORER|
> > SA_RESTART, 0xf0000000fc0c748}, {SIG_DFL, [], 0}, 8) = 0
> > 2370  setitimer(ITIMER_VIRTUAL, {it_interval={0, 0}, it_value={1, 0}},
> > NULL) = 0
> > 2370  --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> > 2370  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> > 2370  +++ killed by SIGSEGV +++
> > 
> > where /dev/md125 is the device I am trying to setup via luksFormat.
> > 
> > Anyone with any suggestion what the problem can be?
> > 
> > Oh I forgot:
> > cryptsetup --version
> > cryptsetup 1.1.2
> > 
> > Regards
> > 
> > -Sven
> > 
> > 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 17:19 [dm-crypt] crypsetup segfaulting during luksFormat Sven Eschenberg
  2010-07-07 17:36 ` Arno Wagner
@ 2010-07-07 18:36 ` Milan Broz
  1 sibling, 0 replies; 25+ messages in thread
From: Milan Broz @ 2010-07-07 18:36 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt

On 07/07/2010 07:19 PM, Sven Eschenberg wrote:
> Hi list,
> 
> I was just trying to setup a new luks device, and for some reason (I
> cannot determine yet) cryptsetup just segfaults (setting up the old
> mappings works as usual).

Hi,
please send me output of that segfaulting command with added --debug switch,
(and  core file if possible) + on which distro and architecture is it?

That's apparently bug which must be fixed.
Thanks,
Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 17:49   ` Sven Eschenberg
@ 2010-07-07 20:19     ` Arno Wagner
  2010-07-07 20:44       ` Milan Broz
  2010-07-07 20:51       ` Arno Wagner
  0 siblings, 2 replies; 25+ messages in thread
From: Arno Wagner @ 2010-07-07 20:19 UTC (permalink / raw)
  To: dm-crypt

On Wed, Jul 07, 2010 at 07:49:04PM +0200, Sven Eschenberg wrote:
> Hummm,
> 
> should I consider this geek humor or are you serious, the CPU is rather
> slow (imho)?

Par humor, part serious. But if the CPU is rather slow, then
this is somethin else, I guess.
 
> Anyway, tweaking the iterationcount didn't chaneg a thing, messing with
> the params did not help either.

Hmm.

> The Backtrace does not really help a lot, I guess:
> 
> (gdb) bt
> #0  0x000000000040d3fe in sigvtalarm ()
> #1  0x0f0000000fc0c748 in ?? ()
> #2  0x0000000000000000 in ?? ()
> 
> Well at least we know the crash happens in sigvtalarm, which I assume is
> a signal handler of some sort within cryptsetup.

Looks like it. SIGVTALARM is what you get when you set
the ITIMER_VIRTUAL interval timer and it expires. YThe
virtual timer is decremented only when the process runns
that set it. Hence it can be used to measure (userspace)
CPU time of a process. My guess would be that it is
not supposed to expire and handling the SIGVTALARM
somehow fails. 

Arno




 
> Regards
> 
> -Sven
> 
> On Wed, 2010-07-07 at 19:36 +0200, Arno Wagner wrote:
> > Just a guess: Your CPU is too fast, the time
> > measurement for the PKBF2 iterations did not 
> > expect that and does nonesense, resutling in 
> > the segfault.
> > 
> > Tht would be a classic ;-)
> > 
> > Arno
> > 
> > On Wed, Jul 07, 2010 at 07:19:38PM +0200, Sven Eschenberg wrote:
> > > Hi list,
> > > 
> > > I was just trying to setup a new luks device, and for some reason (I
> > > cannot determine yet) cryptsetup just segfaults (setting up the old
> > > mappings works as usual).
> > > 
> > > These are the last couple lines before cryptsetup dies:
> > > 2370  read(4, "|\27\211<e\263\36\35j\201j\204e\326 \270\364\277\373
> > > \300LY[\343d[\215\32\17\315N\333", 32) = 32
> > > 2370  open("/dev/md125", O_RDONLY)      = 5
> > > 2370  ioctl(5, BLKIOMIN, 0x7fff561ea000) = 0
> > > 2370  ioctl(5, BLKIOOPT, 0x7fff561e9ff8) = 0
> > > 2370  ioctl(5, BLKALIGNOFF, 0x7fff561ea00c) = 0
> > > 2370  close(5)                          = 0
> > > 2370  read(4, "\242\270\205|\207@\253\227 \321)\f\r/\"R>EN(wo\224\205\7e
> > > \36\313g\232\n\33", 32) = 32
> > > 2370  rt_sigaction(SIGVTALRM, {0x40d3f0, [VTALRM], SA_RESTORER|
> > > SA_RESTART, 0xf0000000fc0c748}, {SIG_DFL, [], 0}, 8) = 0
> > > 2370  setitimer(ITIMER_VIRTUAL, {it_interval={0, 0}, it_value={1, 0}},
> > > NULL) = 0
> > > 2370  --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> > > 2370  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> > > 2370  +++ killed by SIGSEGV +++
> > > 
> > > where /dev/md125 is the device I am trying to setup via luksFormat.
> > > 
> > > Anyone with any suggestion what the problem can be?
> > > 
> > > Oh I forgot:
> > > cryptsetup --version
> > > cryptsetup 1.1.2
> > > 
> > > Regards
> > > 
> > > -Sven
> > > 
> > > 
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > 
> > 
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 20:19     ` Arno Wagner
@ 2010-07-07 20:44       ` Milan Broz
  2010-07-07 20:58         ` Sven Eschenberg
  2010-07-07 20:51       ` Arno Wagner
  1 sibling, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-07 20:44 UTC (permalink / raw)
  To: dm-crypt

On 07/07/2010 10:19 PM, Arno Wagner wrote:
> On Wed, Jul 07, 2010 at 07:49:04PM +0200, Sven Eschenberg wrote:
>> Hummm,
>>
>> should I consider this geek humor or are you serious, the CPU is rather
>> slow (imho)?
> 
> Par humor, part serious. But if the CPU is rather slow, then
> this is somethin else, I guess.

Don't worry, it was optimized such way that even on slow machines
it crashes quickly :-)

>> Well at least we know the crash happens in sigvtalarm, which I assume is
>> a signal handler of some sort within cryptsetup.

The timer is there intentionally to calculate PBKDF2 iterations per second,
after one second is loop stopped. There is safe limit (IIRC 1000
iterations) - so even on slow CPU it returns safe value.
(iteration count is unsigned 64bit number, it should not overflow,
ale least not in this century :-)

Anyway, I need debug log and info I requested to analyse it.

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 20:19     ` Arno Wagner
  2010-07-07 20:44       ` Milan Broz
@ 2010-07-07 20:51       ` Arno Wagner
  1 sibling, 0 replies; 25+ messages in thread
From: Arno Wagner @ 2010-07-07 20:51 UTC (permalink / raw)
  To: dm-crypt

Ok, I had a look at the code. The problem seems to happen
in PBKDF2_performance_check. This seems to run PBKDF2
until 1 sec of CPU time has been consumed and then
terminates, reporting the iteration count.

One potential implementation error I see is that in line 204,
__PBKDF2_performance being non-zero is used as termination
criterium, which in turn is set in the signal handler.
This is at least conceptually wrong, as it assumes
__PBKDF2_global_j is non-zero at the time the signal handler
is called. But whether this is true depends entirely on the
timing of PBKDF2. This should be fixed to use a flag
telling BKPDF2 that the signal handler has been called.
Otherwise the 1 second of CPU time being reached before one
PBKDF2 iteration is complete, will result in the benchmark
running forever? At least the iteration count variable 
is an unsigned int set to -1 ... 

I do not see how this can cause a segmentation fault, though.
The signal handler does a straight assignment between
two static volatile uint64_t variables, no pointers involved
at all.

Maybe some obscure compiler or linker problem?

Incidentially, this code was not changed in 1.1.2...

Arno




On Wed, Jul 07, 2010 at 10:19:55PM +0200, Arno Wagner wrote:
> On Wed, Jul 07, 2010 at 07:49:04PM +0200, Sven Eschenberg wrote:
> > Hummm,
> > 
> > should I consider this geek humor or are you serious, the CPU is rather
> > slow (imho)?
> 
> Par humor, part serious. But if the CPU is rather slow, then
> this is somethin else, I guess.
>  
> > Anyway, tweaking the iterationcount didn't chaneg a thing, messing with
> > the params did not help either.
> 
> Hmm.
> 
> > The Backtrace does not really help a lot, I guess:
> > 
> > (gdb) bt
> > #0  0x000000000040d3fe in sigvtalarm ()
> > #1  0x0f0000000fc0c748 in ?? ()
> > #2  0x0000000000000000 in ?? ()
> > 
> > Well at least we know the crash happens in sigvtalarm, which I assume is
> > a signal handler of some sort within cryptsetup.
> 
> Looks like it. SIGVTALARM is what you get when you set
> the ITIMER_VIRTUAL interval timer and it expires. YThe
> virtual timer is decremented only when the process runns
> that set it. Hence it can be used to measure (userspace)
> CPU time of a process. My guess would be that it is
> not supposed to expire and handling the SIGVTALARM
> somehow fails. 
> 
> Arno
> 
> 
> 
> 
>  
> > Regards
> > 
> > -Sven
> > 
> > On Wed, 2010-07-07 at 19:36 +0200, Arno Wagner wrote:
> > > Just a guess: Your CPU is too fast, the time
> > > measurement for the PKBF2 iterations did not 
> > > expect that and does nonesense, resutling in 
> > > the segfault.
> > > 
> > > Tht would be a classic ;-)
> > > 
> > > Arno
> > > 
> > > On Wed, Jul 07, 2010 at 07:19:38PM +0200, Sven Eschenberg wrote:
> > > > Hi list,
> > > > 
> > > > I was just trying to setup a new luks device, and for some reason (I
> > > > cannot determine yet) cryptsetup just segfaults (setting up the old
> > > > mappings works as usual).
> > > > 
> > > > These are the last couple lines before cryptsetup dies:
> > > > 2370  read(4, "|\27\211<e\263\36\35j\201j\204e\326 \270\364\277\373
> > > > \300LY[\343d[\215\32\17\315N\333", 32) = 32
> > > > 2370  open("/dev/md125", O_RDONLY)      = 5
> > > > 2370  ioctl(5, BLKIOMIN, 0x7fff561ea000) = 0
> > > > 2370  ioctl(5, BLKIOOPT, 0x7fff561e9ff8) = 0
> > > > 2370  ioctl(5, BLKALIGNOFF, 0x7fff561ea00c) = 0
> > > > 2370  close(5)                          = 0
> > > > 2370  read(4, "\242\270\205|\207@\253\227 \321)\f\r/\"R>EN(wo\224\205\7e
> > > > \36\313g\232\n\33", 32) = 32
> > > > 2370  rt_sigaction(SIGVTALRM, {0x40d3f0, [VTALRM], SA_RESTORER|
> > > > SA_RESTART, 0xf0000000fc0c748}, {SIG_DFL, [], 0}, 8) = 0
> > > > 2370  setitimer(ITIMER_VIRTUAL, {it_interval={0, 0}, it_value={1, 0}},
> > > > NULL) = 0
> > > > 2370  --- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
> > > > 2370  --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> > > > 2370  +++ killed by SIGSEGV +++
> > > > 
> > > > where /dev/md125 is the device I am trying to setup via luksFormat.
> > > > 
> > > > Anyone with any suggestion what the problem can be?
> > > > 
> > > > Oh I forgot:
> > > > cryptsetup --version
> > > > cryptsetup 1.1.2
> > > > 
> > > > Regards
> > > > 
> > > > -Sven
> > > > 
> > > > 
> > > > _______________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt@saout.de
> > > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > > 
> > > 
> > 
> > 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> 
> -- 
> Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
> GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
> ----
> Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
> 
> If it's in the news, don't worry about it.  The very definition of 
> "news" is "something that hardly ever happens." -- Bruce Schneier 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 20:44       ` Milan Broz
@ 2010-07-07 20:58         ` Sven Eschenberg
  2010-07-07 21:22           ` Milan Broz
  2010-07-07 21:40           ` Milan Broz
  0 siblings, 2 replies; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-07 20:58 UTC (permalink / raw)
  To: dm-crypt

For starters, I'll select the easiest case, plain luksFormat without
special params:

cryptsetup luksFormat /dev/md125

# Allocating crypt device /dev/md125 context.
# Trying to open and read device /dev/md125.
# Initialising device-mapper backend, UDEV is enabled.
# Detected dm-crypt target of version 1.7.0.
# Timeout set to 0 miliseconds.
# Password retry count set to 0.
# Iteration time set to 1000 miliseconds.
# Password verification disabled.
Enter LUKS passphrase: 
Verify passphrase: 
# Formatting device /dev/md125 as type LUKS1.
# Initializing crypto backend (using secure memory).
# Topology: IO (524288/1048576), offset = 0; Required alignment is
1048576 bytes.
# Generating LUKS header version 1 using hash sha1, aes,
cbc-essiv:sha256, MK 32 bytes
Segmentation fault

For the fun of it, I will now set the iteration time to 10 000ms. (-i
10000)

# Allocating crypt device /dev/md125 context.
# Trying to open and read device /dev/md125.
# Initialising device-mapper backend, UDEV is enabled.
# Detected dm-crypt target of version 1.7.0.
# Timeout set to 0 miliseconds.
# Password retry count set to 0.
# Iteration time set to 10000 miliseconds.
# Password verification disabled.
Enter LUKS passphrase: 
Verify passphrase: 
# Formatting device /dev/md125 as type LUKS1.
# Initializing crypto backend (using secure memory).
# Topology: IO (524288/1048576), offset = 0; Required alignment is
1048576 bytes.
# Generating LUKS header version 1 using hash sha1, aes,
cbc-essiv:sha256, MK 32 bytes
Segmentation fault

The segfault comes way before the 10 seconds are over, that much I can
tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
on amd64 arch.

Currently no core file is created, can you give me a fast shot on how I
force the system to write out a core?

The alignment (btw) is obviously detected correctly, that much I can
tell you too.


Regards

-Sven

P.S.: Sorry it took some time to provide the additional info, I was away
watching the soccer game :-).


On Wed, 2010-07-07 at 22:44 +0200, Milan Broz wrote:
> On 07/07/2010 10:19 PM, Arno Wagner wrote:
> > On Wed, Jul 07, 2010 at 07:49:04PM +0200, Sven Eschenberg wrote:
> >> Hummm,
> >>
> >> should I consider this geek humor or are you serious, the CPU is rather
> >> slow (imho)?
> > 
> > Par humor, part serious. But if the CPU is rather slow, then
> > this is somethin else, I guess.
> 
> Don't worry, it was optimized such way that even on slow machines
> it crashes quickly :-)
> 
> >> Well at least we know the crash happens in sigvtalarm, which I assume is
> >> a signal handler of some sort within cryptsetup.
> 
> The timer is there intentionally to calculate PBKDF2 iterations per second,
> after one second is loop stopped. There is safe limit (IIRC 1000
> iterations) - so even on slow CPU it returns safe value.
> (iteration count is unsigned 64bit number, it should not overflow,
> ale least not in this century :-)
> 
> Anyway, I need debug log and info I requested to analyse it.
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 20:58         ` Sven Eschenberg
@ 2010-07-07 21:22           ` Milan Broz
  2010-07-08  2:22             ` Sven Eschenberg
  2010-07-07 21:40           ` Milan Broz
  1 sibling, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-07 21:22 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt


On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> For starters, I'll select the easiest case, plain luksFormat without
> special params:

thanks.

> The segfault comes way before the 10 seconds are over, that much I can
> tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
> on amd64 arch.

Which gcc? (gcc --version)

I am using gentoo (but 32bit) for upstream testing,
it should fail here too then:)

But that code is there since 1.1.0, so I guess something in gcc/glibc
changed probably and uncovers some old bug.

> Currently no core file is created, can you give me a fast shot on how I
> force the system to write out a core?

try "ulimit -c unlimited" and then run that command

you can also define where core files will be stored using
sysctl kernel.core_pattern=/<some dir>/core 

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 20:58         ` Sven Eschenberg
  2010-07-07 21:22           ` Milan Broz
@ 2010-07-07 21:40           ` Milan Broz
  2010-07-07 22:00             ` Arno Wagner
  1 sibling, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-07 21:40 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt



On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> For starters, I'll select the easiest case, plain luksFormat without
> special params:

http://bugs.gentoo.org/show_bug.cgi?id=283470

(maybe someone form Gentoo can send such bugs upstream?
it is there for so long time...)

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 21:40           ` Milan Broz
@ 2010-07-07 22:00             ` Arno Wagner
  2010-07-08  2:13               ` Sven Eschenberg
  0 siblings, 1 reply; 25+ messages in thread
From: Arno Wagner @ 2010-07-07 22:00 UTC (permalink / raw)
  To: dm-crypt

Ooooh, some interference between gcc and the hardening.
And here I was expecting some nice, obscure algorithmic 
problem...

But reported in 2007? I have to agree there seems to be
an issue with propagatong bugs from Gentoo.

Arno



On Wed, Jul 07, 2010 at 11:40:51PM +0200, Milan Broz wrote:
> 
> 
> On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > For starters, I'll select the easiest case, plain luksFormat without
> > special params:
> 
> http://bugs.gentoo.org/show_bug.cgi?id=283470
> 
> (maybe someone form Gentoo can send such bugs upstream?
> it is there for so long time...)
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 22:00             ` Arno Wagner
@ 2010-07-08  2:13               ` Sven Eschenberg
  0 siblings, 0 replies; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-08  2:13 UTC (permalink / raw)
  To: dm-crypt

Bummers,

I really though I hit something somewhat new. Whoever said you cannot
teach old dogs new tricks was wrong.

I really did not imagine this one goes back this far.

Anyways, is there any hope, we can fix this?

I am happy to provide whatever it needs to fix this one ...

Regards

-Sven

(P.S. I feel kinda stuck right here...)



On Thu, 2010-07-08 at 00:00 +0200, Arno Wagner wrote:
> Ooooh, some interference between gcc and the hardening.
> And here I was expecting some nice, obscure algorithmic 
> problem...
> 
> But reported in 2007? I have to agree there seems to be
> an issue with propagatong bugs from Gentoo.
> 
> Arno
> 
> 
> 
> On Wed, Jul 07, 2010 at 11:40:51PM +0200, Milan Broz wrote:
> > 
> > 
> > On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > > For starters, I'll select the easiest case, plain luksFormat without
> > > special params:
> > 
> > http://bugs.gentoo.org/show_bug.cgi?id=283470
> > 
> > (maybe someone form Gentoo can send such bugs upstream?
> > it is there for so long time...)
> > 
> > Milan
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-07 21:22           ` Milan Broz
@ 2010-07-08  2:22             ` Sven Eschenberg
  2010-07-08  9:30               ` Arno Wagner
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-08  2:22 UTC (permalink / raw)
  To: Milan Broz; +Cc: dm-crypt

Okay, let's see:

gcc --version 
gcc (Gentoo Hardened 4.4.4-r1 p1.0, pie-0.4.5) 4.4.4
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is
NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE.

I am not really sure it is a problem in the source, maybe some awkward
scenario.

If you say, the source did not change, maybe you could recommend some
sort of testing scenario. I'd feel happy to contribute here. Currently I
feel just stuck...

Regards

-Sven

P.S.: Last time I had to set up a mapping(initially) (as in luksFormat)
was like 1 year ago (or somethign amog these figures)... so maybe I just
did not hit the bug (before) in the last couple of months ...





On Wed, 2010-07-07 at 23:22 +0200, Milan Broz wrote:
> On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > For starters, I'll select the easiest case, plain luksFormat without
> > special params:
> 
> thanks.
> 
> > The segfault comes way before the 10 seconds are over, that much I can
> > tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
> > on amd64 arch.
> 
> Which gcc? (gcc --version)
> 
> I am using gentoo (but 32bit) for upstream testing,
> it should fail here too then:)
> 
> But that code is there since 1.1.0, so I guess something in gcc/glibc
> changed probably and uncovers some old bug.
> 
> > Currently no core file is created, can you give me a fast shot on how I
> > force the system to write out a core?
> 
> try "ulimit -c unlimited" and then run that command
> 
> you can also define where core files will be stored using
> sysctl kernel.core_pattern=/<some dir>/core 
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08  2:22             ` Sven Eschenberg
@ 2010-07-08  9:30               ` Arno Wagner
  2010-07-08 13:37                 ` Sven Eschenberg
  0 siblings, 1 reply; 25+ messages in thread
From: Arno Wagner @ 2010-07-08  9:30 UTC (permalink / raw)
  To: dm-crypt

Lets give Milan a few days. I expect he will figure it out.
I admit, I do not understand what the issue really is either 
and I do not have gentoo. Mazbe I should take a look at it ...

Arno

On Thu, Jul 08, 2010 at 04:22:35AM +0200, Sven Eschenberg wrote:
> Okay, let's see:
> 
> gcc --version 
> gcc (Gentoo Hardened 4.4.4-r1 p1.0, pie-0.4.5) 4.4.4
> Copyright (C) 2010 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is
> NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
> 
> I am not really sure it is a problem in the source, maybe some awkward
> scenario.
> 
> If you say, the source did not change, maybe you could recommend some
> sort of testing scenario. I'd feel happy to contribute here. Currently I
> feel just stuck...
> 
> Regards
> 
> -Sven
> 
> P.S.: Last time I had to set up a mapping(initially) (as in luksFormat)
> was like 1 year ago (or somethign amog these figures)... so maybe I just
> did not hit the bug (before) in the last couple of months ...
> 
> 
> 
> 
> 
> On Wed, 2010-07-07 at 23:22 +0200, Milan Broz wrote:
> > On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > > For starters, I'll select the easiest case, plain luksFormat without
> > > special params:
> > 
> > thanks.
> > 
> > > The segfault comes way before the 10 seconds are over, that much I can
> > > tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
> > > on amd64 arch.
> > 
> > Which gcc? (gcc --version)
> > 
> > I am using gentoo (but 32bit) for upstream testing,
> > it should fail here too then:)
> > 
> > But that code is there since 1.1.0, so I guess something in gcc/glibc
> > changed probably and uncovers some old bug.
> > 
> > > Currently no core file is created, can you give me a fast shot on how I
> > > force the system to write out a core?
> > 
> > try "ulimit -c unlimited" and then run that command
> > 
> > you can also define where core files will be stored using
> > sysctl kernel.core_pattern=/<some dir>/core 
> > 
> > Milan
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08  9:30               ` Arno Wagner
@ 2010-07-08 13:37                 ` Sven Eschenberg
  2010-07-08 14:16                   ` Milan Broz
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-08 13:37 UTC (permalink / raw)
  To: dm-crypt

Just for the record:

The crash happens with other gcc versions as well. As the gentoo bug
report suggests, it seems to be a problem when the executeable is linked
statically on hardened profiles.
And yes, in my case compiling it dynamically resolves the segfault
aswell.

In the src the following variables are used in the handler:

static volatile uint64_t __PBKDF2_global_j = 0;
static volatile uint64_t __PBKDF2_performance = 0;

Since they are used in the sighandler, they would better not just be
volatile but sig_atomic_t, to avoid possible races.

But this should not have any influence on the segfault as far as I can
tell.

Oh, and better use sigaction() instead of signal().

I think I possibly found the problem:

In static int pkcs5_pbkdf2() in pbkdf.c:

size_t tmplen = Slen + 4;
tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...

Later:

tmp[Slen + 3] = (i & 0x000000ff) >> 0;

Ohoh, implicit cast to smaller Type, here we trash the stackframe since
tmp was allocated on the stack.

Regards

-Sven


On Thu, 2010-07-08 at 11:30 +0200, Arno Wagner wrote:
> Lets give Milan a few days. I expect he will figure it out.
> I admit, I do not understand what the issue really is either 
> and I do not have gentoo. Mazbe I should take a look at it ...
> 
> Arno
> 
> On Thu, Jul 08, 2010 at 04:22:35AM +0200, Sven Eschenberg wrote:
> > Okay, let's see:
> > 
> > gcc --version 
> > gcc (Gentoo Hardened 4.4.4-r1 p1.0, pie-0.4.5) 4.4.4
> > Copyright (C) 2010 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is
> > NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > PURPOSE.
> > 
> > I am not really sure it is a problem in the source, maybe some awkward
> > scenario.
> > 
> > If you say, the source did not change, maybe you could recommend some
> > sort of testing scenario. I'd feel happy to contribute here. Currently I
> > feel just stuck...
> > 
> > Regards
> > 
> > -Sven
> > 
> > P.S.: Last time I had to set up a mapping(initially) (as in luksFormat)
> > was like 1 year ago (or somethign amog these figures)... so maybe I just
> > did not hit the bug (before) in the last couple of months ...
> > 
> > 
> > 
> > 
> > 
> > On Wed, 2010-07-07 at 23:22 +0200, Milan Broz wrote:
> > > On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > > > For starters, I'll select the easiest case, plain luksFormat without
> > > > special params:
> > > 
> > > thanks.
> > > 
> > > > The segfault comes way before the 10 seconds are over, that much I can
> > > > tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
> > > > on amd64 arch.
> > > 
> > > Which gcc? (gcc --version)
> > > 
> > > I am using gentoo (but 32bit) for upstream testing,
> > > it should fail here too then:)
> > > 
> > > But that code is there since 1.1.0, so I guess something in gcc/glibc
> > > changed probably and uncovers some old bug.
> > > 
> > > > Currently no core file is created, can you give me a fast shot on how I
> > > > force the system to write out a core?
> > > 
> > > try "ulimit -c unlimited" and then run that command
> > > 
> > > you can also define where core files will be stored using
> > > sysctl kernel.core_pattern=/<some dir>/core 
> > > 
> > > Milan
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt@saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> > 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt@saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> 

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08 13:37                 ` Sven Eschenberg
@ 2010-07-08 14:16                   ` Milan Broz
  2010-07-08 14:29                     ` Sven Eschenberg
  0 siblings, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-08 14:16 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt

On 07/08/2010 03:37 PM, Sven Eschenberg wrote:
> Just for the record:
> 
> The crash happens with other gcc versions as well. As the gentoo bug
> report suggests, it seems to be a problem when the executeable is linked
> statically on hardened profiles.
> And yes, in my case compiling it dynamically resolves the segfault
> aswell.

I am compiling static version quite often, so hardened profile probably uses
some not common compiled switch for static version.

> In the src the following variables are used in the handler:
> 
> static volatile uint64_t __PBKDF2_global_j = 0;
> static volatile uint64_t __PBKDF2_performance = 0;
> 
> Since they are used in the sighandler, they would better not just be
> volatile but sig_atomic_t, to avoid possible races.

yes

> But this should not have any influence on the segfault as far as I can
> tell.
> 
> Oh, and better use sigaction() instead of signal().

why? should be no problem here. (that code is ugly anyway, I just polished
it some time ago when replacing pbkdf2 with gcrypt version...)


> I think I possibly found the problem:
> 
> In static int pkcs5_pbkdf2() in pbkdf.c:
> 
> size_t tmplen = Slen + 4;
> tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...

so problem is implicit type cast? interesting...

seems to be some relict from former implementation, I am always
trying to avoid alloca() in code... :)
(wonder if valgrind find that)


Thanks!
Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08 14:16                   ` Milan Broz
@ 2010-07-08 14:29                     ` Sven Eschenberg
  2010-07-08 15:11                       ` Milan Broz
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-08 14:29 UTC (permalink / raw)
  To: dm-crypt

Hi Milan,

Even worse, actually byte swappig is done to store the int in a big
endian manner, unfortunately, since it is done wrong, on big endian
systems all block indeces would be zero and they are part of the Derived
Key. I wonder if this has a security impact as in quality of derived key
on big endian systems.

On Thu, 2010-07-08 at 16:16 +0200, Milan Broz wrote:
> On 07/08/2010 03:37 PM, Sven Eschenberg wrote:
> > Just for the record:
> > 
> > The crash happens with other gcc versions as well. As the gentoo bug
> > report suggests, it seems to be a problem when the executeable is linked
> > statically on hardened profiles.
> > And yes, in my case compiling it dynamically resolves the segfault
> > aswell.
> 
> I am compiling static version quite often, so hardened profile probably uses
> some not common compiled switch for static version.

Yes, a hardened glibc behaves quite differently, esp. layout of
allocated memory and maybe(?) this has influence on an alloca as well,
who knows.

> 
> > In the src the following variables are used in the handler:
> > 
> > static volatile uint64_t __PBKDF2_global_j = 0;
> > static volatile uint64_t __PBKDF2_performance = 0;
> > 
> > Since they are used in the sighandler, they would better not just be
> > volatile but sig_atomic_t, to avoid possible races.
> 
> yes
> 
> > But this should not have any influence on the segfault as far as I can
> > tell.
> > 
> > Oh, and better use sigaction() instead of signal().
> 
> why? should be no problem here. (that code is ugly anyway, I just polished
> it some time ago when replacing pbkdf2 with gcrypt version...)
> 

well, okay, cryptsetup is linux specific, so, yes, we can agree that
portability is meaningless, but it is stated:

The only portable use of signal() is to set a signal's disposition to
SIG_DFL or SIG_IGN. The semantics when using signal() to establish a
signal handler vary across systems (and POSIX.1 explicitly permits this
variation); do not use it for this purpose.

So, it's more a question of style in our case.
> 
> > I think I possibly found the problem:
> > 
> > In static int pkcs5_pbkdf2() in pbkdf.c:
> > 
> > size_t tmplen = Slen + 4;
> > tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...
> 
> so problem is implicit type cast? interesting...
> 
> seems to be some relict from former implementation, I am always
> trying to avoid alloca() in code... :)
> (wonder if valgrind find that)
> 

Yes, funny thing is, this is a typical usage (try making it endian
safe), and I've seen the exactly same mistake (diffferent lengths of
types and thus running out of bound) over and over again. Last time it
was pretty much the same thing, code worked (luckily) and when it moved
to another OS with other allocation - segmentation fault.
> 
> Thanks!
> Milan

You are welcome, I did not modify the lines yet for testing, it just
popped up my eyes when I looked at the code there.

-Sven

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08 14:29                     ` Sven Eschenberg
@ 2010-07-08 15:11                       ` Milan Broz
  2010-07-08 18:44                         ` Sven Eschenberg
  0 siblings, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-08 15:11 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt

On 07/08/2010 04:29 PM, Sven Eschenberg wrote:
> Hi Milan,
> 
> Even worse, actually byte swappig is done to store the int in a big
> endian manner, unfortunately, since it is done wrong, on big endian
> systems all block indeces would be zero and they are part of the Derived
> Key. I wonder if this has a security impact as in quality of derived key
> on big endian systems.

You mean when int is big endian and 64bit? Do you see system where it is wrong
or just guessing?

There is no direct key encryption derived using this code, it is just keyslot
obfuscation + keyslot passphrase verification (master key in LUKS is generated from RNG)
(plain crypt do not use this at all)

That algorithm is not new and passed PBKDF2 test vectors (I will probably add this test
to api check also).

I run test on several architectures - all keyslot operation should fail on prepared image
if there is such bug.

(But it should explicitly use uint32 there.)

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08 15:11                       ` Milan Broz
@ 2010-07-08 18:44                         ` Sven Eschenberg
  2010-07-09  7:22                           ` Milan Broz
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-08 18:44 UTC (permalink / raw)
  To: Milan Broz; +Cc: dm-crypt

Hi Milan,

On Thu, 2010-07-08 at 17:11 +0200, Milan Broz wrote:
> On 07/08/2010 04:29 PM, Sven Eschenberg wrote:
> > Hi Milan,
> > 
> > Even worse, actually byte swappig is done to store the int in a big
> > endian manner, unfortunately, since it is done wrong, on big endian
> > systems all block indeces would be zero and they are part of the Derived
> > Key. I wonder if this has a security impact as in quality of derived key
> > on big endian systems.
> 
> You mean when int is big endian and 64bit? Do you see system where it is wrong
> or just guessing?
> 

Should happen on 32 Bit already, okay, let's see, AA BB CC DD, most
significant byte is AA (mask BB CC DD, shift right logically, so we will
get AA as value), which in little endian looks like AA 00 00 00, which
is copied into position Slen+0, the 4 bytes will be filled with AA 00 00
00, next step would be copying BB 00 00 00 into the next position Slen
+1, notice that here the last zero byte exceeds the buffer on the stack
already ...
After those 4 Ops we will have AA BB CC DD in the buffer and will have
written 4 zeros past the end of the buffer.
On big endian, the most significant byte AA is represented (after the
shift to the value AA) as 00 00 00 AA, we will copy this to slen+0, thus
filling it with 00 00 00 AA, next step fill 00 00 00 BB into slen+1 ...
and so on.
We will end up with all zeros for the index and will have written
various values in the 4 bytes past the buffer.

This is of course assuming an int is 4 bytes, if it is 64 bit, we will
corrupt the stack even more, as far as I can tell.

> There is no direct key encryption derived using this code, it is just keyslot
> obfuscation + keyslot passphrase verification (master key in LUKS is generated from RNG)
> (plain crypt do not use this at all)
> 
> That algorithm is not new and passed PBKDF2 test vectors (I will probably add this test
> to api check also).
> 
> I run test on several architectures - all keyslot operation should fail on prepared image
> if there is such bug.
> 
> (But it should explicitly use uint32 there.)
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

Regards

-Sven

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-08 18:44                         ` Sven Eschenberg
@ 2010-07-09  7:22                           ` Milan Broz
  2010-07-09 12:18                             ` Sven Eschenberg
  0 siblings, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-09  7:22 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt

On 07/08/2010 08:44 PM, Sven Eschenberg wrote:
> On Thu, 2010-07-08 at 17:11 +0200, Milan Broz wrote:
>> On 07/08/2010 04:29 PM, Sven Eschenberg wrote:
>>> Hi Milan,
>>>
>>> Even worse, actually byte swappig is done to store the int in a big
>>> endian manner, unfortunately, since it is done wrong, on big endian
>>> systems all block indeces would be zero and they are part of the Derived
>>> Key. I wonder if this has a security impact as in quality of derived key
>>> on big endian systems.
>>
>> You mean when int is big endian and 64bit? Do you see system where it is wrong
>> or just guessing?
>>
> 
> Should happen on 32 Bit already, okay, let's see, AA BB CC DD, most
> significant byte is AA (mask BB CC DD, shift right logically, so we will
> get AA as value), which in little endian looks like AA 00 00 00, which
> is copied into position Slen+0, the 4 bytes will be filled with AA 00 00
> 00, next step would be copying BB 00 00 00 into the next position Slen
> +1, notice that here the last zero byte exceeds the buffer on the stack
> already ...

Swen,

please look at RFC2898, PBKDF2 definition, search for

"Here, INT (i) is a four-octet encoding of the integer i, most significant octet first."

Is it better now? And compiler interprets mask correctly according to endian type.
There should be no stack overflow, it works with char array, not int array.

FYI the same code is in gnutls, see lib/x509/pbkdf2-sha1.c

Do it still segfaults after your suggested change? Did you tried it?

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-09  7:22                           ` Milan Broz
@ 2010-07-09 12:18                             ` Sven Eschenberg
  2010-07-09 12:46                               ` Milan Broz
  0 siblings, 1 reply; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-09 12:18 UTC (permalink / raw)
  To: dm-crypt

Hi Milan,

seems you are right. I really feared the implicit cast is insecure (I 
guess it would have been for an untyped tmp pointer).

Shame on me :-(.

Okay, back to the beginning, which options do we (I) have to investigate 
this further?

Regards

-Sven

Milan Broz schrieb:
> On 07/08/2010 08:44 PM, Sven Eschenberg wrote:
>> On Thu, 2010-07-08 at 17:11 +0200, Milan Broz wrote:
>>> On 07/08/2010 04:29 PM, Sven Eschenberg wrote:
>>>> Hi Milan,
>>>>
>>>> Even worse, actually byte swappig is done to store the int in a big
>>>> endian manner, unfortunately, since it is done wrong, on big endian
>>>> systems all block indeces would be zero and they are part of the Derived
>>>> Key. I wonder if this has a security impact as in quality of derived key
>>>> on big endian systems.
>>> You mean when int is big endian and 64bit? Do you see system where it is wrong
>>> or just guessing?
>>>
>> Should happen on 32 Bit already, okay, let's see, AA BB CC DD, most
>> significant byte is AA (mask BB CC DD, shift right logically, so we will
>> get AA as value), which in little endian looks like AA 00 00 00, which
>> is copied into position Slen+0, the 4 bytes will be filled with AA 00 00
>> 00, next step would be copying BB 00 00 00 into the next position Slen
>> +1, notice that here the last zero byte exceeds the buffer on the stack
>> already ...
> 
> Swen,
> 
> please look at RFC2898, PBKDF2 definition, search for
> 
> "Here, INT (i) is a four-octet encoding of the integer i, most significant octet first."
> 
> Is it better now? And compiler interprets mask correctly according to endian type.
> There should be no stack overflow, it works with char array, not int array.
> 
> FYI the same code is in gnutls, see lib/x509/pbkdf2-sha1.c
> 
> Do it still segfaults after your suggested change? Did you tried it?
> 
> Milan
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-09 12:18                             ` Sven Eschenberg
@ 2010-07-09 12:46                               ` Milan Broz
  2010-07-14 18:14                                 ` Christoph Anton Mitterer
  0 siblings, 1 reply; 25+ messages in thread
From: Milan Broz @ 2010-07-09 12:46 UTC (permalink / raw)
  To: Sven Eschenberg; +Cc: dm-crypt

On 07/09/2010 02:18 PM, Sven Eschenberg wrote:
> Okay, back to the beginning, which options do we (I) have to investigate 
> this further?

I will eventually install hardened gentoo here, I see no apparent bug in code,
so need reproducer.

Just I am busy with lvm urgent problems, so this is low priority currently
(it is in Gentoo bz since 2008 or so... not many people are affected probably)

If you have failing system and an idea or patch which helps - send it to me.
(It can be bug in signal handling in hardened patches or so... no idea,
probably best start with fixed iteration count in code and then try adding
what part of code cause segfault.)

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-09 12:46                               ` Milan Broz
@ 2010-07-14 18:14                                 ` Christoph Anton Mitterer
  2010-07-14 18:24                                   ` Milan Broz
  2010-07-14 18:56                                   ` Sven Eschenberg
  0 siblings, 2 replies; 25+ messages in thread
From: Christoph Anton Mitterer @ 2010-07-14 18:14 UTC (permalink / raw)
  To: dm-crypt; +Cc: Sven Eschenberg

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

Hey.


Just a question on this...

For those people that did not hit the crash on luksFormat.... are they
in trouble, too?

E.g. by weak PBKDF2 derived keys or that like?

Sven hinted that big endian systems might have even worse problems?


Cheers,
Chris.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3387 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-14 18:14                                 ` Christoph Anton Mitterer
@ 2010-07-14 18:24                                   ` Milan Broz
  2010-07-14 18:56                                   ` Sven Eschenberg
  1 sibling, 0 replies; 25+ messages in thread
From: Milan Broz @ 2010-07-14 18:24 UTC (permalink / raw)
  To: Christoph Anton Mitterer; +Cc: dm-crypt, Sven Eschenberg

On 07/14/2010 08:14 PM, Christoph Anton Mitterer wrote:
> For those people that did not hit the crash on luksFormat.... are they
> in trouble, too?
> 
> E.g. by weak PBKDF2 derived keys or that like?
> 
> Sven hinted that big endian systems might have even worse problems?

no. there is no bug at all with overflow or endianess, please read my replies.

(And I was not able reproduce it on Gentoo also, but I have slightly different
hardened config.)

Milan

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [dm-crypt] crypsetup segfaulting during luksFormat
  2010-07-14 18:14                                 ` Christoph Anton Mitterer
  2010-07-14 18:24                                   ` Milan Broz
@ 2010-07-14 18:56                                   ` Sven Eschenberg
  1 sibling, 0 replies; 25+ messages in thread
From: Sven Eschenberg @ 2010-07-14 18:56 UTC (permalink / raw)
  To: Christoph Anton Mitterer; +Cc: dm-crypt

That was unfortunately a misjudgment, I though there was an overflow
issue due to an implicit typecast, but everything seems to work as
expected at the specific source.

Regards

-Sven

On Wed, 2010-07-14 at 20:14 +0200, Christoph Anton Mitterer wrote:
> Hey.
> 
> 
> Just a question on this...
> 
> For those people that did not hit the crash on luksFormat.... are they
> in trouble, too?
> 
> E.g. by weak PBKDF2 derived keys or that like?
> 
> Sven hinted that big endian systems might have even worse problems?
> 
> 
> Cheers,
> Chris.
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt

^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2010-07-14 18:57 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-07-07 17:19 [dm-crypt] crypsetup segfaulting during luksFormat Sven Eschenberg
2010-07-07 17:36 ` Arno Wagner
2010-07-07 17:49   ` Sven Eschenberg
2010-07-07 20:19     ` Arno Wagner
2010-07-07 20:44       ` Milan Broz
2010-07-07 20:58         ` Sven Eschenberg
2010-07-07 21:22           ` Milan Broz
2010-07-08  2:22             ` Sven Eschenberg
2010-07-08  9:30               ` Arno Wagner
2010-07-08 13:37                 ` Sven Eschenberg
2010-07-08 14:16                   ` Milan Broz
2010-07-08 14:29                     ` Sven Eschenberg
2010-07-08 15:11                       ` Milan Broz
2010-07-08 18:44                         ` Sven Eschenberg
2010-07-09  7:22                           ` Milan Broz
2010-07-09 12:18                             ` Sven Eschenberg
2010-07-09 12:46                               ` Milan Broz
2010-07-14 18:14                                 ` Christoph Anton Mitterer
2010-07-14 18:24                                   ` Milan Broz
2010-07-14 18:56                                   ` Sven Eschenberg
2010-07-07 21:40           ` Milan Broz
2010-07-07 22:00             ` Arno Wagner
2010-07-08  2:13               ` Sven Eschenberg
2010-07-07 20:51       ` Arno Wagner
2010-07-07 18:36 ` Milan Broz

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.