AW:Getting stacktrace for userspace applications on crash(SIGSEGV)

AW:Getting stacktrace for userspace applications on crash(SIGSEGV)

[Johannes Thoma]

An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160909/bc77ad91/attachment.html 

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Daniel.]

Yeap it is. The protocol takes 4 threads the other are user's threads.
Still, when getting some core dump and fatal error log I was expecting
to get backtrace from the thread responsible for the segmentation
fault? Am I wrong?

Thanks for the help and best regards!!!

2016-09-09 18:42 GMT-03:00 Johannes Thoma <johannes@johannesthoma.com>:
> Hi,
>
> Is it a multi threading application? Type
>
> info threads
>
> in gdb to find out. With
>
> thread threadnum
>
> you can switch between threads and then display the other backtraces with
>
> bt
>
> Hope that helps.
>
> Best
>
> Johannes



-- 
"Do or do not. There is no try"
  Yoda Master

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Johannes Thoma]

Hi,

Am 10.09.16 um 05:18 schrieb Daniel.:
> Yeap it is. The protocol takes 4 threads the other are user's threads.
> Still, when getting some core dump and fatal error log I was expecting
> to get backtrace from the thread responsible for the segmentation
> fault? Am I wrong?
AFAIK a signal (like SIGSEGV) is always sent to the whole process not to
a specific thread. Therefore gdb cannot know which thread caused the
segfault (except if it would analyse the assembly which it doesn't). So,
unfortunately you'll have to find the misbehaving thread by yourself ..

Best,

- Johannes
> Thanks for the help and best regards!!!
;)

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Daniel.]

Ohhh, I see, Thansk Johannes, now everything make sense. If this was a
post I would mark it as SOLVED :)

Best regards,

2016-09-11 14:39 GMT-03:00 Johannes Thoma <johannes@johannesthoma.com>:
> Hi,
>
> Am 10.09.16 um 05:18 schrieb Daniel.:
>>
>> Yeap it is. The protocol takes 4 threads the other are user's threads.
>> Still, when getting some core dump and fatal error log I was expecting
>> to get backtrace from the thread responsible for the segmentation
>> fault? Am I wrong?
>
> AFAIK a signal (like SIGSEGV) is always sent to the whole process not to
> a specific thread. Therefore gdb cannot know which thread caused the
> segfault (except if it would analyse the assembly which it doesn't). So,
> unfortunately you'll have to find the misbehaving thread by yourself ..
>
> Best,
>
> - Johannes
>>
>> Thanks for the help and best regards!!!
>
> ;)
>



-- 
"Do or do not. There is no try"
  Yoda Master

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Daniel.]

Hi Harish, thank you for replying me :)

Here it is a sample of what I see:

(gdb) bt
#0  0x76e5ed64 in __GI_raise (sig=sig at entry=6)
    at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x76e627f8 in __GI_abort () at abort.c:89
#2  0x76831b7c in ?? () from /opt/jdk1.8.0_60/jre/lib/arm/client/libjvm.so
#3  0x76831b7c in ?? () from /opt/jdk1.8.0_60/jre/lib/arm/client/libjvm.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)


Best regards,

2016-09-09 17:00 GMT-03:00 Harish <harish.b310@gmail.com>:
>
>    My guess is that gdb is failing to walk the Java frames. Could you post what you see in the gdb when you ask for the backtrace? I can possibly try and help then.
>
> Regards,
> Harish
>
>> On 10-Sep-2016, at 12:46 AM, Rami Rosen <roszenrami@gmail.com> wrote:
>>
>> Hi,
>> Do u know where in the kernel it crashes and can add dump_stack()
>> there and try ? I used dump_stack() in the past many times and it
>> worked for me. IIRC, kernel hacking->Kernel debugging was selected in
>> my kernekl config, and I did not had to to anything else
>>
>> Regards,
>> Rami Rosen
>> http://ramirose.wix.com/ramirosen
>>
>>
>>> On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
>>> Hi everybody,
>>>
>>> I'm debugging a library that implement some nasty protocol. This
>>> library is used by some JNI library which exports the native library
>>> to Java world. By the way I'm running on ARMv7.
>>>
>>> Well, some times, at random times, that library crashes. The problem
>>> is that I can't get stacktrace. Not from JVM's fatal errors logs, not
>>> from core dumps, not from libSegFault... I'm using "-g -rdynamic
>>> -funwind-tables -mapcs-frame" compiler options but still I can't get a
>>> stacktrace. I've tried using glibc backtrace too. What I don't tried
>>> yet: libunwind, valgrid and making kernel do the stacktrace. The cool
>>> part is that when I ran my library outside from the JVM (with native
>>> application) I do get backtraces! [Damn Java]
>>>
>>> I'm here to ask: Is it possible to make kernel print the stacktrace
>>> for some userspace application when it crashes?
>>>
>>> I saw USER_STACKTRACE_SUPPORT but can't find any help from it's
>>> Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this
>>> http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kernel
>>> but can't find "Kernel hacking -> Verbose kernel error messages" even
>>> with "Kernel hacking -> Kernel Debuggin" enabled.
>>>
>>> Any help is appreciated! Thanks !!!
>>>
>>> --
>>> "Do or do not. There is no try"
>>>  Yoda Master
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies



-- 
"Do or do not. There is no try"
  Yoda Master

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Harish]

   My guess is that gdb is failing to walk the Java frames. Could you post what you see in the gdb when you ask for the backtrace? I can possibly try and help then.

Regards,
Harish

> On 10-Sep-2016, at 12:46 AM, Rami Rosen <roszenrami@gmail.com> wrote:
> 
> Hi,
> Do u know where in the kernel it crashes and can add dump_stack()
> there and try ? I used dump_stack() in the past many times and it
> worked for me. IIRC, kernel hacking->Kernel debugging was selected in
> my kernekl config, and I did not had to to anything else
> 
> Regards,
> Rami Rosen
> http://ramirose.wix.com/ramirosen
> 
> 
>> On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
>> Hi everybody,
>> 
>> I'm debugging a library that implement some nasty protocol. This
>> library is used by some JNI library which exports the native library
>> to Java world. By the way I'm running on ARMv7.
>> 
>> Well, some times, at random times, that library crashes. The problem
>> is that I can't get stacktrace. Not from JVM's fatal errors logs, not
>> from core dumps, not from libSegFault... I'm using "-g -rdynamic
>> -funwind-tables -mapcs-frame" compiler options but still I can't get a
>> stacktrace. I've tried using glibc backtrace too. What I don't tried
>> yet: libunwind, valgrid and making kernel do the stacktrace. The cool
>> part is that when I ran my library outside from the JVM (with native
>> application) I do get backtraces! [Damn Java]
>> 
>> I'm here to ask: Is it possible to make kernel print the stacktrace
>> for some userspace application when it crashes?
>> 
>> I saw USER_STACKTRACE_SUPPORT but can't find any help from it's
>> Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this
>> http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kernel
>> but can't find "Kernel hacking -> Verbose kernel error messages" even
>> with "Kernel hacking -> Kernel Debuggin" enabled.
>> 
>> Any help is appreciated! Thanks !!!
>> 
>> --
>> "Do or do not. There is no try"
>>  Yoda Master
>> 
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Daniel.]

Hi Rami
Thanks for your reply

I'm asking if I can use kernel to dump a stacktrace for a userspace
app instead of kernel code. I'm trying that as last resource since
gdb, libSegFault and JVM's fatal error log had failed,


Best regards,

2016-09-09 16:16 GMT-03:00 Rami Rosen <roszenrami@gmail.com>:
> Hi,
> Do u know where in the kernel it crashes and can add dump_stack()
> there and try ? I used dump_stack() in the past many times and it
> worked for me. IIRC, kernel hacking->Kernel debugging was selected in
> my kernekl config, and I did not had to to anything else
>
> Regards,
> Rami Rosen
> http://ramirose.wix.com/ramirosen
>
>
> On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
>> Hi everybody,
>>
>> I'm debugging a library that implement some nasty protocol. This
>> library is used by some JNI library which exports the native library
>> to Java world. By the way I'm running on ARMv7.
>>
>> Well, some times, at random times, that library crashes. The problem
>> is that I can't get stacktrace. Not from JVM's fatal errors logs, not
>> from core dumps, not from libSegFault... I'm using "-g -rdynamic
>> -funwind-tables -mapcs-frame" compiler options but still I can't get a
>> stacktrace. I've tried using glibc backtrace too. What I don't tried
>> yet: libunwind, valgrid and making kernel do the stacktrace. The cool
>> part is that when I ran my library outside from the JVM (with native
>> application) I do get backtraces! [Damn Java]
>>
>> I'm here to ask: Is it possible to make kernel print the stacktrace
>> for some userspace application when it crashes?
>>
>> I saw USER_STACKTRACE_SUPPORT but can't find any help from it's
>> Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this
>> http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kernel
>> but can't find "Kernel hacking -> Verbose kernel error messages" even
>> with "Kernel hacking -> Kernel Debuggin" enabled.
>>
>> Any help is appreciated! Thanks !!!
>>
>> --
>> "Do or do not. There is no try"
>>   Yoda Master
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies



-- 
"Do or do not. There is no try"
  Yoda Master

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Rami Rosen]

Hi,
Do u know where in the kernel it crashes and can add dump_stack()
there and try ? I used dump_stack() in the past many times and it
worked for me. IIRC, kernel hacking->Kernel debugging was selected in
my kernekl config, and I did not had to to anything else

Regards,
Rami Rosen
http://ramirose.wix.com/ramirosen


On 9 September 2016 at 21:21, Daniel. <danielhilst@gmail.com> wrote:
> Hi everybody,
>
> I'm debugging a library that implement some nasty protocol. This
> library is used by some JNI library which exports the native library
> to Java world. By the way I'm running on ARMv7.
>
> Well, some times, at random times, that library crashes. The problem
> is that I can't get stacktrace. Not from JVM's fatal errors logs, not
> from core dumps, not from libSegFault... I'm using "-g -rdynamic
> -funwind-tables -mapcs-frame" compiler options but still I can't get a
> stacktrace. I've tried using glibc backtrace too. What I don't tried
> yet: libunwind, valgrid and making kernel do the stacktrace. The cool
> part is that when I ran my library outside from the JVM (with native
> application) I do get backtraces! [Damn Java]
>
> I'm here to ask: Is it possible to make kernel print the stacktrace
> for some userspace application when it crashes?
>
> I saw USER_STACKTRACE_SUPPORT but can't find any help from it's
> Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this
> http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kernel
> but can't find "Kernel hacking -> Verbose kernel error messages" even
> with "Kernel hacking -> Kernel Debuggin" enabled.
>
> Any help is appreciated! Thanks !!!
>
> --
> "Do or do not. There is no try"
>   Yoda Master
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Getting stacktrace for userspace applications on crash(SIGSEGV)

[Daniel.]

Hi everybody,

I'm debugging a library that implement some nasty protocol. This
library is used by some JNI library which exports the native library
to Java world. By the way I'm running on ARMv7.

Well, some times, at random times, that library crashes. The problem
is that I can't get stacktrace. Not from JVM's fatal errors logs, not
from core dumps, not from libSegFault... I'm using "-g -rdynamic
-funwind-tables -mapcs-frame" compiler options but still I can't get a
stacktrace. I've tried using glibc backtrace too. What I don't tried
yet: libunwind, valgrid and making kernel do the stacktrace. The cool
part is that when I ran my library outside from the JVM (with native
application) I do get backtraces! [Damn Java]

I'm here to ask: Is it possible to make kernel print the stacktrace
for some userspace application when it crashes?

I saw USER_STACKTRACE_SUPPORT but can't find any help from it's
Kconfig. I saw this https://kernelnewbies.org/FAQ/StackTrace and this
http://processors.wiki.ti.com/index.php/Enabling_Stack_Dumping_in_Linux_Kernel
but can't find "Kernel hacking -> Verbose kernel error messages" even
with "Kernel hacking -> Kernel Debuggin" enabled.

Any help is appreciated! Thanks !!!

-- 
"Do or do not. There is no try"
  Yoda Master