All of lore.kernel.org
 help / color / mirror / Atom feed
* [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native
@ 2015-11-06 11:44 Kaaria, Erkka
  2015-11-06 14:40 ` Otavio Salvador
  0 siblings, 1 reply; 4+ messages in thread
From: Kaaria, Erkka @ 2015-11-06 11:44 UTC (permalink / raw)
  To: openembedded-devel

Hi everyone,

There seems to be something wrong with cacao-initial-native JIT compiler that causes sporadic segfaults when compiling jamvm-native or classpath-native. This is fairly rare, I estimate around 1% of the builds fail due to segfault. I use 64 bit Ubuntu 15.04 with GCC 4.9.2 as build machine.

Example do_compile log:

make[6]: Entering directory '<path-snip>/build/tmp/work/x86_64-linux/jamvm-native/1.5.5+1.6.0-devel+gitAUTOINC+ebd11bde0a-r0/build/src/classlib/gnuclasspath/lib'
mkdir classes
<snip, ecj-initial compiling lots of java files>
LOG: [0x00007f32c46a7700] We received a SIGSEGV and tried to handle it, but we were
LOG: [0x00007f32c46a7700] unable to find a Java method at:
LOG: [0x00007f32c46a7700]
LOG: [0x00007f32c46a7700] PC=0x0000000000440c09
LOG: [0x00007f32c46a7700]
LOG: [0x00007f32c46a7700] Dumping the current stacktrace:
    <<No stacktrace available>>
LOG: [0x00007f32c46a7700] Exiting...
Aborted (core dumped)
Makefile:662: recipe for target 'classes.zip' failed


Examination of the core dump in this particular case indicates that the segfault is caused by null pointer dereference in builtin_new at builtin.c:764:

<snip, signal handlers, java stacktrace code etc>
#5 <signal handler called>
#6 builtin_new (c=0x0) at builtin.c:764
#7 0x00007f23badb5a5f in ?? ()
#8 0x0000000000000000 in ?? ()

Gdb is unable to show the calling method, and judging by the fact that the old instruction pointer does not match any shared library addresses, I think it originates from JIT compiled code.

This http://c1.complang.tuwien.ac.at/pipermail/cacao/2011-March/001350.html thread seems to indicate that there could be a race condition in cacao JIT compiler when multiple Java threads are involved. I forced the ecj to only use one thread and segfaulting seems to have stopped.

Given the non-deterministic nature of the bug, I am not sure if I actually managed to fix this or if I have just been lucky. As such, I request help to

a) Verify the presence of the bug
b) Verify that forcing ecj to only use single thread fixes the issue.

Multihreading can be disabled by adding -Djdt.compiler.useSingleThread=true to the third line in "ecj-initial" script in build/tmp/sysroots/<build machine folder>/usr/bin/

example line:
${RUNTIME} -Xmx512m -Djdt.compiler.useSingleThread=true -cp ${ECJ_JAR} org.eclipse.jdt.internal.compiler.batch.Main ${1+"$@"}


Script I use for the recompilation. I usually get a failure within 90 minutes, but this naturally varies between computers. Run from the build folder

#!/bin/bash
while [ true ]; do
  echo $(date)
  echo "Cleaning..."
  bitbake -c cleansstate jamvm-native > /dev/null
  echo "Compiling..."
  bitbake -c compile jamvm-native > /dev/null   || { echo "Stopping"; exit 1; }
done

Best regards,
Erkka Kääriä
---------------------------------------------------------------------
Intel Finland Oy
Registered Address: PL 281, 00181 Helsinki 
Business Identity Code: 0357606 - 4 
Domiciled in Helsinki 

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native
  2015-11-06 11:44 [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native Kaaria, Erkka
@ 2015-11-06 14:40 ` Otavio Salvador
  2015-11-09  8:10   ` Kaaria, Erkka
  0 siblings, 1 reply; 4+ messages in thread
From: Otavio Salvador @ 2015-11-06 14:40 UTC (permalink / raw)
  To: OpenEmbedded Devel List

On Fri, Nov 6, 2015 at 9:44 AM, Kaaria, Erkka <erkka.kaaria@intel.com> wrote:
> There seems to be something wrong with cacao-initial-native JIT compiler that causes sporadic segfaults when compiling jamvm-native or classpath-native. This is fairly rare, I estimate around 1% of the builds fail due to segfault. I use 64 bit Ubuntu 15.04 with GCC 4.9.2 as build machine.

Did you check if cacao-staging tree does not include a fix for it?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native
  2015-11-06 14:40 ` Otavio Salvador
@ 2015-11-09  8:10   ` Kaaria, Erkka
  2015-11-09 10:18     ` Otavio Salvador
  0 siblings, 1 reply; 4+ messages in thread
From: Kaaria, Erkka @ 2015-11-09  8:10 UTC (permalink / raw)
  To: openembedded-devel

> From: openembedded-devel-bounces@lists.openembedded.org
> [mailto:openembedded-devel-bounces@lists.openembedded.org] On
> Behalf Of Otavio Salvador
> Sent: Friday, November 6, 2015 4:40 PM
> To: OpenEmbedded Devel List <openembedded-
> devel@lists.openembedded.org>
> Subject: Re: [oe] [meta-java] Sporadic segfaults when compiling jamvm-
> native and classpath-native
> 
> On Fri, Nov 6, 2015 at 9:44 AM, Kaaria, Erkka <erkka.kaaria@intel.com>
> wrote:
> > There seems to be something wrong with cacao-initial-native JIT compiler
> that causes sporadic segfaults when compiling jamvm-native or classpath-
> native. This is fairly rare, I estimate around 1% of the builds fail due to
> segfault. I use 64 bit Ubuntu 15.04 with GCC 4.9.2 as build machine.
> 
> Did you check if cacao-staging tree does not include a fix for it?
> 

Hi,

I briefly glanced at the cacao-staging commit history, but I didn't find any patches that would fix this segfault specifically. There are several patches that fix issues and race conditions in the jit compiler, for example https://bitbucket.org/cacaovm/cacao-staging/commits/7e6eef2b4c94 and https://bitbucket.org/cacaovm/cacao-staging/commits/56697326cf27. One of these patches may very well fix the segfault, but the problem is that the cacao version that is used in the bootstrapping process is positively ancient (0.98, released in 2007). The linked patches for example are patching files that are either substantially different from the ones in 0.98 or don't even exist in 0.98. Backporting any patches to would not be easy. 

While the workaround would leave the underlying cause unfixed, it only requires changing one line. Since the cacao-initial-native is only used briefly during the build process, I think small workaround is better than large scale changes to the old codebase. I also didn't notice any difference in classpath-native or jamvm-native build times, so the performance impact is negligible.  

On a related note, I ran the clean\recompile script over the weekend and saw no segfaults, so I think the workaround works. 

Best Regards,
Erkka Kääriä
---------------------------------------------------------------------
Intel Finland Oy
Registered Address: PL 281, 00181 Helsinki 
Business Identity Code: 0357606 - 4 
Domiciled in Helsinki 

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native
  2015-11-09  8:10   ` Kaaria, Erkka
@ 2015-11-09 10:18     ` Otavio Salvador
  0 siblings, 0 replies; 4+ messages in thread
From: Otavio Salvador @ 2015-11-09 10:18 UTC (permalink / raw)
  To: OpenEmbedded Devel List

On Mon, Nov 9, 2015 at 6:10 AM, Kaaria, Erkka <erkka.kaaria@intel.com> wrote:
> While the workaround would leave the underlying cause unfixed, it only requires changing one line. Since the cacao-initial-native is only used briefly during the build process, I think small workaround is better than large scale changes to the old codebase. I also didn't notice any difference in classpath-native or jamvm-native build times, so the performance impact is negligible.
>
> On a related note, I ran the clean\recompile script over the weekend and saw no segfaults, so I think the workaround works.

So please cook a patch adding the workaround and add a proper comment
in the recipe with FIXME so we are aware of it.

If possible, consider the possibility of packaging the staging
repository so we could use a newer release, after that,  and likely
drop patches :)

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2015-11-09 10:18 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-06 11:44 [meta-java] Sporadic segfaults when compiling jamvm-native and classpath-native Kaaria, Erkka
2015-11-06 14:40 ` Otavio Salvador
2015-11-09  8:10   ` Kaaria, Erkka
2015-11-09 10:18     ` Otavio Salvador

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.