armv6k support in OE for raspberrypi and s3c6410

armv6k support in OE for raspberrypi and s3c6410

[Peter A. Bigot]

The topic from 
http://lists.openembedded.org/pipermail/openembedded-core/2014-August/095830.html 
is causing trouble again, specifically with meta-raspberrypi and 
boost-1.56.  Here're the details:

For the purposes of defining preprocessor symbols like 
__GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 gcc 4.9.1 partitions ARM (non-M) 
architectures into three groups:

1) armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5e armv5t armv5te 
iwmmxt iwmmxt2
2) armv6 armv6j armv6t2 armv6z
3) armv6k armv6zk armv7-a armv7-r armv7ve armv8-a armv8-a+crc

These indicate increasing levels of support for atomic instructions, and 
perhaps other differences. (There are other preprocessor defines like 
__ARCH_ARM_FOO that distinguish within these groups; here I'm focusing 
only on a specific known problem using __GCC_-prefixed symbols.)

Unlike normal builds of a gcc toolchain, OE builds the runtime libraries 
separately in gcc-runtime and using the machine's tuning flags which 
include the architecture.  The flags affect how atomic operations are 
implemented in the libraries.

Concurrrent applications built to link against the libraries must be 
compiled to use compatible atomic operations, or they do not work. While 
this happens naturally for applications built under OE that use the 
machine-specific tuning flags, it does not occur for on-target builds or 
for builds using the OE toolchain outside of the OE environment, which 
instead normally default to the least capable set of features.

The patch I added to gcc-configure-common.inc attempted to fix this by 
configuring the compiler using --with-arch=foo in cases where the target 
used -march=foo.

This turns out to be unstable as the rules for inferring defaults are 
complex.  From gcc/config/arm/arm.h:

/* Support for a compile-time default CPU, et cetera.  The rules are:
    --with-arch is ignored if -march or -mcpu are specified.
    --with-cpu is ignored if -march or -mcpu are specified, and is 
overridden
     by --with-arch.
    --with-tune is ignored if -mtune or -mcpu are specified (but not 
affected
      by -march).
    --with-float is ignored if -mfloat-abi is specified.
    --with-fpu is ignored if -mfpu is specified.
    --with-abi is ignored if -mabi is specified.
    --with-tls is ignored if -mtls-dialect is specified. */

As an example, while -mtune=arm1176jzf-s by itself would imply 
-march=armv6zk for a normally-built gcc, when gcc is built with 
--with-arch=armv6 the more capable implied architecture is not selected: 
the --with-arch setting takes precedence.

Which would be fine, except that meta-raspberrypi calls the arm1176jzf-s 
(which is armv6zk) an armv6 chip so gcc was built targeting a 
less-capable architecture. You have to explicitly add -march=armv6zk (or 
-march=armv6k) to make it work.

So it appears that, in order to correctly support the Raspberry PI, OE 
needs to have meta/conf/machine/include/arm/arch-armv7a.inc refactored 
to pull out armv6k, and any overrides related to atomic instructions 
updated to include armv6k as an option.

Or possibly meta-raspberrypi could just claim to be an armv7a platform, 
but that seems wrong.

Anybody have a better idea?  Is there somebody who can step up and bell 
this cat?  I'm stretching the limits of my cross-ARM familiarity here 
and really don't feel comfortable taking on this one.

Peter

PS: Lest anybody think this is a new issue that's the fault of gcc 4.9: 
it's not.  It was there with gcc-4.8 and maybe even gcc 4.7 as well, but 
only a few people noticed and none of them told OE about it. 
https://groups.google.com/forum/#!msg/automatak-dnp3/Jisp_zGhd5I/qJy1jHLFQoUJ

Re: armv6k support in OE for raspberrypi and s3c6410

[Richard Purdie]

On Fri, 2014-08-29 at 22:45 -0500, Peter A. Bigot wrote:
> Unlike normal builds of a gcc toolchain, OE builds the runtime libraries 
> separately in gcc-runtime and using the machine's tuning flags which 
> include the architecture.  The flags affect how atomic operations are 
> implemented in the libraries.

Which are you trying to fix? The on target gcc or the SDK one?

In the SDK case, we have environment files which do explicitly specify
the compiler flags.

On target is harder however the on target gcc is compiled to a specific
PACKAGE_ARCH so we should be able to put specific tuning into that gcc.
It does sound like the changes to gcc-configure-common.inc were not the
way to resolve this though, I'd misunderstood what the patches were
doing.

Can we fix this by adjusting gcc itself (the on target version)?

Its even a very old bug:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=139

but we've not actually hit issues due to this before or at least they've
not been reported in these terms.

Cheers,

Richard

Re: armv6k support in OE for raspberrypi and s3c6410

[Peter A. Bigot]

On 08/30/2014 02:49 AM, Richard Purdie wrote:
> On Fri, 2014-08-29 at 22:45 -0500, Peter A. Bigot wrote:
>> Unlike normal builds of a gcc toolchain, OE builds the runtime libraries
>> separately in gcc-runtime and using the machine's tuning flags which
>> include the architecture.  The flags affect how atomic operations are
>> implemented in the libraries.
> Which are you trying to fix? The on target gcc or the SDK one?

On-target.

I want compiler invocations like "g++ -std=c++1y -pthread test.cc" with 
the on-target OE-supplied gcc to work at least as well as the same 
invocation using the same version of gcc built on-target.  For this to 
happen, the OE gcc default architecture must be compatible with flags 
used to build the OE gcc-runtime library, just as gcc built on-target is 
compatible with gcc libraries built on-target.

There are at least two ways to make that happen: change the compiler 
default architecture to match the target-specific flags used for 
gcc-runtime, or remove the target-specific flags when building 
gcc-runtime.  The latter approach makes things more consistent with how 
gcc is built and used outside of OE, but is not how OE has traditionally 
done things so the impact of the change might be significant.  We've 
been trying the former solution, which also makes the on-target compiler 
generate native-optimized code by default (a bonus, fixing the bug you 
mention below).

> On target is harder however the on target gcc is compiled to a specific
> PACKAGE_ARCH so we should be able to put specific tuning into that gcc.
> It does sound like the changes to gcc-configure-common.inc were not the
> way to resolve this though, I'd misunderstood what the patches were
> doing.

It may still be the right way to solve it.  It breaks with 
meta-raspberrypi because that arm1176jzf-s is not optimally 
DEFAULTTUNE="armv6" that it's currently using: -mtune=arm1176jzf-s 
-march=armv6 is not the same as -mtune=arm1176jzf-s alone.  It should be 
DEFAULTTUNE="armv6k", but OE doesn't have that as an option.  If it did 
we could build gcc --with-arch=armv6k and it'd work as well as the 
armv7a platforms.

Unfortunately, I think this would have to end up with creating new 
armv6k-vfp-poky-linux-gnueabi packages, which probably wouldn't fly.  So 
it might be necessary to remove the TUNE_CCARGS flags from gcc-runtime 
after all.

> Can we fix this by adjusting gcc itself (the on target version)?

Maybe.  It makes me uncomfortable to have the on-target compiler use 
different target configuration options than the cross-compilers: I'd 
expect that to produce even more hidden inconsistencies.

I suspect there are issues with statically linked SDK-built applications 
with libstdc++ because the SDK gcc runtime libraries aren't built with 
the same TUNE_CCARGS flags applied by gcc-runtime for the on-target 
libraries, though maybe not because in that case they're built without 
target-specific assumptions.

> Its even a very old bug:
>
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=139
>
> but we've not actually hit issues due to this before or at least they've
> not been reported in these terms.

Yes, but that bug relates to not generating target-optimized code by 
default: programs work, but aren't as fast as they could be.  Now that 
people are using concurrent programs and the compilers are better, we're 
hitting the issues related to inconsistently identifying which 
instructions are available on the target.

Peter

Re: armv6k support in OE for raspberrypi and s3c6410

[Peter A. Bigot]

On 08/30/2014 02:49 AM, Richard Purdie wrote:
> On target is harder however the on target gcc is compiled to a specific
> PACKAGE_ARCH so we should be able to put specific tuning into that gcc.
> It does sound like the changes to gcc-configure-common.inc were not the
> way to resolve this though, I'd misunderstood what the patches were
> doing.

Sorry; I misunderstood what -mtune was meant to do and made it sound 
worse than it is.  -mtune must be used in conjunction with -march and 
does not provide a default architecture as I had expected.  It's -mcpu 
that defaults -march.

--with-arch=foo in gcc's configuration switches is doing exactly what it 
should and I believe it's the best approach with minimal impact on how 
OE toolchains have historically been built and invoked.  I have no 
concerns about this resolution.

(If others are less trusting, I have patches to revert the --with-arch 
change and remove the TARGET_CC_ARCH stuff from gcc-runtime, which is an 
alternative approach that also meets my requirement.  But that has a 
much wider impact than what we're doing now and I don't think it's a 
good approach.)

On full investigation, the Boost issue is completely unrelated. Boost 
1.56 simply will not work with any platform that explicitly uses 
-march=armv6.  That needs to be fixed in Boost; reverting gcc won't help.

Alternatively meta-raspberrypi could set -march=armv6k and deal with the 
ramifications of doing that (viz making PACKAGE_ARCH reflect the 
difference from armv6).  There's nothing at all wrong about it being 
armv6 as it is now: it just wouldn't be the best choice for atomic 
operations if armv6k were a supported option.

Peter