[Buildroot] PowerPC64 toolchain not building

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello Cody,

Back in 2014, you added support for PowerPC64 (BE and LE) in Buildroot.
Today, I wanted to build a PowerPC64 toolchain, and it appears that:

 * The default configuration (gcc 4.9, binutils 2.24)
 * gcc 4.9 + binutils 2.25
 * gcc 5.3 + binutils 2.25

do not build, all are failing with the following error message during
the host-gcc-final build:

/tmp/ccQDkG0a.s: Assembler messages:
/tmp/ccQDkG0a.s:19: Error: operand out of domain (7 is not a multiple of 2)

Note that I have only tested BE and not LE, so I don't know if LE is
also affected or if LE works fine.

Would you have the possibility to look into this issue ?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] PowerPC64 toolchain not building

[Waldemar Brodkorb]

Hi Thomas,
Thomas Petazzoni wrote,

> Hello Cody,
> 
> Back in 2014, you added support for PowerPC64 (BE and LE) in Buildroot.
> Today, I wanted to build a PowerPC64 toolchain, and it appears that:
> 
>  * The default configuration (gcc 4.9, binutils 2.24)
>  * gcc 4.9 + binutils 2.25
>  * gcc 5.3 + binutils 2.25
> 
> do not build, all are failing with the following error message during
> the host-gcc-final build:
> 
> /tmp/ccQDkG0a.s: Assembler messages:
> /tmp/ccQDkG0a.s:19: Error: operand out of domain (7 is not a multiple of 2)
> 
> Note that I have only tested BE and not LE, so I don't know if LE is
> also affected or if LE works fine.
> 
> Would you have the possibility to look into this issue ?
> 
> Thanks!

Could you provide a config and full log? I couldn't reproduce it here.

best regards
 Waldemar

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello,

On Sun, 24 Jan 2016 20:12:37 +0100, Waldemar Brodkorb wrote:

> Could you provide a config and full log? I couldn't reproduce it here.

Sure.

Here is an example:

BR2_powerpc64=y
BR2_powerpc_power8=y
BR2_BINUTILS_VERSION_2_25_X=y
BR2_GCC_VERSION_5_X=y

(but it also failed with binutils 2.24 and gcc 4.9). Maybe the
BR2_powerpc_power8=y part is the important part, I don't know.

The log looks like this:

libtool: compile:  /home/test/outputs/ppc64/build/host-gcc-final-5.3.0/build/./gcc/xgcc -B/home/test/outputs/ppc64/build/host-gcc-final-5.3.0/build/./gcc/ -B/home/test/outputs/ppc64/host
/usr/powerpc64-buildroot-linux-gnu/bin/ -B/home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/lib/ -isystem /home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/in
clude -isystem /home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/sys-include -DHAVE_CONFIG_H -I../../../libatomic/config/powerpc -I../../../libatomic/config/posix -I../../.
./libatomic -I. -Wall -Werror -pthread -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -MT exch_16_.lo -MD -MP -MF .deps/exch_16_.lo.Ppo -DN=16 -c ../../../libatomic
/exch_n.c  -fPIC -DPIC -o .libs/exch_16_.o
[...]
/tmp/ccQDkG0a.s: Assembler messages:
/tmp/ccQDkG0a.s:19: Error: operand out of domain (7 is not a multiple of 2)
make[6]: *** [exch_16_.lo] Error 1
make[6]: *** Waiting for unfinished jobs....
[...]
make[5]: *** [all-recursive] Error 1
make[4]: *** [all] Error 2
make[3]: *** [all-target-libatomic] Error 2
make[2]: *** [all] Error 2
make[1]: *** [/home/test/outputs/ppc64/build/host-gcc-final-5.3.0/.stamp_built] Error 2
make: *** [_all] Error 2

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] PowerPC64 toolchain not building

[Sam Bobroff]

On Sun, Jan 24, 2016 at 09:36:07PM +0100, Thomas Petazzoni wrote:
> Hello,
> 
> On Sun, 24 Jan 2016 20:12:37 +0100, Waldemar Brodkorb wrote:
> 
> > Could you provide a config and full log? I couldn't reproduce it here.
> 
> Sure.
> 
> Here is an example:
> 
> BR2_powerpc64=y
> BR2_powerpc_power8=y
> BR2_BINUTILS_VERSION_2_25_X=y
> BR2_GCC_VERSION_5_X=y
> 
> (but it also failed with binutils 2.24 and gcc 4.9). Maybe the
> BR2_powerpc_power8=y part is the important part, I don't know.
> 
> The log looks like this:
> 
> libtool: compile:  /home/test/outputs/ppc64/build/host-gcc-final-5.3.0/build/./gcc/xgcc -B/home/test/outputs/ppc64/build/host-gcc-final-5.3.0/build/./gcc/ -B/home/test/outputs/ppc64/host
> /usr/powerpc64-buildroot-linux-gnu/bin/ -B/home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/lib/ -isystem /home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/in
> clude -isystem /home/test/outputs/ppc64/host/usr/powerpc64-buildroot-linux-gnu/sys-include -DHAVE_CONFIG_H -I../../../libatomic/config/powerpc -I../../../libatomic/config/posix -I../../.
> ./libatomic -I. -Wall -Werror -pthread -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -MT exch_16_.lo -MD -MP -MF .deps/exch_16_.lo.Ppo -DN=16 -c ../../../libatomic
> /exch_n.c  -fPIC -DPIC -o .libs/exch_16_.o
> [...]
> /tmp/ccQDkG0a.s: Assembler messages:
> /tmp/ccQDkG0a.s:19: Error: operand out of domain (7 is not a multiple of 2)
> make[6]: *** [exch_16_.lo] Error 1
> make[6]: *** Waiting for unfinished jobs....
> [...]
> make[5]: *** [all-recursive] Error 1
> make[4]: *** [all] Error 2
> make[3]: *** [all-target-libatomic] Error 2
> make[2]: *** [all] Error 2
> make[1]: *** [/home/test/outputs/ppc64/build/host-gcc-final-5.3.0/.stamp_built] Error 2
> make: *** [_all] Error 2
> 
> Thomas

Hi Thomas,

I have a bit of experience with buildroot on PowerPC64 so I also attempted to
replicate this, but I could not. I don't know the rest of your config but I
did match the values you posted above. An operand error like that could be
related to the specific architecture version the assembler is expecting, so
changing the "target architecture variant" is probably worth trying (try
"generic").

If your buildroot is up to date, and you've done a completely clean build
(either without ccache or after clearing ccache's cache), and this still happens
then I'd be happy to try replicating your build more closely.

If you can get at the content of the temporary file it's failing on I'd be
happy to take a look at the assembly causing the error. Maybe it will provide a
clue :-)

Cheers,
Sam.

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello,

On Wed, 27 Jan 2016 15:38:58 +1100, Sam Bobroff wrote:

> I have a bit of experience with buildroot on PowerPC64 so I also attempted to
> replicate this, but I could not. I don't know the rest of your config but I
> did match the values you posted above. An operand error like that could be
> related to the specific architecture version the assembler is expecting, so
> changing the "target architecture variant" is probably worth trying (try
> "generic").

Thanks a lot for having looked at the problem!

You need to select the power8 processor for the issue to reproduce. It
doesn't appear with power7 for example.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] PowerPC64 toolchain not building

[Sam Bobroff]

On Wed, Jan 27, 2016 at 09:32:56AM +0100, Thomas Petazzoni wrote:
> Hello,
> 
> On Wed, 27 Jan 2016 15:38:58 +1100, Sam Bobroff wrote:
> 
> > I have a bit of experience with buildroot on PowerPC64 so I also attempted to
> > replicate this, but I could not. I don't know the rest of your config but I
> > did match the values you posted above. An operand error like that could be
> > related to the specific architecture version the assembler is expecting, so
> > changing the "target architecture variant" is probably worth trying (try
> > "generic").
> 
> Thanks a lot for having looked at the problem!
> 
> You need to select the power8 processor for the issue to reproduce. It
> doesn't appear with power7 for example.

Hmm, while that's not surprising, I've used that combination of options (power8
big-endian with gcc 4.9.x and gcc 5.x) and it works fine for me. Have you tried
building from completely clean, including clearing or disabling ccache?

Maybe the host architecture matters? Is that x86_64? What gcc do you have on
the host?

Cheers,
Sam.

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello,

On Thu, 28 Jan 2016 13:44:42 +1100, Sam Bobroff wrote:

> > You need to select the power8 processor for the issue to reproduce. It
> > doesn't appear with power7 for example.
> 
> Hmm, while that's not surprising, I've used that combination of options (power8
> big-endian with gcc 4.9.x and gcc 5.x) and it works fine for me. Have you tried
> building from completely clean, including clearing or disabling ccache?

Yes, I'm doing completely clean builds, I never use ccache, and always
start my builds from scratch.

Waldemar, are you able to reproduce the build failure on Power8 ?

> Maybe the host architecture matters? Is that x86_64? What gcc do you have on
> the host?

Yes, x86_64, the host gcc is quite old (4.4) since I'm using Debian 6.0
on this build server instance.

I've restarted a build on my laptop, which has a more modern distro,
I'll let you know the results.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello,

On Thu, 28 Jan 2016 09:17:42 +0100, Thomas Petazzoni wrote:

> I've restarted a build on my laptop, which has a more modern distro,
> I'll let you know the results.

The following defconfig:

BR2_powerpc64=y
BR2_powerpc_power8=y

fails to build on my machine, with the same error:

/tmp/ccZgjbrm.s: Assembler messages:
/tmp/ccZgjbrm.s:19: Error: operand out of domain (7 is not a multiple of 2)

Here is the offending assembly file:

	.file	"exch_n.c"
	.section	".toc","aw"
	.section	".text"
	.align 2
	.globl libat_exchange_16
	.hidden	libat_exchange_16
	.section	".opd","aw"
	.align 3
libat_exchange_16:
	.quad	.L.libat_exchange_16,.TOC. at tocbase,0
	.previous
	.type	libat_exchange_16, @function
.L.libat_exchange_16:
	sync
	mr 9,3
	mr 11,5
	mr 10,4
.L2:
	lqarx 7,0,9
	mr 3,7
	mr 4,8
	stqcx. 10,0,9
	bne- 0,.L2
	isync
	blr
	.long 0
	.byte 0,0,0,0,0,0,0,0
	.size	libat_exchange_16,.-.L.libat_exchange_16
	.globl __atomic_exchange_16
	.set	__atomic_exchange_16,libat_exchange_16
	.ident	"GCC: (Buildroot 2016.02-git-00995-g390d559) 4.9.3"

So the instruction "lqarx 7,0,9" is at fault.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] PowerPC64 toolchain not building

[Waldemar Brodkorb]

Hi Sam,
Sam Bobroff wrote,

> On Wed, Jan 27, 2016 at 09:32:56AM +0100, Thomas Petazzoni wrote:
> > Hello,
> > 
> > On Wed, 27 Jan 2016 15:38:58 +1100, Sam Bobroff wrote:
> > 
> > > I have a bit of experience with buildroot on PowerPC64 so I also attempted to
> > > replicate this, but I could not. I don't know the rest of your config but I
> > > did match the values you posted above. An operand error like that could be
> > > related to the specific architecture version the assembler is expecting, so
> > > changing the "target architecture variant" is probably worth trying (try
> > > "generic").
> > 
> > Thanks a lot for having looked at the problem!
> > 
> > You need to select the power8 processor for the issue to reproduce. It
> > doesn't appear with power7 for example.
> 
> Hmm, while that's not surprising, I've used that combination of options (power8
> big-endian with gcc 4.9.x and gcc 5.x) and it works fine for me. Have you tried
> building from completely clean, including clearing or disabling ccache?
> 
> Maybe the host architecture matters? Is that x86_64? What gcc do you have on
> the host?

At gcc-final time?
I tried on Ubuntu 14.04 and Debian/wheezy with the same result:
gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc-4.8.real
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.8.4-2ubuntu1~14.04'
--with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs
--enable-languages=c,c++,java,go,d,fortran,objc,obj-c++
--prefix=/usr --program-suffix=-4.8 --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix
--with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-libstdcxx-time=yes
--enable-gnu-unique-object --disable-libmudflap --enable-plugin
--with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
--enable-gtk-cairo
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre
--enable-java-home
--with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64
--with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64
--with-arch-directory=amd64
--with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc
--enable-multiarch --disable-werror --with-arch-32=i686
--with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic
--enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04) 
$ ld -v
GNU ld (GNU Binutils for Ubuntu) 2.24

That is rather strange. Could you provide a full build log
and .config?

make clean
make V=1 &>log
cat .config >> log

best regards
 Waldemar

[Buildroot] PowerPC64 toolchain not building

[Sam Bobroff]

On Thu, Jan 28, 2016 at 10:20:44AM +0100, Thomas Petazzoni wrote:
> Hello,
> 
> On Thu, 28 Jan 2016 09:17:42 +0100, Thomas Petazzoni wrote:
> 
> > I've restarted a build on my laptop, which has a more modern distro,
> > I'll let you know the results.
> 
> The following defconfig:
> 
> BR2_powerpc64=y
> BR2_powerpc_power8=y
> 
> fails to build on my machine, with the same error:
> 
> /tmp/ccZgjbrm.s: Assembler messages:
> /tmp/ccZgjbrm.s:19: Error: operand out of domain (7 is not a multiple of 2)
> 
> Here is the offending assembly file:
> 
> 	.file	"exch_n.c"
> 	.section	".toc","aw"
> 	.section	".text"
> 	.align 2
> 	.globl libat_exchange_16
> 	.hidden	libat_exchange_16
> 	.section	".opd","aw"
> 	.align 3
> libat_exchange_16:
> 	.quad	.L.libat_exchange_16,.TOC. at tocbase,0
> 	.previous
> 	.type	libat_exchange_16, @function
> .L.libat_exchange_16:
> 	sync
> 	mr 9,3
> 	mr 11,5
> 	mr 10,4
> .L2:
> 	lqarx 7,0,9
> 	mr 3,7
> 	mr 4,8
> 	stqcx. 10,0,9
> 	bne- 0,.L2
> 	isync
> 	blr
> 	.long 0
> 	.byte 0,0,0,0,0,0,0,0
> 	.size	libat_exchange_16,.-.L.libat_exchange_16
> 	.globl __atomic_exchange_16
> 	.set	__atomic_exchange_16,libat_exchange_16
> 	.ident	"GCC: (Buildroot 2016.02-git-00995-g390d559) 4.9.3"
> 
> So the instruction "lqarx 7,0,9" is at fault.
> 
> Best regards,
> 
> Thomas

I've managed to replicate your problem!

The key to reproducing it seems to be to optimise for size: I compared your
compile line to mine, and the only difference was -Os vs. -O3 :-)

So try optimizing for speed instead: this obviously works around it for me ;-)

And yes, it's the lqarx instruction causing the problem: it's first argument must
be even but it's 7, so this looks like a bug in gcc.

I did some searching around and it looks like this has been a problem in the
past and it's been fixed, but it looks like optimizing for size causes it to
re-appear. There's a bit of related info here:

http://comments.gmane.org/gmane.comp.gcc.bugs/388637

Would you like to raise it on https://gcc.gnu.org/bugzilla/ ?
Alternativly, it's no problem for me handle it if you like (I work on PowerKVM
:-).

Cheers,
Sam.

[Buildroot] PowerPC64 toolchain not building

[Thomas Petazzoni]

Hello Sam,

On Fri, 29 Jan 2016 11:55:30 +1100, Sam Bobroff wrote:

> I've managed to replicate your problem!
> 
> The key to reproducing it seems to be to optimise for size: I compared your
> compile line to mine, and the only difference was -Os vs. -O3 :-)
> 
> So try optimizing for speed instead: this obviously works around it for me ;-)
> 
> And yes, it's the lqarx instruction causing the problem: it's first argument must
> be even but it's 7, so this looks like a bug in gcc.
> 
> I did some searching around and it looks like this has been a problem in the
> past and it's been fixed, but it looks like optimizing for size causes it to
> re-appear. There's a bit of related info here:
> 
> http://comments.gmane.org/gmane.comp.gcc.bugs/388637

Thanks for looking into the problem.

> Would you like to raise it on https://gcc.gnu.org/bugzilla/ ?
> Alternativly, it's no problem for me handle it if you like (I work on PowerKVM
> :-).

Since you actually work on PowerPC, it would be good if you could
report the problem yourself.

Thanks a lot!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com