[Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

[Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

[Arguin, Christopher P UTAS]

Hello all,

I think I've uncovered a bug where targeting special instruction sets within the x86_64 family is not always getting treated as a cross-compile. 

I'm targeting a 4th Gen i7 board. We had everything working, but had the generic "nocona"  architecture instead of the more specific 'core-avx2'. Our application would benefit from the AVX2 instruction set, so I switched it. After switching everything built fine on my laptop, which also happens to be a 4th Gen i7.

I then migrated my build environment onto our build server, which is Xeon processor based. Suddenly the build starting failing with "illegal instruction" errors during the compile process. I switched the target architecture back to "nocona" and everything works in both environments.

The only change between working and not-working is this entry in the config file:
   BR2_x86_core_avx2=y

Currently there are two failure points, the second of which is fatal. A full working build takes about 61 minutes and the failing build took 52 minutes, so it got through most of the stuff before giving up.

The first sign of problems occurs when building openssl, and it may be that it is incorrectly trying to use just-built code to generate or verify certificates:
   Doing certs/demo
   Illegal instruction (core dumped)
   Illegal instruction (core dumped)
   WARNING: Skipping duplicate certificate pca-cert.pem
   Illegal instruction (core dumped)
   WARNING: Skipping duplicate certificate ca-cert.pem
   Illegal instruction (core dumped)


The second failure is on ncftp, and it fails immediately and catastrophically, as if the compiler is not working at all:
   Makefile:62: recipe for target 'DStrFree.o' failed
   make[4]: *** [DStrFree.o] Illegal instruction (core dumped)
   make[4]: *** Waiting for unfinished jobs....
   Makefile:62: recipe for target 'strtokc.o' failed
   make[4]: *** [strtokc.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'Strncpy.o' failed
   make[4]: *** [Strncpy.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'Strnpcpy.o' failed
   make[4]: *** [Strnpcpy.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'StrFree.o' failed
   make[4]: *** [StrFree.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'DStrCpyList.o' failed
   make[4]: *** [DStrCpyList.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'Strncat.o' failed
   make[4]: *** [Strncat.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'DStrCpy.o' failed
   make[4]: *** [DStrCpy.o] Illegal instruction (core dumped)
   Makefile:62: recipe for target 'DStrInit.o' failed
   make[4]: *** [DStrInit.o] Illegal instruction (core dumped)
   Makefile:32: recipe for target 'libs' failed
   make[3]: *** [libs] Error 2

This despite the configure output looking pretty reasonable:
   checking for x86_64-buildroot-linux-gnu-gcc... /home/jenkins/workspace/LinuxOS/output/svme1908/host/usr/bin/x86_64-buildroot-linux-gnu-gcc
   checking whether the C compiler works... yes
   checking for C compiler default output file name... a.out
   checking for suffix of executables... 
   checking whether we are cross compiling... yes

I'll start delving into this myself, but I wanted to check this mailing list to see if this is some "known-and-too-hard-to-solve" problem. I couldn't find it mentioned anywhere.

Thanks,
Chris Arguin

[Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

[Thomas Petazzoni]

Hello,

First of all, thanks for your report!

On Tue, 19 Jul 2016 18:13:31 +0000, Arguin, Christopher P     UTAS
wrote:

> I think I've uncovered a bug where targeting special instruction sets within the x86_64 family is not always getting treated as a cross-compile. 
> 
> I'm targeting a 4th Gen i7 board. We had everything working, but had the generic "nocona"  architecture instead of the more specific 'core-avx2'. Our application would benefit from the AVX2 instruction set, so I switched it. After switching everything built fine on my laptop, which also happens to be a 4th Gen i7.
> 
> I then migrated my build environment onto our build server, which is Xeon processor based. Suddenly the build starting failing with "illegal instruction" errors during the compile process. I switched the target architecture back to "nocona" and everything works in both environments.
> 
> The only change between working and not-working is this entry in the config file:
>    BR2_x86_core_avx2=y
> 
> Currently there are two failure points, the second of which is fatal. A full working build takes about 61 minutes and the failing build took 52 minutes, so it got through most of the stuff before giving up.
> 
> The first sign of problems occurs when building openssl, and it may be that it is incorrectly trying to use just-built code to generate or verify certificates:
>    Doing certs/demo
>    Illegal instruction (core dumped)
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate pca-cert.pem
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate ca-cert.pem
>    Illegal instruction (core dumped)
> 
> 
> The second failure is on ncftp, and it fails immediately and catastrophically, as if the compiler is not working at all:
>    Makefile:62: recipe for target 'DStrFree.o' failed
>    make[4]: *** [DStrFree.o] Illegal instruction (core dumped)
>    make[4]: *** Waiting for unfinished jobs....
>    Makefile:62: recipe for target 'strtokc.o' failed
>    make[4]: *** [strtokc.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncpy.o' failed
>    make[4]: *** [Strncpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strnpcpy.o' failed
>    make[4]: *** [Strnpcpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'StrFree.o' failed
>    make[4]: *** [StrFree.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpyList.o' failed
>    make[4]: *** [DStrCpyList.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncat.o' failed
>    make[4]: *** [Strncat.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpy.o' failed
>    make[4]: *** [DStrCpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrInit.o' failed
>    make[4]: *** [DStrInit.o] Illegal instruction (core dumped)
>    Makefile:32: recipe for target 'libs' failed
>    make[3]: *** [libs] Error 2

I've just built the following defconfig:

BR2_x86_64=y
BR2_x86_core_avx2=y
BR2_PACKAGE_NCFTP=y

On a Xeon machine:

$ cat /proc/cpuinfo  | grep "model name"
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz

And it built just fine.

Could you provide us a minimal configuration, which exhibits the ncftp
problem in your case?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Arguin, Christopher P UTAS]

Thomas,

Thanks for the quick reply.

I'm working on a minimal set now. I've stripped out practically everything and still see the same issue, but I'll also upgrade buildroot from 2016.02 to 2016.05 and move away from the BR2_EXTERNAL setup I have to make it as simple as possible. 

I looked up your Xeon processor and it still supports the AVX extensions, although not AVX2. Mine is an "Intel(R) Xeon(R) CPU           E5649  @ 2.53GHz", which doesn't support AVX at all.

That said, I can't imagine why compiling ncftp of all things would be a problem.

Thanks,
Chris



-----Original Message-----
From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com] 
Sent: Tuesday, July 19, 2016 6:02 PM
To: Arguin, Christopher P UTAS
Cc: buildroot at busybox.net
Subject: [External] Re: [Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

Hello,

First of all, thanks for your report!

On Tue, 19 Jul 2016 18:13:31 +0000, Arguin, Christopher P     UTAS
wrote:

> I think I've uncovered a bug where targeting special instruction sets within the x86_64 family is not always getting treated as a cross-compile. 
> 
> I'm targeting a 4th Gen i7 board. We had everything working, but had the generic "nocona"  architecture instead of the more specific 'core-avx2'. Our application would benefit from the AVX2 instruction set, so I switched it. After switching everything built fine on my laptop, which also happens to be a 4th Gen i7.
> 
> I then migrated my build environment onto our build server, which is Xeon processor based. Suddenly the build starting failing with "illegal instruction" errors during the compile process. I switched the target architecture back to "nocona" and everything works in both environments.
> 
> The only change between working and not-working is this entry in the config file:
>    BR2_x86_core_avx2=y
> 
> Currently there are two failure points, the second of which is fatal. A full working build takes about 61 minutes and the failing build took 52 minutes, so it got through most of the stuff before giving up.
> 
> The first sign of problems occurs when building openssl, and it may be that it is incorrectly trying to use just-built code to generate or verify certificates:
>    Doing certs/demo
>    Illegal instruction (core dumped)
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate pca-cert.pem
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate ca-cert.pem
>    Illegal instruction (core dumped)
> 
> 
> The second failure is on ncftp, and it fails immediately and catastrophically, as if the compiler is not working at all:
>    Makefile:62: recipe for target 'DStrFree.o' failed
>    make[4]: *** [DStrFree.o] Illegal instruction (core dumped)
>    make[4]: *** Waiting for unfinished jobs....
>    Makefile:62: recipe for target 'strtokc.o' failed
>    make[4]: *** [strtokc.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncpy.o' failed
>    make[4]: *** [Strncpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strnpcpy.o' failed
>    make[4]: *** [Strnpcpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'StrFree.o' failed
>    make[4]: *** [StrFree.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpyList.o' failed
>    make[4]: *** [DStrCpyList.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncat.o' failed
>    make[4]: *** [Strncat.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpy.o' failed
>    make[4]: *** [DStrCpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrInit.o' failed
>    make[4]: *** [DStrInit.o] Illegal instruction (core dumped)
>    Makefile:32: recipe for target 'libs' failed
>    make[3]: *** [libs] Error 2

I've just built the following defconfig:

BR2_x86_64=y
BR2_x86_core_avx2=y
BR2_PACKAGE_NCFTP=y

On a Xeon machine:

$ cat /proc/cpuinfo  | grep "model name"
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz

And it built just fine.

Could you provide us a minimal configuration, which exhibits the ncftp problem in your case?

Thanks,

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Arguin, Christopher P UTAS]

I dug into it more and found the source of the problem, which is in ncftp itself. As part of the 'configure' script it actually creates a 'ccdv.c' file and compiles it, apparently using the wrong compiler. Then this 'ccdv' tool is used throughout the build process instead of CC. It was to make the compilation output prettier.

It's not directly AVX-related, but from the BMI instruction set... the actual failure is the "andn" instruction. I still need to dig into it more to understand what's going on there.

There is a configure option to disable use of ccdv, so I may give that shot. Then I have to go look at what is going on in openssl. I guess you can probably expect a patch or two from me this week :)

Thanks,
Chris

-- 
Christopher Arguin - Software Architect - ISR Systems
UTC AEROSPACE SYSTEMS
7 Technology Park Drive, Westford, MA 01886
Tel: +1 978 490 2278  Mobile: +1 978 495 2542  Fax: +1 978 490 2806
christopher.arguin at utas.utc.com  www.utcaerospacesystems.com

CONFIDENTIALITY WARNING: This message may contain proprietary and/or privileged information of UTC Aerospace Systems and its affiliated companies. If you are not the intended recipient please 1) do not disclose, copy, distribute or use this message or its contents, 2) advise the sender by return e-mail, and 3) delete all copies (including all attachments) from your computer. Your cooperation is greatly appreciated.


-----Original Message-----
From: Arguin, Christopher P UTAS 
Sent: Tuesday, July 19, 2016 10:08 PM
To: 'Thomas Petazzoni'
Cc: buildroot at busybox.net
Subject: RE: [External] Re: [Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

Thomas,

Thanks for the quick reply.

I'm working on a minimal set now. I've stripped out practically everything and still see the same issue, but I'll also upgrade buildroot from 2016.02 to 2016.05 and move away from the BR2_EXTERNAL setup I have to make it as simple as possible. 

I looked up your Xeon processor and it still supports the AVX extensions, although not AVX2. Mine is an "Intel(R) Xeon(R) CPU           E5649  @ 2.53GHz", which doesn't support AVX at all.

That said, I can't imagine why compiling ncftp of all things would be a problem.

Thanks,
Chris



-----Original Message-----
From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com] 
Sent: Tuesday, July 19, 2016 6:02 PM
To: Arguin, Christopher P UTAS
Cc: buildroot at busybox.net
Subject: [External] Re: [Buildroot] Issues with cross-compiling from x86_64 to a different x86_64

Hello,

First of all, thanks for your report!

On Tue, 19 Jul 2016 18:13:31 +0000, Arguin, Christopher P     UTAS
wrote:

> I think I've uncovered a bug where targeting special instruction sets within the x86_64 family is not always getting treated as a cross-compile. 
> 
> I'm targeting a 4th Gen i7 board. We had everything working, but had the generic "nocona"  architecture instead of the more specific 'core-avx2'. Our application would benefit from the AVX2 instruction set, so I switched it. After switching everything built fine on my laptop, which also happens to be a 4th Gen i7.
> 
> I then migrated my build environment onto our build server, which is Xeon processor based. Suddenly the build starting failing with "illegal instruction" errors during the compile process. I switched the target architecture back to "nocona" and everything works in both environments.
> 
> The only change between working and not-working is this entry in the config file:
>    BR2_x86_core_avx2=y
> 
> Currently there are two failure points, the second of which is fatal. A full working build takes about 61 minutes and the failing build took 52 minutes, so it got through most of the stuff before giving up.
> 
> The first sign of problems occurs when building openssl, and it may be that it is incorrectly trying to use just-built code to generate or verify certificates:
>    Doing certs/demo
>    Illegal instruction (core dumped)
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate pca-cert.pem
>    Illegal instruction (core dumped)
>    WARNING: Skipping duplicate certificate ca-cert.pem
>    Illegal instruction (core dumped)
> 
> 
> The second failure is on ncftp, and it fails immediately and catastrophically, as if the compiler is not working at all:
>    Makefile:62: recipe for target 'DStrFree.o' failed
>    make[4]: *** [DStrFree.o] Illegal instruction (core dumped)
>    make[4]: *** Waiting for unfinished jobs....
>    Makefile:62: recipe for target 'strtokc.o' failed
>    make[4]: *** [strtokc.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncpy.o' failed
>    make[4]: *** [Strncpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strnpcpy.o' failed
>    make[4]: *** [Strnpcpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'StrFree.o' failed
>    make[4]: *** [StrFree.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpyList.o' failed
>    make[4]: *** [DStrCpyList.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'Strncat.o' failed
>    make[4]: *** [Strncat.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrCpy.o' failed
>    make[4]: *** [DStrCpy.o] Illegal instruction (core dumped)
>    Makefile:62: recipe for target 'DStrInit.o' failed
>    make[4]: *** [DStrInit.o] Illegal instruction (core dumped)
>    Makefile:32: recipe for target 'libs' failed
>    make[3]: *** [libs] Error 2

I've just built the following defconfig:

BR2_x86_64=y
BR2_x86_core_avx2=y
BR2_PACKAGE_NCFTP=y

On a Xeon machine:

$ cat /proc/cpuinfo  | grep "model name"
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
model name      : Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz

And it built just fine.

Could you provide us a minimal configuration, which exhibits the ncftp problem in your case?

Thanks,

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Thomas Petazzoni]

Hello,

On Wed, 20 Jul 2016 02:49:45 +0000, Arguin, Christopher P     UTAS
wrote:

> I dug into it more and found the source of the problem, which is in
> ncftp itself. As part of the 'configure' script it actually creates a
> 'ccdv.c' file and compiles it, apparently using the wrong compiler.
> Then this 'ccdv' tool is used throughout the build process instead of
> CC. It was to make the compilation output prettier.

Thanks for the investigation!

> It's not directly AVX-related, but from the BMI instruction set...
> the actual failure is the "andn" instruction. I still need to dig
> into it more to understand what's going on there.
> 
> There is a configure option to disable use of ccdv, so I may give
> that shot. Then I have to go look at what is going on in openssl. I
> guess you can probably expect a patch or two from me this week :)

Looks like indeed passing --disable-ccdv is the best way forward.

However, I don't understand why on my system, ccdv doesn't get built,
while it does get built for you.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Arguin, Christopher P UTAS]

Thomas,

> However, I don't understand why on my system, ccdv doesn't get built, while it does get built for you.

I just stumbled into the answer to that. I pulled buildroot from git so I could start making changes, and did a 'mostly-default' build first. Much to my surprise, the ccdv executable didn't get built.

Switching from the default uclibc target to glibc target changed that.

Basically, the configure script in ncftp tries it's hardest to build ccdv, but if it doesn't generate a working executable it gives up on it. In my case the executable worked well enough if you didn't pass any arguments, so it passed the configure test... but then died when you try to actually use it. With uclibc as the target, ccdv may not have been successfully built because the C library was mismatched.

Now that I know what to look for, I see that you predicted this a few years ago. From http://lists.busybox.net/pipermail/buildroot/2014-February/089526.html:
> That being said, I agree that the cross-compilation test is stupid: it
> builds a program, and tries to run it. If it runs, then the configure
> script concludes that we're doing native compilation, if it doesn't
> run, we're doing cross-compilation. Except that of course when you're
> building x86 or x86-64 on x86-64 and both the target and host use
> glibc, the program may very well run.

That addressed the detection of cross-compilation in ncftp, but not the use of the host-tool.  I managed to have just the right setup to hit a pretty obscure corner case. 

Thanks,
Chris

-----Original Message-----
From: Thomas Petazzoni [mailto:thomas.petazzoni at free-electrons.com] 
Sent: Wednesday, July 20, 2016 4:03 PM
To: Arguin, Christopher P UTAS
Cc: buildroot at busybox.net
Subject: Re: [Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

Hello,

On Wed, 20 Jul 2016 02:49:45 +0000, Arguin, Christopher P     UTAS
wrote:

> I dug into it more and found the source of the problem, which is in 
> ncftp itself. As part of the 'configure' script it actually creates a 
> 'ccdv.c' file and compiles it, apparently using the wrong compiler.
> Then this 'ccdv' tool is used throughout the build process instead of 
> CC. It was to make the compilation output prettier.

Thanks for the investigation!

> It's not directly AVX-related, but from the BMI instruction set...
> the actual failure is the "andn" instruction. I still need to dig into 
> it more to understand what's going on there.
> 
> There is a configure option to disable use of ccdv, so I may give that 
> shot. Then I have to go look at what is going on in openssl. I guess 
> you can probably expect a patch or two from me this week :)

Looks like indeed passing --disable-ccdv is the best way forward.

However, I don't understand why on my system, ccdv doesn't get built, while it does get built for you.

Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Thomas Petazzoni]

Hello,

On Wed, 20 Jul 2016 23:12:13 +0000, Arguin, Christopher P     UTAS
wrote:

> I just stumbled into the answer to that. I pulled buildroot from git
> so I could start making changes, and did a 'mostly-default' build
> first. Much to my surprise, the ccdv executable didn't get built.
> 
> Switching from the default uclibc target to glibc target changed that.
> 
> Basically, the configure script in ncftp tries it's hardest to build
> ccdv, but if it doesn't generate a working executable it gives up on
> it. In my case the executable worked well enough if you didn't pass
> any arguments, so it passed the configure test... but then died when
> you try to actually use it. With uclibc as the target, ccdv may not
> have been successfully built because the C library was mismatched.

Aah, makes sense.

> Now that I know what to look for, I see that you predicted this a few
> years ago. From
> http://lists.busybox.net/pipermail/buildroot/2014-February/089526.html:
> > That being said, I agree that the cross-compilation test is stupid:
> > it builds a program, and tries to run it. If it runs, then the
> > configure script concludes that we're doing native compilation, if
> > it doesn't run, we're doing cross-compilation. Except that of
> > course when you're building x86 or x86-64 on x86-64 and both the
> > target and host use glibc, the program may very well run.  

Wow, I did some useful predictions two years ago that turn out to be
true today? I wish I had instead predicted the winning numbers at the
lottery. I would then have been able to work full time on Buildroot...
or just enjoy Pi?a Coladas on the beach.

> That addressed the detection of cross-compilation in ncftp, but not
> the use of the host-tool.  I managed to have just the right setup to
> hit a pretty obscure corner case. 

Indeed. Will you submit a patch for --disable-ccdv ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Romain NAOUR]

Hi Christopher, Thomas, All,

Le 2016-07-21 09:22, Thomas Petazzoni a ?crit :

> Hello,
> 
> On Wed, 20 Jul 2016 23:12:13 +0000, Arguin, Christopher P UTAS
> wrote:
> 
>> I just stumbled into the answer to that. I pulled buildroot from git 
>> so
>> I could start making changes, and did a 'mostly-default' build first.
>> Much to my surprise, the ccdv executable didn't get built. Switching
>> from the default uclibc target to glibc target changed that. 
>> Basically,
>> the configure script in ncftp tries it's hardest to build ccdv, but 
>> if
>> it doesn't generate a working executable it gives up on it. In my 
>> case
>> the executable worked well enough if you didn't pass any arguments, 
>> so
>> it passed the configure test... but then died when you try to 
>> actually
>> use it. With uclibc as the target, ccdv may not have been 
>> successfully
>> built because the C library was mismatched.
> 
> Aah, makes sense.

Good catch!

> 
>> Now that I know what to look for, I see that you predicted this a few
>> years ago. From
>> http://lists.busybox.net/pipermail/buildroot/2014-February/089526.html:
>> [1]
>> 
>>> That being said, I agree that the cross-compilation test is stupid:
>>> it builds a program, and tries to run it. If it runs, then the
>>> configure script concludes that we're doing native compilation, if 
>>> it
>>> doesn't run, we're doing cross-compilation. Except that of course
>>> when you're building x86 or x86-64 on x86-64 and both the target and
>>> host use glibc, the program may very well run.
> 
> Wow, I did some useful predictions two years ago that turn out to be
> true today? I wish I had instead predicted the winning numbers at the
> lottery. I would then have been able to work full time on Buildroot...
> or just enjoy Pi&n> das on the beach. That addressed the detection of
> cross-compilation in

Actually I didn't noticed ccdv tool and that it was used and may cause 
some
issue for cross-compiling. As you can see the configure script bundled 
with
the archive have a similar issue, that is why we have to regenerate it 
:-/

>> ncftp, but not the use of the host-tool. I managed to have just the
>> right setup
> ty obscure corner case.
> 
> Indeed. Will you submit a patch for --disable-ccdv ?

I reproduced the issue here and indeed --disable-ccdv fix the build.

Best regards,
Romain

> 
> Thanks,
> 
> Thomas

[Buildroot] [External] Re: Issues with cross-compiling from x86_64 to a different x86_64

[Arguin, Christopher P UTAS]

Thomas and Romain,

>>> ncftp, but not the use of the host-tool. I managed to have just the 
>>> right setup
>> ty obscure corner case.
>> 
>> Indeed. Will you submit a patch for --disable-ccdv ?
>
> I reproduced the issue here and indeed --disable-ccdv fix the build.

I've got a patch ready to submit, but I have to jump through some corporate hoops to release it. I know it's a trivial one-liner with that barely qualifies as code... but there you are.

So it'll be along in a little while...

Thanks,
Chris