[PATCH for-6.1] configure: Fix endianess test with LTO

[PATCH for-6.1] configure: Fix endianess test with LTO

[Thomas Huth]

If a user is trying to compile QEMU with link-time optimization
enabled by running the configure script like this:

 .../configure --extra-cflags="-flto"

then the endianess test is failing since the magic values do not
show up in the intermediate object files there. If the host is
a big endian machine (like s390x), the QEMU binary is then unusable
since the corresponding variable "bigendian" is pre-initialized
with "no".

To fix this issue, we should rather create a full binary and look
for the magic strings there instead.
And we really should not continue the build if the endianess check
failed, to make it clear right from the start that something went
wrong here, thus let's also add some "exit 1" statements here
after emitting the error message.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 configure | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/configure b/configure
index eee1356814..b3f37a3bf1 100755
--- a/configure
+++ b/configure
@@ -2349,24 +2349,27 @@ feature_not_found() {
 # ---
 # big/little endian test
 cat > $TMPC << EOF
+#include <stdio.h>
 short big_endian[] = { 0x4269, 0x4765, 0x4e64, 0x4961, 0x4e00, 0, };
 short little_endian[] = { 0x694c, 0x7454, 0x654c, 0x6e45, 0x6944, 0x6e41, 0, };
-extern int foo(short *, short *);
-int main(int argc, char *argv[]) {
-    return foo(big_endian, little_endian);
+int main(int argc, char *argv[])
+{
+    return printf("%s %s\n", (char *)big_endian, (char *)little_endian);
 }
 EOF
 
-if compile_object ; then
-    if strings -a $TMPO | grep -q BiGeNdIaN ; then
+if compile_prog ; then
+    if strings -a $TMPE | grep -q BiGeNdIaN ; then
         bigendian="yes"
-    elif strings -a $TMPO | grep -q LiTtLeEnDiAn ; then
+    elif strings -a $TMPE | grep -q LiTtLeEnDiAn ; then
         bigendian="no"
     else
         echo big/little test failed
+        exit 1
     fi
 else
     echo big/little test failed
+    exit 1
 fi
 
 ##########################################
-- 
2.27.0

Re: [PATCH for-6.1] configure: Fix endianess test with LTO

[Richard Henderson]

On 7/15/21 1:39 AM, Thomas Huth wrote:
> If a user is trying to compile QEMU with link-time optimization
> enabled by running the configure script like this:
> 
>   .../configure --extra-cflags="-flto"
> 
> then the endianess test is failing since the magic values do not
> show up in the intermediate object files there. If the host is
> a big endian machine (like s390x), the QEMU binary is then unusable
> since the corresponding variable "bigendian" is pre-initialized
> with "no".
> 
> To fix this issue, we should rather create a full binary and look
> for the magic strings there instead.
> And we really should not continue the build if the endianess check
> failed, to make it clear right from the start that something went
> wrong here, thus let's also add some "exit 1" statements here
> after emitting the error message.
> 
> Signed-off-by: Thomas Huth<thuth@redhat.com>
> ---
>   configure | 15 +++++++++------
>   1 file changed, 9 insertions(+), 6 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~