[Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

I was too optimistic, my 64-bit Windows build of the ARM emulator is not functional. :-(

I also tested other 64-bit builds of qemu-system-arm made with mingw-w64 and they are also not functional, they generally crash with some nasty DLL errors when emulation is about to start (for example 0xc0000028, which is a stack error).

I tried the following:
- the 64-bit qemu-system-arm that comes with MSYS2 as a binary package, 
- a source build on Arch (Arch because it already has mingw-w64 packages for all needed libs) 
- my source build on Ubuntu 14 LTS, with all libraries built by my script and the result as static.

the result is the same, i.e. starting the program with -version works (so no missing DLLs), but when trying to start emulation, with -kernel test.elf, the program crashes.

for comparison, on the same machine, in the same testing conditions, with the same image, but executed with the 32-bit version of the emulator, everything works just fine.


I could not identify if the problem is within qemu or mingw-w64, but I would not rule out a problem in qemu. I would first suspect a customisation made especially for mingw-w32, which is not appropriate for the 64-bit version of mingw-w64.

unfortunately my testing capabilities on Windows are limited, and cannot go much further.


it would be great if someone could investigate this problem, mingw-w64 seems a promising building tool. as development platform I would recommend Arch for Linux users or MSYS2 for Windows users.


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 7 April 2015 at 22:21, Liviu Ionescu <ilg@livius.net> wrote:
> I could not identify if the problem is within qemu or mingw-w64,
> but I would not rule out a problem in qemu. I would first suspect
> a customisation made especially for mingw-w32, which is not
> appropriate for the 64-bit version of mingw-w64.

I have successfully run QEMU in the past built with the Ubuntu
gcc-mingw-w64 compiler. My guess is that it's more likely
that the runtime is picking up the wrong version of some mingw
runtime DLL (likely the threading related one), or possibly
that you compiled against the wrong threading DLL.

> unfortunately my testing capabilities on Windows are limited, and
> cannot go much further.

> it would be great if someone could investigate this problem,
> mingw-w64 seems a promising building tool. as development platform
> I would recommend Arch for Linux users or MSYS2 for Windows users.

Unfortunately most active QEMU developers care primarily
about Linux (and the few who do have some need for Windows
builds generally have a working toolchain setup, obviously).
If you want to see this fixed you're probably going to need
to investigate it yourself :-(

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 00:30, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> On 7 April 2015 at 22:21, Liviu Ionescu <ilg@livius.net> wrote:
>> I could not identify if the problem is within qemu or mingw-w64,
>> but I would not rule out a problem in qemu. I would first suspect
>> a customisation made especially for mingw-w32, which is not
>> appropriate for the 64-bit version of mingw-w64.
> 
> I have successfully run QEMU in the past built with the Ubuntu
> gcc-mingw-w64 compiler.

yes, if you compile the 32-bit version with the mingw-w64, it works.

problems occur only with the 64-bit version.

those interested can easily reproduce the problem, both on MSYS2 and Arch.

if your tests were indeed done with mingw-w64, the 64-bit toolchain, I guess it was an older version of qemu. all my tests were done with the latest git sources (2.2.92).

> My guess is that it's more likely
> that the runtime is picking up the wrong version of some mingw
> runtime DLL (likely the threading related one), or possibly
> that you compiled against the wrong threading DLL.

it's not only about my builds, the one included in MSYS2 has the same problem. those interested can also test it easily.

> ... and the few who do have some need for Windows
> builds generally have a working toolchain setup, obviously).

yes, the 64-bit Windows build prepared by Stefan Weil seems to be done with the old mingw-w32 on Windows, not mingw-w64.

> If you want to see this fixed you're probably going to need
> to investigate it yourself :-(

the chances of succeeding this on my Mac are slim...

regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 00:46, Liviu Ionescu <ilg@livius.net> wrote:
> 
> ... the 64-bit Windows build prepared by Stefan Weil seems to be done with the old mingw-w32 on Windows, not mingw-w64.

correction, I'm no longer convinced this is true, the build might be done with mingw-w64, but with a lot of extra options (that I could not find in the doc pages...)

regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 7 April 2015 at 22:46, Liviu Ionescu <ilg@livius.net> wrote:
>
>> On 08 Apr 2015, at 00:30, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On 7 April 2015 at 22:21, Liviu Ionescu <ilg@livius.net> wrote:
>>> I could not identify if the problem is within qemu or mingw-w64,
>>> but I would not rule out a problem in qemu. I would first suspect
>>> a customisation made especially for mingw-w32, which is not
>>> appropriate for the 64-bit version of mingw-w64.
>>
>> I have successfully run QEMU in the past built with the Ubuntu
>> gcc-mingw-w64 compiler.
>
> yes, if you compile the 32-bit version with the mingw-w64, it works.
>
> problems occur only with the 64-bit version.

No,  I built an actual 64 bit Windows binary and it worked.

> those interested can easily reproduce the problem, both on MSYS2 and Arch.
>
> if your tests were indeed done with mingw-w64, the 64-bit toolchain,
> I guess it was an older version of qemu. all my tests were done with
> the latest git sources (2.2.92).

It was a while back. So it could possibly be a newly added
issue. Or it could be that my setup and your setup are different.
In particular it would have been somewhere around commit
ff4873cb8c, because that is specifically fixing a bug which
meant that 64-bit Windows builds didn't work, and once that
was fixed they did work. You could see if that commit works
for you, and if it does then bisect.

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 01:36, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> No,  I built an actual 64 bit Windows binary and it worked.

ok, then it must be only a matter of configure, which does not detect running on mingw-w64 to set the required options (perhaps like -mthreads  -D_POSIX=1), and I need to set them manually.

I'll look at Stephen Weil's options and try again.

> ... somewhere around commit
> ff4873cb8c,

ok, thank you, I'll check it.


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 7 April 2015 at 23:45, Liviu Ionescu <ilg@livius.net> wrote:
>> On 08 Apr 2015, at 01:36, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> No,  I built an actual 64 bit Windows binary and it worked.
>
> ok, then it must be only a matter of configure, which does not
> detect running on mingw-w64 to set the required options
> (perhaps like -mthreads  -D_POSIX=1), and I need to set them
> manually.

I didn't need to set any odd configure options. I dug out my
notes on my setup at the time. There's stuff here about setting
up the 32-bit toolchain as well as the 64-bit one which I haven't
bothered to edit out; ignore the bits you don't care about.
I haven't checked if all the zipfile links are still valid;
but at the time (about June last year) this was a reproducible
recipe that other people were able to use to build working
w64 binaries.

My top two guesses for your situation would be:
(1) did you install the right libpthreads compile
time package on the build host, and copy its runtime
dlls across to the windows box?
(2) did you make sure that if the windows box had
any mingw install on it already that those directories
were removed from the %PATH% to ensure that Windows didn't
try to load bogus DLLs from them?

Notes below:

===begin===
My build machine was Ubuntu Raring. The Windows machine
I tested on ran Windows 7.

You need the following packages on your build host:
binutils-mingw-w64
gcc-mingw-w64
libpthreads-mingw-w64
mingw-w64-i686-dev
mingw-w64-x86-64-dev

(in addition to all the usual things you'd need for native QEMU builds).

These install cross tools from the 'mingw-w64' project for i686 and
x86-64 targets. Don't get confused and install any 'mingw32' tools:
that is a separate project, and at least in Ubuntu the compiler is
very old and doesn't support thread local storage, so trying to use
it results in a QEMU that crashes at startup.

You'll also need to get hold of the glib dev packages from
http://www.gtk.org/download/win32.php
http://www.gtk.org/download/win64.php

(the ones in the 'gtk+ 3.x' section of the page)

Specifically for win32 you need
http://win32builder.gnome.org/packages/3.6/gettext-dev_0.18.2.1-1_win32.zip
http://win32builder.gnome.org/packages/3.6/libpng-dev_1.5.14-1_win32.zip
http://win32builder.gnome.org/packages/3.6/zlib-dev_1.2.7-1_win32.zip
http://win32builder.gnome.org/packages/3.6/glib-dev_2.34.3-1_win32.zip
http://win32builder.gnome.org/packages/3.6/pixman-dev_0.26.0-1_win32.zip

and for win64 you need:
http://win32builder.gnome.org/packages/3.6/gettext-dev_0.18.2.1-1_win64.zip
http://win32builder.gnome.org/packages/3.6/libpng-dev_1.5.14-1_win64.zip
http://win32builder.gnome.org/packages/3.6/zlib-dev_1.2.7-1_win64.zip
http://win32builder.gnome.org/packages/3.6/glib-dev_2.34.3-1_win64.zip
http://win32builder.gnome.org/packages/3.6/pixman-dev_0.26.0-1_win64.zip

Create directories ~/my-mingw and ~/my-mingw64, and extract
the zipfiles into them (so you should end up with a ~/my-mingw/lib,
and a ~/my-mingw64/lib, and some other dirs).

Create the pkg-config scripts for the two toolchains:

/usr/local/bin/i686-w64-mingw32-pkg-config
===begin===
#!/bin/sh
prefix=/full/path/to/my-mingw
PKG_CONFIG_LIBDIR=$prefix/lib/pkgconfig
export PKG_CONFIG_LIBDIR
exec pkg-config --define-variable=prefix=$prefix $@
===endit===

/usr/local/bin/x86_64-w64-mingw32-pkg-config
===begin===
#!/bin/sh
prefix=/full/path/to/my-mingw64
PKG_CONFIG_LIBDIR=$prefix/lib/pkgconfig
export PKG_CONFIG_LIBDIR
exec pkg-config --define-variable=prefix=$prefix $@
===endit===

Adjust the prefix= setting appropriately in each case.

Now you can do a build. Adjust the paths to the mingw and mingw64
directories to suit.

mkdir build/w32-arm
(cd build/w32-arm && ../../configure --cross-prefix=i686-w64-mingw32-
'--extra-cflags=-I/path/to/my-mingw/include'
'--extra-ldflags=-L/path/to/my-mingw/lib'
'--target-list=aarch64-softmmu,arm-softmmu') && make -C build/w32-arm -j4

mkdir build/w64-arm
(cd build/w64-arm && ../../configure
--cross-prefix=x86_64-w64-mingw32-
'--extra-cflags=-I/path/to/my-mingw64/include'
'--extra-ldflags=-L/path/to/my-mingw64/lib'
'--target-list=aarch64-softmmu,arm-softmmu') && make -C build/w64-arm -j4

That should give you .exe files in the usual places:

build/w32-arm/aarch64-softmmu/qemu-system-aarch64.exe
build/w64-arm/aarch64-softmmu/qemu-system-aarch64.exe

Next up we need to copy those over to a Windows box, together with
all the DLLs needed at runtime. Create directories on the Windows
box, say winqemu\ and w64qemu\, and copy the corresponding exes into them.

Copy across from the source tree the pc-bios/keymaps/ tree, so that
it ends up as winqemu\keymaps\ and w64qemu\keymaps\.

1. MinGW runtime DLLs:
For winqemu:
/usr/lib/gcc/i686-w64-mingw32/4.6/*.dll
/usr/i686-w64-mingw32/lib/*.dll
For w64qemu:
/usr/lib/gcc/x86_64-w64-mingw32/4.6/*.dll
/usr/x86_64-w64-mingw32/lib/*.dll

2. glib etc runtime DLLs:
Grab the 'runtime' zip files corresponding to the dev packages above.
You also need the libiconv runtime:
http://win32builder.gnome.org/packages/3.6/glib_2.34.3-1_win64.zip
http://win32builder.gnome.org/packages/3.6/zlib_1.2.7-1_win64.zip
http://win32builder.gnome.org/packages/3.6/pixman_0.26.0-1_win64.zip
http://win32builder.gnome.org/packages/3.6/libpng_1.5.14-1_win64.zip
http://win32builder.gnome.org/packages/3.6/gettext_0.18.2.1-1_win64.zip
http://win32builder.gnome.org/packages/3.6/libiconv_1.13.1-1_win64.zip

Note that these zipfiles tend to have a bunch of runtime files and
a variety of directory structures inside them. All you want is to grab
all the .dll files and extract those to the top level winqemu\ or
w64qemu\ directories.

Similarly for win32.

If you happen to have a local mingw installation it's probably a good
idea to make sure it is *not* on your PATH, otherwise Windows will try
to fish DLLs out of there.
===endit===

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Stefan Weil]

Am 08.04.2015 um 00:45 schrieb Liviu Ionescu:
>> On 08 Apr 2015, at 01:36, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> No,  I built an actual 64 bit Windows binary and it worked.
> ok, then it must be only a matter of configure, which does not detect running on mingw-w64 to set the required options (perhaps like -mthreads  -D_POSIX=1), and I need to set them manually.
>
> I'll look at Stephen Weil's options and try again.

Here they are:

../../../configure --cross-prefix=x86_64-w64-mingw32- 
--enable-trace-backend=stderr 
--extra-cflags=-Wno-missing-format-attribute && make

So you can see that I build in a subdirectory, enable a trace backend 
(no special reason
for that, just to cover more code during the build) and suppress a 
compiler warning.

And yes, I use mingw-w64.Here is my package list (from Debian Jessie):

ii  binutils-mingw-w64-i686 2.22-8+deb7u2+2+deb7u1        amd64        
Cross-binutils for Win32 (x86) using MinGW-w64
ii  binutils-mingw-w64-x86-64 2.22-8+deb7u2+2+deb7u1        amd64        
Cross-binutils for Win64 (x64) using MinGW-w64
ii  g++-mingw-w64 4.6.3-14+8                    all          GNU C++ 
compiler for MinGW-w64
ii  g++-mingw-w64-i686 4.6.3-14+8                    amd64        GNU 
C++ compiler for MinGW-w64 targeting Win32
ii  g++-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU 
C++ compiler for MinGW-w64 targeting Win64
ii  gcc-mingw-w64 4.6.3-14+8                    all          GNU C 
compiler for MinGW-w64
ii  gcc-mingw-w64-base 4.6.3-14+8                    amd64        GNU 
Compiler Collection for MinGW-w64 (base package)
ii  gcc-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C 
compiler for MinGW-w64 targeting Win32
ii  gcc-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU 
C compiler for MinGW-w64 targeting Win64
ii  gdb-mingw-w64 7.4.1-1.1+5                   amd64        
Cross-debugger for Win32 and Win64 using MinGW-w64
ii  gdb-mingw-w64-target 7.4.1-1.1+5                   all          
Cross-debugger server for Win32 and Win64 using MinGW-w64
ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          
Converted tgz package
ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          
Converted tgz package
ii  libfdt-mingw-w64-i686 1.4.0-2                       all          
Converted tgz package
ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          
Converted tgz package
ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          
POSIX threads library for 32- and 64-bit Windows
ii  mingw-w64 2.0.3-1                       all          Development 
environment targetting 32- and 64-bit Windows
ii  mingw-w64-i686-dev 2.0.3-1                       all          
Development files for MinGW-w64 targeting Win32
ii  mingw-w64-tools 2.0.3-1                       amd64        
Development tools for 32- and 64-bit Windows
ii  mingw-w64-x86-64-dev 2.0.3-1                       all          
Development files for MinGW-w64 targeting Win64

The last time when I had to debug a crash on Windows, I used gdb to find 
the reason.

Regards
Stefan

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 09:20, Stefan Weil <sw@weilnetz.de> wrote:
> 
> ../../../configure --cross-prefix=x86_64-w64-mingw32- --enable-trace-backend=stderr --extra-cflags=-Wno-missing-format-attribute && make
> 
> ... mingw-w64.Here is my package list (from Debian Jessie):

ok

I'm just installing a new Win 8 virtual machine to be sure I do not mess any dlls and try again.

> The last time when I had to debug a crash on Windows, I used gdb to find the reason.

you mean you copied the mingw...-gdb.exe to windows and alone it was enough to debug? good to know it works.


thank you,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 09:20, Stefan Weil <sw@weilnetz.de> wrote:
> 
> ... Here is my package list (from Debian Jessie):
> 
> ii  binutils-mingw-w64-i686 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win32 (x86) using MinGW-w64
> ii  binutils-mingw-w64-x86-64 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win64 (x64) using MinGW-w64
> ii  g++-mingw-w64 4.6.3-14+8                    all          GNU C++ compiler for MinGW-w64
> ii  g++-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win32
> ii  g++-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win64
> ii  gcc-mingw-w64 4.6.3-14+8                    all          GNU C compiler for MinGW-w64
> ii  gcc-mingw-w64-base 4.6.3-14+8                    amd64        GNU Compiler Collection for MinGW-w64 (base package)
> ii  gcc-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win32
> ii  gcc-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win64
> ii  gdb-mingw-w64 7.4.1-1.1+5                   amd64        Cross-debugger for Win32 and Win64 using MinGW-w64
> ii  gdb-mingw-w64-target 7.4.1-1.1+5                   all          Cross-debugger server for Win32 and Win64 using MinGW-w64
> ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          Converted tgz package
> ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          Converted tgz package
> ii  libfdt-mingw-w64-i686 1.4.0-2                       all          Converted tgz package
> ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          Converted tgz package
> ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          POSIX threads library for 32- and 64-bit Windows
> ii  mingw-w64 2.0.3-1                       all          Development environment targetting 32- and 64-bit Windows
> ii  mingw-w64-i686-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win32
> ii  mingw-w64-tools 2.0.3-1                       amd64        Development tools for 32- and 64-bit Windows
> ii  mingw-w64-x86-64-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win64


Hallo Stefan, 

I'm trying to reproduce your build environment, but I'm only partly successful.

I got a fresh Debian 8 (in a virtual environment, but this should not matter):

  docker run --interactive --tty --name "deb8x64_mingw-w64" debian:8 /bin/bash

then, as usually recommended, I updated and upgraded it:

  apt-get -y update
  apt-get -y upgrade
  apt-get -y install lsb-release

after this, I have a confirmation that I have the expected version:

# lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 8.0 (jessie)
Release:	8.0
Codename:	jessie

so far, so good.

then I took your list of packages and first installed the following mingw-w64 packages:

# apt-get -y install \
binutils-mingw-w64-i686 \
binutils-mingw-w64-x86-64 \
g++-mingw-w64 \
g++-mingw-w64-i686 \
g++-mingw-w64-x86-64 \
gcc-mingw-w64 \
gcc-mingw-w64-base \
gcc-mingw-w64-i686 \
gcc-mingw-w64-x86-64 \
gdb-mingw-w64 \
gdb-mingw-w64-target \
mingw-w64 \
mingw-w64-i686-dev \
mingw-w64-tools \
mingw-w64-x86-64-dev \

the result is:

# dpkg -l | grep mingw
ii  binutils-mingw-w64-i686    2.25-5+5.2               amd64        Cross-binutils for Win32 (x86) using MinGW-w64
ii  binutils-mingw-w64-x86-64  2.25-5+5.2               amd64        Cross-binutils for Win64 (x64) using MinGW-w64
ii  g++-mingw-w64              4.9.1-19+14.3            all          GNU C++ compiler for MinGW-w64
ii  g++-mingw-w64-i686         4.9.1-19+14.3            amd64        GNU C++ compiler for MinGW-w64 targeting Win32
ii  g++-mingw-w64-x86-64       4.9.1-19+14.3            amd64        GNU C++ compiler for MinGW-w64 targeting Win64
ii  gcc-mingw-w64              4.9.1-19+14.3            all          GNU C compiler for MinGW-w64
ii  gcc-mingw-w64-base         4.9.1-19+14.3            amd64        GNU Compiler Collection for MinGW-w64 (base package)
ii  gcc-mingw-w64-i686         4.9.1-19+14.3            amd64        GNU C compiler for MinGW-w64 targeting Win32
ii  gcc-mingw-w64-x86-64       4.9.1-19+14.3            amd64        GNU C compiler for MinGW-w64 targeting Win64
ii  gdb-mingw-w64              7.6.2-1.1+9              amd64        Cross-debugger for Win32 and Win64 using MinGW-w64
ii  gdb-mingw-w64-target       7.6.2-1.1+9              all          Cross-debugger server for Win32 and Win64 using MinGW-w64
ii  gfortran-mingw-w64         4.9.1-19+14.3            all          GNU Fortran compiler for MinGW-w64
ii  gfortran-mingw-w64-i686    4.9.1-19+14.3            amd64        GNU Fortran compiler for MinGW-w64 targeting Win32
ii  gfortran-mingw-w64-x86-64  4.9.1-19+14.3            amd64        GNU Fortran compiler for MinGW-w64 targeting Win64
ii  gnat-mingw-w64             4.9.1-3+14               all          GNU Ada compiler for MinGW-w64
ii  gnat-mingw-w64-base        4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 (base package)
ii  gnat-mingw-w64-i686        4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 targeting Win32
ii  gnat-mingw-w64-x86-64      4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 targeting Win64
ii  mingw-w64                  3.2.0-2                  all          Development environment targeting 32- and 64-bit Windows
ii  mingw-w64-common           3.2.0-2                  all          Common files for Mingw-w64
ii  mingw-w64-i686-dev         3.2.0-2                  all          Development files for MinGW-w64 targeting Win32
ii  mingw-w64-tools            3.2.0-2                  amd64        Development tools for 32- and 64-bit Windows
ii  mingw-w64-x86-64-dev       3.2.0-2                  all          Development files for MinGW-w64 targeting Win64


there are slightly more packages than you have, and some are newer, probably because I did the upgrade, but I hope this will not be a problem.


however, trying to bring the rest of the packages, with the libraries, failed:

# apt-get -y install \
> libpthreads-mingw-w64 \
> gtk-mingw-w64-x86-64 \
> gtk2.0-mingw-w64-i686 \
> libfdt-mingw-w64-i686 \
> libfdt-mingw-w64-x86-64 \
> 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package libpthreads-mingw-w64 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  mingw-w64-x86-64-dev mingw-w64-i686-dev

E: Package 'libpthreads-mingw-w64' has no installation candidate
E: Unable to locate package gtk-mingw-w64-x86-64
E: Unable to locate package gtk2.0-mingw-w64-i686
E: Couldn't find any package by regex 'gtk2.0-mingw-w64-i686'
E: Unable to locate package libfdt-mingw-w64-i686
E: Unable to locate package libfdt-mingw-w64-x86-64


can you be more specific how did you install those packages?

my first goal is to reproduce your build environment, and only then proceed with the builds. 

a secondary goal is to also have a clean, documented and reproducible procedure to create a Docker image that can be used for qemu builds (Docker is a great tool for such automated builds).


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 02:10, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> ... I dug out my
> notes on my setup at the time. ...
> but at the time (about June last year) this was a reproducible
> recipe that other people were able to use to build working
> w64 binaries.

thank you, Peter, I'll first try to reproduce Stephen's environment, and then the build.

regards,

Liviu


p.s. I'm always amazed when for exact tasks, like complex builds, people prefer narrative recipes instead of ready to run scripts... 

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 8 April 2015 at 15:13, Liviu Ionescu <ilg@livius.net> wrote:
> p.s. I'm always amazed when for exact tasks, like complex
> builds, people prefer narrative recipes instead of ready
> to run scripts...

I agree generally, though this one is tedious because it
involves several different machines. The reason it ended
up as a narrative is because it's a one-time setup task
that required a bunch of exploration and trials to get
to something that worked. At that point it's in a state
which is "works for me" and it's less effort for me just
to cut-and-paste the notes I made in the exploration
process than to go back and write a script. If I ever
need to do the setup again myself, then I'll probably
script it in the course of doing that second run-through.
But so far I haven't needed to (and it's been nearly a year).

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 08 Apr 2015, at 09:20, Stefan Weil <sw@weilnetz.de> wrote:
> 
> Here is my package list (from Debian Jessie):
> 
> ii  binutils-mingw-w64-i686 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win32 (x86) using MinGW-w64
> ii  binutils-mingw-w64-x86-64 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win64 (x64) using MinGW-w64
> ii  g++-mingw-w64 4.6.3-14+8                    all          GNU C++ compiler for MinGW-w64
> ii  g++-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win32
> ii  g++-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win64
> ii  gcc-mingw-w64 4.6.3-14+8                    all          GNU C compiler for MinGW-w64
> ii  gcc-mingw-w64-base 4.6.3-14+8                    amd64        GNU Compiler Collection for MinGW-w64 (base package)
> ii  gcc-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win32
> ii  gcc-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win64
> ii  gdb-mingw-w64 7.4.1-1.1+5                   amd64        Cross-debugger for Win32 and Win64 using MinGW-w64
> ii  gdb-mingw-w64-target 7.4.1-1.1+5                   all          Cross-debugger server for Win32 and Win64 using MinGW-w64
> ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          Converted tgz package
> ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          Converted tgz package
> ii  libfdt-mingw-w64-i686 1.4.0-2                       all          Converted tgz package
> ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          Converted tgz package
> ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          POSIX threads library for 32- and 64-bit Windows
> ii  mingw-w64 2.0.3-1                       all          Development environment targetting 32- and 64-bit Windows
> ii  mingw-w64-i686-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win32
> ii  mingw-w64-tools 2.0.3-1                       amd64        Development tools for 32- and 64-bit Windows
> ii  mingw-w64-x86-64-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win64
> 

Stefan,

I'm afraid there is a small misunderstanding here, I checked and even without upgrading the packages, the Debian 8 (Jessie) does not include the packages you are referring above, the actual versions I identified are:

# dpkg -l | grep mingw
ii  binutils-mingw-w64-i686    2.25-5+5.2               amd64        Cross-binutils for Win32 (x86) using MinGW-w64
ii  binutils-mingw-w64-x86-64  2.25-5+5.2               amd64        Cross-binutils for Win64 (x64) using MinGW-w64
ii  g++-mingw-w64              4.9.1-19+14.3            all          GNU C++ compiler for MinGW-w64
ii  g++-mingw-w64-i686         4.9.1-19+14.3            amd64        GNU C++ compiler for MinGW-w64 targeting Win32
ii  g++-mingw-w64-x86-64       4.9.1-19+14.3            amd64        GNU C++ compiler for MinGW-w64 targeting Win64
ii  gcc-mingw-w64              4.9.1-19+14.3            all          GNU C compiler for MinGW-w64
ii  gcc-mingw-w64-base         4.9.1-19+14.3            amd64        GNU Compiler Collection for MinGW-w64 (base package)
ii  gcc-mingw-w64-i686         4.9.1-19+14.3            amd64        GNU C compiler for MinGW-w64 targeting Win32
ii  gcc-mingw-w64-x86-64       4.9.1-19+14.3            amd64        GNU C compiler for MinGW-w64 targeting Win64
ii  gdb-mingw-w64              7.6.2-1.1+9              amd64        Cross-debugger for Win32 and Win64 using MinGW-w64
ii  gdb-mingw-w64-target       7.6.2-1.1+9              all          Cross-debugger server for Win32 and Win64 using MinGW-w64
ii  gfortran-mingw-w64         4.9.1-19+14.3            all          GNU Fortran compiler for MinGW-w64
ii  gfortran-mingw-w64-i686    4.9.1-19+14.3            amd64        GNU Fortran compiler for MinGW-w64 targeting Win32
ii  gfortran-mingw-w64-x86-64  4.9.1-19+14.3            amd64        GNU Fortran compiler for MinGW-w64 targeting Win64
ii  gnat-mingw-w64             4.9.1-3+14               all          GNU Ada compiler for MinGW-w64
ii  gnat-mingw-w64-base        4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 (base package)
ii  gnat-mingw-w64-i686        4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 targeting Win32
ii  gnat-mingw-w64-x86-64      4.9.1-3+14               amd64        GNU Ada compiler for MinGW-w64 targeting Win64
ii  mingw-w64                  3.2.0-2                  all          Development environment targeting 32- and 64-bit Windows
ii  mingw-w64-common           3.2.0-2                  all          Common files for Mingw-w64
ii  mingw-w64-i686-dev         3.2.0-2                  all          Development files for MinGW-w64 targeting Win32
ii  mingw-w64-tools            3.2.0-2                  amd64        Development tools for 32- and 64-bit Windows
ii  mingw-w64-x86-64-dev       3.2.0-2                  all          Development files for MinGW-w64 targeting Win64


the differences are major, mingw-w64 was upgraded from 2.0.3 to 3.2.0, the compiler from 4.6.3 to 4.9.1, and so on.

what are the recommended versions to correctly build qemu?

and the question from the previous message is still open, how did you install the following packages? the comment "Converted tgz package" is a good sign these are custom packages.

ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          Converted tgz package
ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          Converted tgz package
ii  libfdt-mingw-w64-i686 1.4.0-2                       all          Converted tgz package
ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          Converted tgz package
ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          POSIX threads library for 32- and 64-bit Windows


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Stefan Weil]

Am 08.04.2015 um 22:27 schrieb Liviu Ionescu:
>> On 08 Apr 2015, at 09:20, Stefan Weil <sw@weilnetz.de> wrote:
>>
>> Here is my package list (from Debian Jessie):
>>
>> ii  binutils-mingw-w64-i686 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win32 (x86) using MinGW-w64
>> ii  binutils-mingw-w64-x86-64 2.22-8+deb7u2+2+deb7u1        amd64        Cross-binutils for Win64 (x64) using MinGW-w64
>> ii  g++-mingw-w64 4.6.3-14+8                    all          GNU C++ compiler for MinGW-w64
>> ii  g++-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win32
>> ii  g++-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C++ compiler for MinGW-w64 targeting Win64
>> ii  gcc-mingw-w64 4.6.3-14+8                    all          GNU C compiler for MinGW-w64
>> ii  gcc-mingw-w64-base 4.6.3-14+8                    amd64        GNU Compiler Collection for MinGW-w64 (base package)
>> ii  gcc-mingw-w64-i686 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win32
>> ii  gcc-mingw-w64-x86-64 4.6.3-14+8                    amd64        GNU C compiler for MinGW-w64 targeting Win64
>> ii  gdb-mingw-w64 7.4.1-1.1+5                   amd64        Cross-debugger for Win32 and Win64 using MinGW-w64
>> ii  gdb-mingw-w64-target 7.4.1-1.1+5                   all          Cross-debugger server for Win32 and Win64 using MinGW-w64
>> ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          Converted tgz package
>> ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          Converted tgz package
>> ii  libfdt-mingw-w64-i686 1.4.0-2                       all          Converted tgz package
>> ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          Converted tgz package
>> ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          POSIX threads library for 32- and 64-bit Windows
>> ii  mingw-w64 2.0.3-1                       all          Development environment targetting 32- and 64-bit Windows
>> ii  mingw-w64-i686-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win32
>> ii  mingw-w64-tools 2.0.3-1                       amd64        Development tools for 32- and 64-bit Windows
>> ii  mingw-w64-x86-64-dev 2.0.3-1                       all          Development files for MinGW-w64 targeting Win64
>>
> Stefan,
>
> I'm afraid there is a small misunderstanding here, I checked and even without upgrading the packages, the Debian 8 (Jessie) does not include the packages you are referring above, the actual versions I identified are:

You are right, I was wrong: my production server qemu.weilnetz.de uses 
Wheezy,
not Jessie, so my list was for Wheezy.

Here is the list for my Jessie system:

ii  binutils-mingw-w64-i686 2.25-5+5.2                          
amd64        Cross-binutils for Win32 (x86) using MinGW-w64
ii  binutils-mingw-w64-x86-64 2.25-5+5.2                          
amd64        Cross-binutils for Win64 (x64) using MinGW-w64
ii  g++-mingw-w64 4.9.1-19+14.3                       all          GNU 
C++ compiler for MinGW-w64
ii  g++-mingw-w64-i686 4.9.1-19+14.3                       amd64        
GNU C++ compiler for MinGW-w64 targeting Win32
ii  g++-mingw-w64-x86-64 4.9.1-19+14.3                       
amd64        GNU C++ compiler for MinGW-w64 targeting Win64
ii  gcc-mingw-w64 4.9.1-19+14.3                       all          GNU C 
compiler for MinGW-w64
ii  gcc-mingw-w64-base 4.9.1-19+14.3                       amd64        
GNU Compiler Collection for MinGW-w64 (base package)
ii  gcc-mingw-w64-i686 4.9.1-19+14.3                       amd64        
GNU C compiler for MinGW-w64 targeting Win32
ii  gcc-mingw-w64-x86-64 4.9.1-19+14.3                       
amd64        GNU C compiler for MinGW-w64 targeting Win64
ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2                    
all          Converted tgz package
ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2                  
all          Converted tgz package
ii  libfdt-mingw-w64-i686 1.4.0-2                             
all          Converted tgz package
ii  libfdt-mingw-w64-x86-64 1.4.0-2                             
all          Converted tgz package
ii  mingw-w64 4.0~rc3-1                           all          
Development environment targeting 32- and 64-bit Windows
ii  mingw-w64-common 4.0~rc3-1                           all          
Common files for Mingw-w64
ii  mingw-w64-i686-dev 4.0~rc3-1                           all          
Development files for MinGW-w64 targeting Win32
ii  mingw-w64-x86-64-dev 4.0~rc3-1                           
all          Development files for MinGW-w64 targeting Win64

Both the Wheezy and the Jessie environment work.

> and the question from the previous message is still open, how did you install the following packages? the comment "Converted tgz package" is a good sign these are custom packages.
>
> ii  gtk-mingw-w64-x86-64 3.6.4-20131201-2              all          Converted tgz package
> ii  gtk2.0-mingw-w64-i686 2.24.10-20120208-2            all          Converted tgz package
> ii  libfdt-mingw-w64-i686 1.4.0-2                       all          Converted tgz package
> ii  libfdt-mingw-w64-x86-64 1.4.0-2                       all          Converted tgz package
> ii  libpthreads-mingw-w64 2.9.1+dfsg-1                  all          POSIX threads library for 32- and 64-bit Windows
>

The first four are custom packages, and you can get them from 
http://qemu.weilnetz.de/debian/.
The last one is Wheezy specific and not needed.

The gtk custom packages were made from gtk.org downloads 
(all-in-one-bundles, for example
from http://www.gtk.org/download/win64.php). That is the same source 
which Peter suggested.

The libfdt custom packages simple packed the results from a QEMU build 
with internal libfdt:
QEMU can be build without any external libfdt and will then compile it 
during the normal build.
As I run many builds and want to avoid this extra compilation step, I 
created that packages.

As you can see in the list above, all custom packages work for Wheezy 
and for Jessie.

Stefan

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 09 Apr 2015, at 08:12, Stefan Weil <sw@weilnetz.de> wrote:
> 
> The first four are custom packages, and you can get them from http://qemu.weilnetz.de/debian/.
> The last one is Wheezy specific and not needed.

ok, got them, installed them and got no errors, they are now listed by dpkg -l

> The gtk custom packages were made from gtk.org downloads (all-in-one-bundles, for example
> from http://www.gtk.org/download/win64.php). That is the same source which Peter suggested.
> 
> The libfdt custom packages simple packed the results from a QEMU build with internal libfdt:
> QEMU can be build without any external libfdt and will then compile it during the normal build.
> As I run many builds and want to avoid this extra compilation step, I created that packages.

ok, we'll return to the procedure you used to build them later, since I want to document it too.

> As you can see in the list above, all custom packages work for Wheezy and for Jessie.

not yet. the packages were installed correctly, I can see the files in the corresponding /usr/x86_64-w64-mingw32 folders, but when you built these packages, you did not update the prefix in the pkg-config files, and most read something like "prefix=/srv/win32builder/fixed_364/build/win64", which might exist on your machine, but does not exist on a clean machine.

I guess you either tweaked the pkg-config to find them, or you set some environment variables before configure.

with the default settings, configure produces:

CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mms-bitfields -I/srv/win32builder/fixed_364/build/win64/include/glib-2.0 -I/srv/win32builder/fixed_364/build/win64/lib/glib-2.0/include  -g 
QEMU_CFLAGS       -I/srv/win32builder/fixed_364/build/win64/include/pixman-1   -m64 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wno-missing-format-attribute  -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong  -I/srv/win32builder/fixed_364/build/win64/include/libpng15 
LDFLAGS           -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g 

which obviously breaks the make.

can you share these details too? I want to reproduce "exactly" your environment.


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 09 Apr 2015, at 10:40, Liviu Ionescu <ilg@livius.net> wrote:

> I guess you either tweaked the pkg-config to find them, or you set some environment variables before configure.
> ...
> can you share these details too? I want to reproduce "exactly" your environment.

the reason I ask for these details is that even after adding a x86_64-w64-mingw32-pkg-config, the configure still differs from yours.

among the differences I spotted were:

- no -mthreads -D_POSIX=1 
- no SDL support 

another problems are related to the setup:
- "make installer" did not pack any DLL inside the setup
- the setup insists on installing in C:\Program Files (x86), although it is not a 32-bit application

my feeling is that you still have additional libraries that you have difficulties to keep track, you possibly use environment variables, and the procedure to generate the setup is not the one in the Makefile.


I would appreciate your help in documenting all build details.

regards,

Liviu


mkdir -p /root/build
cd /root/build
PKG_CONFIG=/root/Host/Work/qemu/gnuarmeclipse-qemu.git/gnuarmeclipse/scripts/x86_64-w64-mingw32-pkg-config \
/root/qemu.git/configure --cross-prefix=x86_64-w64-mingw32- --enable-trace-backend=stderr --extra-cflags=-Wno-missing-format-attribute --target-list="arm-softmmu" | tee build.txt
make
make installer

CFLAGS            -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -mms-bitfields -I/usr/x86_64-w64-mingw32/include/glib-2.0 -I/usr/x86_64-w64-mingw32/lib/glib-2.0/include -g 
QEMU_CFLAGS       -I/usr/x86_64-w64-mingw32/include/pixman-1  -m64 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -Wno-missing-format-attribute  -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-strong  -I/usr/x86_64-w64-mingw32/include/libpng15
LDFLAGS           -Wl,--nxcompat -Wl,--no-seh -Wl,--dynamicbase -Wl,--warn-common -m64 -g 
make              make
install           install
python            python -B
smbd              /usr/sbin/smbd
module support    no
host CPU          x86_64
host big endian   no
target list       arm-softmmu
tcg debug enabled no
gprof enabled     no
sparse enabled    no
strip binaries    yes
profiler          no
static build      no
pixman            system
SDL support       no
GTK support       yes
VTE support       no
curses support    no
curl support      no
mingw32 support   yes
Audio drivers     winwave
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS support    no
VNC support       yes
VNC TLS support   no
VNC SASL support  no
VNC JPEG support  yes
VNC PNG support   yes
VNC WS support    no
xen support       no
brlapi support    no
bluez  support    no
Documentation     yes
GUEST_BASE        yes
PIE               no
vde support       no
netmap support    no
Linux AIO support no
ATTR/XATTR support no
Install blobs     yes
KVM support       no
RDMA support      no
TCG interpreter   no
fdt support       yes
preadv support    no
fdatasync         no
madvise           no
posix_madvise     no
sigev_thread_id   no
uuid support      no
libcap-ng support no
vhost-net support no
vhost-scsi support no
Trace backends    stderr
spice support     no
rbd support       no
xfsctl support    no
nss used          no
libusb            no
usb net redir     no
OpenGL support    no
libiscsi support  no
libnfs support    no
build guest agent yes
QGA VSS support   no
seccomp support   no
coroutine backend win32
coroutine pool    yes
GlusterFS support no
Archipelago support no
gcov              gcov
gcov enabled      no
TPM support       yes
libssh2 support   no
TPM passthrough   no
QOM debugging     yes
vhdx              no
Quorum            no
lzo support       no
snappy support    no
bzip2 support     no
NUMA host support no

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Stefan Weil]

Am 09.04.2015 um 16:07 schrieb Liviu Ionescu
>> On 09 Apr 2015, at 10:40, Liviu Ionescu <ilg@livius.net> wrote:
>> I guess you either tweaked the pkg-config to find them, or you set some environment variables before configure.
>> ...
>> can you share these details too? I want to reproduce "exactly" your environment.


Here is the template for all cross pkg-configs:
http://repo.or.cz/w/qemu/ar7.git/blob_plain/HEAD:/scripts/cross-pkg-config

See also http://patchwork.ozlabs.org/patch/114242/

Maybe I should resend that patch.

> the reason I ask for these details is that even after adding a x86_64-w64-mingw32-pkg-config, the configure still differs from yours.
>
> among the differences I spotted were:
>
> - no -mthreads -D_POSIX=1
> - no SDL support


I think -mthreads is essential (needed for thread local storage),
and -D_POSIX=1 is also needed for 64 bit builds.

Maybe you can try my QEMU fork from http://repo.or.cz/w/qemu/ar7.git
and see whether it works for you. My binaries are based on that code.
If it works, we have to look which patches are missing in the official
QEMU version.

SDL comes from a locally cross built libSDL. I never had time to put
it into a Debian package. Maybe it is better for your tests to build
without SDL support, because the SDL init code redirects stdin and
stdout to files which complicates things even when you run QEMU
with SDL disabled.
.


>   
>
> another problems are related to the setup:
> - "make installer" did not pack any DLL inside the setup

It has a built-in magic: all DLL files from $(SRC_PATH)/dll/w64 are
taken for the 64 bit build (w32 for 32 bit build).

Get a list of all needed DLL files either by try and error or by using
tools and copy them (or make symbolic links) to that directory.
Here are the files for 32 bit for my configuration:

dll/w32/freetype6.dll
dll/w32/pdcurses.dll
dll/w32/libgio-2.0-0.dll
dll/w32/libfontconfig-1.dll
dll/w32/libpangoft2-1.0-0.dll
dll/w32/libpangocairo-1.0-0.dll
dll/w32/libgtk-win32-2.0-0.dll
dll/w32/libatk-1.0-0.dll
dll/w32/libpangowin32-1.0-0.dll
dll/w32/intl.dll
dll/w32/libpango-1.0-0.dll
dll/w32/libgcc_s_sjlj-1.dll
dll/w32/libgdk_pixbuf-2.0-0.dll
dll/w32/libglib-2.0-0.dll
dll/w32/libssp-0.dll
dll/w32/zlib1.dll
dll/w32/libgmodule-2.0-0.dll
dll/w32/libcairo-2.dll
dll/w32/SDL.dll
dll/w32/libgthread-2.0-0.dll
dll/w32/libexpat-1.dll
dll/w32/libpng14-14.dll
dll/w32/libgobject-2.0-0.dll
dll/w32/libgdk-win32-2.0-0.dll
dll/w32/libstdc++-6.dll

> - the setup insists on installing in C:\Program Files (x86), although it is not a 32-bit application
>
> my feeling is that you still have additional libraries that you have difficulties to keep track, you possibly use environment variables, and the procedure to generate the setup is not the one in the Makefile.

There is an nsis macro W64 which switches the program directory (see 
qemu.nsi).
Are you using the packages from Debian (here those for Wheezy):

ii  nsis 2.46-7                        amd64        Nullsoft Scriptable 
Install System (modified for Debian)
ii  nsis-common 2.46-7                        all          Nullsoft 
Scriptable Install System stubs and plugins


Regards
Stefan

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 9 April 2015 at 18:41, Stefan Weil <sw@weilnetz.de> wrote:
> I think -mthreads is essential (needed for thread local storage),
> and -D_POSIX=1 is also needed for 64 bit builds.

My 64-bit builds worked without either of these things...

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 09 Apr 2015, at 20:58, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> On 9 April 2015 at 18:41, Stefan Weil <sw@weilnetz.de> wrote:
>> I think -mthreads is essential (needed for thread local storage),
>> and -D_POSIX=1 is also needed for 64 bit builds.
> 
> My 64-bit builds worked without either of these things...

hmmm... things get complicated, even messier. :-(

first I was confident, I wrote my multi-platform build script that worked for all platforms, except win64; then I reverted, trying to build the official version, and since you remember having successful win64 builds, I took it as reference. unfortunately the build was not successful, due to the missing libraries; then I considered Stefan's binaries to be the 'official' QEMU releases for windows, just to find out that they are actually based on a fork, that I'm invited to check for differences...

it seems to never end...

the fact is: the public version from the project git, cross built for 64-bit on a fresh Debian 8, with the (old) packages from Stefan, and with the configuration you suggested, builds successfully, but... fails to run. the 32-bit version, built in the exactly the same conditions, is ok.


I don't exclude a systematic error that I made in my builds, but, from what I see, my tests are far more rigorous than what I found up to now, and I'd rather suspect the problem to be somewhere else.

it might very well be a problem with the treading support, slightly differently implemented on win64.

---

> SDL comes from a locally cross built libSDL. 

ok, I expected this. :-(

> Maybe it is better for your tests to build
without SDL support,

in GNU ARM Eclipse QEMU I do not use SDL, but I just wanted to reproduce the exact conditions you use for your builds.

> > - "make installer" did not pack any DLL inside the setup

> It has a built-in magic: all DLL files from $(SRC_PATH)/dll/w64 are
> taken for the 64 bit build (w32 for 32 bit build).

great, I like magic, but what I do not understand is why this magic is not in the public git, for us all to use it?

> Get a list of all needed DLL files either by try and error or by using
> tools and copy them (or make symbolic links) to that directory.

I did this for my builds, I just wanted to clarify things for the official build.

is it that complicated to share **all** build scripts and related details, preferably to contribute them back to the project?

> There is an nsis macro W64 which switches the program directory (see qemu.nsi).

I have a far more elaborate nsis configuration for my builds; the problem is that the official 'make install' does not generate a functional setup (the dlls are not there, and the setup does not write the program in the proper place), and since your setup seems functional, it would be great to update the project.

> Maybe you can try my QEMU fork from http://repo.or.cz/w/qemu/ar7.git
> and see whether it works for you.

are your actual build scripts in the repo? or in some public place?

all my scripts are in the GNU ARM Eclipse QEMU repo, and not only the build script is self contained, but all information necessary to run it is public.

even more, now I'm migrating the build processes to using Docker, which is an even more reproducible and scriptable solution, exactly to avoid this kind of embarrassing situation that we are now in, with the mysterious win64 build.


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Stefan Weil]

Am 09.04.2015 um 21:44 schrieb Liviu Ionescu:
> hmmm... things get complicated, even messier. :-(

Windows support _is_ complicated, yes.

There is MinGW32, MinGW-w64, Cygwin, Cygwin64 (list incomplete).

And some versions of MinGW-w64 are incompatible because they
changed their name mangling, so there are pre-compiled libraries
with and without leading underscores in symbol names.

Different models for exception handling exist and require different
toolchains.

The dependencies required for QEMU must be collected from different
sources, and maybe they depend on different variants of zlib or libz.
Or you have to get the sources and build one after the other.

And all this runs on a closed source system which sometimes
remembers a black hole.

> first I was confident, I wrote my multi-platform build script that worked for all platforms, except win64; then I reverted, trying to build the official version, and since you remember having successful win64 builds, I took it as reference. unfortunately the build was not successful, due to the missing libraries; then I considered Stefan's binaries to be the 'official' QEMU releases for windows, just to find out that they are actually based on a fork, that I'm invited to check for differences...

I thought the documentation on my website were clear, but if it is not,
suggestions for improvement are welcome.

I try to keep my fork close to the official version. As this is voluntary
work done in my free time, there can be delays in the synchronization.
My fork also contains additional code which would need much work
to integrate it in the official QEMU (MIPS AR7 with WLAN router emulation,
Raspberry Pi emulation and other parts which are not always working).
I don't think those differences are important for your problem, but if
you would run a short test, we could be sure.

By the way: I still don't have your test setting, so I have no chance to
reproduce it. Is the test.elf which you use publicly available?

> it seems to never end...
>
> the fact is: the public version from the project git, cross built for 64-bit on a fresh Debian 8, with the (old) packages from Stefan, and with the configuration you suggested, builds successfully, but... fails to run. the 32-bit version, built in the exactly the same conditions, is ok.
>
> I don't exclude a systematic error that I made in my builds, but, from what I see, my tests are far more rigorous than what I found up to now, and I'd rather suspect the problem to be somewhere else.
>
> it might very well be a problem with the treading support, slightly differently implemented on win64.
>
> ---
>
>> SDL comes from a locally cross built libSDL.
> ok, I expected this. :-(

:-) (my binaries for SDL are online now, http://qemu.weilnetz.de/debian/)

>
>> Maybe it is better for your tests to build
> without SDL support,
>
> in GNU ARM Eclipse QEMU I do not use SDL, but I just wanted to reproduce the exact conditions you use for your builds.
>
>>> - "make installer" did not pack any DLL inside the setup
>> It has a built-in magic: all DLL files from $(SRC_PATH)/dll/w64 are
>> taken for the 64 bit build (w32 for 32 bit build).
> great, I like magic, but what I do not understand is why this magic is not in the public git, for us all to use it?

It is. Not many people want to build QEMU installers for Windows,
and those who do usually find a way how to do it.

As I said above: suggestions for improvement are welcome.


>
>> Get a list of all needed DLL files either by try and error or by using
>> tools and copy them (or make symbolic links) to that directory.
> I did this for my builds, I just wanted to clarify things for the official build.
>
> is it that complicated to share **all** build scripts and related details, preferably to contribute them back to the project?

As you see, I am willing to share any information which is needed.

>> There is an nsis macro W64 which switches the program directory (see qemu.nsi).
> I have a far more elaborate nsis configuration for my builds; the problem is that the official 'make install' does not generate a functional setup (the dlls are not there, and the setup does not write the program in the proper place), and since your setup seems functional, it would be great to update the project.

'make install' installs (in the wrong place). You are invited to fix it. 
I never use it.

'make installer' creates an installer.

>
>> Maybe you can try my QEMU fork from http://repo.or.cz/w/qemu/ar7.git
>> and see whether it works for you.
> are your actual build scripts in the repo? or in some public place?

I already told you my configure + make commands - nothing magic with them
and they create working binaries here.

> all my scripts are in the GNU ARM Eclipse QEMU repo, and not only the build script is self contained, but all information necessary to run it is public.
>
> even more, now I'm migrating the build processes to using Docker, which is an even more reproducible and scriptable solution, exactly to avoid this kind of embarrassing situation that we are now in, with the mysterious win64 build.

That's great.

Cheers
Stefan

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

> On 09 Apr 2015, at 20:58, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> On 9 April 2015 at 18:41, Stefan Weil <sw@weilnetz.de> wrote:
>> I think -mthreads is essential (needed for thread local storage),
>> and -D_POSIX=1 is also needed for 64 bit builds.
> 
> My 64-bit builds worked without either of these things...

I tried with these settings and it still does not work.

> On 09 Apr 2015, at 20:41, Stefan Weil <sw@weilnetz.de> wrote:
> 
> Maybe you can try my QEMU fork from http://repo.or.cz/w/qemu/ar7.git
> and see whether it works for you. My binaries are based on that code.
> If it works, we have to look which patches are missing in the official
> QEMU version.

there are probably hundreds of differences between your fork ant the official branch.

in 'configure' too there are quite a lot of differences. apart from the above CFLAGS, I could not identify something special.


regards,

Liviu

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Peter Maydell]

On 9 April 2015 at 21:43, Stefan Weil <sw@weilnetz.de> wrote:
> My fork also contains additional code which would need much work
> to integrate it in the official QEMU (MIPS AR7 with WLAN router emulation,
> Raspberry Pi emulation and other parts which are not always working).

As an aside, if you're interested in trying to get rpi support
upstream I'm happy to help -- we do seem to get a fair amount
of "how do I emulate rpi in QEMU" questions so it seems
worthwhile. If you don't want to (or if this is just somebody
else's project/patches you're carrying in your tree) I
understand that too -- I have a bunch of omap3 patches
out-of-tree I've never got round to upstreaming either...

-- PMM

Re: [Qemu-devel] 64-bit build of qemu-system-arm with mingw-w64 not functional

[Liviu Ionescu]

first of all, for those interested in a reproducible build procedure for Windows, I have a single and complete script, building the 32/64-bit versions from the original repository or from Stefan's repository.

the short story is:

- install Docker from Docker.com
- download the build script from https://sourceforge.net/p/gnuarmeclipse/qemu/ci/gnuarmeclipse-dev/tree/gnuarmeclipse/scripts/build-orig-qemu.sh (use the Download this file link)
- run the script with 

	$ bash build-orig-qemu.sh --orig --win64
	$ bash build-orig-qemu.sh --orig --win32
	$ bash build-orig-qemu.sh --stefan --win64

the script will leave the build results in ${HOME}/Work/orig-qemu or ${HOME}/Work/stefan-qemu.

(the 32-bit build with Stefan's repository failed).


at first run, the script will download the custom Debian 8 image from 

	https://registry.hub.docker.com/u/ilegeul/qemu-builds/

the image was prepared using the Dokerfile published on GitHub

	https://github.com/ilg-ul/docker/blob/master/builds/d8_x64_mingw/Dockerfile


the long story is documented in a wiki page

	http://gnuarmeclipse.livius.net/wiki/How_to_build_the_original_QEMU_binaries

-------------

after fixing the build details, I was able to further investigate the windows 64-bit failures, which are present on all builds (the original version and Stefan's fork and my fork).

the problem seems to be triggered by the use of SysTick.

I reached this conclusion after spliting my test in several simpler tests; after enabling the timer, the emulator crashes if built for 64-bit.

the public QEMU also crashes when semihosting is used, I guess semihosting support is incomplete because we no longer applied the patches related to the command line. in my fork, semihosting is fully functional, and uses -semihosting-cmdline.


I'll try to check the SysTick code, but I'm not sure when this will happen, since I lost quite a lot of time with these Windows builds.

if someone is interested in the the test images, I can make them public.


regards,

Liviu