Futex problems with firebird3.0 and openjdk-8 on m68k

Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

Hi!

We're having an odd issue on Debian/m68k with both firebird3.0 and openjdk-8
hanging during build until the build process is killed.

With openjdk-8, it looks like this [1]:

checking if we are using ecj as javac... [debian/buildwatch.sh] compiler/java/jar running ...
[debian/buildwatch.sh] compiler/java/jar running ...
[debian/buildwatch.sh] compiler/java/jar running ...
[debian/buildwatch.sh] compiler/java/jar running ...
[debian/buildwatch.sh] compiler/java/jar running ...
[debian/buildwatch.sh] compiler/java/jar running ...
E: ABORT: Received INT signal (requesting cleanup and shutdown)

Which means that running "javac" just hangs forever. It seems that Thorsten Glaser
has seen this problem in the past before [2] but I don't know whether he ever
managed to fix it. The openjdk-8 package that currently exists for m68k was
actually re-packed by me by replacing libjvm.so from Andreas Schwab's build
(see below why).

And with firebird3.0 we have [3]:

make[4]: Entering directory '/<<PKGBUILDDIR>>/gen'
sh -x -c "lockfile -1 /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock && /<<PKGBUILDDIR>>/gen/Release/firebird/bin/gpre_current -m -z -n
/<<PKGBUILDDIR>>/src/misc/codes.epp /<<PKGBUILDDIR>>/temp/Release/misc/codes.cpp; res=\$?; rm -f /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock; exit
\$res"
+ lockfile -1 /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock
+ /<<PKGBUILDDIR>>/gen/Release/firebird/bin/gpre_current -m -z -n /<<PKGBUILDDIR>>/src/misc/codes.epp /<<PKGBUILDDIR>>/temp/Release/misc/codes.cpp
gpre version LI-V3.0.1.32609 Firebird 3.0
    Version(s) for database "msg.fdb"
        Firebird/Linux/M68k (access method), version "LI-V3.0.1.32609 Firebird 3.0"
        on disk structure version 12.0
+ res=0
+ rm -f /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock
+ exit 0
g++ -I/<<PKGBUILDDIR>>/src/include/gen -I/<<PKGBUILDDIR>>/src/include -Wdate-time -D_FORTIFY_SOURCE=2 -ggdb -O3 -DLINUX -pipe -MMD -fPIC
-DFB_SEND_FLAGS=MSG_NOSIGNAL -pthread -fno-rtti -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security
-std=gnu++98 -fno-lifetime-dse -fno-delete-null-pointer-checks -fno-strict-aliasing  -c /<<PKGBUILDDIR>>/temp/Release/misc/codes.cpp -o
/<<PKGBUILDDIR>>/temp/Release/misc/codes.o
g++  -g -O2 -fdebug-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -Wformat -Werror=format-security -std=gnu++98 -fno-lifetime-dse
-fno-delete-null-pointer-checks -fno-strict-aliasing -fPIE -pie -Wl,-z,relro -Wl,-z,now -pthread    -Wl,--version-script,empty.vers
/<<PKGBUILDDIR>>/temp/Release/misc/codes.o /<<PKGBUILDDIR>>/temp/Release/common.a -o /<<PKGBUILDDIR>>/gen/Release/firebird/bin/codes
-L/<<PKGBUILDDIR>>/gen/Release/firebird/lib -lfbclient -ltommath -latomic_ops -lm -ldl   -lcurses
sh -x -c "lockfile -1 /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock && /<<PKGBUILDDIR>>/gen/Release/firebird/bin/codes
/<<PKGBUILDDIR>>/src/include/gen /<<PKGBUILDDIR>>/lang_helpers; res=\$?; rm -f /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock; exit \$res"
+ lockfile -1 /<<PKGBUILDDIR>>/gen/Release/firebird/bin/build-db.lock
+ /<<PKGBUILDDIR>>/gen/Release/firebird/bin/codes /<<PKGBUILDDIR>>/src/include/gen /<<PKGBUILDDIR>>/lang_helpers
(hangs here)

In both cases, attaching to the running process shows we're hanging in the futex() system call:

root@mama:~# strace -p 28109
strace: Process 28109 attached
futex(0xc5aff4f8, FUTEX_WAIT, 28113, NULL

For the openjdk-8, the issue can be worked around by replacing libjvm.so with a copy
from Andreas Schwab's openjdk-8 build for openSUSE/m68k, so it seems like the JVM
gets miscompiled on Debian/m68k.

Something similar applies to firebird3.0. When building the package in an older
change root (with glibc2.19), the problem [4].

I have tested both on Aranym and qemu-m68k, both show the same behavior. I have
also rebuilt glibc with an upstream patch from Andreas which fixed 64-bit
atomics in glibc [5] but that didn't help either. I also tried rebuilding
glibc on Aranym to make sure it wasn't miscompiled on qemu-m68k on the
buildds.

Does anyone have a suggestion where to look for problems here? I am really
running out of ideas. It's one of the weirdest problems I have ever seen.

Adrian

> [1] https://people.debian.org/~glaubitz/openjdk-8_8u131-b11-1_m68k.build
> [2] https://lists.debian.org/debian-68k/2012/08/msg00023.html
> [3] https://people.debian.org/~glaubitz/firebird3.0_3.0.1.32609.ds4-14_m68k.build
> [4] https://buildd.debian.org/status/fetch.php?pkg=firebird3.0&arch=m68k&ver=3.0.1.32609.ds4-12%2Bb1&stamp=1481424918&raw=0
> [5] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=64ae9fe45662c8994b0e56ab469b01967408a154

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Thorsten Glaser]

John Paul Adrian Glaubitz dixit:

>actually re-packed by me by replacing libjvm.so from Andreas Schwab's build

Ouch.

Does diffing the disassembly of (parts of) that give any meaningful
information? Maybe rather the individual .a files in libjvm.a?

As for the main problem, no idea. I don’t even know what a futex is.
Only that it has something to do with multiple CPUs. (And wasn’t m68k
Linux currently single-CPU only?)

bye,
//mirabilos
-- 
21:49⎜<allamoox:#sendmail> I have a question guys,
     ⎜    Can I use my PC as SMTP server, I use Windows 7 .
     ⎜    Already googled and Installed IIS
     ⎜    but Still I can't send E-mail

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/10/2017 11:21 PM, Thorsten Glaser wrote:
> Does diffing the disassembly of (parts of) that give any meaningful
> information? Maybe rather the individual .a files in libjvm.a?

Hmm, that could be an idea. However, I don't have a static library at
the moment, just the dynamic one.

Here's a tarball containing the working one and the broken one, with
the SUSE one being the one that works:

> https://people.debian.org/~glaubitz/libjvm-m68k.tgz

> As for the main problem, no idea. I don’t even know what a futex is.
> Only that it has something to do with multiple CPUs. (And wasn’t m68k
> Linux currently single-CPU only?)

Well, futexes are used for locking and since the code in question
is capable of multi-threading in general, it would still contain
futex calls, wouldn't it?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Finn Thain]

On Wed, 10 May 2017, John Paul Adrian Glaubitz wrote:

> On 05/10/2017 11:21 PM, Thorsten Glaser wrote:

> > Does diffing the disassembly of (parts of) that give any meaningful 
> > information? Maybe rather the individual .a files in libjvm.a?
> 
> Hmm, that could be an idea. However, I don't have a static library at 
> the moment, just the dynamic one.
> 
> Here's a tarball containing the working one and the broken one, with the 
> SUSE one being the one that works:
> 
> > https://people.debian.org/~glaubitz/libjvm-m68k.tgz
> 

Or maybe a race condition in the kernel or glibc futex implementation? If 
so, the problem won't be apparent from comparing libjvm builds. Are there 
any other known issues with libjvm on m68k?

I can run the kernel futex test suite under QEMU once this debootstrap 
issue is sorted:

dpkg: error processing package mac-fdisk (--configure):
 subprocess installed post-installation script returned error exit status 2
Errors were encountered while processing:
 mac-fdisk
Setting up mac-fdisk (0.1-16+b1) ...
dpkg: error: unknown option --print-installation-architecture

(Or maybe on some other system, given remote access.)

-- 

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/11/2017 04:46 AM, Finn Thain wrote:
> Or maybe a race condition in the kernel or glibc futex implementation?

On qemu-m68k, the kernel isn't even running. And on Aranym, I'm running
the same kernel on which it works with an older chroot. And, as I said,
when I replace the libjvm.so itself, it works. So, it seems the library
is getting somehow miscompiled.

> If so, the problem won't be apparent from comparing libjvm builds. Are there 
> any other known issues with libjvm on m68k?

Not that I know of. However, since I haven't managed yet to do a clean
openjdk build on Debian/m68k, I can't really say.

> I can run the kernel futex test suite under QEMU once this debootstrap 
> issue is sorted:
> 
> dpkg: error processing package mac-fdisk (--configure):
>  subprocess installed post-installation script returned error exit status 2
> Errors were encountered while processing:
>  mac-fdisk
> Setting up mac-fdisk (0.1-16+b1) ...

I'm not sure why you keep including mac-fdisk in your debbootstrap run,
it's not required at all. A regular debootstrap works without problems,
it's performed regularly on the buildds:

# debootstrap --no-check-gpg --include=apt,nano,aptitude,vim unstable unstable-m68k ftp://ftp.debian-ports.org/debian

Or, when cross-bootstrapping:

# debootstrap --no-check-gpg --arch=m68k --foreign --include=apt,nano,aptitude,vim unstable unstable-m68k ftp://ftp.debian-ports.org/debian

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Finn Thain]

On Thu, 11 May 2017, John Paul Adrian Glaubitz wrote:

> On 05/11/2017 04:46 AM, Finn Thain wrote:
> > Or maybe a race condition in the kernel or glibc futex implementation?
> 
> On qemu-m68k, the kernel isn't even running.

That's promising, but it doesn't eliminate glibc as a potential problem.
Do we have glibc test suite results?

> And on Aranym, I'm running the same kernel on which it works with an 
> older chroot. And, as I said, when I replace the libjvm.so itself, it 
> works. So, it seems the library is getting somehow miscompiled.
> 

... and hence my race condition hypothesis. But I'm guessing.

> > If so, the problem won't be apparent from comparing libjvm builds. Are 
> > there any other known issues with libjvm on m68k?
> 
> Not that I know of. However, since I haven't managed yet to do a clean 
> openjdk build on Debian/m68k, I can't really say.
> 
> > I can run the kernel futex test suite under QEMU once this debootstrap 
> > issue is sorted:
> > 
> > dpkg: error processing package mac-fdisk (--configure):
> >  subprocess installed post-installation script returned error exit status 2
> > Errors were encountered while processing:
> >  mac-fdisk
> > Setting up mac-fdisk (0.1-16+b1) ...
> 
> I'm not sure why you keep including mac-fdisk in your debbootstrap run,

AFAIK I didn't include it. How would it get included?

> it's not required at all.

It shouldn't be; it's not useful on most m68k platforms.

> A regular debootstrap works without problems, it's performed regularly 
> on the buildds:
> 
> # debootstrap --no-check-gpg --include=apt,nano,aptitude,vim unstable unstable-m68k ftp://ftp.debian-ports.org/debian
> 
> Or, when cross-bootstrapping:
> 
> # debootstrap --no-check-gpg --arch=m68k --foreign --include=apt,nano,aptitude,vim unstable unstable-m68k ftp://ftp.debian-ports.org/debian
> 

Here's the command I used:

debootstrap --arch=m68k --foreign --no-check-gpg --include=apt,nano unstable /export/debian-68k http://ftp.de.debian.org/debian-ports

Full disclosure:

# which debootstrap
/usr/sbin/debootstrap
# equery belongs /usr/sbin/debootstrap
 * Searching for /usr/sbin/debootstrap ... 
dev-util/debootstrap-1.0.87 (/usr/sbin/debootstrap)
# 

Yep, it's Gentoo. But to be fair, it performed a native debootstrap just 
fine. Next time I'll use the native chroot to debootstrap the m68k one.

-- 

> Adrian
> 
> 

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/11/2017 08:27 AM, Finn Thain wrote:
> That's promising, but it doesn't eliminate glibc as a potential problem.
> Do we have glibc test suite results?

Well, it's potentially a glibc problem. But I haven't fully ruled out
other packages yet. All I know is that it works with an older chroot
which has glibc-2.19. But it may be related to gcc as well, no idea.

>> And on Aranym, I'm running the same kernel on which it works with an 
>> older chroot. And, as I said, when I replace the libjvm.so itself, it 
>> works. So, it seems the library is getting somehow miscompiled.
>>
> 
> ... and hence my race condition hypothesis. But I'm guessing.

But wouldn't it then show in the old chroot sometimes as well? It never
shows there, just with the recent chroot.

>>> I can run the kernel futex test suite under QEMU once this debootstrap 
>>> issue is sorted:
>>>
>>> dpkg: error processing package mac-fdisk (--configure):
>>>  subprocess installed post-installation script returned error exit status 2
>>> Errors were encountered while processing:
>>>  mac-fdisk
>>> Setting up mac-fdisk (0.1-16+b1) ...
>>
>> I'm not sure why you keep including mac-fdisk in your debbootstrap run,
> 
> AFAIK I didn't include it. How would it get included?

It shouldn't. It's not necessary at all. Try bootstrapping with --variant=minimal.

> Here's the command I used:
> 
> debootstrap --arch=m68k --foreign --no-check-gpg --include=apt,nano unstable /export/debian-68k http://ftp.de.debian.org/debian-ports

Mmh, it doesn't work at the moment because I uploaded a patched glibc package
to "unreleased" and debootstrap doesn't support that. You need a patched
version of debootstrap to make this work.

> Yep, it's Gentoo. But to be fair, it performed a native debootstrap just 
> fine. Next time I'll use the native chroot to debootstrap the m68k one.

Unless Gentoo modified debootstrap in some way, that shouldn't be a problem.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On Thu, May 11, 2017 at 08:38:42AM +0200, John Paul Adrian Glaubitz wrote:
> Mmh, it doesn't work at the moment because I uploaded a patched glibc package
> to "unreleased" and debootstrap doesn't support that. You need a patched
> version of debootstrap to make this work.

FWIW, you can just download this older chroot I created in 2015 [1],
unpack it, copy qemu-m68k-static into the usr/bin sub-directory and
perform a dist-upgrade.

Use this updated sources.list [2], the old one doesn't work anymore.

Btw, if you build firebird3.0 in this chroot without updating the
glibc, the problem will actually go away. However, since there are a
lot of other packages tied to the version of glibc, especially Perl, I
cannot rule out it's not some other package causing the issue.

Adrian

> [1] https://people.debian.org/~glaubitz/chroots/unstable-m68k-20150315.tar.gz
> [2] https://people.debian.org/~glaubitz/sources.list

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Finn Thain]

On Thu, 11 May 2017, John Paul Adrian Glaubitz wrote:

> On Thu, May 11, 2017 at 08:38:42AM +0200, John Paul Adrian Glaubitz 
> wrote:
> > Mmh, it doesn't work at the moment because I uploaded a patched glibc 
> > package to "unreleased" and debootstrap doesn't support that. You need 
> > a patched version of debootstrap to make this work.
> 
> FWIW, you can just download this older chroot I created in 2015 [1], 
> unpack it, copy qemu-m68k-static into the usr/bin sub-directory and 
> perform a dist-upgrade.
> 

OK.

I'm using Laurent's QEMU in Quadra 800 system mode (not user mode) which 
is a bit inefficient. But it performs well and it will exercise kernel 
code which is of interest to me.

> Use this updated sources.list [2], the old one doesn't work anymore.
> 

Done.

> Btw, if you build firebird3.0 in this chroot without updating the glibc, 
> the problem will actually go away.

OK, I will try that too. But first I will try to reproduce the bug.

What's the quickest way to reproduce the build failure for the firebird3.0 
deb?

> However, since there are a lot of other packages tied to the version of 
> glibc, especially Perl, I cannot rule out it's not some other package 
> causing the issue.
> 

The dist-upgrade finished without any problems. Thanks!

root@panmac:~# dpkg-query -W
acl     2.2.52-3
adduser 3.115
apt     1.4.4
apt-utils       1.4.4
aptitude        0.8.7-1
aptitude-common 0.8.7-1
base-files      9.9
base-passwd     3.5.43
bash    4.4-5
bsd-mailx       8.1.2-0.20160123cvs-4
bsdmainutils    9.0.12+nmu1
bsdutils        1:2.29.2-1
coreutils       8.26-3
cpio    2.11+dfsg-6
cron    3.0pl1-128
dash    0.5.8-2.4
debconf 1.5.60
debconf-i18n    1.5.60
debian-archive-keyring  2014.3
debianutils     4.8.1.1
delo    0.8-2.6
diffutils       1:3.5-3
dirmngr 2.1.18-8
dmsetup 2:1.02.137-2
dpkg    1.18.24
e2fslibs:m68k   1.43.4-2
e2fsprogs       1.43.4-2
exim4-base      4.89-2
exim4-config    4.89-2
exim4-daemon-light      4.89-2
findutils       4.6.0+git+20161106-2
gcc-4.7-base:m68k       4.7.4-3
gcc-4.8-base:m68k       4.8.4-1
gcc-4.9-base:m68k       4.9.2-10
gcc-6-base:m68k 6.3.0-18
gnupg   2.1.18-8
gnupg-agent     2.1.18-8
gnupg-l10n      2.1.18-8
gpgv    2.1.18-8
grep    2.27-2
groff-base      1.22.3-9
gzip    1.6-5
hostname        3.18
ifupdown        0.8.19
init    1.48
init-system-helpers     1.48
initscripts     2.88dsf-59.9
insserv 1.14.0-5.4
iproute2        4.9.0-1
iptables        1.6.0+snapshot20161117-6
iputils-ping    3:20161105-1
isc-dhcp-client 4.3.5-3
isc-dhcp-common 4.3.5-3
kmod    24-1
less    481-2.1
libacl1:m68k    2.2.52-3
libapparmor1:m68k       2.11.0-3
libapt-inst1.5:m68k     1.0.9.7
libapt-inst2.0:m68k     1.4.4
libapt-pkg4.12:m68k     1.0.9.7
libapt-pkg5.0:m68k      1.4.4
libassuan0:m68k 2.4.3-2
libattr1:m68k   1:2.4.47-2
libaudit-common 1:2.6.7-2
libaudit1:m68k  1:2.6.7-2
libblkid1:m68k  2.29.2-1
libboost-filesystem1.62.0:m68k  1.62.0+dfsg-4
libboost-iostreams1.49.0        1.49.0-4+b1
libboost-iostreams1.62.0:m68k   1.62.0+dfsg-4
libboost-system1.62.0:m68k      1.62.0+dfsg-4
libbsd0:m68k    0.8.3-1
libbz2-1.0:m68k 1.0.6-8.1
libc-bin        2.24-10+m68k
libc6:m68k      2.24-10+m68k
libcap-ng0:m68k 0.7.7-3
libcap2:m68k    1:2.25-1
libcap2-bin     1:2.25-1
libcomerr2:m68k 1.43.4-2
libcryptsetup4:m68k     2:1.7.3-4
libcwidget3:m68k        0.5.17-2
libcwidget3v5:m68k      0.5.17-4+b1
libdb5.3:m68k   5.3.28-12+b1
libdebconfclient0:m68k  0.227
libdevmapper1.02.1:m68k 2:1.02.137-2
libdns-export100        1:9.9.5.dfsg-9
libdns-export162        1:9.10.3.dfsg.P4-12.3
libelf1:m68k    0.168-0.2
libestr0        0.1.10-2
libfastjson4:m68k       0.99.4-1
libfdisk1:m68k  2.29.2-1
libffi6:m68k    3.2.1-6
libgcc2:m68k    1:6.3.0-18
libgcrypt20:m68k        1.7.6-1
libgdbm3:m68k   1.8.3-14
libgmp10:m68k   2:6.1.2+dfsg-1
libgnutls-deb0-28:m68k  3.3.8-2
libgnutls-openssl27:m68k        3.5.8-5
libgnutls30:m68k        3.5.8-5
libgpg-error0:m68k      1.26-2
libgpm2:m68k    1.20.4-6.2
libhogweed2:m68k        2.7.1-5
libhogweed4:m68k        3.3-1
libicu52:m68k   52.1-4
libidn11:m68k   1.33-1
libidn2-0:m68k  0.16-1
libip4tc0:m68k  1.6.0+snapshot20161117-6
libip6tc0:m68k  1.6.0+snapshot20161117-6
libiptc0:m68k   1.6.0+snapshot20161117-6
libirs-export91 1:9.9.5.dfsg-9
libisc-export160        1:9.10.3.dfsg.P4-12.3
libisc-export95 1:9.9.5.dfsg-9
libisccfg-export90      1:9.9.5.dfsg-9
libjson-c2:m68k 0.11-4
libkmod2:m68k   24-1
libksba8:m68k   1.3.5-2
libldap-2.4-2:m68k      2.4.44+dfsg-4+b1
libldap-common  2.4.44+dfsg-4
liblocale-gettext-perl  1.07-3+b1
liblockfile-bin 1.14-1
liblockfile1:m68k       1.14-1
liblogging-stdlog0:m68k 1.0.6-1
liblognorm1:m68k        1.0.1-3
liblognorm5:m68k        2.0.1-1.1
liblz4-1:m68k   0.0~r131-2
liblzma5:m68k   5.2.2-1.2
libmnl0:m68k    1.0.4-2
libmount1:m68k  2.29.2-1
libncurses5:m68k        6.0+20161126-1
libncursesw5:m68k       6.0+20161126-1
libnetfilter-acct1:m68k 1.0.2-1.1
libnetfilter-conntrack3:m68k    1.0.6-2
libnettle4:m68k 2.7.1-5
libnettle6:m68k 3.3-1
libnewt0.52:m68k        0.52.20-1
libnfnetlink0:m68k      1.0.1-3
libnpth0:m68k   1.3-1
libp11-kit0:m68k        0.23.3-5
libpam-modules:m68k     1.1.8-3.5
libpam-modules-bin      1.1.8-3.5
libpam-runtime  1.1.8-3.5
libpam0g:m68k   1.1.8-3.5
libpcre3:m68k   2:8.39-3
libpipeline1:m68k       1.4.1-2
libpopt0:m68k   1.16-10+b2
libprocps3:m68k 2:3.3.9-8
libprocps6:m68k 2:3.3.12-3
libpsl0:m68k    0.5.1-1
libpsl5:m68k    0.17.0-4
libreadline6:m68k       6.3-9
libreadline7:m68k       7.0-3
libsasl2-2:m68k 2.1.27~101-g0780600+dfsg-3
libsasl2-modules:m68k   2.1.27~101-g0780600+dfsg-3
libsasl2-modules-db:m68k        2.1.27~101-g0780600+dfsg-3
libselinux1:m68k        2.6-3
libsemanage-common      2.6-2
libsemanage1:m68k       2.6-2
libsepol1:m68k  2.6-2
libsigc++-2.0-0c2a:m68k 2.4.0-1
libsigc++-2.0-0v5:m68k  2.10.0-1
libslang2:m68k  2.3.1-5
libsmartcols1:m68k      2.29.2-1
libsqlite3-0:m68k       3.16.2-3
libss2:m68k     1.43.4-2
libssl1.0.0:m68k        1.0.1k-1
libssl1.0.2:m68k        1.0.2k-1
libssl1.1:m68k  1.1.0e-2
libstdc++6:m68k 6.3.0-18
libsystemd0:m68k        232-23
libtasn1-6:m68k 4.10-1
libtext-charwidth-perl  0.04-7+b5
libtext-iconv-perl      1.7-5+b5
libtext-wrapi18n-perl   0.06-7.1
libtinfo5:m68k  6.0+20161126-1
libudev0:m68k   175-7.2
libudev1:m68k   232-23
libunistring0:m68k      0.9.6+really0.9.3-0.1
libusb-0.1-4:m68k       2:0.1.12-31
libustr-1.0-1:m68k      1.0.4-6
libuuid1:m68k   2.29.2-1
libxapian22     1.2.18-1
libxapian30:m68k        1.4.3-2
libxtables10    1.4.21-2
libxtables12:m68k       1.6.0+snapshot20161117-6
login   1:4.4-4
logrotate       3.11.0-0.1
lsb-base        9.20161125
mac-fdisk       0.1-16+b1
man-db  2.7.6.1-2
manpages        4.10-2
mawk    1.3.3-17
mount   2.29.2-1
multiarch-support       2.24-10+m68k
nano    2.7.4-1
ncurses-base    6.0+20161126-1
ncurses-bin     6.0+20161126-1
net-tools       1.60+git20161116.90da8a0-1
netbase 5.4
netcat-traditional      1.10-41.1
nfacct  1.0.1-1.1
passwd  1:4.4-4
perl-base       5.24.1-2
pinentry-curses 1.0.0-2
procps  2:3.3.12-3
psmisc  22.21-2.1
publicsuffix    20170424.0717-1
readline-common 7.0-3
rsyslog 8.24.0-1
sed     4.4-1
sensible-utils  0.0.9
startpar        0.59-3.1
systemd 232-23
systemd-sysv    232-23
sysv-rc 2.88dsf-59.9
sysvinit-utils  2.88dsf-59.9
tar     1.29b-1.1
tasksel 3.39
tasksel-data    3.39
traceroute      1:2.1.0-2
tzdata  2017b-1
udev    232-23
util-linux      2.29.2-1
vim     2:8.0.0197-4
vim-common      2:8.0.0197-4
vim-runtime     2:8.0.0197-4
vim-tiny        2:8.0.0197-4
wget    1.19.1-3
whiptail        0.52.20-1
xxd     2:8.0.0197-4
zlib1g:m68k     1:1.2.8.dfsg-5

-- 

> Adrian
> 
> > [1] https://people.debian.org/~glaubitz/chroots/unstable-m68k-20150315.tar.gz
> > [2] https://people.debian.org/~glaubitz/sources.list
> 
> 

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/20/2017 08:53 AM, Finn Thain wrote:
>> Btw, if you build firebird3.0 in this chroot without updating the glibc, 
>> the problem will actually go away.
> 
> OK, I will try that too. But first I will try to reproduce the bug.
> 
> What's the quickest way to reproduce the build failure for the firebird3.0 
> deb?

1. Make sure you have the necessary "deb-src" entries in your sources.list:

# binary default
deb http://ftp.ports.debian.org/debian-ports/ unstable main
deb http://incoming.ports.debian.org/buildd/ unstable main
deb http://ftp.ports.debian.org/debian-ports/ unreleased main

# source
deb-src http://ftp.debian.org/debian/ unstable main
deb-src http://incoming.debian.org/debian-buildd/ buildd-unstable main

2. Run "apt update"

3. apt install devscripts && apt build-dep firebird3.0

4. dget -u http://deb.debian.org/debian/pool/main/f/firebird3.0/firebird3.0_3.0.2.32703.ds4-1.dsc

5. cd firebird3.0-3.0.2.32703.ds4/

6. ./debian/rules binary-arch

It should hang at some point when running the compiled binary on Aranym
or qemu-system. On qemu-user, it will just crash but for other reasons.

Although I didn't test it with qemu-system, I am very confident it should
hang the same way as on Aranym. The qemu-user crash is related to the
broken atomics in qemu-user.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Andreas Schwab]

On Mai 21 2017, John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> wrote:

> 5. cd firebird3.0-3.0.2.32703.ds4/
>
> 6. ./debian/rules binary-arch
>
> It should hang at some point when running the compiled binary on Aranym
> or qemu-system. On qemu-user, it will just crash but for other reasons.

Firebird 3.0.1.32609 built successfully on aranym for me.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/21/2017 09:31 AM, Andreas Schwab wrote:
> Firebird 3.0.1.32609 built successfully on aranym for me.

Yes, and I am fully aware of that. It also used to built fine on Debian
but it no longer does and we need to figure out why. There is a bug hidden
somewhere that you apparently haven't run into yet.

Be it because you built glibc or the toolchain with different options or
be it because you're using an older version of any of the relevant packages
as compared to Debian.

All I know so far is that it builds fine in an outdated chroot [1] on
Aranym and it breaks the moment one performs a dist-upgrade. I haven't
done enough testing yet though that I can say which package it is
the cause in particular. It's unfortunately not so easy to selectively
install an older version of glibc as all of Perl is built against it
and downgrading glibc is therefore not trivially possible.

Adrian

> [1] https://people.debian.org/~glaubitz/chroots/unstable-m68k-20150315.tar.gz

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[John Paul Adrian Glaubitz]

On 05/21/2017 01:15 PM, John Paul Adrian Glaubitz wrote:
> On 05/21/2017 09:31 AM, Andreas Schwab wrote:
>> Firebird 3.0.1.32609 built successfully on aranym for me.
> 
> Yes, and I am fully aware of that. It also used to built fine on Debian
> but it no longer does and we need to figure out why. There is a bug hidden
> somewhere that you apparently haven't run into yet.

It seems pypy is affected as well [1]:

********#%*********%#%%%%%[144e6] translation-task}

[translation:info] inserting stack checks...
[144f8] {translation-task
starting stackcheckinsertion_lltype
**
[rtyper] specializing: 190000 / 190002 blocks   (99%)
[rtyper] -=- specialized 7 more blocks -=-
E: Build killed with signal TERM after 284 minutes of inactivity

Built tested on Aranym. Or maybe the timeout was insufficent.

Adrian

> [1] https://people.debian.org/~glaubitz/pypy_5.6.0+dfsg-4_m68k.build

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Futex problems with firebird3.0 and openjdk-8 on m68k

[Finn Thain]

On Sun, 21 May 2017, John Paul Adrian Glaubitz wrote:
> On 05/20/2017 08:53 AM, Finn Thain wrote:
> >> Btw, if you build firebird3.0 in this chroot without updating the 
> >> glibc, the problem will actually go away.
> > 
> > OK, I will try that too. But first I will try to reproduce the bug.
> > 
> > What's the quickest way to reproduce the build failure for the 
> > firebird3.0 deb?
> 
> 1. Make sure you have the necessary "deb-src" entries in your 
>    sources.list:
> 
> # binary default
> deb http://ftp.ports.debian.org/debian-ports/ unstable main
> deb http://incoming.ports.debian.org/buildd/ unstable main
> deb http://ftp.ports.debian.org/debian-ports/ unreleased main
> 
> # source
> deb-src http://ftp.debian.org/debian/ unstable main
> deb-src http://incoming.debian.org/debian-buildd/ buildd-unstable main
> 
> 2. Run "apt update"
> 
> 3. apt install devscripts && apt build-dep firebird3.0
> 
> 4. dget -u http://deb.debian.org/debian/pool/main/f/firebird3.0/firebird3.0_3.0.2.32703.ds4-1.dsc
> 
> 5. cd firebird3.0-3.0.2.32703.ds4/
> 
> 6. ./debian/rules binary-arch
> 
> It should hang at some point when running the compiled binary on Aranym 
> or qemu-system. On qemu-user, it will just crash but for other reasons.
> 
> Although I didn't test it with qemu-system, I am very confident it 
> should hang the same way as on Aranym. The qemu-user crash is related to 
> the broken atomics in qemu-user.
> 

The build didn't hang when I ran it (twice) on vanilla kernel v4.11.0 on 
Laurent's qemu-system. This rootfs is fully up-to-date, as I described in 
earlier in this thread.

The build stops here:

[snip]
touch build-firebird-stamp
touch build-stamp
dh_testdir
dh_testroot
dh_prep
dh_installdirs
sh debian/make_packages.sh
Creating firebird3.0-server-core content
Creating firebird3.0-server content
stdin not open for reading!
stdin not open for reading!
dh_installman: man --recode UTF-8 ./fbtracemgr\.1 > fbtracemgr\.1\.new 
returned exit code 2
debian/rules:217: recipe for target 'install-stamp' failed
make: *** [install-stamp] Error 2

Let me know if you would like me to send the entire log.

It may be that following the same steps would yield the same result under 
Aranym that I got under qemu (being that Andreas was able to build 
3.0.1.32609 under Aranym).

Or it may be that you, Andreas and I have attempted to build three 
different versions of firebird3.0, meaning that our results can't be 
compared.

But my guess is that your build chroots and/or their host kernel is 
different to this system in some small but significant way.

P.S. This thread could perhaps be confined to the debian-68k list?

-- 

> Adrian
> 
>