Re: [Qemu-devel] [qemu-s390x] linux-user: s390x issue on Fedora 30 (dynamic library loader?)

[David Hildenbrand <david@redhat.com>]

On 19.08.19 13:55, David Hildenbrand wrote:
> On 19.08.19 12:32, Laurent Vivier wrote:
>> Le 17/08/2019 à 18:51, David Hildenbrand a écrit :
>>> On 17.08.19 18:22, Laurent Vivier wrote:
>>>> Le 17/08/2019 à 18:14, David Hildenbrand a écrit :
>>>>> On 17.08.19 17:59, David Hildenbrand wrote:
>>>>>> Hi everybody,
>>>>>>
>>>>>> I was just trying to run qemu-s390x (linux-user) with a very simple
>>>>>> binary (gzip + lib/ld64.so.1, compiled under Fedora 27). This used to
>>>>>> work just fine a while ago (especially when I was working on vector
>>>>>> instructions using QEMU v3.1). However, now I can't get past a SEGFAULT
>>>>>> in the dynamic library loader (I assume it is trying to locate glibc). I
>>>>>> tried a couple of other binaries that definitely used to work (from
>>>>>> Fedora 30).
>>>>>>
>>>>>> I checked QEMU v4.1, v4.0 and v3.1. All are broken for me. Which is
>>>>>> weird - because it used to work :/
>>>>>>
>>>>>> I remember that I was running Fedora 29 the last time I had it running,
>>>>>> so my gut feeling is that this is related to some other system library
>>>>>> (but which?). I am running on an up-to-date Fedora 30 x86-64 now.
>>>>>>
>>>>>> Any ideas? Has this been reported already? (not sure if this is a Fedora
>>>>>> 30 issue)
>>>>>>
>>>>>> LANG=C ~/git/qemu/s390x-linux-user/qemu-s390x -d in_asm -L . gzip --help
>>>>>>
>>>>>> ----------------
>>>>>> IN: _dl_load_cache_lookup
>>>>>> 0x00000040008854c2:  larl       %r1,0x4000895030
>>>>>> 0x00000040008854c8:  lg %r8,264(%r11)
>>>>>> 0x00000040008854ce:  mvghi      0(%r1),-1
>>>>>> 0x00000040008854d4:  la %r3,0(%r3,%r8)
>>>>>> 0x00000040008854d8:  l  %r7,12(%r8)
>>>>>> 0x00000040008854dc:  llgfr      %r2,%r7
>>>>>> 0x00000040008854e0:  sllg       %r1,%r2,1
>>>>>> 0x00000040008854e6:  agr        %r1,%r2
>>>>>> 0x00000040008854ea:  sllg       %r1,%r1,2
>>>>>> 0x00000040008854f0:  la %r6,16(%r1,%r8)
>>>>>> 0x00000040008854f4:  sgr        %r3,%r6
>>>>>> 0x00000040008854f8:  stg        %r3,256(%r11)
>>>>>> 0x00000040008854fe:  ahi        %r7,-1
>>>>>> 0x0000004000885502:  jl 0x40008850f0
>>>>>>
>>>>>> ----------------
>>>>>> IN: _dl_load_cache_lookup
>>>>>> 0x0000004000885506:  srak       %r10,%r7,1
>>>>>> 0x000000400088550c:  lgfr       %r2,%r10
>>>>>> 0x0000004000885510:  sllg       %r1,%r2,1
>>>>>> 0x0000004000885516:  agr        %r1,%r2
>>>>>> 0x000000400088551a:  sllg       %r1,%r1,2
>>>>>> 0x0000004000885520:  l  %r1,20(%r1,%r8)
>>>>>> 0x0000004000885524:  clrjhe     %r1,%r3,0x40008850f0
>>>>>>
>>>>>> Segmentation fault (Speicherabzug geschrieben)
>>>>>>
>>>>>>
>>>>>> Core was generated by
>>>>>> `/home/dhildenb/git/qemu/s390x-linux-user/qemu-s390x -d in_asm -L . gzip
>>>>>> --help'.
>>>>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>>>>> #0  0x00007fdc5d7c3232 in sigsuspend () from /lib64/libc.so.6
>>>>>> [Current thread is 1 (Thread 0x7fdc5d7127c0 (LWP 31072))]
>>>>>> Missing separate debuginfos, use: dnf debuginfo-install
>>>>>> glib2-2.60.6-1.fc30.x86_64 glibc-2.29-15.fc30.x86_64
>>>>>> libgcc-9.1.1-1.fc30.x86_64 libstdc++-9.1.1-1.fc30.x86_64
>>>>>> pcre-8.43-2.fc30.x86_64 zlib-1.2.11-16.fc30.x86_64
>>>>>> (gdb) bt
>>>>>> #0  0x00007fdc5d7c3232 in sigsuspend () from /lib64/libc.so.6
>>>>>> #1  0x000055f826135a9c in dump_core_and_abort
>>>>>> (target_sig=target_sig@entry=11)
>>>>>>     at /home/dhildenb/git/qemu/linux-user/signal.c:613
>>>>>> #2  0x000055f826135e37 in handle_pending_signal
>>>>>> (cpu_env=cpu_env@entry=0x55f8292cec48, sig=sig@entry=11,
>>>>>>     k=k@entry=0x55f8292d7df0) at
>>>>>> /home/dhildenb/git/qemu/linux-user/signal.c:877
>>>>>> #3  0x000055f826136edd in process_pending_signals
>>>>>> (cpu_env=cpu_env@entry=0x55f8292cec48)
>>>>>>     at /home/dhildenb/git/qemu/linux-user/signal.c:953
>>>>>> #4  0x000055f82613a13a in cpu_loop (env=0x55f8292cec48) at
>>>>>> /home/dhildenb/git/qemu/linux-user/s390x/cpu_loop.c:150
>>>>>> #5  0x000055f8260ce2ba in main (argc=<optimized out>,
>>>>>> argv=0x7fff587a69d8, envp=<optimized out>)
>>>>>>     at /home/dhildenb/git/qemu/linux-user/main.c:819
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>
>>>>> CCing QEMU-devel + use my proper dev mail address (I need more coffee :))
>>>>>
>>>>
>>>> I generally test qemu-s390x before my PR. Last time, I have tested with
>>>> Fedora 30 x86_64 but my target are always debian.
>>>
>>> What puzzles me is that it used to work just fine about 3-4 months ago.
>>> I still have the binaries/libs lying around that I used back then (when
>>> debugging a vector instruction-related issue). Whatever binary/QEMU
>>> version I try now, it keeps segfaulting.
>>>
>>> Via qemu-system-s390x, inside a Fedora guest, the binaries work
>>> perfectly fine. So I really suspect that this has to do with my host system.
>>>
>>>>
>>>> So I guess the problem is with the target.
>>>>
>>>> I will have a look on Monday.
>>>
>>
>> I'm not able to reproduce it. In a chroot or using "-L" with only
>> /bin/gzip, /lib64/ld64.so.1 and /lib/libc.so.6 in the directory.
>>
>> My host is Fedora 30 (updated today) and the target directory is Fedora
>> 30 too (from
>> https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/30/Container/s390x/images/Fedora-Container-Minimal-Base-30-1.2.s390x.tar.xz)
>>
>> Do you have more details?
> 
> Thanks for having a look!
> 
> I just tried with the Fedora 30 target files they seem to work just fine
> indeed (chroot and -L after manually copying the two lib files).
> 
> Now, I took the same files from RHEL8 (everything is built with
> z13/vector instruction support enforced under RHEL8). (I copied them
> from a running guest using scp) - rhel-guest-image-8.0-1854.s390x.qcow2.
> 
> 
> Trying to run them results in the reported issue.
> 
> ~/git/qemu/s390x-linux-user/qemu-s390x  -L . gzip
> Segmentation fault (Speicherabzug geschrieben)
> 
> I can spot one main difference:
> 
> /lib64/libc-2.28.so: ELF 64-bit MSB shared object, IBM S/390, version 1
> (GNU/Linux), dynamically linked, interpreter /lib/ld64.so.1,
> BuildID[sha1]=f2ed86061df0bad28270244424e58eb95f60c00c, for GNU/Linux
> 3.2.0, not stripped, too many notes (256)
> 
> /lib64/ld-2.28.so: ELF 64-bit MSB shared object, IBM S/390, version 1
> (SYSV), dynamically linked,
> BuildID[sha1]=6a58fd6ea86ec36455655ff718509ee4320cefc2, not stripped,
> too many notes (256)
> 
> The libraries are not completely stripped. But even when stripping, I
> get the same issue when trying to load them.
> 
> The binaries/libraries work perfectly fine within qemu-system-s390x.
> 

One more thing:

Replacing /lib/ld64.so.1 from RHEL-8 by the one from Fedora 30 does the
trick for me. The libc from RHEL-8 can be loaded and used sucessfully.

-- 

Thanks,

David / dhildenb