clang build error on i686

clang build error on i686

[Cole Robinson]

Hi, I'm hitting build errors with clang on i686 userspace on x86_64
kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
clang-12.0.1~rc3-1.fc35.i686.

Full build log from the 6.0.0 build:
https://gist.githubusercontent.com/crobinso/7b1206044eac7326490b2adce829e861/raw/9dddef968051fd6383ba7adb9e595081ad4f8fa4/gistfile1.txt

Lots of errors like:

/usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
function `helper_atomic_cmpxchgq_le_mmu':
/builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
undefined reference to `__atomic_compare_exchange_8'
/usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
function `helper_atomic_xchgq_le_mmu':
/builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
undefined reference to `__atomic_exchange_8'

Also warnings like:

/builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
misaligned atomic operation may incur significant performance penalty;
the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
[-Watomic-alignment]
    uint64_t orig = qatomic_read__nocheck(&s->value);
                    ^
/builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
expanded from macro 'qatomic_read__nocheck'
    __atomic_load_n(ptr, __ATOMIC_RELAXED)

Search the log for __atomic for other hits.

Thanks,
Cole

Re: clang build error on i686

[Philippe Mathieu-Daudé]

On 7/3/21 4:34 PM, Cole Robinson wrote:
> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
> clang-12.0.1~rc3-1.fc35.i686.

> /builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
> misaligned atomic operation may incur significant performance penalty;
> the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
> [-Watomic-alignment]
>     uint64_t orig = qatomic_read__nocheck(&s->value);
>                     ^
> /builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
> expanded from macro 'qatomic_read__nocheck'
>     __atomic_load_n(ptr, __ATOMIC_RELAXED)

Ah I hit this one few months ago using Clang10 to build i386 guest
on mips64el host.

In my notes I see I tried:

-- >8 --
diff --git a/include/qemu/stats64.h b/include/qemu/stats64.h
index fdd3d1b8f98..df9962add4e 100644
--- a/include/qemu/stats64.h
+++ b/include/qemu/stats64.h
@@ -37,7 +37,7 @@ static inline void stat64_init(Stat64 *s, uint64_t value)

 static inline uint64_t stat64_get(const Stat64 *s)
 {
-    return qatomic_read__nocheck(&s->value);
+    return qatomic_read_u64(&s->value);
 }

 static inline void stat64_add(Stat64 *s, uint64_t value)
@@ -47,7 +47,7 @@ static inline void stat64_add(Stat64 *s, uint64_t value)

 static inline void stat64_min(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = qatomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read_u64(&s->value);
     while (orig > value) {
         orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
@@ -55,7 +55,7 @@ static inline void stat64_min(Stat64 *s, uint64_t value)

 static inline void stat64_max(Stat64 *s, uint64_t value)
 {
-    uint64_t orig = qatomic_read__nocheck(&s->value);
+    uint64_t orig = qatomic_read_u64(&s->value);
     while (orig < value) {
         orig = qatomic_cmpxchg__nocheck(&s->value, orig, value);
     }
---

TBH today I don't remember much, I think the change wasn't making much
sense, then I stopped using Clang there.

Ah wait, there is also another one (which I reverted later):

-- >8 --
diff --git a/meson.build b/meson.build
index 372576f82c5..1a5da5ee56b 100644
--- a/meson.build
+++ b/meson.build
@@ -161,6 +161,9 @@
   error('Multipath is supported only on Linux')
 endif

+if 'CONFIG_ATOMIC64' in config_host
+  atomic = cc.find_library('atomic', required: config_host['ARCH'] in
['i386', 'arm', 'riscv32'])
+endif
 m = cc.find_library('m', required: false)
 util = cc.find_library('util', required: false)
 winmm = []
@@ -1534,7 +1537,7 @@
 util_ss = util_ss.apply(config_all, strict: false)
 libqemuutil = static_library('qemuutil',
                              sources: util_ss.sources() +
stub_ss.sources() + genh,
-                             dependencies: [util_ss.dependencies(), m,
glib, socket, malloc])
+                             dependencies: [util_ss.dependencies(),
atomic, m, glib, socket, malloc])
 qemuutil = declare_dependency(link_with: libqemuutil,
                               sources: genh + version_res)

---

No clue neither. Posting in case it ring a bell to someone.

Re: clang build error on i686

[Philippe Mathieu-Daudé]

On 7/3/21 6:20 PM, Philippe Mathieu-Daudé wrote:
> On 7/3/21 4:34 PM, Cole Robinson wrote:
>> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
>> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
>> clang-12.0.1~rc3-1.fc35.i686.
> 
>> /builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
>> misaligned atomic operation may incur significant performance penalty;
>> the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
>> [-Watomic-alignment]
>>     uint64_t orig = qatomic_read__nocheck(&s->value);
>>                     ^
>> /builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
>> expanded from macro 'qatomic_read__nocheck'
>>     __atomic_load_n(ptr, __ATOMIC_RELAXED)
> 
> Ah I hit this one few months ago using Clang10 to build i386 guest
> on mips64el host.

> Ah wait, there is also another one (which I reverted later):
> 
> -- >8 --
> diff --git a/meson.build b/meson.build
> index 372576f82c5..1a5da5ee56b 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -161,6 +161,9 @@
>    error('Multipath is supported only on Linux')
>  endif
> 
> +if 'CONFIG_ATOMIC64' in config_host
> +  atomic = cc.find_library('atomic', required: config_host['ARCH'] in
> ['i386', 'arm', 'riscv32'])
> +endif
>  m = cc.find_library('m', required: false)
>  util = cc.find_library('util', required: false)
>  winmm = []
> @@ -1534,7 +1537,7 @@
>  util_ss = util_ss.apply(config_all, strict: false)
>  libqemuutil = static_library('qemuutil',
>                               sources: util_ss.sources() +
> stub_ss.sources() + genh,
> -                             dependencies: [util_ss.dependencies(), m,
> glib, socket, malloc])
> +                             dependencies: [util_ss.dependencies(),
> atomic, m, glib, socket, malloc])
>  qemuutil = declare_dependency(link_with: libqemuutil,
>                                sources: genh + version_res)
> 
> ---
> 
> No clue neither. Posting in case it ring a bell to someone.

I suppose I dropped it after reading this thread:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg445404.html

Re: clang build error on i686

[Philippe Mathieu-Daudé]

On 7/3/21 4:34 PM, Cole Robinson wrote:
> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
> clang-12.0.1~rc3-1.fc35.i686.
> 
> Full build log from the 6.0.0 build:
> https://gist.githubusercontent.com/crobinso/7b1206044eac7326490b2adce829e861/raw/9dddef968051fd6383ba7adb9e595081ad4f8fa4/gistfile1.txt
> 
> Lots of errors like:
> 
> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> function `helper_atomic_cmpxchgq_le_mmu':
> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
> undefined reference to `__atomic_compare_exchange_8'
> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> function `helper_atomic_xchgq_le_mmu':
> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
> undefined reference to `__atomic_exchange_8'

Marc-André reported this one recently:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg814163.html

Peter once replied smth related:
https://lists.gnu.org/archive/html/qemu-devel/2018-01/msg02103.html

Re: clang build error on i686

[Peter Maydell]

On Sat, 3 Jul 2021 at 15:37, Cole Robinson <crobinso@redhat.com> wrote:
>
> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
> clang-12.0.1~rc3-1.fc35.i686.
>
> Full build log from the 6.0.0 build:
> https://gist.githubusercontent.com/crobinso/7b1206044eac7326490b2adce829e861/raw/9dddef968051fd6383ba7adb9e595081ad4f8fa4/gistfile1.txt
>
> Lots of errors like:
>
> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> function `helper_atomic_cmpxchgq_le_mmu':
> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
> undefined reference to `__atomic_compare_exchange_8'
> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> function `helper_atomic_xchgq_le_mmu':
> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
> undefined reference to `__atomic_exchange_8'
>
> Also warnings like:
>
> /builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
> misaligned atomic operation may incur significant performance penalty;
> the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
> [-Watomic-alignment]
>     uint64_t orig = qatomic_read__nocheck(&s->value);
>                     ^
> /builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
> expanded from macro 'qatomic_read__nocheck'
>     __atomic_load_n(ptr, __ATOMIC_RELAXED)

I think at least part of what is happening here is that this compiler/host
doesn't support native 64-bit atomics, but configure has selected
CONFIG_ATOMIC64 anyway. I've cc'd Paolo and Richard who 'git blame'
suggests have touched the configure test and the stats64.h code in the past.

(The compiler provides fallbacks in libatomic for the 64-bit atomics,
but we deliberately do not link against libatomic because we do not
want to use them.)

thanks
-- PMM

Re: clang build error on i686

[Richard Henderson]

On 7/3/21 10:45 AM, Peter Maydell wrote:
> On Sat, 3 Jul 2021 at 15:37, Cole Robinson <crobinso@redhat.com> wrote:
>>
>> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
>> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
>> clang-12.0.1~rc3-1.fc35.i686.
>>
>> Full build log from the 6.0.0 build:
>> https://gist.githubusercontent.com/crobinso/7b1206044eac7326490b2adce829e861/raw/9dddef968051fd6383ba7adb9e595081ad4f8fa4/gistfile1.txt
>>
>> Lots of errors like:
>>
>> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
>> function `helper_atomic_cmpxchgq_le_mmu':
>> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
>> undefined reference to `__atomic_compare_exchange_8'
>> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
>> function `helper_atomic_xchgq_le_mmu':
>> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
>> undefined reference to `__atomic_exchange_8'
>>
>> Also warnings like:
>>
>> /builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
>> misaligned atomic operation may incur significant performance penalty;
>> the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
>> [-Watomic-alignment]
>>      uint64_t orig = qatomic_read__nocheck(&s->value);
>>                      ^
>> /builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
>> expanded from macro 'qatomic_read__nocheck'
>>      __atomic_load_n(ptr, __ATOMIC_RELAXED)
> 
> I think at least part of what is happening here is that this compiler/host
> doesn't support native 64-bit atomics, but configure has selected
> CONFIG_ATOMIC64 anyway.

Not true.  The host certainly supports it.

This is a new alignment warning in clang-12 wrt the alignment of the atomic operation. 
Which may be complicated by the fact that the i386 abi does not normally align structures 
beyond 4 bytes.

We may need to disable this warning for i386 (but not x86_64).


r~

Re: clang build error on i686

[Peter Maydell]

On Sun, 11 Jul 2021 at 16:17, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 7/3/21 10:45 AM, Peter Maydell wrote:
> > On Sat, 3 Jul 2021 at 15:37, Cole Robinson <crobinso@redhat.com> wrote:
> >>
> >> Hi, I'm hitting build errors with clang on i686 userspace on x86_64
> >> kernel. Affects both qemu 6.0.0 and qemu.git, tested with fedora
> >> clang-12.0.1~rc3-1.fc35.i686.
> >>
> >> Full build log from the 6.0.0 build:
> >> https://gist.githubusercontent.com/crobinso/7b1206044eac7326490b2adce829e861/raw/9dddef968051fd6383ba7adb9e595081ad4f8fa4/gistfile1.txt
> >>
> >> Lots of errors like:
> >>
> >> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> >> function `helper_atomic_cmpxchgq_le_mmu':
> >> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
> >> undefined reference to `__atomic_compare_exchange_8'
> >> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
> >> function `helper_atomic_xchgq_le_mmu':
> >> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
> >> undefined reference to `__atomic_exchange_8'
> >>
> >> Also warnings like:
> >>
> >> /builddir/build/BUILD/qemu-6.0.0/include/qemu/stats64.h:58:21: warning:
> >> misaligned atomic operation may incur significant performance penalty;
> >> the expected alignment (8 bytes) exceeds the actual alignment (4 bytes)
> >> [-Watomic-alignment]
> >>      uint64_t orig = qatomic_read__nocheck(&s->value);
> >>                      ^
> >> /builddir/build/BUILD/qemu-6.0.0/include/qemu/atomic.h:129:5: note:
> >> expanded from macro 'qatomic_read__nocheck'
> >>      __atomic_load_n(ptr, __ATOMIC_RELAXED)
> >
> > I think at least part of what is happening here is that this compiler/host
> > doesn't support native 64-bit atomics, but configure has selected
> > CONFIG_ATOMIC64 anyway.
>
> Not true.  The host certainly supports it.
>
> This is a new alignment warning in clang-12 wrt the alignment of the atomic operation.
> Which may be complicated by the fact that the i386 abi does not normally align structures
> beyond 4 bytes.
>
> We may need to disable this warning for i386 (but not x86_64).

The first part of the problem isn't just a warning, though -- it's clang
actually emitting calls to libatomic.

-- PMM

Re: clang build error on i686

[Richard Henderson]

On 7/11/21 10:50 AM, Peter Maydell wrote:
>>>> Lots of errors like:
>>>>
>>>> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
>>>> function `helper_atomic_cmpxchgq_le_mmu':
>>>> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:86:
>>>> undefined reference to `__atomic_compare_exchange_8'
>>>> /usr/bin/ld: libqemu-aarch64-softmmu.fa.p/accel_tcg_cputlb.c.o: in
>>>> function `helper_atomic_xchgq_le_mmu':
>>>> /builddir/build/BUILD/qemu-6.0.0/accel/tcg/atomic_template.h:134:
>>>> undefined reference to `__atomic_exchange_8'
...
> The first part of the problem isn't just a warning, though -- it's clang
> actually emitting calls to libatomic.

I wasn't able to reproduce this part, only the compile-time warning.


r~