Re: [PATCH v5 17/31] target/arm: Enforce alignment for LDM/STM

Re: [PATCH v5 17/31] target/arm: Enforce alignment for LDM/STM

[Nathan Chancellor]

Hi Richard,

On Mon, Apr 19, 2021 at 01:22:43PM -0700, Richard Henderson wrote:
> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  target/arm/translate.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/target/arm/translate.c b/target/arm/translate.c
> index 29fbbb84b2..f58ac4f018 100644
> --- a/target/arm/translate.c
> +++ b/target/arm/translate.c
> @@ -7868,7 +7868,7 @@ static bool op_stm(DisasContext *s, arg_ldst_block *a, int min_n)
>          } else {
>              tmp = load_reg(s, i);
>          }
> -        gen_aa32_st32(s, tmp, addr, mem_idx);
> +        gen_aa32_st_i32(s, tmp, addr, mem_idx, MO_UL | MO_ALIGN);
>          tcg_temp_free_i32(tmp);
>  
>          /* No need to add after the last transfer.  */
> @@ -7943,7 +7943,7 @@ static bool do_ldm(DisasContext *s, arg_ldst_block *a, int min_n)
>          }
>  
>          tmp = tcg_temp_new_i32();
> -        gen_aa32_ld32u(s, tmp, addr, mem_idx);
> +        gen_aa32_ld_i32(s, tmp, addr, mem_idx, MO_UL | MO_ALIGN);
>          if (user) {
>              tmp2 = tcg_const_i32(i);
>              gen_helper_set_user_reg(cpu_env, tmp2, tmp);
> -- 
> 2.25.1

I just bisected a boot hang with an LLVM-built multi_v7_defconfig +
CONFIG_THUMB2_KERNEL=y kernel down to this commit. I do not see the same
hang when the kernel is compiled with GCC 11.2.0 and binutils 2.37 nor
do I see a hang with multi_v7_defconfig by itself. Is there something
that LLVM is doing wrong when compiling/assembling/linking the kernel or
is there something wrong/too aggressive with this commit? I can
reproduce this with current QEMU HEAD (ad22d05833).

My QEMU invocation is:

$ qemu-system-arm \
    -append "console=ttyAMA0 earlycon" \
    -display none \
    -initrd rootfs.cpio \
    -kernel zImage \
    -M virt \
    -m 512m \
    -nodefaults \
    -no-reboot \
    -serial mon:stdio

and the rootfs.cpio and zImage files can be found here:

https://github.com/nathanchance/bug-files/tree/15c1fd6e44622a3c27823d2c5c3083dfc7246146/qemu-2e1f39e29bf9a6b28eaee9fc0949aab50dbad94a

Cheers,
Nathan

Re: [PATCH v5 17/31] target/arm: Enforce alignment for LDM/STM

[Richard Henderson]

On 8/31/21 2:51 AM, Nathan Chancellor wrote:
> I just bisected a boot hang with an LLVM-built multi_v7_defconfig +
> CONFIG_THUMB2_KERNEL=y kernel down to this commit. I do not see the same
> hang when the kernel is compiled with GCC 11.2.0 and binutils 2.37 nor
> do I see a hang with multi_v7_defconfig by itself. Is there something
> that LLVM is doing wrong when compiling/assembling/linking the kernel or
> is there something wrong/too aggressive with this commit? I can
> reproduce this with current QEMU HEAD (ad22d05833).
> 
> My QEMU invocation is:
> 
> $ qemu-system-arm \
>      -append "console=ttyAMA0 earlycon" \
>      -display none \
>      -initrd rootfs.cpio \
>      -kernel zImage \
>      -M virt \
>      -m 512m \
>      -nodefaults \
>      -no-reboot \
>      -serial mon:stdio
> 
> and the rootfs.cpio and zImage files can be found here:
> 
> https://github.com/nathanchance/bug-files/tree/15c1fd6e44622a3c27823d2c5c3083dfc7246146/qemu-2e1f39e29bf9a6b28eaee9fc0949aab50dbad94a

Hmm.  I see

IN:
0xc13038e2:  e890 008c  ldm.w    r0, {r2, r3, r7}

R00=c13077ca R01=c11a8058 R02=c11a8058 R03=c031737f
R04=48379000 R05=00000024 R06=c031748d R07=c03174bb
R08=412fc0f1 R09=c0ce9308 R10=50c5387d R11=00000000
R12=00000009 R13=c1501f88 R14=c0301739 R15=c13038e2
PSR=200001f3 --C- T svc32
Taking exception 4 [Data Abort]
...from EL1 to EL1
...with ESR 0x25/0x9600003f
...with DFSR 0x1 DFAR 0xc13077ca

So, yes, it's a ldm from an address % 4 = 2, so it is correct that we should trap.  You 
should see the same trap on real hw.


r~

Re: [PATCH v5 17/31] target/arm: Enforce alignment for LDM/STM

[Nick Desaulniers]

On Tue, Sep 7, 2021 at 6:44 AM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> On 8/31/21 2:51 AM, Nathan Chancellor wrote:
> > I just bisected a boot hang with an LLVM-built multi_v7_defconfig +
> > CONFIG_THUMB2_KERNEL=y kernel down to this commit. I do not see the same
> > hang when the kernel is compiled with GCC 11.2.0 and binutils 2.37 nor
> > do I see a hang with multi_v7_defconfig by itself. Is there something
> > that LLVM is doing wrong when compiling/assembling/linking the kernel or
> > is there something wrong/too aggressive with this commit? I can
> > reproduce this with current QEMU HEAD (ad22d05833).
> >
> > My QEMU invocation is:
> >
> > $ qemu-system-arm \
> >      -append "console=ttyAMA0 earlycon" \
> >      -display none \
> >      -initrd rootfs.cpio \
> >      -kernel zImage \
> >      -M virt \
> >      -m 512m \
> >      -nodefaults \
> >      -no-reboot \
> >      -serial mon:stdio
> >
> > and the rootfs.cpio and zImage files can be found here:
> >
> > https://github.com/nathanchance/bug-files/tree/15c1fd6e44622a3c27823d2c5c3083dfc7246146/qemu-2e1f39e29bf9a6b28eaee9fc0949aab50dbad94a
>
> Hmm.  I see
>
> IN:
> 0xc13038e2:  e890 008c  ldm.w    r0, {r2, r3, r7}
>
> R00=c13077ca R01=c11a8058 R02=c11a8058 R03=c031737f
> R04=48379000 R05=00000024 R06=c031748d R07=c03174bb
> R08=412fc0f1 R09=c0ce9308 R10=50c5387d R11=00000000
> R12=00000009 R13=c1501f88 R14=c0301739 R15=c13038e2
> PSR=200001f3 --C- T svc32
> Taking exception 4 [Data Abort]
> ...from EL1 to EL1
> ...with ESR 0x25/0x9600003f
> ...with DFSR 0x1 DFAR 0xc13077ca
>
> So, yes, it's a ldm from an address % 4 = 2, so it is correct that we should trap.  You
> should see the same trap on real hw.

Makes sense. I guess if we can find which label that's in, we can look
closer into the code generated by the compiler.
scripts/extract-vmlinux doesn't seem to be able to extract a vmlinux
from either zImage artifact though; not sure yet we'll be able to
disassemble those.

Oh, I guess GDB can show us. Looks like 0xc13038e2 is...hard to tell,
there's no debug info so we just have jumps to addresses in hex, not
symbols.  I don't know my way around GDB well enough to get a sense
for where we are in the source code.
https://gist.github.com/nickdesaulniers/764ac9afab04775846ffa6c90c5a266a

If I rebuild QEMU from source, I don't get any disassembly that looks
similar, probably as a result of different compiler versions, and
maybe adding debug info.

--
Thanks,
~Nick Desaulniers