pxa crashes with qemu v5.2 when executing xscale operations

pxa crashes with qemu v5.2 when executing xscale operations

[Guenter Roeck]

Hi,

I noticed that booting Linux on PXA emulations no longer works with qemu v5.2.
When trying to boot akita, borzoi, or similar emulations, I get the following crash.

[    0.965279] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT ARM
[    0.967273] Modules linked in:
[    0.967875] CPU: 0 PID: 1 Comm: swapper Not tainted 5.10.2-rc1-00017-gc96cfd687a3f #1
[    0.968101] Hardware name: SHARP Akita
[    0.968676] PC is at xscale_cp0_init+0x84/0x114
[    0.968815] LR is at do_one_initcall+0x60/0x290
[    0.968997] pc : [<c08ce068>]    lr : [<c000a2dc>]    psr: 60000013
[    0.969186] sp : c0bdfec8  ip : c0bdfee0  fp : c0bdfedc
[    0.969332] r10: c08fe834  r9 : c09f2000  r8 : c08c6a9c
[    0.969498] r7 : c09e0c00  r6 : 00000000  r5 : 00002041  r4 : 00002040
[    0.969679] r3 : 00000100  r2 : 00000000  r1 : 69052000  r0 : 00000000
[    0.969892] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[    0.970123] Control: 00007977  Table: a22e0000  DAC: 00000071
[    0.970349] Process swapper (pid: 1, stack limit = 0x(ptrval))
[    0.970564] Stack: (0xc0bdfec8 to 0xc0be0000)
[    0.970818] fec0:                   c08cdfe4 ffffe000 c0bdff4c c0bdfee0 c000a2dc c08cdff0
[    0.971144] fee0: c004b8a0 c08c9458 c0bdfee4 00000000 00000dc0 00000000 00000007 00000007
[    0.971438] ff00: 00000000 c07fc180 c08c944c c08c6a9c c017213c c0c07c4c c0c07c54 c092d028
[    0.971728] ff20: 00000000 000000a0 c0c07c20 c092d028 c09293fc c0c07c20 00000008 c08fe854
[    0.972026] ff40: c0bdff94 c0bdff50 c08ca220 c000a288 00000007 00000007 00000000 c08c944c
[    0.972315] ff60: ffffe000 000000a0 c0bdff8c 00000000 c0671de8 00000000 00000000 00000000
[    0.972603] ff80: 00000000 00000000 c0bdffac c0bdff98 c0671e00 c08ca0d8 00000000 c0671de8
[    0.972891] ffa0: 00000000 c0bdffb0 c0008360 c0671df4 00000000 00000000 00000000 00000000
[    0.973199] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    0.973488] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
[    0.973763] Backtrace:
[    0.974001] [<c08cdfe4>] (xscale_cp0_init) from [<c000a2dc>] (do_one_initcall+0x60/0x290)
[    0.974282]  r5:ffffe000 r4:c08cdfe4
[    0.974564] [<c000a27c>] (do_one_initcall) from [<c08ca220>] (kernel_init_freeable+0x154/0x1dc)
[    0.974791]  r7:c08fe854 r6:00000008 r5:c0c07c20 r4:c09293fc
[    0.974948] [<c08ca0cc>] (kernel_init_freeable) from [<c0671e00>] (kernel_init+0x18/0x110)
[    0.975191]  r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0671de8
[    0.975394]  r4:00000000
[    0.975476] [<c0671de8>] (kernel_init) from [<c0008360>] (ret_from_fork+0x14/0x34)
[    0.975695] Exception stack(0xc0bdffb0 to 0xc0bdfff8)
[    0.975885] ffa0:                                     00000000 00000000 00000000 00000000
[    0.976184] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    0.976458] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000
[    0.976657]  r5:c0671de8 r4:00000000
[    0.976961] Code: e1a03003 e24ff004 e3a02000 e3a03c01 (ec432000)

The code is:

  70:	ee1f3f11 	mrc	15, 0, r3, cr15, cr1, {0}
  74:	e1a03003 	mov	r3, r3
  78:	e24ff004 	sub	pc, pc, #4
  7c:	e3a02000 	mov	r2, #0
  80:	e3a03c01 	mov	r3, #256	; 0x100
  84:	ec432000 	mar	acc0, r2, r3
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is supposed to be a DSP or iWMMXt coprocessor instruction.
I did notice that the code to support xscale instructions has changed significantly
in qemu v5.2.
Does this mean that there is a bug, that the affected emulations are no longer
supported, that I now have to specify some new option on the qemu command line,
or something else ?

Thanks,
Guenter

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Philippe Mathieu-Daudé]

On 12/21/20 6:28 AM, Guenter Roeck wrote:
> Hi,
> 
> I noticed that booting Linux on PXA emulations no longer works with qemu v5.2.
> When trying to boot akita, borzoi, or similar emulations, I get the following crash.
> 
> [    0.965279] Internal error: Oops - undefined instruction: 0 [#1] PREEMPT ARM
> [    0.967273] Modules linked in:
> [    0.967875] CPU: 0 PID: 1 Comm: swapper Not tainted 5.10.2-rc1-00017-gc96cfd687a3f #1
> [    0.968101] Hardware name: SHARP Akita
> [    0.968676] PC is at xscale_cp0_init+0x84/0x114
> [    0.968815] LR is at do_one_initcall+0x60/0x290
> [    0.968997] pc : [<c08ce068>]    lr : [<c000a2dc>]    psr: 60000013
> [    0.969186] sp : c0bdfec8  ip : c0bdfee0  fp : c0bdfedc
> [    0.969332] r10: c08fe834  r9 : c09f2000  r8 : c08c6a9c
> [    0.969498] r7 : c09e0c00  r6 : 00000000  r5 : 00002041  r4 : 00002040
> [    0.969679] r3 : 00000100  r2 : 00000000  r1 : 69052000  r0 : 00000000
> [    0.969892] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> [    0.970123] Control: 00007977  Table: a22e0000  DAC: 00000071
> [    0.970349] Process swapper (pid: 1, stack limit = 0x(ptrval))
> [    0.970564] Stack: (0xc0bdfec8 to 0xc0be0000)
> [    0.970818] fec0:                   c08cdfe4 ffffe000 c0bdff4c c0bdfee0 c000a2dc c08cdff0
> [    0.971144] fee0: c004b8a0 c08c9458 c0bdfee4 00000000 00000dc0 00000000 00000007 00000007
> [    0.971438] ff00: 00000000 c07fc180 c08c944c c08c6a9c c017213c c0c07c4c c0c07c54 c092d028
> [    0.971728] ff20: 00000000 000000a0 c0c07c20 c092d028 c09293fc c0c07c20 00000008 c08fe854
> [    0.972026] ff40: c0bdff94 c0bdff50 c08ca220 c000a288 00000007 00000007 00000000 c08c944c
> [    0.972315] ff60: ffffe000 000000a0 c0bdff8c 00000000 c0671de8 00000000 00000000 00000000
> [    0.972603] ff80: 00000000 00000000 c0bdffac c0bdff98 c0671e00 c08ca0d8 00000000 c0671de8
> [    0.972891] ffa0: 00000000 c0bdffb0 c0008360 c0671df4 00000000 00000000 00000000 00000000
> [    0.973199] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> [    0.973488] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
> [    0.973763] Backtrace:
> [    0.974001] [<c08cdfe4>] (xscale_cp0_init) from [<c000a2dc>] (do_one_initcall+0x60/0x290)
> [    0.974282]  r5:ffffe000 r4:c08cdfe4
> [    0.974564] [<c000a27c>] (do_one_initcall) from [<c08ca220>] (kernel_init_freeable+0x154/0x1dc)
> [    0.974791]  r7:c08fe854 r6:00000008 r5:c0c07c20 r4:c09293fc
> [    0.974948] [<c08ca0cc>] (kernel_init_freeable) from [<c0671e00>] (kernel_init+0x18/0x110)
> [    0.975191]  r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c0671de8
> [    0.975394]  r4:00000000
> [    0.975476] [<c0671de8>] (kernel_init) from [<c0008360>] (ret_from_fork+0x14/0x34)
> [    0.975695] Exception stack(0xc0bdffb0 to 0xc0bdfff8)
> [    0.975885] ffa0:                                     00000000 00000000 00000000 00000000
> [    0.976184] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> [    0.976458] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000
> [    0.976657]  r5:c0671de8 r4:00000000
> [    0.976961] Code: e1a03003 e24ff004 e3a02000 e3a03c01 (ec432000)
> 
> The code is:
> 
>   70:	ee1f3f11 	mrc	15, 0, r3, cr15, cr1, {0}
>   74:	e1a03003 	mov	r3, r3
>   78:	e24ff004 	sub	pc, pc, #4
>   7c:	e3a02000 	mov	r2, #0
>   80:	e3a03c01 	mov	r3, #256	; 0x100
>   84:	ec432000 	mar	acc0, r2, r3
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> This is supposed to be a DSP or iWMMXt coprocessor instruction.
> I did notice that the code to support xscale instructions has changed significantly
> in qemu v5.2.

Indeed a lot changed...

I had a quick look. The instruction is decoded in aa32 as LDR_ri.

Without looking at the spec, I simply googled the instruction and
got this link:
https://developer.arm.com/documentation/ddi0406/c/Application-Level-Architecture/ARM-Instruction-Set-Encoding/Load-store-word-and-unsigned-byte
which is for ARMv7-[AR] so I tried this dumb diff to skip the
LDR_ri decoding:

-- >8 --
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -6562,6 +6562,10 @@ static bool op_load_ri(DisasContext *s,
arg_ldst_ri *a,
     ISSInfo issinfo = make_issinfo(s, a->rt, a->p, a->w);
     TCGv_i32 addr, tmp;

+    if (!ENABLE_ARCH_7) {
+        return false;
+    }
+
     addr = op_addr_ri_pre(s, a);

     tmp = tcg_temp_new_i32();
@@ -6583,6 +6587,10 @@ static bool op_store_ri(DisasContext *s,
arg_ldst_ri *a,
     ISSInfo issinfo = make_issinfo(s, a->rt, a->p, a->w) | ISSIsWrite;
     TCGv_i32 addr, tmp;

+    if (!ENABLE_ARCH_7) {
+        return false;
+    }
+
     addr = op_addr_ri_pre(s, a);

     tmp = load_reg(s, a->rt);
---

Then realized disas_xscale_insn() isn't reached. In case that helped,
I focused on these commits:

590057d969a ("target/arm: Simplify disas_arm_insn")
19c23a9baaf ("target/arm: Separate decode from handling of coproc insns")
7b4f933db86 ("target/arm: Pull handling of XScale insns out of
disas_coproc_insn()")

> Does this mean that there is a bug, that the affected emulations are no longer
> supported, that I now have to specify some new option on the qemu command line,
> or something else ?

The command line is likely correct, all the pxa270* CPUs have the same
architectural features. It is not yet listed as "no longer supported"
but certainly "not enough tested". Good news, you seem to have an easy
reproducible test.

> 
> Thanks,
> Guenter
> 

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Guenter Roeck]

Hi Philippe,

On 12/21/20 7:24 AM, Philippe Mathieu-Daudé wrote:
[ ... ])
> 
>> Does this mean that there is a bug, that the affected emulations are no longer
>> supported, that I now have to specify some new option on the qemu command line,
>> or something else ?
> 
> The command line is likely correct, all the pxa270* CPUs have the same
> architectural features. It is not yet listed as "no longer supported"
> but certainly "not enough tested". Good news, you seem to have an easy
> reproducible test.
> 

Something like the following should do.

qemu-system-arm -M z2 -kernel arch/arm/boot/zImage -no-reboot \
    -initrd rootfs-armv5.cpio \
    --append "rdinit=/sbin/init console=ttyS0" \
    -nographic -monitor null -serial stdio

where the kernel is built with pxa_defconfig.
Machine name can be any of the pxa machines (akita, borzoi, spitz,
tosa, terrier, z2, or mainstone). The initrd is from:
https://github.com/groeck/linux-build-test/blob/master/rootfs/arm/rootfs-armv5.cpio.gz

Guenter

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Peter Maydell]

On Mon, 21 Dec 2020 at 16:01, Guenter Roeck <linux@roeck-us.net> wrote:
> Something like the following should do.
>
> qemu-system-arm -M z2 -kernel arch/arm/boot/zImage -no-reboot \
>     -initrd rootfs-armv5.cpio \
>     --append "rdinit=/sbin/init console=ttyS0" \
>     -nographic -monitor null -serial stdio
>
> where the kernel is built with pxa_defconfig.
> Machine name can be any of the pxa machines (akita, borzoi, spitz,
> tosa, terrier, z2, or mainstone). The initrd is from:
> https://github.com/groeck/linux-build-test/blob/master/rootfs/arm/rootfs-armv5.cpio.gz

Do you have a zImage that exhibits this so I don't have to build
my own, please?

thanks
-- PMM

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Guenter Roeck]

[-- Attachment #1: Type: text/plain, Size: 740 bytes --]

On 1/8/21 9:25 AM, Peter Maydell wrote:
> On Mon, 21 Dec 2020 at 16:01, Guenter Roeck <linux@roeck-us.net> wrote:
>> Something like the following should do.
>>
>> qemu-system-arm -M z2 -kernel arch/arm/boot/zImage -no-reboot \
>>     -initrd rootfs-armv5.cpio \
>>     --append "rdinit=/sbin/init console=ttyS0" \
>>     -nographic -monitor null -serial stdio
>>
>> where the kernel is built with pxa_defconfig.
>> Machine name can be any of the pxa machines (akita, borzoi, spitz,
>> tosa, terrier, z2, or mainstone). The initrd is from:
>> https://github.com/groeck/linux-build-test/blob/master/rootfs/arm/rootfs-armv5.cpio.gz
> 
> Do you have a zImage that exhibits this so I don't have to build
> my own, please?
> 

Attached.

Guenter

[-- Attachment #2: zImage --]
[-- Type: application/octet-stream, Size: 4764520 bytes --]

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Peter Maydell]

On Fri, 8 Jan 2021 at 18:56, Guenter Roeck <linux@roeck-us.net> wrote:
>
> On 1/8/21 9:25 AM, Peter Maydell wrote:
> > On Mon, 21 Dec 2020 at 16:01, Guenter Roeck <linux@roeck-us.net> wrote:
> >> Something like the following should do.
> >>
> >> qemu-system-arm -M z2 -kernel arch/arm/boot/zImage -no-reboot \
> >>     -initrd rootfs-armv5.cpio \
> >>     --append "rdinit=/sbin/init console=ttyS0" \
> >>     -nographic -monitor null -serial stdio
> >>
> >> where the kernel is built with pxa_defconfig.
> >> Machine name can be any of the pxa machines (akita, borzoi, spitz,
> >> tosa, terrier, z2, or mainstone). The initrd is from:
> >> https://github.com/groeck/linux-build-test/blob/master/rootfs/arm/rootfs-armv5.cpio.gz
> >
> > Do you have a zImage that exhibits this so I don't have to build
> > my own, please?
> >
>
> Attached.

Thanks. Yeah, this is a bug in my refactoring of the coprocessor
insn handling :-(  I've just sent a patch which fixes it.

-- PMM

Re: pxa crashes with qemu v5.2 when executing xscale operations

[Peter Maydell]

On Mon, 21 Dec 2020 at 15:24, Philippe Mathieu-Daudé <f4bug@amsat.org> wrote:
> On 12/21/20 6:28 AM, Guenter Roeck wrote:
> >   84: ec432000        mar     acc0, r2, r3
> >         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> > This is supposed to be a DSP or iWMMXt coprocessor instruction.
> > I did notice that the code to support xscale instructions has changed significantly
> > in qemu v5.2.
>
> Indeed a lot changed...
>
> I had a quick look. The instruction is decoded in aa32 as LDR_ri.

It isn't, incidentally. LDR_ri has 010 in bits [27:25], and this
insn has 110.

thanks
-- PMM