Re: debian bootup hangs on qemu-system-mips64el

Re: debian bootup hangs on qemu-system-mips64el

[Jun Sun]

Cross-post to the *right* linux-mips list with the *right*
"plain/text" format (sigh)... and hopefully we can have more expert
eyes looking at this ...

(Really sorry to qemu-discuss and debian-mips list ...)

Thanks.

Jun


On Wed, Jan 1, 2020 at 6:50 PM Jun Sun <jsun@junsun.net> wrote:
>
> Hi, all,
>
> Debian/mips64el runs well on qemu-system-mips64el until I turned on smp option with a number of 2 or higher.  See the error message below.  It usually hangs pretty early on.
>
> I have experimented with different CPU types (e.g., 5KEc) and different memory amont with no avail. Does anybody have an idea?
>
> I attached the kernel config for reference.  It appears SMP kernel configs are enabled properly.
>
> Thanks a lot!
>
> Jun
>
> qemu-system-mips64el -M malta -smp 2 -cpu MIPS64R2-generic -m 1314 -hda hda.qcow2 -kernel vmlinux -initrd initrd.img -append 'root=/dev/sda1 console=ttyS0 nokaslr' -nographic -net user,hostfwd=tcp::2222-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8443-:443 -net nic -serial mon:stdio
> qemu-system-mips64el: warning: SPD cannot represent final 2MB of SDRAM
> [    0.000000] Linux version 4.19.0-6-5kc-malta (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.67-2+deb10u1 (2019-09-20)
> [    0.000000] earlycon: uart8250 at I/O port 0x3f8 (options '38400n8')
> [    0.000000] bootconsole [uart8250] enabled
> [    0.000000] CPU0 revision is: 00010000 (MIPS GENERIC QEMU)
> [    0.000000] FPU revision is: 007f0000
> [    0.000000] Checking for the multiply/shift bug... no.
> [    0.000000] Checking for the daddiu bug... no.
> [    0.000000] MIPS: machine is mti,malta
> [    0.000000] Software DMA cache coherency enabled
> [    0.000000] Determined physical RAM map:
> [    0.000000]  memory: 0000000010000000 @ 0000000000000000 (usable)
> [    0.000000]  memory: 0000000042200000 @ 0000000090000000 (usable)
>
> [HANG FOREVER AT HERE ...]
>

Re: debian bootup hangs on qemu-system-mips64el

[Aurelien Jarno]

On 2020-01-04 20:46, Jun Sun wrote:
> On Wed, Jan 1, 2020 at 6:50 PM Jun Sun <jsun@junsun.net> wrote:
> >
> > Hi, all,
> >
> > Debian/mips64el runs well on qemu-system-mips64el until I turned on smp option with a number of 2 or higher.  See the error message below.  It usually hangs pretty early on.
> >

QEMU/mips doesn't really support SMP. The -smp option used to be for
SMT, but I am not even sure it really worked.

All that said, it's still a bug in QEMU to accept this option.

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: debian bootup hangs on qemu-system-mips64el

[Jun Sun]

Are you sure QEMU/MIPS does not support SMP?  Below link written in
2016 clearly shows something was working then.

https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/

Also, as Lixing mentioned in our private exchange, there is SMP
support for loongson.  Lixing, I assume loongson SMP support is
already merged in QEMU repo, right?  Can you point out the relevant
source code directories?

Would love to get more pointers here!  Thanks!

Jun

On Thu, Jan 9, 2020 at 1:24 PM Aurelien Jarno <aurelien@aurel32.net> wrote:
>
> On 2020-01-04 20:46, Jun Sun wrote:
> > On Wed, Jan 1, 2020 at 6:50 PM Jun Sun <jsun@junsun.net> wrote:
> > >
> > > Hi, all,
> > >
> > > Debian/mips64el runs well on qemu-system-mips64el until I turned on smp option with a number of 2 or higher.  See the error message below.  It usually hangs pretty early on.
> > >
>
> QEMU/mips doesn't really support SMP. The -smp option used to be for
> SMT, but I am not even sure it really worked.
>
> All that said, it's still a bug in QEMU to accept this option.
>
> --
> Aurelien Jarno                          GPG: 4096R/1DDD8C9B
> aurelien@aurel32.net                 http://www.aurel32.net

Re: debian bootup hangs on qemu-system-mips64el

[Aurelien Jarno]

Hi,

On 2020-01-12 11:34, Jun Sun wrote:
> Are you sure QEMU/MIPS does not support SMP?  Below link written in
> 2016 clearly shows something was working then.
> 
> https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/

It's not supported for the malta/mips64r2 combination you are trying to
use. It's clearly possible that it is supported for mips64r6, but it's a
different instruction set, so you can't run a standard Debian
distribution on it.

> Also, as Lixing mentioned in our private exchange, there is SMP
> support for loongson.  Lixing, I assume loongson SMP support is
> already merged in QEMU repo, right?  Can you point out the relevant
> source code directories?

Lixing mentioned virtualization. In that case it is indeed supported
like on other mips64r5 CPUs. For that you need to use add the -enable-kvm
option to the qemu command.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: debian bootup hangs on qemu-system-mips64el

[Jiaxun Yang]

于 2020年1月13日 GMT+08:00 上午3:34:18, Jun Sun <jsun@junsun.net> 写到:
>Are you sure QEMU/MIPS does not support SMP?  Below link written in
>2016 clearly shows something was working then.
>
>https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>
>Also, as Lixing mentioned in our private exchange, there is SMP
>support for loongson.  Lixing, I assume loongson SMP support is
>already merged in QEMU repo, right?  Can you point out the relevant
>source code directories?
Hi Jun,

As far as I known, Loongson is using there own SMP IPI implementation instead of MIPS MT One.
There QEMU implementation could be found here [1].
QEMU have implemented MIPS MT ASE for SMP.

Thanks
>
>Would love to get more pointers here!  Thanks!
>
>Jun
>
>On Thu, Jan 9, 2020 at 1:24 PM Aurelien Jarno <aurelien@aurel32.net>
>wrote:
>>
>> On 2020-01-04 20:46, Jun Sun wrote:
>> > On Wed, Jan 1, 2020 at 6:50 PM Jun Sun <jsun@junsun.net> wrote:
>> > >
>> > > Hi, all,
>> > >
>> > > Debian/mips64el runs well on qemu-system-mips64el until I turned
>on smp option with a number of 2 or higher.  See the error message
>below.  It usually hangs pretty early on.
>> > >
>>
>> QEMU/mips doesn't really support SMP. The -smp option used to be for
>> SMT, but I am not even sure it really worked.
>>
>> All that said, it's still a bug in QEMU to accept this option.
>>
>> --
>> Aurelien Jarno                          GPG: 4096R/1DDD8C9B
>> aurelien@aurel32.net                 http://www.aurel32.net

[1]: https://gitee.com/loongsonlab/qemu

-- 
Jiaxun Yang

Re: debian bootup hangs on qemu-system-mips64el

[Jun Sun]

See below.

On Sun, Jan 12, 2020 at 11:58 AM Aurelien Jarno <aurelien@aurel32.net> wrote:
>
> Hi,
>
> On 2020-01-12 11:34, Jun Sun wrote:
> > Are you sure QEMU/MIPS does not support SMP?  Below link written in
> > 2016 clearly shows something was working then.
> >
> > https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>
> It's not supported for the malta/mips64r2 combination you are trying to
> use. It's clearly possible that it is supported for mips64r6, but it's a
> different instruction set, so you can't run a standard Debian
> distribution on it.
>

How do I specify mips64r6 cpu to start qemu?  A simple cpu listing
below shows no such CPU.  If mips64r6 is indeed supported with SMP, I
would think mips64r2 userland should run fine.

$ qemu-system-mips64el -cpu ?
MIPS '4Kc'
MIPS '4Km'
MIPS '4KEcR1'
MIPS '4KEmR1'
MIPS '4KEc'
MIPS '4KEm'
MIPS '24Kc'
MIPS '24KEc'
MIPS '24Kf'
MIPS '34Kf'
MIPS '74Kf'
MIPS 'M14K'
MIPS 'M14Kc'
MIPS 'P5600'
MIPS 'mips32r6-generic'
MIPS 'R4000'
MIPS 'VR5432'
MIPS '5Kc'
MIPS '5Kf'
MIPS '20Kc'
MIPS 'MIPS64R2-generic'
MIPS '5KEc'
MIPS '5KEf'
MIPS 'I6400'
MIPS 'Loongson-2E'
MIPS 'Loongson-2F'
MIPS 'mips64dspr2'

> > Also, as Lixing mentioned in our private exchange, there is SMP
> > support for loongson.  Lixing, I assume loongson SMP support is
> > already merged in QEMU repo, right?  Can you point out the relevant
> > source code directories?
>
> Lixing mentioned virtualization. In that case it is indeed supported
> like on other mips64r5 CPUs. For that you need to use add the -enable-kvm
> option to the qemu command.

I'm not sure if loongson support has to be kvm.  Maybe Lixing can clarify.

Re: debian bootup hangs on qemu-system-mips64el

[Jun Sun]

Thanks, Jiaxun.  See more questions below.

Jun

On Sun, Jan 12, 2020 at 5:17 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
>
>
>
> 于 2020年1月13日 GMT+08:00 上午3:34:18, Jun Sun <jsun@junsun.net> 写到:
> >Are you sure QEMU/MIPS does not support SMP?  Below link written in
> >2016 clearly shows something was working then.
> >
> >https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
> >
> >Also, as Lixing mentioned in our private exchange, there is SMP
> >support for loongson.  Lixing, I assume loongson SMP support is
> >already merged in QEMU repo, right?  Can you point out the relevant
> >source code directories?
> Hi Jun,
>
> As far as I known, Loongson is using there own SMP IPI implementation instead of MIPS MT One.
> There QEMU implementation could be found here [1].
> QEMU have implemented MIPS MT ASE for SMP.
>

Do you know how to enable and run MIPS MT ASE in qemu?  For with CPU options?

Re: debian bootup hangs on qemu-system-mips64el

[Huacai Chen]

Hi, all

I'm working on Loongson-3's SMP support in QEMU, and I'll send my patches soon.

Huacai

On Mon, Jan 13, 2020 at 10:35 AM Jun Sun <jsun@junsun.net> wrote:
>
> Thanks, Jiaxun.  See more questions below.
>
> Jun
>
> On Sun, Jan 12, 2020 at 5:17 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
> >
> >
> >
> > 于 2020年1月13日 GMT+08:00 上午3:34:18, Jun Sun <jsun@junsun.net> 写到:
> > >Are you sure QEMU/MIPS does not support SMP?  Below link written in
> > >2016 clearly shows something was working then.
> > >
> > >https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
> > >
> > >Also, as Lixing mentioned in our private exchange, there is SMP
> > >support for loongson.  Lixing, I assume loongson SMP support is
> > >already merged in QEMU repo, right?  Can you point out the relevant
> > >source code directories?
> > Hi Jun,
> >
> > As far as I known, Loongson is using there own SMP IPI implementation instead of MIPS MT One.
> > There QEMU implementation could be found here [1].
> > QEMU have implemented MIPS MT ASE for SMP.
> >
>
> Do you know how to enable and run MIPS MT ASE in qemu?  For with CPU options?

Re: debian bootup hangs on qemu-system-mips64el

[Jiaxun Yang]

于 2020年1月13日 GMT+08:00 上午10:35:18, Jun Sun <jsun@junsun.net> 写到:
>Thanks, Jiaxun.  See more questions below.
>
>Jun
>
>On Sun, Jan 12, 2020 at 5:17 PM Jiaxun Yang <jiaxun.yang@flygoat.com>
>wrote:
>>
>>
>>
>> 于 2020年1月13日 GMT+08:00 上午3:34:18, Jun Sun <jsun@junsun.net> 写到:
>> >Are you sure QEMU/MIPS does not support SMP?  Below link written in
>> >2016 clearly shows something was working then.
>> >
>>
>>https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
>> >
>> >Also, as Lixing mentioned in our private exchange, there is SMP
>> >support for loongson.  Lixing, I assume loongson SMP support is
>> >already merged in QEMU repo, right?  Can you point out the relevant
>> >source code directories?
>> Hi Jun,
>>
>> As far as I known, Loongson is using there own SMP IPI implementation
>instead of MIPS MT One.
>> There QEMU implementation could be found here [1].
>> QEMU have implemented MIPS MT ASE for SMP.
>>
>
>Do you know how to enable and run MIPS MT ASE in qemu?  For with CPU
>options?

I tried maxcpus=2 cpu=34kf for malta long ago and it was working.
But I'm not sure whether it still works now.


-- 
Jiaxun Yang

Re: debian bootup hangs on qemu-system-mips64el

[Paul Burton]

Hi Jun Sun,

On Sun, Jan 12, 2020 at 06:14:12PM -0800, Jun Sun wrote:
> On Sun, Jan 12, 2020 at 11:58 AM Aurelien Jarno <aurelien@aurel32.net> wrote:
> > On 2020-01-12 11:34, Jun Sun wrote:
> > > Are you sure QEMU/MIPS does not support SMP?  Below link written in
> > > 2016 clearly shows something was working then.
> > >
> > > https://www.mips.com/blog/how-to-run-smp-linux-in-qemu-on-a-mips64-release-6-cpu/
> >
> > It's not supported for the malta/mips64r2 combination you are trying to
> > use. It's clearly possible that it is supported for mips64r6, but it's a
> > different instruction set, so you can't run a standard Debian
> > distribution on it.
> 
> How do I specify mips64r6 cpu to start qemu?  A simple cpu listing
> below shows no such CPU.  If mips64r6 is indeed supported with SMP, I
> would think mips64r2 userland should run fine.

I know that using QEMU's I6400 or I6500 CPU emulation works fine with
SMP as far as -smp 8. It emulates a single core system with up to 8 VPs
(hardware threads), and Linux works fine with it. I most often use the
Boston board emulation with a 64r6el_defconfig kernel for this, but it
ought to work for Malta too.

Both I6400 & I6500 are MIPS64r6 CPUs, so as Aurelien said you'd need to
ensure you're using a suitable MIPSr6 root filesystem.

If you wanted to run as I do it'd go something like this:

$ qemu-system-mips64el \
    -M boston \
    -cpu I6500 \
    -smp 8 \
    -kernel arch/mips/boot/vmlinux.gz.itb

Where that's run from the root of a kernel source tree within which
you've built a 64r6el_defconfig kernel.

Not sure what the behavior is with older CPUs; QEMU's CM & CPC emulation
is somewhat incomplete, and whilst I6400 & I6500 have a relatively
straightforward VP model older CPUs have the more complex MT ASE with
VPEs & TCs to worry about.

Thanks,
    Paul