Qemu system mode emulation for heterogeneous SOC

Qemu system mode emulation for heterogeneous SOC

[Ancuta, Cristian]

[-- Attachment #1: Type: text/plain, Size: 464 bytes --]

Hi all,

As the subject says, I was wondering if QEMU execution model allows the full system emulation of a custom heterogeneous SoC architecture.

I'm currently looking at the VersatilePB board as an example of how to define a board, it's CPUs, peripherals, memory maps, etc, but it is just one ARM core so I'm not sure whether that's the best example to start with.

As a related question, is MMIO supported in user mode emulation?

Thanks,
Cristian


[-- Attachment #2: Type: text/html, Size: 7564 bytes --]

Re: Qemu system mode emulation for heterogeneous SOC

[Peter Maydell]

On Tue, 14 Apr 2020 at 23:08, Ancuta, Cristian
<cristian.ancuta@intel.com> wrote:
> As the subject says, I was wondering if QEMU execution model allows the full system emulation of a custom heterogeneous SoC architecture.

Only in the very limited sense "you can have two different CPUs which
are of the same architecture", eg an Arm M-class core and an A-class
core. You can't have two CPUs which are entirely different architectures
(eg PPC and Arm). In principle this would be nice to be able to do, but
in practice QEMU's current source tree assumes that various properties
of the guest CPU are compile-time fixed, and making those all be
"this is actually a variable property that applies only to some of the
CPUs being emulated" rather than just "this is set by a #define" would
be a massive effort.

> As a related question, is MMIO supported in user mode emulation?

No. Much of the infrastructure and framework for device emulation
is only in the -softmmu binaries. The user-mode emulation stuff
is intended for emulating single Linux userspace processes, which
don't have direct access to device hardware.

thanks
-- PMM

RE: Qemu system mode emulation for heterogeneous SOC

[Ancuta, Cristian]

Hi, thanks for taking the time.

So just to make sure I understand the MMIO stuff correctly, there's no user mode support because user mode applications are talking to hardware through the OS drivers and syscalls (which would run in system mode) anyway?

Also, is there a multi core board in the source tree that I could look at to get me started ? And if there is, when emulating multiple guest cores, do they each get their own threads, or are they run in a round-robin fashion on the same qemu thread?

Thanks,
Cristian

-----Original Message-----
From: Peter Maydell <peter.maydell@linaro.org> 
Sent: Wednesday, April 15, 2020 11:46 AM
To: Ancuta, Cristian <cristian.ancuta@intel.com>
Cc: qemu-devel@nongnu.org
Subject: Re: Qemu system mode emulation for heterogeneous SOC

On Tue, 14 Apr 2020 at 23:08, Ancuta, Cristian <cristian.ancuta@intel.com> wrote:
> As the subject says, I was wondering if QEMU execution model allows the full system emulation of a custom heterogeneous SoC architecture.

Only in the very limited sense "you can have two different CPUs which are of the same architecture", eg an Arm M-class core and an A-class core. You can't have two CPUs which are entirely different architectures (eg PPC and Arm). In principle this would be nice to be able to do, but in practice QEMU's current source tree assumes that various properties of the guest CPU are compile-time fixed, and making those all be "this is actually a variable property that applies only to some of the CPUs being emulated" rather than just "this is set by a #define" would be a massive effort.

> As a related question, is MMIO supported in user mode emulation?

No. Much of the infrastructure and framework for device emulation is only in the -softmmu binaries. The user-mode emulation stuff is intended for emulating single Linux userspace processes, which don't have direct access to device hardware.

thanks
-- PMM

Re: Qemu system mode emulation for heterogeneous SOC

[Peter Maydell]

On Wed, 15 Apr 2020 at 14:27, Ancuta, Cristian
<cristian.ancuta@intel.com> wrote:
> So just to make sure I understand the MMIO stuff correctly, there's no user mode support because user mode applications are talking to hardware through the OS drivers and syscalls (which would run in system mode) anyway?

Correct.

> Also, is there a multi core board in the source tree that I could look at to get me started ?

Lots; any board that sets its max_cpus field to something other than 1.
Almost all of these are "all CPUs are identical and see basically
the same view of the system" setups. About the only heterogenous
board I think is xlnx-zcu102 (which has some A-class and some
R-class Arm cores).

> And if there is, when emulating multiple guest cores, do they each get their own threads, or are they run in a round-robin fashion on the same qemu thread?

That depends. By default, if this host-and-guest-architecture
combination can handle it then we will create one host thread
per vCPU (there's a command line flag to turn this off, I forget
its name). Otherwise we fall back to one thread and round-robin.

thanks
-- PMM

RE: Qemu system mode emulation for heterogeneous SOC

[Ancuta, Cristian]

> Almost all of these are "all CPUs are identical and see basically the same view of the system" setups"
On a second thought, that setup would suffice for my use case. Heterogeneous was just a "nice to have".

Thanks,
Cristian

-----Original Message-----
From: Peter Maydell <peter.maydell@linaro.org> 
Sent: Wednesday, April 15, 2020 4:40 PM
To: Ancuta, Cristian <cristian.ancuta@intel.com>
Cc: qemu-devel@nongnu.org
Subject: Re: Qemu system mode emulation for heterogeneous SOC

On Wed, 15 Apr 2020 at 14:27, Ancuta, Cristian <cristian.ancuta@intel.com> wrote:
> So just to make sure I understand the MMIO stuff correctly, there's no user mode support because user mode applications are talking to hardware through the OS drivers and syscalls (which would run in system mode) anyway?

Correct.

> Also, is there a multi core board in the source tree that I could look at to get me started ?

Lots; any board that sets its max_cpus field to something other than 1.
Almost all of these are "all CPUs are identical and see basically the same view of the system" setups. About the only heterogenous board I think is xlnx-zcu102 (which has some A-class and some R-class Arm cores).

> And if there is, when emulating multiple guest cores, do they each get their own threads, or are they run in a round-robin fashion on the same qemu thread?

That depends. By default, if this host-and-guest-architecture combination can handle it then we will create one host thread per vCPU (there's a command line flag to turn this off, I forget its name). Otherwise we fall back to one thread and round-robin.

thanks
-- PMM