[RFC 00/52] Introduce hybrid CPU topology

[RFC 00/52] Introduce hybrid CPU topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Hi list,

This RFC series is to introduce hybrid topology in QEMU, and is based
on our previous patches about cluster support for x86 [1] ([PATCH RESEND
00/18] Support smp.clusters for x86).

Though our cluster support patches [1] has not yet accepted, we are
still posting the hybrid topology support patches in advance, hoping to
get discussion and feedbacks.

In this RFC series, we mainly have the following work:
* Introduce the new cpu topology under heterogeneous architecture, which
  we called "hybrid" cpu topology.
      - since the name "hmp" is already used in QEMU and "hybrid" is
        also a synonym for heterogeneous.
* Reorganized the relevant topology information in qemu to adapt to the
  hybrid topology.
* Introduce a new command "hybrid", which allows user to set a custom
  hybrid topology.
      - currently limited to custom cluster and custom core
* Support hybrid topology in i386 PC machine.
* Expose basic topology information (topology type and core type) in
  query_cpus_fast.

We hope that the hybrid topology can be general enough to be compatible
with the needs of hybrids with various architectures in the future.

Welcome your feedbacks!


# 1. Background

Heterogeneous computing refers to systems that use more than one kind of
processor or cores [2]. Now there are more and more heterogeneous
computing architectures, typically including arm's big.LITTLE [3] and
intel hybrid architecture [4].

The development of heterogeneous computing brings new challenges to
virtualization. The first problem we face is how to support the vCPU
topology under the heterogeneous architecture.

Here, to distinguish it from "smp" cpu topology, we call it "hybrid" cpu
topology.


# 2. Motivation

In a typical hybrid system, different types of core may have different
capabilities. This difference may be reflected in different capacities,
performance or power efficiencies, and even different instruction-per-
cycle (IPC) capabilities. The direct consequence of these differences is
to affect the performance of the workload.

For a virtual machines, vCPUs are just threads, although we can bind
vCPUs to different cores to benefit from the different advantages of
these physical cores, it's obviously not enough since CPU topology (and
cache topology) will still have a significant impact on scheduling
performance.

In this case (binding vCPU to pCPU) - actually it's a very common
application example, especially on the client side - it is very
necessary for vCPU to obtain the same (or similar) topology as pCPU.

Unfortunately, the current "smp" command cannot do this. For example,
Alder Lake P [5] has 6 Core cores (2 threads per Core) and 8 Atom cores
(1 threads per Atom), and each Core has a L2 cache as well as every 4
Atoms (in the same module) share a L2 cache. "smp" cannot set the
correct cluster topology for "Atom" vCPU or "Core" vCPU.

Therefore, we need a new way to set up hybrid cpu topology for virtual
machines.


# 3. Design overview

## 3.1. General topology structure

Considering that not only intel has implemented the hybrid architecture,
but also other architectures have related implementations, such as arm,
we hope to make the hybrid topology more general, so we changed the
topology structure in MachineState to make it compatible with smp or
hybrid, which also avoids the conflict between the new topology in an
arch-specific machine and the general MachineState.smp.

The new topology structure is as follows:

typedef struct CpuTopology {
    unsigned int cpus;
    unsigned int max_cpus;
    CpuTopoType topo_type;
    union {
        SmpCpuTopology smp;
        HybridCpuTopology hybrid;
    };
} CpuTopology;

Here "smp" and "hybrid" are wrapped in a union, and the new topology
flag "topo_type" identifies whether it is currently smp or hybrid.

Please note, the hybrid topology can be compatible with the smp topology
type, that is, different parts on the same level of the hybrid topology
can set to be the same, but the hybrid topology will introduce more
complexities:
* need to allocate more memory, organized with array or linked-list.
* if someone wants specific cluster/core/thread information, the cpu
  index is required.

These complexities are obviously very different from the usage habits
of "smp", so the original smp topology support is retained while
introducing the hybrid topology, and the hybrid topology is only built
when the hybrid is explicitly required.

Because of the complexity brought by the new topology type and union, we
also wrap the access to this structure into a series of helpers for other
modules to call:

- General interfaces, no need to worry about whether the underlying
  topology is smp or hybrid:

* machine_topo_get_cpus()
* machine_topo_get_max_cpus()
* machine_topo_is_smp()
* machine_topo_get_sockets()
* machine_topo_get_dies()
* machine_topo_get_clusters()
* machine_topo_get_cores()
* machine_topo_get_threads()
* machine_topo_get_threads_by_idx()
* machine_topo_get_cores_per_socket()
* machine_topo_get_threads_per_socket()

- smp-specific interfaces - provided for the cases that are clearly
known to be smp topology:

* machine_topo_get_smp_cores()
* machine_topo_get_smp_threads()

- hybrid-specific interface:

* machine_topo_get_hybrid_core_type()


## 3.2. hybrid topology structures

For most hybrid systems, the threads are the same. The difference is
mainly at the core level: different cores may have different numbers of
threads, or may belong to different core type. So we introduce the new
hybrid core topology structure:

typedef struct HybridCore {
    unsigned int threads;
    unsigned int core_type;
} HybridCore;

At the same time, it should be noted that since the cores are different,
the clusters containing different cores will also be different.
Correspondingly, we also introduce a new hybrid cluster topology:

typedef struct HybridCluster {
    unsigned int cores;
    HybridCore *core_list;
    QSLIST_HEAD(, HybridCorePack) core_pack_list;
} HybridCluster;

With the single linked list "core_pack_list" in cluster structure, the
different cores can be inserted into specific clusters. This provides
much flexibility for the "hybrid" command. After "hybrid" parsing, the
"core_list" array will be built from "core_pack_list", and this is the
topological form of the core level maintained by MachineState since
core-id can be used well on the array.

As of now, hybrid core and hybrid cluster are enough for us, so we limit
the dies and sockets in the current hybrid topology to be the same
respectively. In order to list clusters in one die, we define an array
of clusters in the general hybrid topology structure. The reason why the
linked list is not used here is that the array is simple enough and can
already meet the needs of the "hybrid" command.

typedef struct HybridCpuTopology {
    unsigned int sockets;
    unsigned int dies;
    unsigned int clusters;
    HybridCluster *cluster_list;
} HybridCpuTopology;


## 3.3. "-hybrid" command

For hybrid cpu topology configuration, the original "-smp" lack of
flexibility to expand, and unables to customize different cores.

So we introduce "-hybrid" command and implement it as the multi-
line command. The multi-line command format is more complex than the
single-line one, but it can bring stronger scalability and
intuitiveness. In the future, it can also be easily extended to more
heterogeneous topology levels.

"-hybrid" command is as follows:

-hybrid socket,sockets=n
-hybrid die,dies=n
-hybrid cluster,clusters=n
-hybrid core,cores=n,type=core_type[,threads=threads]
        [,clusterid=cluster]

Here, we first define the corresponding qapi options for these 4
topology levels: core, cluster, die and socket.

We doesn't need a thread level since thread doesn't have different
type.

For example:

-hybrid socket,sockets=1
-hybrid die,dies=1
-hybrid cluster,clusters=4
-hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
-hybrid core,cores=4,coretype="atom",threads=1

Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
cores with 2 threads.

Please note this example is not an actual machine topology, but it shows
the powerful flexibility of "hybrid" command.


## 3.4. Reorganized topology information in CPUState

CPUState maintains nr_cores and nr_threads for a long time, but
there're some issues:
* The naming of nr_cores and nr_threads is not clear enough to cause
  misunderstandings in topology-related modifications.
* smp has already supported more topology levels, but these added
  topology information is stored in the architecture-specific code,
  therefore, this results in the fragmentation of topology information,
  which is not conducive to maintenance.
    - For example, i386 store nr_dies and nr_modules in CPUX86State.

So, we reorganized the CPUState topology information and cleaned up the
topology information fragments scattered in the specific architecture
code (i386's CPUX86State).

The new topology structure in CPUState is as follows:

typedef struct TopologyState {
    int sockets;
    int cores_per_socket;
    int threads_per_socket;
    int dies_per_socket;
    int clusters_per_die;
    int cores_per_cluster;
    int threads_per_core;
} TopologyState;


Since then, the different topologies in qemu have the clear purposes and
scopes:
* MachineState.topo maintains the topology information of the
entire machine.
* CPUState.topo maintains the topology information required by the
current CPU.


## 3.5. topology information in query_cpus_fast

The topology type and core type are useful to users, so expose them
in query_cpus_fast.

Since the hybrid core is pre-set and does not support hotplug for
the entire core, the topology type and core type are not exposed in
query-hotpluggable-cpus.

For example:

{
    "cpu-index": 0,
    "props": {
        "cluster-id": 0,
        "core-id": 0,
        "socket-id": 0,
        "thread-id": 0
    },
    "qom-path": "/machine/unattached/device[0]",
    "target": "x86_64",
    "thread-id": 313464,
    "topo": {
        "core-type": "core",
        "topo-type": "hybrid"
    }
}


# 4. Patch summary

- patch 1-4 fix historical topology-related issues, which will affect
            subsequent hybrid topology support.
- patch 5-9 add the new general topology structure support which
            is compatible with hybrid and smp.
- patch 10-27 replace origial smp topology structure with the new
              general topology structure. For generic code, hybrid
              topology must be respected, so case by case is required
              for replacement (patch 10-18). As for arch-specific code,
              restricting the topology to smp is also enough until the
              arch supports hybrid (patch 19-27).
- pacth 28-29 add complete topology information in CPUState and remove
              the nr_dies and nr_modules in CPUX86State. This avoid
              fragmentation of topology information and facilitate
              subsequent maintenance. Since then, MachineState.topo
              keeps the system topology information, and CPUState.topo
              owns the topology information for current CPU.
- patch 30-34 cleanups X86CPUTopoInfo uses. X86CPUTopoInfo original was
              used to keep topology informations and used to generate
              APIC ID. For hybrid topology, these two functions cannot
              be preserved at the same time. So make it only used to do
              APIC ID related topology work, and specifc topology
              information should get from CPUState.topo or
              MachineState.topo.
- patch 35-39 add the hybrid topology related support for i386.
- patch 40-50 introduce the "hybrid" command.
- patch 51-52 expose topology type and core type in query_cpus_fast.


# 5. TODOs

- Add the unit test for "hybrid" command.
- Support complete online cpu custom configuration for "hybrid" command.
- Support coretype and core level can be omitted.
- Add a limit on the maximum apicid.


# 6. References

[1]: [PATCH RESEND 00/18] Support smp.clusters for x86:
     https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03184.html
[2]: https://en.wikipedia.org/wiki/Heterogeneous_computing
[3]: https://www.arm.com/technologies/big-little
[4]: https://www.intel.com/content/www/us/en/support/articles/000091896/processors.html
[5]: https://www.intel.com/content/www/us/en/products/platforms/details/alder-lake-p.html

---
Zhao Liu (52):
  hw/smbios: Fix smbios_smp_sockets caculation
  hw/smbios: Fix thread count in type4
  hw/smbios: Fix core count in type4
  i386/WHPX: Fix error message when fail to set ProcessorCount
  hw/core/machine: Rename machine-smp.c to machine-topo.c
  hw/cpu: Introduce hybrid CPU topology
  hw/core/machine: Add the new topology support in MachineState
  machine: Add helpers to get cpu topology info from MachineState.topo
  hw/machine: Introduce core type for hybrid topology
  machine: Replace MachineState.topo.smp access with topology helpers
  accel/kvm: Add hybrid info when check cpu num
  hw/acpi: Replace MachineState.smp access with topology helpers
  cpu/core: Use generic topology helper for "help" to set nr_threads
  hw/smbios: Use generic topology name and helper
  migration/postcopy-ram: Use generic topology name and helper
  plugins: Use generic topology name and helper
  softmmu/cpus: Use generic topology helper in vcpus initialization
  general: Replace MachineState.smp access with topology helpers
  i386: Replace MachineState.smp access with topology helpers
  s390x: Replace MachineState.smp access with topology helpers
  ppc: Replace MachineState.smp access with topology helpers
  riscv: Replace MachineState.smp access with topology helpers
  arm: Replace MachineState.smp access with topology helpers
  loongarch: Replace MachineState.smp access with topology helpers
  mips: Replace MachineState.smp access with topology helpers
  hw: Replace MachineState.smp access with topology helpers for all
    remaining archs
  test/test-smp-parse: Check fields of MachineState.topo.smp
  hw/core/machine: Remove support of MachineState.smp
  hw/core/cpu: Introduce TopologyState in CPUState
  i386: Drop nr_dies and nr_modules CPUX86State
  i386/cpu: Use CPUState.topo to replace X86CPUTopoInfo to get topology
    info
  i386: Rename X86CPUTopoInfo and its members to reflect relationship
    with APIC ID
  i386: Rename init_topo_info() to init_apic_topo_info()
  i386: Rename variable topo_info to apicid_topo
  i386: Support APIC ID topology for hybrid CPU topology
  i386: Use init_apicid_topo_info() to initialize APIC ID topology for
    system emulator
  i386: Update X86CPUTopoIDs generating rule for hybrid topology
  i386: Introduce hybrid_core_type to CPUX86State
  i386/cpu: Add Intel hybrid related CPUID support
  qapi: Introduce hybrid options
  machine: Introduce core_type() hook
  hw/machine: Add hybrid_supported in generic topo properties
  hw/machine: Rename MachineClass.smp_props to MachineClass.topo_props
  machine: Add "-hybrid" parsing rule
  hw/machine: Add hybrid cpu topology validation
  hw/machine: build core level hybrid topology form HybridCorePack
  hw/machine: Use opts_visitor to parse hybrid topo
  machine: Support "-hybrid" command
  i386/pc: Support hybrid cpu topology
  qemu-options: Add the document of hybrid command
  qapi: Expose CPU topology info in query_cpus_fast
  i386: Support cpu_index_to_core_type() for x86

 MAINTAINERS                          |   3 +-
 accel/kvm/kvm-all.c                  |   5 +-
 accel/tcg/tcg-accel-ops-mttcg.c      |   2 +-
 accel/tcg/tcg-all.c                  |   2 +-
 backends/hostmem.c                   |   2 +-
 contrib/plugins/hwprofile.c          |   2 +-
 gdbstub/gdbstub.c                    |   2 +-
 hw/acpi/aml-build.c                  |   4 +-
 hw/alpha/dp264.c                     |   2 +-
 hw/arm/fsl-imx6.c                    |   4 +-
 hw/arm/fsl-imx6ul.c                  |   4 +-
 hw/arm/fsl-imx7.c                    |   4 +-
 hw/arm/highbank.c                    |   2 +-
 hw/arm/realview.c                    |   2 +-
 hw/arm/sbsa-ref.c                    |   8 +-
 hw/arm/vexpress.c                    |   2 +-
 hw/arm/virt-acpi-build.c             |   4 +-
 hw/arm/virt.c                        |  56 +-
 hw/arm/xlnx-zynqmp.c                 |   6 +-
 hw/core/cpu-common.c                 |  14 +-
 hw/core/machine-qmp-cmds.c           |  11 +
 hw/core/machine-smp.c                | 198 -------
 hw/core/machine-topo.c               | 853 +++++++++++++++++++++++++++
 hw/core/machine.c                    |  33 +-
 hw/core/meson.build                  |   2 +-
 hw/core/numa.c                       |   2 +-
 hw/cpu/core.c                        |   6 +-
 hw/hppa/machine.c                    |   6 +-
 hw/i386/acpi-build.c                 |   2 +-
 hw/i386/kvmvapic.c                   |   4 +-
 hw/i386/microvm.c                    |   4 +-
 hw/i386/pc.c                         |   5 +-
 hw/i386/pc_piix.c                    |   2 +-
 hw/i386/pc_q35.c                     |   2 +-
 hw/i386/x86.c                        | 255 ++++++--
 hw/i386/xen/xen-hvm.c                |   4 +-
 hw/intc/spapr_xive_kvm.c             |   3 +-
 hw/intc/xics_kvm.c                   |   3 +-
 hw/loongarch/acpi-build.c            |   4 +-
 hw/loongarch/fw_cfg.c                |   4 +-
 hw/loongarch/virt.c                  |  30 +-
 hw/mips/boston.c                     |   8 +-
 hw/mips/loongson3_bootp.c            |   7 +-
 hw/mips/loongson3_virt.c             |   8 +-
 hw/mips/malta.c                      |  22 +-
 hw/openrisc/openrisc_sim.c           |   2 +-
 hw/openrisc/virt.c                   |   2 +-
 hw/ppc/e500.c                        |   6 +-
 hw/ppc/mac_newworld.c                |  16 +-
 hw/ppc/mac_oldworld.c                |  12 +-
 hw/ppc/pnv.c                         |  11 +-
 hw/ppc/prep.c                        |   3 +-
 hw/ppc/spapr.c                       |  39 +-
 hw/ppc/spapr_rtas.c                  |  17 +-
 hw/riscv/microchip_pfsoc.c           |  11 +-
 hw/riscv/numa.c                      |  21 +-
 hw/riscv/opentitan.c                 |   8 +-
 hw/riscv/shakti_c.c                  |   2 +-
 hw/riscv/sifive_e.c                  |  10 +-
 hw/riscv/sifive_u.c                  |  28 +-
 hw/riscv/virt.c                      |  24 +-
 hw/s390x/s390-virtio-ccw.c           |   8 +-
 hw/s390x/sclp.c                      |   3 +-
 hw/smbios/smbios.c                   |  22 +-
 hw/sparc/sun4m.c                     |   4 +-
 hw/sparc64/sun4u.c                   |   6 +-
 hw/virtio/virtio-pci.c               |   2 +-
 hw/xtensa/sim.c                      |   2 +-
 hw/xtensa/xtfpga.c                   |   2 +-
 include/hw/arm/virt.h                |   2 +-
 include/hw/boards.h                  |  93 ++-
 include/hw/core/cpu.h                |  26 +-
 include/hw/cpu/cpu-topology.h        | 137 +++++
 include/hw/i386/topology.h           | 128 ++--
 include/hw/i386/x86.h                |   3 +-
 include/qemu/qemu-plugin.h           |   4 +-
 migration/postcopy-ram.c             |  24 +-
 plugins/api.c                        |   4 +-
 plugins/loader.c                     |   4 +-
 qapi/machine.json                    | 119 +++-
 qemu-options.hx                      |  84 +++
 softmmu/cpus.c                       |  18 +-
 softmmu/dirtylimit.c                 |   8 +-
 softmmu/vl.c                         |  35 +-
 target/arm/cpu.c                     |   2 +-
 target/arm/cpu_tcg.c                 |   2 +-
 target/arm/kvm.c                     |   2 +-
 target/i386/cpu.c                    | 172 ++++--
 target/i386/cpu.h                    |  16 +-
 target/i386/hax/hax-all.c            |   2 +-
 target/i386/hvf/x86_emu.c            |   6 +-
 target/i386/kvm/kvm.c                |   8 +-
 target/i386/tcg/sysemu/misc_helper.c |   2 +-
 target/i386/whpx/whpx-all.c          |   6 +-
 target/mips/tcg/sysemu/cp0_helper.c  |   4 +-
 target/openrisc/sys_helper.c         |   2 +-
 target/ppc/compat.c                  |   2 +-
 target/s390x/cpu-sysemu.c            |   2 +-
 target/s390x/kvm/kvm.c               |  15 +-
 target/s390x/tcg/excp_helper.c       |   2 +-
 tests/unit/meson.build               |   2 +-
 tests/unit/test-smp-parse.c          |  74 ++-
 tests/unit/test-x86-apicid.c         | 211 ++++---
 103 files changed, 2250 insertions(+), 840 deletions(-)
 delete mode 100644 hw/core/machine-smp.c
 create mode 100644 hw/core/machine-topo.c
 create mode 100644 include/hw/cpu/cpu-topology.h

-- 
2.34.1

[RFC 01/52] hw/smbios: Fix smbios_smp_sockets caculation

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Here're 2 mistakes:
1. 003f230 (machine: Tweak the order of topology members in struct
   CpuTopology) changes the meaning of smp.cores but doesn't fix
   original smp.cores uses. And because of the introduction of cluster,
   now smp.cores means the number of cores in one cluster. So smp.cores
   * smp.threads just means the cpus in a cluster not in a socket.
2. smp.cpus means the number of initial online cpus, not the total
   number of cpus. For such topology calculation, smp.max_cpus
   should be considered.

Since the number of sockets has already been recorded in smp structure,
use smp.sockets directly.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Ani Sinha <ani@anisinha.ca>
Fixes: 003f230 (machine: Tweak the order of topology members in struct CpuTopology)
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/smbios/smbios.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index 4869566cf5df..993e8d899b9a 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -1086,8 +1086,7 @@ void smbios_get_tables(MachineState *ms,
         smbios_build_type_2_table();
         smbios_build_type_3_table();
 
-        smbios_smp_sockets = DIV_ROUND_UP(ms->smp.cpus,
-                                          ms->smp.cores * ms->smp.threads);
+        smbios_smp_sockets = ms->smp.sockets;
         assert(smbios_smp_sockets >= 1);
 
         for (i = 0; i < smbios_smp_sockets; i++) {
-- 
2.34.1

[RFC 02/52] hw/smbios: Fix thread count in type4

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

From SMBIOS 3.0 specification, thread count field means:

Thread Count is the total number of threads detected by the BIOS for
this processor socket. It is a processor-wide count, not a
thread-per-core count. [1]

So here we should use threads per socket other than threads per core.

[1] SMBIOS 3.0.0, section 7.5.8, Processor Information - Thread Count

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Ani Sinha <ani@anisinha.ca>
Fixes: c97294e (SMBIOS: Build aggregate smbios tables and entry point)
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/smbios/smbios.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index 993e8d899b9a..5e6d4584edbb 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -713,6 +713,7 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
 {
     char sock_str[128];
     size_t tbl_len = SMBIOS_TYPE_4_LEN_V28;
+    unsigned cpus_per_socket = ms->smp.max_cpus / ms->smp.sockets;
 
     if (smbios_ep_type == SMBIOS_ENTRY_POINT_TYPE_64) {
         tbl_len = SMBIOS_TYPE_4_LEN_V30;
@@ -752,8 +753,8 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
 
     t->core_count2 = t->core_enabled2 = cpu_to_le16(ms->smp.cores);
 
-    t->thread_count = (ms->smp.threads > 255) ? 0xFF : ms->smp.threads;
-    t->thread_count2 = cpu_to_le16(ms->smp.threads);
+    t->thread_count = (cpus_per_socket > 255) ? 0xFF : cpus_per_socket;
+    t->thread_count2 = cpu_to_le16(cpus_per_socket);
 
     t->processor_characteristics = cpu_to_le16(0x02); /* Unknown */
     t->processor_family2 = cpu_to_le16(0x01); /* Other */
-- 
2.34.1

[RFC 03/52] hw/smbios: Fix core count in type4

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

From SMBIOS 3.0 specification, core count field means:

Core Count is the number of cores detected by the BIOS for this
processor socket. [1]

Before 003f230 (machine: Tweak the order of topology members in struct
CpuTopology), MachineState.smp.cores means "the number of cores in one
package", and it's correct to use smp.cores for core count.

But 003f230 changes the smp.cores' meaning to "the number of cores in
one die" and doesn't change the original smp.cores' use in smbios as
well, which makes core count in type4 go wrong.

Fix this issue with the correct "cores per socket" caculation.

[1] SMBIOS 3.0.0, section 7.5.6, Processor Information - Core Count

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Ani Sinha <ani@anisinha.ca>
Fixes: 003f230 (machine: Tweak the order of topology members in struct CpuTopology)
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/smbios/smbios.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index 5e6d4584edbb..cd4aa331cb3e 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -714,6 +714,7 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
     char sock_str[128];
     size_t tbl_len = SMBIOS_TYPE_4_LEN_V28;
     unsigned cpus_per_socket = ms->smp.max_cpus / ms->smp.sockets;
+    unsigned cores_per_socket = cpus_per_socket / ms->smp.threads;
 
     if (smbios_ep_type == SMBIOS_ENTRY_POINT_TYPE_64) {
         tbl_len = SMBIOS_TYPE_4_LEN_V30;
@@ -748,10 +749,10 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
     SMBIOS_TABLE_SET_STR(4, asset_tag_number_str, type4.asset);
     SMBIOS_TABLE_SET_STR(4, part_number_str, type4.part);
 
-    t->core_count = (ms->smp.cores > 255) ? 0xFF : ms->smp.cores;
+    t->core_count = (cores_per_socket > 255) ? 0xFF : cores_per_socket;
     t->core_enabled = t->core_count;
 
-    t->core_count2 = t->core_enabled2 = cpu_to_le16(ms->smp.cores);
+    t->core_count2 = t->core_enabled2 = cpu_to_le16(cores_per_socket);
 
     t->thread_count = (cpus_per_socket > 255) ? 0xFF : cpus_per_socket;
     t->thread_count2 = cpu_to_le16(cpus_per_socket);
-- 
2.34.1

[RFC 04/52] i386/WHPX: Fix error message when fail to set ProcessorCount

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

003f230 (machine: Tweak the order of topology members in struct
CpuTopology) changes the meaning of MachineState.smp.cores from "the
number of cores in one package" to "the number of cores in one die"
and doesn't fix other uses of MachineState.smp.cores. And because of
the introduction of cluster, now smp.cores just means "the number of
cores in one cluster". This clearly does not fit the semantics here.

And before this error message, WHvSetPartitionProperty() is called to
set prop.ProcessorCount.

So the error message should show the prop.ProcessorCount other than
"cores per cluster" or "cores per package".

Cc: Sunil Muthuswamy <sunilmut@microsoft.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 target/i386/whpx/whpx-all.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index e738d83e8191..fc349f887e47 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -2613,8 +2613,8 @@ static int whpx_accel_init(MachineState *ms)
         sizeof(WHV_PARTITION_PROPERTY));
 
     if (FAILED(hr)) {
-        error_report("WHPX: Failed to set partition core count to %d,"
-                     " hr=%08lx", ms->smp.cores, hr);
+        error_report("WHPX: Failed to set partition processor count to %d,"
+                     " hr=%08lx", prop.ProcessorCount, hr);
         ret = -EINVAL;
         goto error;
     }
-- 
2.34.1

[RFC 05/52] hw/core/machine: Rename machine-smp.c to machine-topo.c

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Hybrid topology support will also be added to this file.

In order to maintain the semantic consistency of the file name and
content, the file name is changed to "machine-topo.c".

This file will handle all cpu topology related things, thus also change
the file description comment.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                               | 2 +-
 hw/core/{machine-smp.c => machine-topo.c} | 2 +-
 hw/core/meson.build                       | 2 +-
 tests/unit/meson.build                    | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)
 rename hw/core/{machine-smp.c => machine-topo.c} (99%)

diff --git a/MAINTAINERS b/MAINTAINERS
index 71c1bc24371b..58794885ced3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1733,7 +1733,7 @@ F: cpu.c
 F: hw/core/cpu.c
 F: hw/core/machine-qmp-cmds.c
 F: hw/core/machine.c
-F: hw/core/machine-smp.c
+F: hw/core/machine-topo.c
 F: hw/core/null-machine.c
 F: hw/core/numa.c
 F: hw/cpu/cluster.c
diff --git a/hw/core/machine-smp.c b/hw/core/machine-topo.c
similarity index 99%
rename from hw/core/machine-smp.c
rename to hw/core/machine-topo.c
index 3fd9e641efde..8066d2c46bef 100644
--- a/hw/core/machine-smp.c
+++ b/hw/core/machine-topo.c
@@ -1,5 +1,5 @@
 /*
- * QEMU Machine core (related to -smp parsing)
+ * QEMU Machine core (related to CPU topology)
  *
  * Copyright (c) 2021 Huawei Technologies Co., Ltd
  *
diff --git a/hw/core/meson.build b/hw/core/meson.build
index 7a4d02b6c050..71d523dc6826 100644
--- a/hw/core/meson.build
+++ b/hw/core/meson.build
@@ -23,7 +23,7 @@ else
 endif
 
 common_ss.add(files('cpu-common.c'))
-common_ss.add(files('machine-smp.c'))
+common_ss.add(files('machine-topo.c'))
 softmmu_ss.add(when: 'CONFIG_FITLOADER', if_true: files('loader-fit.c'))
 softmmu_ss.add(when: 'CONFIG_GENERIC_LOADER', if_true: files('generic-loader.c'))
 softmmu_ss.add(when: ['CONFIG_GUEST_LOADER', fdt], if_true: files('guest-loader.c'))
diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index a9df2843e92e..1720a769f1a2 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -46,7 +46,7 @@ tests = {
   'test-uuid': [],
   'ptimer-test': ['ptimer-test-stubs.c', meson.project_source_root() / 'hw/core/ptimer.c'],
   'test-qapi-util': [],
-  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c'],
+  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-topo.c'],
   'test-interval-tree': [],
 }
 
-- 
2.34.1

[RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For smp systems, the parts in one topology level are the same. But now
there are more and more systems with hybrid architectures. Different
parts of the same topology level may have differences. For example,
Intel's Alder Lake series CPU has two types of cores, so the CPU
topology is no longer symmetrical.

The hybrid topology is compatible with the smp topology type, that is,
different parts on the same level of the hybrid topology can set to be
the same, but the hybrid topology will introduce more complexity (need
to allocate more memory, organized with array or linked-list), so the
original smp topology support is retained while introducing the hybrid
topology, and the hybrid topology is only built when the hybrid is
explicitly required.

Therefore, we introduce the definition support of hybrid cpu topology
here. At the same time, in order to unify with the original smp, we
introduce a new cpu topology structure that can support smp topology
or hybrid topology. This structure will replace the CpuTopology type (in
include/hw/boards.h) used by MachineState.smp.

As for now, we only support two hybrid topology levels: core and
cluster.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 MAINTAINERS                   |   1 +
 include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
 qapi/machine.json             |  12 ++++
 3 files changed, 130 insertions(+)
 create mode 100644 include/hw/cpu/cpu-topology.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 58794885ced3..918a9418d98e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
 F: include/hw/boards.h
 F: include/hw/core/cpu.h
 F: include/hw/cpu/cluster.h
+F: include/hw/cpu/cpu-topology.h
 F: include/sysemu/numa.h
 F: tests/unit/test-smp-parse.c
 T: git https://gitlab.com/ehabkost/qemu.git machine-next
diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
new file mode 100644
index 000000000000..8268ea3a8569
--- /dev/null
+++ b/include/hw/cpu/cpu-topology.h
@@ -0,0 +1,117 @@
+/*
+ * CPU topology defination for Machine core
+ *
+ * Copyright (c) 2023 Intel Corporation
+ * Author: Zhao Liu <zhao1.liu@intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef CPU_TOPOLOGY_H
+#define CPU_TOPOLOGY_H
+
+#include "qemu/queue.h"
+
+/**
+ * SmpCpuTopology - smp cpu topology defination.
+ *
+ * For smp system, the parts in one topology level are the same.
+ *
+ * @sockets: the number of sockets on the machine
+ * @dies: the number of dies in one socket
+ * @clusters: the number of clusters in one die
+ * @cores: the number of cores in one cluster
+ * @threads: the number of threads in one core
+ */
+typedef struct SmpCpuTopology {
+    unsigned int sockets;
+    unsigned int dies;
+    unsigned int clusters;
+    unsigned int cores;
+    unsigned int threads;
+} SmpCpuTopology;
+
+/**
+ * HybridCore - hybrid core topology defination:
+ * @threads: the number of threads in one core.
+ */
+typedef struct HybridCore {
+    unsigned int threads;
+} HybridCore;
+
+/**
+ * HybridCluster - hybrid cluster topology defination:
+ * @cores: the number of cores in current cluster.
+ * @core_list: the array includes all the cores that belong to current
+ *             cluster.
+ */
+typedef struct HybridCluster {
+    unsigned int cores;
+    HybridCore *core_list;
+} HybridCluster;
+
+/**
+ * HybridCpuTopology - hybrid cpu topology defination.
+ *
+ * At present we only support two heterogeneous topology levels: core
+ * and cluster. For heterogeneous levels, we need additional structs
+ * to define their custom internal topology. So here we defines
+ * symmetric topology levels, and use a list to point to heterogeneous
+ * levels.
+ *
+ * @sockets: the number of sockets on the machine. All sockets are the
+ *           same.
+ * @dies: the number of dies in one socket. All dies are the same.
+ * @clusters: the number of clusters in one die. Cluster may be
+ *            different. This field indicates the length of
+ *            cluster_list.
+ * @cluster_list: the array includes all the clusters in one die.
+ */
+typedef struct HybridCpuTopology {
+    unsigned int sockets;
+    unsigned int dies;
+    unsigned int clusters;
+    HybridCluster *cluster_list;
+} HybridCpuTopology;
+
+/**
+ * GeneralCpuTopology - General cpu topology defination.
+ *
+ * It supports one of two topologies: smp topology or hybrid topology.
+ *
+ * @cpus: the number of present logical processors on the machine
+ * @max_cpus: the maximum number of logical processors on the machine
+ * @topo_type: the topology type of the machine and this decides which
+ *             member of the union to visit: smp or hybrid.
+ * @smp: the smp cpu topology informantion. Only valid when topo_type is
+ *       CPU_TOPO_TYPE_SMP.
+ * @hybrid: the hybrid cpu topology informantion. Only valid when
+ *          topo_type is CPU_TOPO_TYPE_HYBRID.
+ */
+typedef struct GeneralCpuTopology {
+    unsigned int cpus;
+    unsigned int max_cpus;
+    CpuTopoType topo_type;
+    union {
+        SmpCpuTopology smp;
+        HybridCpuTopology hybrid;
+    };
+} GeneralCpuTopology; /*
+                       * TODO: This name is temporary, just to distinguish it
+                       * from the CpuTopology in boards.h. When CpuTopology in
+                       * boards.h is merged here, it will be uniformly named as
+                       * CpuTopology.
+                       */
+
+#endif /* CPU_TOPOLOGY_H */
diff --git a/qapi/machine.json b/qapi/machine.json
index b9228a5e4616..bd7303f34497 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -36,6 +36,18 @@
              'sh4eb', 'sparc', 'sparc64', 'tricore',
              'x86_64', 'xtensa', 'xtensaeb' ] }
 
+##
+# @CpuTopoType:
+#
+# An enumeration of cpu topology type
+# TODO: Expose topology type in query-cpus-fast
+#
+# Since: 8.0
+##
+{ 'enum': 'CpuTopoType',
+  'prefix': 'CPU_TOPO_TYPE',
+  'data': [ 'smp', 'hybrid' ] }
+
 ##
 # @CpuS390State:
 #
-- 
2.34.1

[RFC 07/52] hw/core/machine: Add the new topology support in MachineState

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Add MachineState.topo to represent CPU topology, and initialize this
field as smp topology.

Also collect topology information from MachineState.topo in
machine_get_smp().

Additionally, check the validity of MachineState.topo instead of
MachineState.smp.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 26 +++++++++++++++++---------
 hw/core/machine.c      | 23 ++++++++++++++++-------
 include/hw/boards.h    |  2 ++
 3 files changed, 35 insertions(+), 16 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 8066d2c46bef..7223f73f99b0 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -31,18 +31,18 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
     MachineClass *mc = MACHINE_GET_CLASS(ms);
     GString *s = g_string_new(NULL);
 
-    g_string_append_printf(s, "sockets (%u)", ms->smp.sockets);
+    g_string_append_printf(s, "sockets (%u)", ms->topo.smp.sockets);
 
     if (mc->smp_props.dies_supported) {
-        g_string_append_printf(s, " * dies (%u)", ms->smp.dies);
+        g_string_append_printf(s, " * dies (%u)", ms->topo.smp.dies);
     }
 
     if (mc->smp_props.clusters_supported) {
-        g_string_append_printf(s, " * clusters (%u)", ms->smp.clusters);
+        g_string_append_printf(s, " * clusters (%u)", ms->topo.smp.clusters);
     }
 
-    g_string_append_printf(s, " * cores (%u)", ms->smp.cores);
-    g_string_append_printf(s, " * threads (%u)", ms->smp.threads);
+    g_string_append_printf(s, " * cores (%u)", ms->topo.smp.cores);
+    g_string_append_printf(s, " * threads (%u)", ms->topo.smp.threads);
 
     return g_string_free(s, false);
 }
@@ -159,6 +159,14 @@ void machine_parse_smp_config(MachineState *ms,
     ms->smp.threads = threads;
     ms->smp.max_cpus = maxcpus;
 
+    ms->topo.cpus = cpus;
+    ms->topo.max_cpus = maxcpus;
+    ms->topo.smp.sockets = sockets;
+    ms->topo.smp.dies = dies;
+    ms->topo.smp.clusters = clusters;
+    ms->topo.smp.cores = cores;
+    ms->topo.smp.threads = threads;
+
     mc->smp_props.has_clusters = config->has_clusters;
 
     /* sanity-check of the computed topology */
@@ -180,18 +188,18 @@ void machine_parse_smp_config(MachineState *ms,
         return;
     }
 
-    if (ms->smp.cpus < mc->min_cpus) {
+    if (ms->topo.cpus < mc->min_cpus) {
         error_setg(errp, "Invalid SMP CPUs %d. The min CPUs "
                    "supported by machine '%s' is %d",
-                   ms->smp.cpus,
+                   ms->topo.cpus,
                    mc->name, mc->min_cpus);
         return;
     }
 
-    if (ms->smp.max_cpus > mc->max_cpus) {
+    if (ms->topo.max_cpus > mc->max_cpus) {
         error_setg(errp, "Invalid SMP CPUs %d. The max CPUs "
                    "supported by machine '%s' is %d",
-                   ms->smp.max_cpus,
+                   ms->topo.max_cpus,
                    mc->name, mc->max_cpus);
         return;
     }
diff --git a/hw/core/machine.c b/hw/core/machine.c
index f73fc4c45c41..56e796c18873 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -829,13 +829,13 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,
 {
     MachineState *ms = MACHINE(obj);
     SMPConfiguration *config = &(SMPConfiguration){
-        .has_cpus = true, .cpus = ms->smp.cpus,
-        .has_sockets = true, .sockets = ms->smp.sockets,
-        .has_dies = true, .dies = ms->smp.dies,
-        .has_clusters = true, .clusters = ms->smp.clusters,
-        .has_cores = true, .cores = ms->smp.cores,
-        .has_threads = true, .threads = ms->smp.threads,
-        .has_maxcpus = true, .maxcpus = ms->smp.max_cpus,
+        .has_cpus = true, .cpus = ms->topo.cpus,
+        .has_sockets = true, .sockets = ms->topo.smp.sockets,
+        .has_dies = true, .dies = ms->topo.smp.dies,
+        .has_clusters = true, .clusters = ms->topo.smp.clusters,
+        .has_cores = true, .cores = ms->topo.smp.cores,
+        .has_threads = true, .threads = ms->topo.smp.threads,
+        .has_maxcpus = true, .maxcpus = ms->topo.max_cpus,
     };
 
     if (!visit_type_SMPConfiguration(v, name, &config, &error_abort)) {
@@ -1101,6 +1101,15 @@ static void machine_initfn(Object *obj)
     ms->smp.cores = 1;
     ms->smp.threads = 1;
 
+    ms->topo.cpus = mc->default_cpus;
+    ms->topo.max_cpus = mc->default_cpus;
+    ms->topo.topo_type = CPU_TOPO_TYPE_SMP;
+    ms->topo.smp.sockets = 1;
+    ms->topo.smp.dies = 1;
+    ms->topo.smp.clusters = 1;
+    ms->topo.smp.cores = 1;
+    ms->topo.smp.threads = 1;
+
     machine_copy_boot_config(ms, &(BootConfiguration){ 0 });
 }
 
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6fbbfd56c808..0a61855499e3 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -10,6 +10,7 @@
 #include "qemu/module.h"
 #include "qom/object.h"
 #include "hw/core/cpu.h"
+#include "hw/cpu/cpu-topology.h"
 
 #define TYPE_MACHINE_SUFFIX "-machine"
 
@@ -360,6 +361,7 @@ struct MachineState {
     AccelState *accelerator;
     CPUArchIdList *possible_cpus;
     CpuTopology smp;
+    GeneralCpuTopology topo; /* TODO: Completely replace MachineState.smp */
     struct NVDIMMState *nvdimms_state;
     struct NumaState *numa_state;
 };
-- 
2.34.1

[RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. In the general case (hybrid or smp topology),
accessing the topology information needs to determine whether it is
currently smp or hybrid topology, and then access the corresponding
MachineState.topo.smp or MachineState.topo.hybrid.

The best way to do this is to wrap the access to the topology to
avoid having to check each time it is accessed.

The following helpers are provided here:

- General interfaces - no need to worry about whether the underlying
  topology is smp or hybrid:

* machine_topo_get_cpus()
* machine_topo_get_max_cpus()
* machine_topo_is_smp()
* machine_topo_get_sockets()
* machine_topo_get_dies()
* machine_topo_get_clusters()
* machine_topo_get_threads();
* machine_topo_get_cores();
* machine_topo_get_threads_by_idx()
* machine_topo_get_cores_by_idx()
* machine_topo_get_cores_per_socket()
* machine_topo_get_threads_per_socket()

- SMP-specific interfaces - provided for the cases that are clearly
known to be smp topology:

* machine_topo_get_smp_cores()
* machine_topo_get_smp_threads()

Since for hybrid topology, each core may has different threads, if
someone wants "cpus per core", the cpu_index is need to target a
specific core (machine_topo_get_threads_by_idx()). But for smp, there is
no need to be so troublesome, so for this case, we provide smp-specific
interfaces.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 142 +++++++++++++++++++++++++++++++++++++++++
 include/hw/boards.h    |  35 ++++++++++
 2 files changed, 177 insertions(+)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 7223f73f99b0..b20160479629 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -21,6 +21,148 @@
 #include "hw/boards.h"
 #include "qapi/error.h"
 
+unsigned int machine_topo_get_sockets(const MachineState *ms)
+{
+    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
+                                     ms->topo.hybrid.sockets;
+}
+
+unsigned int machine_topo_get_dies(const MachineState *ms)
+{
+    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
+                                     ms->topo.hybrid.dies;
+}
+
+unsigned int machine_topo_get_clusters(const MachineState *ms)
+{
+    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
+                                     ms->topo.hybrid.clusters;
+}
+
+unsigned int machine_topo_get_smp_cores(const MachineState *ms)
+{
+    g_assert(machine_topo_is_smp(ms));
+    return ms->topo.smp.cores;
+}
+
+unsigned int machine_topo_get_smp_threads(const MachineState *ms)
+{
+    g_assert(machine_topo_is_smp(ms));
+    return ms->topo.smp.threads;
+}
+
+unsigned int machine_topo_get_threads(const MachineState *ms,
+                                      unsigned int cluster_id,
+                                      unsigned int core_id)
+{
+    if (machine_topo_is_smp(ms)) {
+        return ms->topo.smp.threads;
+    } else {
+        return ms->topo.hybrid.cluster_list[cluster_id]
+                   .core_list[core_id].threads;
+    }
+
+    return 0;
+}
+
+unsigned int machine_topo_get_cores(const MachineState *ms,
+                                    unsigned int cluster_id)
+{
+    if (machine_topo_is_smp(ms)) {
+        return ms->topo.smp.cores;
+    } else {
+        return ms->topo.hybrid.cluster_list[cluster_id].cores;
+    }
+}
+
+unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
+                                             unsigned int cpu_index)
+{
+    unsigned cpus_per_die;
+    unsigned tmp_idx;
+    HybridCluster *cluster;
+    HybridCore *core;
+
+    if (machine_topo_is_smp(ms)) {
+        return ms->topo.smp.threads;
+    }
+
+    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
+                            ms->topo.hybrid.dies);
+    tmp_idx = cpu_index % cpus_per_die;
+
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        cluster = &ms->topo.hybrid.cluster_list[i];
+
+        for (int j = 0; j < cluster->cores; j++) {
+            core = &cluster->core_list[j];
+
+            if (tmp_idx < core->threads) {
+                return core->threads;
+            } else {
+                tmp_idx -= core->threads;
+            }
+        }
+    }
+
+    return 0;
+}
+
+unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
+                                           unsigned int cpu_index)
+{
+    unsigned cpus_per_die;
+    unsigned tmp_idx;
+    HybridCluster *cluster;
+    HybridCore *core;
+
+    if (machine_topo_is_smp(ms)) {
+        return ms->topo.smp.cores;
+    }
+
+    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
+                            ms->topo.hybrid.dies);
+    tmp_idx = cpu_index % cpus_per_die;
+
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        cluster = &ms->topo.hybrid.cluster_list[i];
+
+        for (int j = 0; j < cluster->cores; j++) {
+            core = &cluster->core_list[j];
+
+            if (tmp_idx < core->threads) {
+                return cluster->cores;
+            } else {
+                tmp_idx -= core->threads;
+            }
+        }
+    }
+
+    return 0;
+}
+
+unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
+{
+    unsigned int cores_per_die = 0;
+
+    if (machine_topo_is_smp(ms)) {
+        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
+    }
+
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
+    }
+
+    return cores_per_die * ms->topo.hybrid.dies;
+}
+
+unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
+{
+    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
+                           ms->topo.hybrid.sockets;
+    return ms->topo.max_cpus / sockets;
+}
+
 /*
  * Report information of a machine's supported CPU topology hierarchy.
  * Topology members will be ordered from the largest to the smallest
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 0a61855499e3..34b64b012022 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;
 extern GlobalProperty hw_compat_2_1[];
 extern const size_t hw_compat_2_1_len;
 
+static inline
+unsigned int machine_topo_get_cpus(const MachineState *ms)
+{
+    return ms->topo.cpus;
+}
+
+static inline
+unsigned int machine_topo_get_max_cpus(const MachineState *ms)
+{
+    return ms->topo.max_cpus;
+}
+
+static inline
+bool machine_topo_is_smp(const MachineState *ms)
+{
+    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP;
+}
+
+unsigned int machine_topo_get_sockets(const MachineState *ms);
+unsigned int machine_topo_get_dies(const MachineState *ms);
+unsigned int machine_topo_get_clusters(const MachineState *ms);
+unsigned int machine_topo_get_smp_cores(const MachineState *ms);
+unsigned int machine_topo_get_smp_threads(const MachineState *ms);
+unsigned int machine_topo_get_threads(const MachineState *ms,
+                                      unsigned int cluster_id,
+                                      unsigned int core_id);
+unsigned int machine_topo_get_cores(const MachineState *ms,
+                                    unsigned int cluster_id);
+unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
+                                             unsigned int cpu_index);
+unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
+                                           unsigned int cpu_index);
+unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
+unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
+
 #endif
-- 
2.34.1

[RFC 09/52] hw/machine: Introduce core type for hybrid topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Under the hybrid cpu topology, some CPUs care about the core type.
For example, Intel's Alder Lake series CPU contains two types of cores:
Intel Core and Intel Atom. The type information of these two types is
exposed in 1A leaf of CPUID.

Core type should also be part of the hybrid topology, and
MachineState.cpu_type cannot provide different type information for
different cpus in the same machine, so add a type field for the core
level in the hybrid cpu topology.

Additionally, add a helper to get core type information from
MachineState. Though core_type is only used in hybrid case, don't
use assert since it may be used to initialize some generic fields.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c        | 12 ++++++++++++
 include/hw/boards.h           |  3 +++
 include/hw/cpu/cpu-topology.h |  2 ++
 3 files changed, 17 insertions(+)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index b20160479629..e0ec07b53d42 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -51,6 +51,18 @@ unsigned int machine_topo_get_smp_threads(const MachineState *ms)
     return ms->topo.smp.threads;
 }
 
+unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
+                                               unsigned int cluster_id,
+                                               unsigned int core_id)
+{
+    if (!machine_topo_is_smp(ms)) {
+        return ms->topo.hybrid.cluster_list[cluster_id]
+                       .core_list[core_id].core_type;
+    } else {
+        return 0;
+    }
+}
+
 unsigned int machine_topo_get_threads(const MachineState *ms,
                                       unsigned int cluster_id,
                                       unsigned int core_id)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 34b64b012022..78e52af38cb1 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -484,6 +484,9 @@ unsigned int machine_topo_get_dies(const MachineState *ms);
 unsigned int machine_topo_get_clusters(const MachineState *ms);
 unsigned int machine_topo_get_smp_cores(const MachineState *ms);
 unsigned int machine_topo_get_smp_threads(const MachineState *ms);
+unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
+                                               unsigned int cluster_id,
+                                               unsigned int core_id);
 unsigned int machine_topo_get_threads(const MachineState *ms,
                                       unsigned int cluster_id,
                                       unsigned int core_id);
diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
index 8268ea3a8569..87d832556229 100644
--- a/include/hw/cpu/cpu-topology.h
+++ b/include/hw/cpu/cpu-topology.h
@@ -45,9 +45,11 @@ typedef struct SmpCpuTopology {
 /**
  * HybridCore - hybrid core topology defination:
  * @threads: the number of threads in one core.
+ * @core_type: the type of current core.
  */
 typedef struct HybridCore {
     unsigned int threads;
+    unsigned int core_type;
 } HybridCore;
 
 /**
-- 
2.34.1

[RFC 10/52] machine: Replace MachineState.topo.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

In cpu_hierarchy_to_string(), the access to MachineState.topo.smp can
be replaced by topology helpers.

But because machine_parse_smp_config() is used to adjust
MachineState.topo.smp, there's no need to replace with helpers.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index e0ec07b53d42..cdbe65ffd9fd 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -185,18 +185,20 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
     MachineClass *mc = MACHINE_GET_CLASS(ms);
     GString *s = g_string_new(NULL);
 
-    g_string_append_printf(s, "sockets (%u)", ms->topo.smp.sockets);
+    g_string_append_printf(s, "sockets (%u)", machine_topo_get_sockets(ms));
 
     if (mc->smp_props.dies_supported) {
-        g_string_append_printf(s, " * dies (%u)", ms->topo.smp.dies);
+        g_string_append_printf(s, " * dies (%u)", machine_topo_get_dies(ms));
     }
 
     if (mc->smp_props.clusters_supported) {
-        g_string_append_printf(s, " * clusters (%u)", ms->topo.smp.clusters);
+        g_string_append_printf(s, " * clusters (%u)",
+                               machine_topo_get_clusters(ms));
     }
 
-    g_string_append_printf(s, " * cores (%u)", ms->topo.smp.cores);
-    g_string_append_printf(s, " * threads (%u)", ms->topo.smp.threads);
+    g_string_append_printf(s, " * cores (%u)", machine_topo_get_smp_cores(ms));
+    g_string_append_printf(s, " * threads (%u)",
+                           machine_topo_get_smp_threads(ms));
 
     return g_string_free(s, false);
 }
-- 
2.34.1

[RFC 11/52] accel/kvm: Add hybrid info when check cpu num

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

As the generic code, kvm should respect the different topologies: smp
or hybrid.

So improve the topology related informantion here.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 accel/kvm/kvm-all.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 9b2658265561..e62df7b579d0 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2361,12 +2361,13 @@ static int kvm_init(MachineState *ms)
     static const char upgrade_note[] =
         "Please upgrade to at least kernel 2.6.29 or recent kvm-kmod\n"
         "(see http://sourceforge.net/projects/kvm).\n";
+    const char *cpu_name = machine_topo_is_smp(ms) ? "SMP" : "Hybrid";
     struct {
         const char *name;
         int num;
     } num_cpus[] = {
-        { "SMP",          ms->smp.cpus },
-        { "hotpluggable", ms->smp.max_cpus },
+        { cpu_name,       machine_topo_get_cpus(ms) },
+        { "hotpluggable", machine_topo_get_max_cpus(ms) },
         { NULL, }
     }, *nc = num_cpus;
     int soft_vcpus_limit, hard_vcpus_limit;
-- 
2.34.1

[RFC 12/52] hw/acpi: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

At present, in QEMU only arm needs PPTT table to build cpu topology.

Before QEMU's arm supports hybrid architectures, it's enough to limit
the cpu topology of PPTT to smp type through the explicit smp interface
(machine_topo_get_smp_threads()).

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Ani Sinha <ani@anisinha.ca>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/acpi/aml-build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index ea331a20d131..693bd8833d10 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
             cluster_offset = socket_offset;
         }
 
-        if (ms->smp.threads == 1) {
+        if (machine_topo_get_smp_threads(ms) == 1) {
             build_processor_hierarchy_node(table_data,
                 (1 << 1) | /* ACPI Processor ID valid */
                 (1 << 3),  /* Node is a Leaf */
-- 
2.34.1

[RFC 13/52] cpu/core: Use generic topology helper for "help" to set nr_threads

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The commit 0b47ec4 (cpu/core: Fix "help" of CPU core device types) sets
nr_threads with current_machine's smp.threads for "help".

This will break if current_machine has hybrid topology. Though core
device now is only used by PPC, as the general code, hybrid topology
deserves to be handled more gracefully and does not depend on whether
the specific architecture supports hybrid topology.

So use a generic topology helper to get threads info (of CPU 0) from
current_machine. For smp case, this helper returns "threads per core"
as the original smp.threads, and for hybrid case, it returns threads of
the core that CPU 0 is located.

Since that nr_threads is just to make "help" happy and it isn't used
afterwards, give it the threads of CPU 0 can work.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/cpu/core.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/hw/cpu/core.c b/hw/cpu/core.c
index 987607515574..bc5ab4e01038 100644
--- a/hw/cpu/core.c
+++ b/hw/cpu/core.c
@@ -71,10 +71,12 @@ static void cpu_core_instance_init(Object *obj)
     /*
      * Only '-device something-cpu-core,help' can get us there before
      * the machine has been created. We don't care to set nr_threads
-     * in this case since it isn't used afterwards.
+     * in this case since it isn't used afterwards, so we fetch the
+     * threads of the core where the CPU 0 is located, no matter whether
+     * the topology is smp or hybrid.
      */
     if (current_machine) {
-        core->nr_threads = current_machine->smp.threads;
+        core->nr_threads = machine_topo_get_threads_by_idx(current_machine, 0);
     }
 }
 
-- 
2.34.1

[RFC 14/52] hw/smbios: Use generic topology name and helper

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

As the generic code, smbios should respect the different topologies: smp
or hybrid.

So rename smbios_smp_sockets to smbios_topo_sockets to decouple with smp
topology.

And use generic topology helpers to get topology information.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Ani Sinha <ani@anisinha.ca>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/smbios/smbios.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/hw/smbios/smbios.c b/hw/smbios/smbios.c
index cd4aa331cb3e..2ce1f2b1d94e 100644
--- a/hw/smbios/smbios.c
+++ b/hw/smbios/smbios.c
@@ -70,7 +70,8 @@ static SmbiosEntryPoint ep;
 static int smbios_type4_count = 0;
 static bool smbios_immutable;
 static bool smbios_have_defaults;
-static uint32_t smbios_cpuid_version, smbios_cpuid_features, smbios_smp_sockets;
+static uint32_t smbios_cpuid_version, smbios_cpuid_features;
+static uint32_t smbios_topo_sockets;
 
 static DECLARE_BITMAP(have_binfile_bitmap, SMBIOS_MAX_TYPE+1);
 static DECLARE_BITMAP(have_fields_bitmap, SMBIOS_MAX_TYPE+1);
@@ -461,7 +462,7 @@ opts_init(smbios_register_config);
 static void smbios_validate_table(MachineState *ms)
 {
     uint32_t expect_t4_count = smbios_legacy ?
-                                        ms->smp.cpus : smbios_smp_sockets;
+                               machine_topo_get_cpus(ms) : smbios_topo_sockets;
 
     if (smbios_type4_count && smbios_type4_count != expect_t4_count) {
         error_report("Expected %d SMBIOS Type 4 tables, got %d instead",
@@ -713,8 +714,8 @@ static void smbios_build_type_4_table(MachineState *ms, unsigned instance)
 {
     char sock_str[128];
     size_t tbl_len = SMBIOS_TYPE_4_LEN_V28;
-    unsigned cpus_per_socket = ms->smp.max_cpus / ms->smp.sockets;
-    unsigned cores_per_socket = cpus_per_socket / ms->smp.threads;
+    unsigned cpus_per_socket = machine_topo_get_threads_per_socket(ms);
+    unsigned cores_per_socket = machine_topo_get_cores_per_socket(ms);
 
     if (smbios_ep_type == SMBIOS_ENTRY_POINT_TYPE_64) {
         tbl_len = SMBIOS_TYPE_4_LEN_V30;
@@ -1088,10 +1089,10 @@ void smbios_get_tables(MachineState *ms,
         smbios_build_type_2_table();
         smbios_build_type_3_table();
 
-        smbios_smp_sockets = ms->smp.sockets;
-        assert(smbios_smp_sockets >= 1);
+        smbios_topo_sockets = machine_topo_get_sockets(ms);
+        assert(smbios_topo_sockets >= 1);
 
-        for (i = 0; i < smbios_smp_sockets; i++) {
+        for (i = 0; i < smbios_topo_sockets; i++) {
             smbios_build_type_4_table(ms, i);
         }
 
-- 
2.34.1

[RFC 15/52] migration/postcopy-ram: Use generic topology name and helper

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

As the generic code, here we should respect the different topologies:
smp or hybrid.

So rename PostcopyBlocktimeContext.smp_cpus_down to
PostcopyBlocktimeContext.cpus_down, and also rename other local
variables from smp_cpus to cpus_num, to decouple with smp topology.

And use generic topology helpers to get topology information.

Cc: Juan Quintela <quintela@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 migration/postcopy-ram.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
index 53299b7a5ebd..1e861e313258 100644
--- a/migration/postcopy-ram.c
+++ b/migration/postcopy-ram.c
@@ -122,7 +122,7 @@ typedef struct PostcopyBlocktimeContext {
     /* point in time when last page fault was initiated */
     uint32_t last_begin;
     /* number of vCPU are suspended */
-    int smp_cpus_down;
+    int cpus_down;
     uint64_t start_time;
 
     /*
@@ -150,11 +150,11 @@ static void migration_exit_cb(Notifier *n, void *data)
 static struct PostcopyBlocktimeContext *blocktime_context_new(void)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int cpus_num = machine_topo_get_cpus(ms);
     PostcopyBlocktimeContext *ctx = g_new0(PostcopyBlocktimeContext, 1);
-    ctx->page_fault_vcpu_time = g_new0(uint32_t, smp_cpus);
-    ctx->vcpu_addr = g_new0(uintptr_t, smp_cpus);
-    ctx->vcpu_blocktime = g_new0(uint32_t, smp_cpus);
+    ctx->page_fault_vcpu_time = g_new0(uint32_t, cpus_num);
+    ctx->vcpu_addr = g_new0(uintptr_t, cpus_num);
+    ctx->vcpu_blocktime = g_new0(uint32_t, cpus_num);
 
     ctx->exit_notifier.notify = migration_exit_cb;
     ctx->start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
@@ -168,7 +168,7 @@ static uint32List *get_vcpu_blocktime_list(PostcopyBlocktimeContext *ctx)
     uint32List *list = NULL;
     int i;
 
-    for (i = ms->smp.cpus - 1; i >= 0; i--) {
+    for (i = machine_topo_get_cpus(ms) - 1; i >= 0; i--) {
         QAPI_LIST_PREPEND(list, ctx->vcpu_blocktime[i]);
     }
 
@@ -798,7 +798,7 @@ static void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
 
     low_time_offset = get_low_time_offset(dc);
     if (dc->vcpu_addr[cpu] == 0) {
-        qatomic_inc(&dc->smp_cpus_down);
+        qatomic_inc(&dc->cpus_down);
     }
 
     qatomic_xchg(&dc->last_begin, low_time_offset);
@@ -814,7 +814,7 @@ static void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
     if (already_received) {
         qatomic_xchg(&dc->vcpu_addr[cpu], 0);
         qatomic_xchg(&dc->page_fault_vcpu_time[cpu], 0);
-        qatomic_dec(&dc->smp_cpus_down);
+        qatomic_dec(&dc->cpus_down);
     }
     trace_mark_postcopy_blocktime_begin(addr, dc, dc->page_fault_vcpu_time[cpu],
                                         cpu, already_received);
@@ -852,7 +852,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr)
     MigrationIncomingState *mis = migration_incoming_get_current();
     PostcopyBlocktimeContext *dc = mis->blocktime_ctx;
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int cpus_num = machine_topo_get_cpus(ms);
     int i, affected_cpu = 0;
     bool vcpu_total_blocktime = false;
     uint32_t read_vcpu_time, low_time_offset;
@@ -866,7 +866,7 @@ static void mark_postcopy_blocktime_end(uintptr_t addr)
      * that algorithm looks straightforward, but it's not
      * optimal, more optimal algorithm is keeping tree or hash
      * where key is address value is a list of  */
-    for (i = 0; i < smp_cpus; i++) {
+    for (i = 0; i < cpus_num; i++) {
         uint32_t vcpu_blocktime = 0;
 
         read_vcpu_time = qatomic_fetch_add(&dc->page_fault_vcpu_time[i], 0);
@@ -881,14 +881,14 @@ static void mark_postcopy_blocktime_end(uintptr_t addr)
          * faulted page, another possible case it's prefetched
          * page and in that case we shouldn't be here */
         if (!vcpu_total_blocktime &&
-            qatomic_fetch_add(&dc->smp_cpus_down, 0) == smp_cpus) {
+            qatomic_fetch_add(&dc->cpus_down, 0) == cpus_num) {
             vcpu_total_blocktime = true;
         }
         /* continue cycle, due to one page could affect several vCPUs */
         dc->vcpu_blocktime[i] += vcpu_blocktime;
     }
 
-    qatomic_sub(&dc->smp_cpus_down, affected_cpu);
+    qatomic_sub(&dc->cpus_down, affected_cpu);
     if (vcpu_total_blocktime) {
         dc->total_blocktime += low_time_offset - qatomic_fetch_add(
                 &dc->last_begin, 0);
-- 
2.34.1

[RFC 16/52] plugins: Use generic topology name and helper

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

As the generic code, here we should respect the different topologies:
smp or hybrid.

So rename qemu_info_t.system.smp_vcpus to qemu_info_t.system.vcpus to
decouple with smp topology.

And use generic topology helpers to get topology information.

Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Alexandre Iooss <erdnaxe@crans.org>
Cc: Mahmoud Mandour <ma.mandourr@gmail.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 contrib/plugins/hwprofile.c | 2 +-
 include/qemu/qemu-plugin.h  | 4 ++--
 plugins/api.c               | 4 ++--
 plugins/loader.c            | 4 ++--
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/contrib/plugins/hwprofile.c b/contrib/plugins/hwprofile.c
index 691d4edb0c67..839175396932 100644
--- a/contrib/plugins/hwprofile.c
+++ b/contrib/plugins/hwprofile.c
@@ -307,7 +307,7 @@ int qemu_plugin_install(qemu_plugin_id_t id, const qemu_info_t *info,
     }
 
     /* Just warn about overflow */
-    if (info->system.smp_vcpus > 64 ||
+    if (info->system.vcpus > 64 ||
         info->system.max_vcpus > 64) {
         fprintf(stderr, "hwprofile: can only track up to 64 CPUs\n");
     }
diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index d0e9d03adfe3..529fcc54e43e 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -74,8 +74,8 @@ typedef struct qemu_info_t {
     union {
         /** @system: information relevant to system emulation */
         struct {
-            /** @system.smp_vcpus: initial number of vCPUs */
-            int smp_vcpus;
+            /** @system.vcpus: initial number of vCPUs */
+            int vcpus;
             /** @system.max_vcpus: maximum possible number of vCPUs */
             int max_vcpus;
         } system;
diff --git a/plugins/api.c b/plugins/api.c
index 2078b16edb02..22018083d0cd 100644
--- a/plugins/api.c
+++ b/plugins/api.c
@@ -374,7 +374,7 @@ int qemu_plugin_n_vcpus(void)
 #ifdef CONFIG_USER_ONLY
     return -1;
 #else
-    return get_ms()->smp.cpus;
+    return machine_topo_get_cpus(get_ms());
 #endif
 }
 
@@ -383,7 +383,7 @@ int qemu_plugin_n_max_vcpus(void)
 #ifdef CONFIG_USER_ONLY
     return -1;
 #else
-    return get_ms()->smp.max_cpus;
+    return machine_topo_get_max_cpus(get_ms());
 #endif
 }
 
diff --git a/plugins/loader.c b/plugins/loader.c
index 88c30bde2d6a..c17ece9acc49 100644
--- a/plugins/loader.c
+++ b/plugins/loader.c
@@ -298,8 +298,8 @@ int qemu_plugin_load_list(QemuPluginList *head, Error **errp)
 #ifndef CONFIG_USER_ONLY
     MachineState *ms = MACHINE(qdev_get_machine());
     info->system_emulation = true;
-    info->system.smp_vcpus = ms->smp.cpus;
-    info->system.max_vcpus = ms->smp.max_cpus;
+    info->system.vcpus = machine_topo_get_cpus(ms);
+    info->system.max_vcpus = machine_topo_get_max_cpus(ms);
 #else
     info->system_emulation = false;
 #endif
-- 
2.34.1

[RFC 17/52] softmmu/cpus: Use generic topology helper in vcpus initialization

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

As for the generic vcpu initialization, qemu_init_vcpu() should respect
the different topologies: smp or hybrid.

Use generic topology helpers to get topology information.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 softmmu/cpus.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 9996e6a3b295..7892da9c82b5 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -630,8 +630,8 @@ void qemu_init_vcpu(CPUState *cpu)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
 
-    cpu->nr_cores = ms->smp.dies * ms->smp.clusters * ms->smp.cores;
-    cpu->nr_threads =  ms->smp.threads;
+    cpu->nr_cores = machine_topo_get_cores_per_socket(ms);
+    cpu->nr_threads = machine_topo_get_threads_by_idx(ms, cpu->cpu_index);
     cpu->stopped = true;
     cpu->random_seed = qemu_guest_random_seed_thread_part1();
 
-- 
2.34.1

[RFC 18/52] general: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When "threads" or "cores" are not asked for, smp and hybrid topology do
not affect the access of topology information.

For these generic codes, it is straightforward to replace topology access
with wrapped interfaces.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 accel/tcg/tcg-accel-ops-mttcg.c | 2 +-
 accel/tcg/tcg-all.c             | 2 +-
 backends/hostmem.c              | 2 +-
 gdbstub/gdbstub.c               | 2 +-
 hw/core/numa.c                  | 2 +-
 hw/virtio/virtio-pci.c          | 2 +-
 softmmu/dirtylimit.c            | 8 ++++----
 softmmu/vl.c                    | 2 +-
 8 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index d50239e0e28f..5fee991108dd 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -140,7 +140,7 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
     char thread_name[VCPU_THREAD_NAME_SIZE];
 
     g_assert(tcg_enabled());
-    tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
+    tcg_cpu_init_cflags(cpu, machine_topo_get_max_cpus(current_machine) > 1);
 
     cpu->thread = g_new0(QemuThread, 1);
     cpu->halt_cond = g_malloc0(sizeof(QemuCond));
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index 30b503fb22fc..088af3b6f103 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -116,7 +116,7 @@ static int tcg_init_machine(MachineState *ms)
 #ifdef CONFIG_USER_ONLY
     unsigned max_cpus = 1;
 #else
-    unsigned max_cpus = ms->smp.max_cpus;
+    unsigned max_cpus = machine_topo_get_max_cpus(ms);
 #endif
 
     tcg_allowed = true;
diff --git a/backends/hostmem.c b/backends/hostmem.c
index 747e7838c031..43fcee4c302d 100644
--- a/backends/hostmem.c
+++ b/backends/hostmem.c
@@ -280,7 +280,7 @@ static void host_memory_backend_init(Object *obj)
     backend->merge = machine_mem_merge(machine);
     backend->dump = machine_dump_guest_core(machine);
     backend->reserve = true;
-    backend->prealloc_threads = machine->smp.cpus;
+    backend->prealloc_threads = machine_topo_get_cpus(machine);
 }
 
 static void host_memory_backend_post_init(Object *obj)
diff --git a/gdbstub/gdbstub.c b/gdbstub/gdbstub.c
index be88ca0d7182..bc283778be86 100644
--- a/gdbstub/gdbstub.c
+++ b/gdbstub/gdbstub.c
@@ -1120,7 +1120,7 @@ static int gdb_handle_vcont(const char *p)
     }
 #else
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
 #endif
     /* uninitialised CPUs stay 0 */
     newstates = g_new0(char, max_cpus);
diff --git a/hw/core/numa.c b/hw/core/numa.c
index d8d36b16d80b..dfe14291c518 100644
--- a/hw/core/numa.c
+++ b/hw/core/numa.c
@@ -68,7 +68,7 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
     uint16_t nodenr;
     uint16List *cpus = NULL;
     MachineClass *mc = MACHINE_GET_CLASS(ms);
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
     NodeInfo *numa_info = ms->numa_state->nodes;
 
     if (node->has_nodeid) {
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 247325c1933c..952fcdcca78e 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -2438,7 +2438,7 @@ unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues)
      * doing so arbitrarily would result in a sudden performance drop once the
      * threshold number of vCPUs is exceeded.
      */
-    unsigned num_queues = current_machine->smp.cpus;
+    unsigned num_queues = machine_topo_get_cpus(current_machine);
 
     /*
      * The maximum number of MSI-X vectors is PCI_MSIX_FLAGS_QSIZE + 1, but the
diff --git a/softmmu/dirtylimit.c b/softmmu/dirtylimit.c
index c56f0f58c8c9..4aa6b405d3be 100644
--- a/softmmu/dirtylimit.c
+++ b/softmmu/dirtylimit.c
@@ -146,7 +146,7 @@ void vcpu_dirty_rate_stat_stop(void)
 void vcpu_dirty_rate_stat_initialize(void)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    int max_cpus = ms->smp.max_cpus;
+    int max_cpus = machine_topo_get_max_cpus(ms);
 
     vcpu_dirty_rate_stat =
         g_malloc0(sizeof(*vcpu_dirty_rate_stat));
@@ -191,7 +191,7 @@ static inline VcpuDirtyLimitState *dirtylimit_vcpu_get_state(int cpu_index)
 void dirtylimit_state_initialize(void)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    int max_cpus = ms->smp.max_cpus;
+    int max_cpus = machine_topo_get_max_cpus(ms);
     int i;
 
     dirtylimit_state = g_malloc0(sizeof(*dirtylimit_state));
@@ -228,7 +228,7 @@ bool dirtylimit_vcpu_index_valid(int cpu_index)
     MachineState *ms = MACHINE(qdev_get_machine());
 
     return !(cpu_index < 0 ||
-             cpu_index >= ms->smp.max_cpus);
+             cpu_index >= machine_topo_get_max_cpus(ms));
 }
 
 static inline int64_t dirtylimit_dirty_ring_full_time(uint64_t dirtyrate)
@@ -390,7 +390,7 @@ void dirtylimit_set_all(uint64_t quota,
                         bool enable)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    int max_cpus = ms->smp.max_cpus;
+    int max_cpus = machine_topo_get_max_cpus(ms);
     int i;
 
     for (i = 0; i < max_cpus; i++) {
diff --git a/softmmu/vl.c b/softmmu/vl.c
index b2ee3fee3f06..0547ad390f52 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1851,7 +1851,7 @@ static void qemu_apply_machine_options(QDict *qdict)
         semihosting_arg_fallback(current_machine->kernel_filename, current_machine->kernel_cmdline);
     }
 
-    if (current_machine->smp.cpus > 1) {
+    if (machine_topo_get_cpus(current_machine) > 1) {
         Error *blocker = NULL;
         error_setg(&blocker, QERR_REPLAY_NOT_SUPPORTED, "smp");
         replay_add_blocker(blocker);
-- 
2.34.1

[RFC 19/52] i386: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

For these i386 codes, it is straightforward to replace topology access
with wrapped interfaces.

Note in x86_cpu_pre_plug(), the caculation of "max_socket" can be
replaced with simpler math, and the calculation of "cores" and "threads"
will use general topology helpers to be compatible with both hybrid
topology and smp topology.

Since X86CPUTopoInfo hasn't supported hybrid case, just use smp specific
helpers to get cores_per_cluster and threads_per_core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/acpi-build.c        |  2 +-
 hw/i386/kvmvapic.c          |  4 +--
 hw/i386/microvm.c           |  4 +--
 hw/i386/x86.c               | 60 ++++++++++++++++++++-----------------
 hw/i386/xen/xen-hvm.c       |  4 +--
 target/i386/cpu.c           |  3 +-
 target/i386/hax/hax-all.c   |  2 +-
 target/i386/whpx/whpx-all.c |  2 +-
 8 files changed, 44 insertions(+), 37 deletions(-)

diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index b67dcbbb37fa..d0971eb4c389 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -157,7 +157,7 @@ static void init_common_fadt_data(MachineState *ms, Object *o,
              * CPUs for more than 8 CPUs, "Clustered Logical" mode has to be
              * used
              */
-            ((ms->smp.max_cpus > 8) ?
+            ((machine_topo_get_max_cpus(ms) > 8) ?
                         (1 << ACPI_FADT_F_FORCE_APIC_CLUSTER_MODEL) : 0),
         .int_model = 1 /* Multiple APIC */,
         .rtc_century = RTC_CENTURY,
diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 43f8a8f679e3..9def3fcd7629 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -449,7 +449,7 @@ static void patch_instruction(VAPICROMState *s, X86CPU *cpu, target_ulong ip)
     VAPICHandlers *handlers;
     PatchInfo *info;
 
-    if (ms->smp.cpus == 1) {
+    if (machine_topo_get_cpus(ms) == 1) {
         handlers = &s->rom_state.up;
     } else {
         handlers = &s->rom_state.mp;
@@ -759,7 +759,7 @@ static void kvmvapic_vm_state_change(void *opaque, bool running,
     }
 
     if (s->state == VAPIC_ACTIVE) {
-        if (ms->smp.cpus == 1) {
+        if (machine_topo_get_cpus(ms) == 1) {
             run_on_cpu(first_cpu, do_vapic_enable, RUN_ON_CPU_HOST_PTR(s));
         } else {
             zero = g_malloc0(s->rom_state.vapic_size);
diff --git a/hw/i386/microvm.c b/hw/i386/microvm.c
index 29f30dd6d34c..8abde895e52d 100644
--- a/hw/i386/microvm.c
+++ b/hw/i386/microvm.c
@@ -320,8 +320,8 @@ static void microvm_memory_init(MicrovmMachineState *mms)
     fw_cfg = fw_cfg_init_io_dma(FW_CFG_IO_BASE, FW_CFG_IO_BASE + 4,
                                 &address_space_memory);
 
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, machine->smp.cpus);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, machine_topo_get_cpus(machine));
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, machine_topo_get_max_cpus(machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i32(fw_cfg, FW_CFG_IRQ0_OVERRIDE, 1);
     fw_cfg_add_file(fw_cfg, "etc/e820", e820_table,
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 20ba2384bbb2..0aa4594455e2 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -70,10 +70,10 @@ inline void init_topo_info(X86CPUTopoInfo *topo_info,
 {
     MachineState *ms = MACHINE(x86ms);
 
-    topo_info->dies_per_pkg = ms->smp.dies;
-    topo_info->modules_per_die = ms->smp.clusters;
-    topo_info->cores_per_module = ms->smp.cores;
-    topo_info->threads_per_core = ms->smp.threads;
+    topo_info->dies_per_pkg = machine_topo_get_dies(ms);
+    topo_info->modules_per_die = machine_topo_get_clusters(ms);
+    topo_info->cores_per_module = machine_topo_get_smp_cores(ms);
+    topo_info->threads_per_core = machine_topo_get_smp_threads(ms);
 }
 
 /*
@@ -126,7 +126,7 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
      * This is used for FW_CFG_MAX_CPUS. See comments on fw_cfg_arch_create().
      */
     x86ms->apic_id_limit = x86_cpu_apic_id_from_index(x86ms,
-                                                      ms->smp.max_cpus - 1) + 1;
+                                machine_topo_get_max_cpus(ms) - 1) + 1;
 
     /*
      * Can we support APIC ID 255 or higher?
@@ -147,7 +147,7 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
     }
 
     possible_cpus = mc->possible_cpu_arch_ids(ms);
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
         x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id, &error_fatal);
     }
 }
@@ -283,8 +283,6 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     CPUX86State *env = &cpu->env;
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
-    unsigned int smp_cores = ms->smp.cores;
-    unsigned int smp_threads = ms->smp.threads;
     X86CPUTopoInfo topo_info;
 
     if (!object_dynamic_cast(OBJECT(cpu), ms->cpu_type)) {
@@ -306,22 +304,22 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
 
     init_topo_info(&topo_info, x86ms);
 
-    env->nr_dies = ms->smp.dies;
-    env->nr_modules = ms->smp.clusters;
+    env->nr_dies = machine_topo_get_dies(ms);
+    env->nr_modules = machine_topo_get_clusters(ms);
 
     /*
      * If APIC ID is not set,
      * set it based on socket/die/cluster/core/thread properties.
      */
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
-        int max_socket = (ms->smp.max_cpus - 1) / smp_threads / smp_cores /
-                                ms->smp.clusters / ms->smp.dies;
+        int sockets, dies, clusters, cores, threads;
 
         /*
          * die-id was optional in QEMU 4.0 and older, so keep it optional
          * if there's only one die per socket.
          */
-        if (cpu->die_id < 0 && ms->smp.dies == 1) {
+        dies = machine_topo_get_dies(ms);
+        if (cpu->die_id < 0 && dies == 1) {
             cpu->die_id = 0;
         }
 
@@ -329,48 +327,56 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
          * cluster-id was optional in QEMU 8.0 and older, so keep it optional
          * if there's only one cluster per die.
          */
-        if (cpu->cluster_id < 0 && ms->smp.clusters == 1) {
+        clusters = machine_topo_get_clusters(ms);
+        if (cpu->cluster_id < 0 && clusters == 1) {
             cpu->cluster_id = 0;
         }
 
+        sockets = machine_topo_get_sockets(ms);
         if (cpu->socket_id < 0) {
             error_setg(errp, "CPU socket-id is not set");
             return;
-        } else if (cpu->socket_id > max_socket) {
+        } else if (cpu->socket_id > (sockets - 1)) {
             error_setg(errp, "Invalid CPU socket-id: %u must be in range 0:%u",
-                       cpu->socket_id, max_socket);
+                       cpu->socket_id, sockets - 1);
             return;
         }
+
         if (cpu->die_id < 0) {
             error_setg(errp, "CPU die-id is not set");
             return;
-        } else if (cpu->die_id > ms->smp.dies - 1) {
+        } else if (cpu->die_id > (dies - 1)) {
             error_setg(errp, "Invalid CPU die-id: %u must be in range 0:%u",
-                       cpu->die_id, ms->smp.dies - 1);
+                       cpu->die_id, dies - 1);
             return;
         }
+
         if (cpu->cluster_id < 0) {
             error_setg(errp, "CPU cluster-id is not set");
             return;
-        } else if (cpu->cluster_id > ms->smp.clusters - 1) {
+        } else if (cpu->cluster_id > (clusters - 1)) {
             error_setg(errp, "Invalid CPU cluster-id: %u must be in range 0:%u",
-                       cpu->cluster_id, ms->smp.clusters - 1);
+                       cpu->cluster_id, clusters - 1);
             return;
         }
+
+        cores = machine_topo_get_cores(ms, cpu->cluster_id);
         if (cpu->core_id < 0) {
             error_setg(errp, "CPU core-id is not set");
             return;
-        } else if (cpu->core_id > (smp_cores - 1)) {
+        } else if (cpu->core_id > (cores - 1)) {
             error_setg(errp, "Invalid CPU core-id: %u must be in range 0:%u",
-                       cpu->core_id, smp_cores - 1);
+                       cpu->core_id, cores - 1);
             return;
         }
+
+        threads = machine_topo_get_threads(ms, cpu->cluster_id, cpu->core_id);
         if (cpu->thread_id < 0) {
             error_setg(errp, "CPU thread-id is not set");
             return;
-        } else if (cpu->thread_id > (smp_threads - 1)) {
+        } else if (cpu->thread_id > (threads - 1)) {
             error_setg(errp, "Invalid CPU thread-id: %u must be in range 0:%u",
-                       cpu->thread_id, smp_threads - 1);
+                       cpu->thread_id, threads - 1);
             return;
         }
 
@@ -488,7 +494,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
 const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
 {
     X86MachineState *x86ms = X86_MACHINE(ms);
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
     X86CPUTopoInfo topo_info;
     int i;
 
@@ -518,11 +524,11 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
                                  &topo_info, &topo_ids);
         ms->possible_cpus->cpus[i].props.has_socket_id = true;
         ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
-        if (ms->smp.dies > 1) {
+        if (machine_topo_get_dies(ms) > 1) {
             ms->possible_cpus->cpus[i].props.has_die_id = true;
             ms->possible_cpus->cpus[i].props.die_id = topo_ids.die_id;
         }
-        if (ms->smp.clusters > 1) {
+        if (machine_topo_get_clusters(ms) > 1) {
             ms->possible_cpus->cpus[i].props.has_cluster_id = true;
             ms->possible_cpus->cpus[i].props.cluster_id = topo_ids.module_id;
         }
diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index b9a6f7f5381e..63a430aa7418 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -757,7 +757,7 @@ static ioreq_t *cpu_get_ioreq_from_shared_memory(XenIOState *state, int vcpu)
 static ioreq_t *cpu_get_ioreq(XenIOState *state)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
     int i;
     evtchn_port_t port;
 
@@ -1392,7 +1392,7 @@ static int xen_map_ioreq_server(XenIOState *state)
 void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory)
 {
     MachineState *ms = MACHINE(pcms);
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
     int i, rc;
     xen_pfn_t ioreq_pfn;
     XenIOState *state;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index cf84c720a431..1aeea0e0ac3f 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6667,7 +6667,8 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     MachineState *ms = MACHINE(qdev_get_machine());
     qemu_register_reset(x86_cpu_machine_reset_cb, cpu);
 
-    if (cpu->env.features[FEAT_1_EDX] & CPUID_APIC || ms->smp.cpus > 1) {
+    if (cpu->env.features[FEAT_1_EDX] & CPUID_APIC ||
+        machine_topo_get_cpus(ms) > 1) {
         x86_cpu_apic_create(cpu, &local_err);
         if (local_err != NULL) {
             goto out;
diff --git a/target/i386/hax/hax-all.c b/target/i386/hax/hax-all.c
index 3e5992a63b63..a33a3b6f1456 100644
--- a/target/i386/hax/hax-all.c
+++ b/target/i386/hax/hax-all.c
@@ -349,7 +349,7 @@ static int hax_init(ram_addr_t ram_size, int max_cpus)
 
 static int hax_accel_init(MachineState *ms)
 {
-    int ret = hax_init(ms->ram_size, (int)ms->smp.max_cpus);
+    int ret = hax_init(ms->ram_size, (int)machine_topo_get_max_cpus(ms));
 
     if (ret && (ret != -ENOSPC)) {
         fprintf(stderr, "No accelerator found.\n");
diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index fc349f887e47..fb3332c7b82b 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -2605,7 +2605,7 @@ static int whpx_accel_init(MachineState *ms)
     }
 
     memset(&prop, 0, sizeof(WHV_PARTITION_PROPERTY));
-    prop.ProcessorCount = ms->smp.cpus;
+    prop.ProcessorCount = machine_topo_get_cpus(ms);
     hr = whp_dispatch.WHvSetPartitionProperty(
         whpx->partition,
         WHvPartitionPropertyCodeProcessorCount,
-- 
2.34.1

[RFC 20/52] s390x: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

In hw/s390x/s390-virtio-ccw.c, s390_init_cpus() needs "threads per core".
Before s390x supports hybrid, here we use smp-specific interface to get
"threads per core".

For other cases, it's straightforward to replace topology access with
wrapped generic interfaces.

Cc: Halil Pasic <pasic@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Eric Farman <farman@linux.ibm.com>
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/s390x/s390-virtio-ccw.c     |  6 +++---
 hw/s390x/sclp.c                |  3 ++-
 target/s390x/cpu-sysemu.c      |  2 +-
 target/s390x/kvm/kvm.c         | 15 +++++++++------
 target/s390x/tcg/excp_helper.c |  2 +-
 5 files changed, 16 insertions(+), 12 deletions(-)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index f22f61b8b6ac..9b5020a0d395 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -88,7 +88,7 @@ static void s390_init_cpus(MachineState *machine)
     S390CcwMachineClass *s390mc = S390_CCW_MACHINE_CLASS(mc);
     int i;
 
-    if (machine->smp.threads > s390mc->max_threads) {
+    if (machine_topo_get_smp_threads(machine) > s390mc->max_threads) {
         error_report("S390 does not support more than %d threads.",
                      s390mc->max_threads);
         exit(1);
@@ -97,7 +97,7 @@ static void s390_init_cpus(MachineState *machine)
     /* initialize possible_cpus */
     mc->possible_cpu_arch_ids(machine);
 
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(machine); i++) {
         s390x_new_cpu(machine->cpu_type, i, &error_fatal);
     }
 }
@@ -540,7 +540,7 @@ static CpuInstanceProperties s390_cpu_index_to_props(MachineState *ms,
 static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
 {
     int i;
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
 
     if (ms->possible_cpus) {
         g_assert(ms->possible_cpus && ms->possible_cpus->len == max_cpus);
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f458..2aef1a57a591 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -127,7 +127,8 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
     prepare_cpu_entries(machine, entries_start, &cpu_count);
     read_info->entries_cpu = cpu_to_be16(cpu_count);
     read_info->offset_cpu = cpu_to_be16(offset_cpu);
-    read_info->highest_cpu = cpu_to_be16(machine->smp.max_cpus - 1);
+    read_info->highest_cpu =
+        cpu_to_be16(machine_topo_get_max_cpus(machine) - 1);
 
     read_info->ibc_val = cpu_to_be32(s390_get_ibc_val());
 
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 948e4bd3e09e..617f23b39913 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -125,7 +125,7 @@ bool s390_cpu_realize_sysemu(DeviceState *dev, Error **errp)
 {
     S390CPU *cpu = S390_CPU(dev);
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
 
     if (cpu->env.core_id >= max_cpus) {
         error_setg(errp, "Unable to add CPU with core-id: %" PRIu32
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 3ac7ec9acf4e..d297daed1117 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -406,9 +406,11 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
 
 int kvm_arch_init_vcpu(CPUState *cs)
 {
-    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
+    unsigned int max_cpus;
     S390CPU *cpu = S390_CPU(cs);
+
     kvm_s390_set_cpu_state(cpu, cpu->env.cpu_state);
+    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
     cpu->irqstate = g_malloc0(VCPU_IRQ_BUF_SIZE(max_cpus));
     return 0;
 }
@@ -2097,14 +2099,15 @@ int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state)
 
 void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu)
 {
-    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
-    struct kvm_s390_irq_state irq_state = {
-        .buf = (uint64_t) cpu->irqstate,
-        .len = VCPU_IRQ_BUF_SIZE(max_cpus),
-    };
+    unsigned int max_cpus;
+    struct kvm_s390_irq_state irq_state;
     CPUState *cs = CPU(cpu);
     int32_t bytes;
 
+    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
+    irq_state.buf = (uint64_t) cpu->irqstate;
+    irq_state.len = VCPU_IRQ_BUF_SIZE(max_cpus);
+
     if (!kvm_check_extension(kvm_state, KVM_CAP_S390_IRQ_STATE)) {
         return;
     }
diff --git a/target/s390x/tcg/excp_helper.c b/target/s390x/tcg/excp_helper.c
index bc767f044381..e396a89d5540 100644
--- a/target/s390x/tcg/excp_helper.c
+++ b/target/s390x/tcg/excp_helper.c
@@ -321,7 +321,7 @@ static void do_ext_interrupt(CPUS390XState *env)
     if ((env->pending_int & INTERRUPT_EMERGENCY_SIGNAL) &&
         (env->cregs[0] & CR0_EMERGENCY_SIGNAL_SC)) {
         MachineState *ms = MACHINE(qdev_get_machine());
-        unsigned int max_cpus = ms->smp.max_cpus;
+        unsigned int max_cpus = machine_topo_get_max_cpus(ms);
 
         lowcore->ext_int_code = cpu_to_be16(EXT_EMERGENCY);
         cpu_addr = find_first_bit(env->emergency_signals, S390_MAX_CPUS);
-- 
2.34.1

[RFC 21/52] ppc: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

Before ppc supports hybrid, here we use smp-specific interface to get
"threads per core" and "cores per cluster".

For other cases, it's straightforward to replace topology access with
wrapped generic interfaces.

Cc: Hervé Poussinea <hpoussin@reactos.or>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Greg Kurz <groug@kaod.org>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/ppc/e500.c         |  6 +++---
 hw/ppc/mac_newworld.c | 16 +++++++++-------
 hw/ppc/mac_oldworld.c | 12 +++++++-----
 hw/ppc/pnv.c          | 11 +++++++----
 hw/ppc/prep.c         |  3 ++-
 hw/ppc/spapr.c        | 37 ++++++++++++++++++++-----------------
 hw/ppc/spapr_rtas.c   | 17 +++++++++--------
 7 files changed, 57 insertions(+), 45 deletions(-)

diff --git a/hw/ppc/e500.c b/hw/ppc/e500.c
index 117c9c08eddf..e999bd27dbc2 100644
--- a/hw/ppc/e500.c
+++ b/hw/ppc/e500.c
@@ -371,7 +371,7 @@ static int ppce500_load_device_tree(PPCE500MachineState *pms,
                                     bool dry_run)
 {
     MachineState *machine = MACHINE(pms);
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     const PPCE500MachineClass *pmc = PPCE500_MACHINE_GET_CLASS(pms);
     CPUPPCState *env = first_cpu->env_ptr;
     int ret = -1;
@@ -808,7 +808,7 @@ static DeviceState *ppce500_init_mpic_qemu(PPCE500MachineState *pms,
     SysBusDevice *s;
     int i, j, k;
     MachineState *machine = MACHINE(pms);
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     const PPCE500MachineClass *pmc = PPCE500_MACHINE_GET_CLASS(pms);
 
     dev = qdev_new(TYPE_OPENPIC);
@@ -915,7 +915,7 @@ void ppce500_init(MachineState *machine)
     struct boot_info *boot_info = NULL;
     int dt_size;
     int i;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     /* irq num for pin INTA, INTB, INTC and INTD is 1, 2, 3 and
      * 4 respectively */
     unsigned int pci_irq_nrs[PCI_NUM_PINS] = {1, 2, 3, 4};
diff --git a/hw/ppc/mac_newworld.c b/hw/ppc/mac_newworld.c
index 460c14b5e368..54212afb29e3 100644
--- a/hw/ppc/mac_newworld.c
+++ b/hw/ppc/mac_newworld.c
@@ -136,7 +136,7 @@ static void ppc_core99_init(MachineState *machine)
     CPUPPCState *env = NULL;
     char *filename;
     IrqLines *openpic_irqs;
-    int i, j, k, ppc_boot_device, machine_arch, bios_size = -1;
+    int i, j, k, cpus, ppc_boot_device, machine_arch, bios_size = -1;
     const char *bios_name = machine->firmware ?: PROM_FILENAME;
     MemoryRegion *bios = g_new(MemoryRegion, 1);
     hwaddr kernel_base = 0, initrd_base = 0, cmdline_base = 0;
@@ -155,8 +155,9 @@ static void ppc_core99_init(MachineState *machine)
     hwaddr nvram_addr = 0xFFF04000;
     uint64_t tbfreq = kvm_enabled() ? kvmppc_get_tbfreq() : TBFREQ;
 
+    cpus = machine_topo_get_cpus(machine);
     /* init CPUs */
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         cpu = POWERPC_CPU(cpu_create(machine->cpu_type));
         env = &cpu->env;
 
@@ -253,9 +254,9 @@ static void ppc_core99_init(MachineState *machine)
         }
     }
 
-    openpic_irqs = g_new0(IrqLines, machine->smp.cpus);
+    openpic_irqs = g_new0(IrqLines, cpus);
     dev = DEVICE(cpu);
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         /* Mac99 IRQ connection between OpenPIC outputs pins
          * and PowerPC input pins
          */
@@ -393,7 +394,7 @@ static void ppc_core99_init(MachineState *machine)
     /* OpenPIC */
     s = SYS_BUS_DEVICE(pic_dev);
     k = 0;
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         for (j = 0; j < OPENPIC_OUTPUT_NB; j++) {
             sysbus_connect_irq(s, k++, openpic_irqs[i].irq[j]);
         }
@@ -472,8 +473,9 @@ static void ppc_core99_init(MachineState *machine)
     sysbus_mmio_map(s, 0, CFG_ADDR);
     sysbus_mmio_map(s, 1, CFG_ADDR + 2);
 
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine->smp.cpus);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
+                   (uint16_t)machine_topo_get_max_cpus(machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i16(fw_cfg, FW_CFG_MACHINE_ID, machine_arch);
     fw_cfg_add_i32(fw_cfg, FW_CFG_KERNEL_ADDR, kernel_base);
diff --git a/hw/ppc/mac_oldworld.c b/hw/ppc/mac_oldworld.c
index 2e4cc3fe0b22..9b715ed6b438 100644
--- a/hw/ppc/mac_oldworld.c
+++ b/hw/ppc/mac_oldworld.c
@@ -90,7 +90,7 @@ static void ppc_heathrow_init(MachineState *machine)
     PowerPCCPU *cpu = NULL;
     CPUPPCState *env = NULL;
     char *filename;
-    int i, bios_size = -1;
+    int i, cpus, bios_size = -1;
     MemoryRegion *bios = g_new(MemoryRegion, 1);
     uint64_t bios_addr;
     uint32_t kernel_base = 0, initrd_base = 0, cmdline_base = 0;
@@ -106,8 +106,9 @@ static void ppc_heathrow_init(MachineState *machine)
     void *fw_cfg;
     uint64_t tbfreq = kvm_enabled() ? kvmppc_get_tbfreq() : TBFREQ;
 
+    cpus = machine_topo_get_cpus(machine);
     /* init CPUs */
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         cpu = POWERPC_CPU(cpu_create(machine->cpu_type));
         env = &cpu->env;
 
@@ -260,7 +261,7 @@ static void ppc_heathrow_init(MachineState *machine)
     }
 
     /* Connect the heathrow PIC outputs to the 6xx bus */
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         switch (PPC_INPUT(env)) {
         case PPC_FLAGS_INPUT_6xx:
             /* XXX: we register only 1 output pin for heathrow PIC */
@@ -315,8 +316,9 @@ static void ppc_heathrow_init(MachineState *machine)
     sysbus_mmio_map(s, 0, CFG_ADDR);
     sysbus_mmio_map(s, 1, CFG_ADDR + 2);
 
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine->smp.cpus);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
+                   (uint16_t)machine_topo_get_max_cpus(machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i16(fw_cfg, FW_CFG_MACHINE_ID, ARCH_HEATHROW);
     fw_cfg_add_i32(fw_cfg, FW_CFG_KERNEL_ADDR, kernel_base);
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 44b1fbbc9329..1bdd3202115c 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -938,8 +938,9 @@ static void pnv_init(MachineState *machine)
         exit(1);
     }
 
-    pnv->num_chips =
-        machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
+    pnv->num_chips = machine_topo_get_max_cpus(machine) /
+                     (machine_topo_get_smp_cores(machine) *
+                     machine_topo_get_smp_threads(machine));
     /*
      * TODO: should we decide on how many chips we can create based
      * on #cores and Venice vs. Murano vs. Naples chip type etc...,
@@ -969,9 +970,11 @@ static void pnv_init(MachineState *machine)
         snprintf(chip_name, sizeof(chip_name), "chip[%d]", i);
         object_property_add_child(OBJECT(pnv), chip_name, chip);
         object_property_set_int(chip, "chip-id", i, &error_fatal);
-        object_property_set_int(chip, "nr-cores", machine->smp.cores,
+        object_property_set_int(chip, "nr-cores",
+                                machine_topo_get_smp_cores(machine),
                                 &error_fatal);
-        object_property_set_int(chip, "nr-threads", machine->smp.threads,
+        object_property_set_int(chip, "nr-threads",
+                                machine_topo_get_smp_threads(machine),
                                 &error_fatal);
         /*
          * The POWER8 machine use the XICS interrupt interface.
diff --git a/hw/ppc/prep.c b/hw/ppc/prep.c
index fcbe4c583765..aefdf986f641 100644
--- a/hw/ppc/prep.c
+++ b/hw/ppc/prep.c
@@ -382,7 +382,8 @@ static void ibm_40p_init(MachineState *machine)
         boot_device = machine->boot_config.order[0];
     }
 
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
+                   (uint16_t)machine_topo_get_max_cpus(machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i16(fw_cfg, FW_CFG_MACHINE_ID, ARCH_PREP);
 
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 4921198b9d6f..6faad75afd70 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -119,7 +119,7 @@
 static int spapr_vcpu_id(SpaprMachineState *spapr, int cpu_index)
 {
     MachineState *ms = MACHINE(spapr);
-    unsigned int smp_threads = ms->smp.threads;
+    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
 
     assert(spapr->vsmt);
     return
@@ -171,7 +171,8 @@ int spapr_max_server_number(SpaprMachineState *spapr)
     MachineState *ms = MACHINE(spapr);
 
     assert(spapr->vsmt);
-    return DIV_ROUND_UP(ms->smp.max_cpus * spapr->vsmt, ms->smp.threads);
+    return DIV_ROUND_UP(machine_topo_get_max_cpus(ms) * spapr->vsmt,
+                        machine_topo_get_smp_threads(ms));
 }
 
 static int spapr_fixup_cpu_smt_dt(void *fdt, int offset, PowerPCCPU *cpu,
@@ -659,8 +660,8 @@ static void spapr_dt_cpu(CPUState *cs, void *fdt, int offset,
     uint32_t cpufreq = kvm_enabled() ? kvmppc_get_clockfreq() : 1000000000;
     uint32_t page_sizes_prop[64];
     size_t page_sizes_prop_size;
-    unsigned int smp_threads = ms->smp.threads;
-    uint32_t vcpus_per_socket = smp_threads * ms->smp.cores;
+    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
+    uint32_t vcpus_per_socket = smp_threads * machine_topo_get_smp_cores(ms);
     uint32_t pft_size_prop[] = {0, cpu_to_be32(spapr->htab_shift)};
     int compat_smt = MIN(smp_threads, ppc_compat_max_vthreads(cpu));
     SpaprDrc *drc;
@@ -866,7 +867,8 @@ static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
         cpu_to_be32(max_device_addr & 0xffffffff),
         cpu_to_be32(SPAPR_MEMORY_BLOCK_SIZE >> 32),
         cpu_to_be32(SPAPR_MEMORY_BLOCK_SIZE & 0xffffffff),
-        cpu_to_be32(ms->smp.max_cpus / ms->smp.threads),
+        cpu_to_be32(machine_topo_get_max_cpus(ms) /
+                    machine_topo_get_smp_threads(ms)),
     };
 
     _FDT(rtas = fdt_add_subnode(fdt, 0, "rtas"));
@@ -936,7 +938,7 @@ static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
      */
     _FDT(fdt_setprop_cell(fdt, rtas, "rtas-size", RTAS_MIN_SIZE +
                           RTAS_ERROR_LOG_MAX +
-                          ms->smp.max_cpus * sizeof(uint64_t) * 2 +
+                          machine_topo_get_max_cpus(ms) * sizeof(uint64_t) * 2 +
                           sizeof(uint64_t)));
     _FDT(fdt_setprop_cell(fdt, rtas, "rtas-error-log-max",
                           RTAS_ERROR_LOG_MAX));
@@ -2503,7 +2505,7 @@ static void spapr_validate_node_memory(MachineState *machine, Error **errp)
 /* find cpu slot in machine->possible_cpus by core_id */
 static CPUArchId *spapr_find_cpu_slot(MachineState *ms, uint32_t id, int *idx)
 {
-    int index = id / ms->smp.threads;
+    int index = id / machine_topo_get_smp_threads(ms);
 
     if (index >= ms->possible_cpus->len) {
         return NULL;
@@ -2522,7 +2524,7 @@ static void spapr_set_vsmt_mode(SpaprMachineState *spapr, Error **errp)
     bool vsmt_user = !!spapr->vsmt;
     int kvm_smt = kvmppc_smt_threads();
     int ret;
-    unsigned int smp_threads = ms->smp.threads;
+    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
 
     if (!kvm_enabled() && (smp_threads > 1)) {
         error_setg(errp, "TCG cannot support more than 1 thread/core "
@@ -2594,9 +2596,9 @@ static void spapr_init_cpus(SpaprMachineState *spapr)
     SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(machine);
     const char *type = spapr_get_cpu_core_type(machine->cpu_type);
     const CPUArchIdList *possible_cpus;
-    unsigned int smp_cpus = machine->smp.cpus;
-    unsigned int smp_threads = machine->smp.threads;
-    unsigned int max_cpus = machine->smp.max_cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
+    unsigned int smp_threads = machine_topo_get_smp_threads(machine);
+    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
     int boot_cores_nr = smp_cpus / smp_threads;
     int i;
 
@@ -4002,7 +4004,7 @@ static void spapr_core_pre_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
     const char *type = object_get_typename(OBJECT(dev));
     CPUArchId *core_slot;
     int index;
-    unsigned int smp_threads = machine->smp.threads;
+    unsigned int smp_threads = machine_topo_get_smp_threads(machine);
 
     if (dev->hotplugged && !mc->has_hotpluggable_cpus) {
         error_setg(errp, "CPU hotplug not supported for this machine");
@@ -4320,16 +4322,16 @@ spapr_cpu_index_to_props(MachineState *machine, unsigned cpu_index)
 
 static int64_t spapr_get_default_cpu_node_id(const MachineState *ms, int idx)
 {
-    return idx / ms->smp.cores % ms->numa_state->num_nodes;
+    return idx / machine_topo_get_smp_cores(ms) % ms->numa_state->num_nodes;
 }
 
 static const CPUArchIdList *spapr_possible_cpu_arch_ids(MachineState *machine)
 {
     int i;
-    unsigned int smp_threads = machine->smp.threads;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_threads = machine_topo_get_smp_threads(machine);
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     const char *core_type;
-    int spapr_max_cores = machine->smp.max_cpus / smp_threads;
+    int spapr_max_cores = machine_topo_get_max_cpus(machine) / smp_threads;
     MachineClass *mc = MACHINE_GET_CLASS(machine);
 
     if (!mc->has_hotpluggable_cpus) {
@@ -4500,7 +4502,8 @@ bool spapr_set_vcpu_id(PowerPCCPU *cpu, int cpu_index, Error **errp)
         error_setg(errp, "Can't create CPU with id %d in KVM", vcpu_id);
         error_append_hint(errp, "Adjust the number of cpus to %d "
                           "or try to raise the number of threads per core\n",
-                          vcpu_id * ms->smp.threads / spapr->vsmt);
+                          vcpu_id * machine_topo_get_smp_threads(ms) /
+                          spapr->vsmt);
         return false;
     }
 
diff --git a/hw/ppc/spapr_rtas.c b/hw/ppc/spapr_rtas.c
index 3f664ea02ccf..e377299a28ee 100644
--- a/hw/ppc/spapr_rtas.c
+++ b/hw/ppc/spapr_rtas.c
@@ -279,14 +279,15 @@ static void rtas_ibm_get_system_parameter(PowerPCCPU *cpu,
 
     switch (parameter) {
     case RTAS_SYSPARM_SPLPAR_CHARACTERISTICS: {
-        g_autofree char *param_val = g_strdup_printf("MaxEntCap=%d,"
-                                                     "DesMem=%" PRIu64 ","
-                                                     "DesProcs=%d,"
-                                                     "MaxPlatProcs=%d",
-                                                     ms->smp.max_cpus,
-                                                     ms->ram_size / MiB,
-                                                     ms->smp.cpus,
-                                                     ms->smp.max_cpus);
+        g_autofree char *param_val =
+            g_strdup_printf("MaxEntCap=%d,"
+                            "DesMem=%" PRIu64 ","
+                            "DesProcs=%d,"
+                            "MaxPlatProcs=%d",
+                            machine_topo_get_max_cpus(ms),
+                            ms->ram_size / MiB,
+                            machine_topo_get_cpus(ms),
+                            machine_topo_get_max_cpus(ms));
         if (pcc->n_host_threads > 0) {
             /*
              * Add HostThrs property. This property is not present in PAPR but
-- 
2.34.1

[RFC 22/52] riscv: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

In the codes of riscv, it's straightforward to replace topology access
with wrapped generic interfaces.

Cc: Bin Meng <bin.meng@windriver.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Alistair Francis <alistair.francis@wdc.com>
CC: Vijai Kumar K <vijai@behindbytes.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/riscv/microchip_pfsoc.c | 11 ++++++-----
 hw/riscv/numa.c            | 21 +++++++++++----------
 hw/riscv/opentitan.c       |  8 ++++----
 hw/riscv/shakti_c.c        |  2 +-
 hw/riscv/sifive_e.c        | 10 ++++++----
 hw/riscv/sifive_u.c        | 28 ++++++++++++++--------------
 hw/riscv/virt.c            | 24 +++++++++++++-----------
 7 files changed, 55 insertions(+), 49 deletions(-)

diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c
index 2b91e49561f1..30295cce17e7 100644
--- a/hw/riscv/microchip_pfsoc.c
+++ b/hw/riscv/microchip_pfsoc.c
@@ -164,7 +164,8 @@ static void microchip_pfsoc_soc_instance_init(Object *obj)
 
     object_initialize_child(OBJECT(&s->u_cluster), "u-cpus", &s->u_cpus,
                             TYPE_RISCV_HART_ARRAY);
-    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
+    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts",
+                         machine_topo_get_cpus(ms) - 1);
     qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
     qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type",
                          TYPE_RISCV_CPU_SIFIVE_U54);
@@ -249,10 +250,10 @@ static void microchip_pfsoc_soc_realize(DeviceState *dev, Error **errp)
 
     /* CLINT */
     riscv_aclint_swi_create(memmap[MICROCHIP_PFSOC_CLINT].base,
-        0, ms->smp.cpus, false);
+        0, machine_topo_get_cpus(ms), false);
     riscv_aclint_mtimer_create(
         memmap[MICROCHIP_PFSOC_CLINT].base + RISCV_ACLINT_SWI_SIZE,
-        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
+        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, machine_topo_get_cpus(ms),
         RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
         CLINT_TIMEBASE_FREQ, false);
 
@@ -276,11 +277,11 @@ static void microchip_pfsoc_soc_realize(DeviceState *dev, Error **errp)
                                 l2lim_mem);
 
     /* create PLIC hart topology configuration string */
-    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
+    plic_hart_config = riscv_plic_hart_config_string(machine_topo_get_cpus(ms));
 
     /* PLIC */
     s->plic = sifive_plic_create(memmap[MICROCHIP_PFSOC_PLIC].base,
-        plic_hart_config, ms->smp.cpus, 0,
+        plic_hart_config, machine_topo_get_cpus(ms), 0,
         MICROCHIP_PFSOC_PLIC_NUM_SOURCES,
         MICROCHIP_PFSOC_PLIC_NUM_PRIORITIES,
         MICROCHIP_PFSOC_PLIC_PRIORITY_BASE,
diff --git a/hw/riscv/numa.c b/hw/riscv/numa.c
index 472010256183..1fabdc42e767 100644
--- a/hw/riscv/numa.c
+++ b/hw/riscv/numa.c
@@ -37,13 +37,14 @@ int riscv_socket_count(const MachineState *ms)
 
 int riscv_socket_first_hartid(const MachineState *ms, int socket_id)
 {
-    int i, first_hartid = ms->smp.cpus;
+    int i, first_hartid, cpus = machine_topo_get_cpus(ms);
 
+    first_hartid = cpus;
     if (!numa_enabled(ms)) {
         return (!socket_id) ? 0 : -1;
     }
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
             continue;
         }
@@ -52,18 +53,18 @@ int riscv_socket_first_hartid(const MachineState *ms, int socket_id)
         }
     }
 
-    return (first_hartid < ms->smp.cpus) ? first_hartid : -1;
+    return (first_hartid < cpus) ? first_hartid : -1;
 }
 
 int riscv_socket_last_hartid(const MachineState *ms, int socket_id)
 {
-    int i, last_hartid = -1;
+    int i, last_hartid = -1, cpus = machine_topo_get_cpus(ms);
 
     if (!numa_enabled(ms)) {
-        return (!socket_id) ? ms->smp.cpus - 1 : -1;
+        return (!socket_id) ? cpus - 1 : -1;
     }
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
             continue;
         }
@@ -72,7 +73,7 @@ int riscv_socket_last_hartid(const MachineState *ms, int socket_id)
         }
     }
 
-    return (last_hartid < ms->smp.cpus) ? last_hartid : -1;
+    return (last_hartid < cpus) ? last_hartid : -1;
 }
 
 int riscv_socket_hart_count(const MachineState *ms, int socket_id)
@@ -80,7 +81,7 @@ int riscv_socket_hart_count(const MachineState *ms, int socket_id)
     int first_hartid, last_hartid;
 
     if (!numa_enabled(ms)) {
-        return (!socket_id) ? ms->smp.cpus : -1;
+        return (!socket_id) ? machine_topo_get_cpus(ms) : -1;
     }
 
     first_hartid = riscv_socket_first_hartid(ms, socket_id);
@@ -208,7 +209,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const MachineState *ms, int idx)
     int64_t nidx = 0;
 
     if (ms->numa_state->num_nodes) {
-        nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes);
+        nidx = idx / (machine_topo_get_cpus(ms) / ms->numa_state->num_nodes);
         if (ms->numa_state->num_nodes <= nidx) {
             nidx = ms->numa_state->num_nodes - 1;
         }
@@ -220,7 +221,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const MachineState *ms, int idx)
 const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms)
 {
     int n;
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
 
     if (ms->possible_cpus) {
         assert(ms->possible_cpus->len == max_cpus);
diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
index 353f030d80c1..fb59c43dcc08 100644
--- a/hw/riscv/opentitan.c
+++ b/hw/riscv/opentitan.c
@@ -143,11 +143,11 @@ static void lowrisc_ibex_soc_realize(DeviceState *dev_soc, Error **errp)
     MachineState *ms = MACHINE(qdev_get_machine());
     LowRISCIbexSoCState *s = RISCV_IBEX_SOC(dev_soc);
     MemoryRegion *sys_mem = get_system_memory();
-    int i;
+    int i, cpus = machine_topo_get_cpus(ms);
 
     object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
                             &error_abort);
-    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
+    object_property_set_int(OBJECT(&s->cpus), "num-harts", cpus,
                             &error_abort);
     object_property_set_int(OBJECT(&s->cpus), "resetvec", s->resetvec,
                             &error_abort);
@@ -186,10 +186,10 @@ static void lowrisc_ibex_soc_realize(DeviceState *dev_soc, Error **errp)
     }
     sysbus_mmio_map(SYS_BUS_DEVICE(&s->plic), 0, memmap[IBEX_DEV_PLIC].base);
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < cpus; i++) {
         CPUState *cpu = qemu_get_cpu(i);
 
-        qdev_connect_gpio_out(DEVICE(&s->plic), ms->smp.cpus + i,
+        qdev_connect_gpio_out(DEVICE(&s->plic), cpus + i,
                               qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
     }
 
diff --git a/hw/riscv/shakti_c.c b/hw/riscv/shakti_c.c
index e43cc9445cb1..34499510e2f0 100644
--- a/hw/riscv/shakti_c.c
+++ b/hw/riscv/shakti_c.c
@@ -110,7 +110,7 @@ static void shakti_c_soc_state_realize(DeviceState *dev, Error **errp)
     sysbus_realize(SYS_BUS_DEVICE(&sss->cpus), &error_abort);
 
     sss->plic = sifive_plic_create(shakti_c_memmap[SHAKTI_C_PLIC].base,
-        (char *)SHAKTI_C_PLIC_HART_CONFIG, ms->smp.cpus, 0,
+        (char *)SHAKTI_C_PLIC_HART_CONFIG, machine_topo_get_cpus(ms), 0,
         SHAKTI_C_PLIC_NUM_SOURCES,
         SHAKTI_C_PLIC_NUM_PRIORITIES,
         SHAKTI_C_PLIC_PRIORITY_BASE,
diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c
index 3e3f4b0088ca..ca3a7a0877cd 100644
--- a/hw/riscv/sifive_e.c
+++ b/hw/riscv/sifive_e.c
@@ -178,7 +178,8 @@ static void sifive_e_soc_init(Object *obj)
     SiFiveESoCState *s = RISCV_E_SOC(obj);
 
     object_initialize_child(obj, "cpus", &s->cpus, TYPE_RISCV_HART_ARRAY);
-    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
+    object_property_set_int(OBJECT(&s->cpus), "num-harts",
+                            machine_topo_get_cpus(ms),
                             &error_abort);
     object_property_set_int(OBJECT(&s->cpus), "resetvec", 0x1004, &error_abort);
     object_initialize_child(obj, "riscv.sifive.e.gpio0", &s->gpio,
@@ -191,6 +192,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error **errp)
     const MemMapEntry *memmap = sifive_e_memmap;
     SiFiveESoCState *s = RISCV_E_SOC(dev);
     MemoryRegion *sys_mem = get_system_memory();
+    unsigned int cpus = machine_topo_get_cpus(ms);
 
     object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
                             &error_abort);
@@ -204,7 +206,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error **errp)
 
     /* MMIO */
     s->plic = sifive_plic_create(memmap[SIFIVE_E_DEV_PLIC].base,
-        (char *)SIFIVE_E_PLIC_HART_CONFIG, ms->smp.cpus, 0,
+        (char *)SIFIVE_E_PLIC_HART_CONFIG, cpus, 0,
         SIFIVE_E_PLIC_NUM_SOURCES,
         SIFIVE_E_PLIC_NUM_PRIORITIES,
         SIFIVE_E_PLIC_PRIORITY_BASE,
@@ -215,10 +217,10 @@ static void sifive_e_soc_realize(DeviceState *dev, Error **errp)
         SIFIVE_E_PLIC_CONTEXT_STRIDE,
         memmap[SIFIVE_E_DEV_PLIC].size);
     riscv_aclint_swi_create(memmap[SIFIVE_E_DEV_CLINT].base,
-        0, ms->smp.cpus, false);
+        0, cpus, false);
     riscv_aclint_mtimer_create(memmap[SIFIVE_E_DEV_CLINT].base +
             RISCV_ACLINT_SWI_SIZE,
-        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
+        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
         RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
         RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, false);
     create_unimplemented_device("riscv.sifive.e.aon",
diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
index d3ab7a9cda5d..c7f969510209 100644
--- a/hw/riscv/sifive_u.c
+++ b/hw/riscv/sifive_u.c
@@ -99,7 +99,7 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
     MachineState *ms = MACHINE(s);
     uint64_t mem_size = ms->ram_size;
     void *fdt;
-    int cpu, fdt_size;
+    int cpu, fdt_size, num_cpus = machine_topo_get_cpus(ms);
     uint32_t *cells;
     char *nodename;
     uint32_t plic_phandle, prci_phandle, gpio_phandle, phandle = 1;
@@ -175,7 +175,7 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
     qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
     qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
 
-    for (cpu = ms->smp.cpus - 1; cpu >= 0; cpu--) {
+    for (cpu = num_cpus - 1; cpu >= 0; cpu--) {
         int cpu_phandle = phandle++;
         nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
         char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
@@ -207,8 +207,8 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
         g_free(nodename);
     }
 
-    cells =  g_new0(uint32_t, ms->smp.cpus * 4);
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    cells =  g_new0(uint32_t, num_cpus * 4);
+    for (cpu = 0; cpu < num_cpus; cpu++) {
         nodename =
             g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
         uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
@@ -227,7 +227,7 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
         0x0, memmap[SIFIVE_U_DEV_CLINT].base,
         0x0, memmap[SIFIVE_U_DEV_CLINT].size);
     qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
-        cells, ms->smp.cpus * sizeof(uint32_t) * 4);
+        cells, num_cpus * sizeof(uint32_t) * 4);
     g_free(cells);
     g_free(nodename);
 
@@ -258,8 +258,8 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
     g_free(nodename);
 
     plic_phandle = phandle++;
-    cells =  g_new0(uint32_t, ms->smp.cpus * 4 - 2);
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    cells =  g_new0(uint32_t, machine_topo_get_cpus(ms) * 4 - 2);
+    for (cpu = 0; cpu < machine_topo_get_cpus(ms); cpu++) {
         nodename =
             g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
         uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
@@ -283,7 +283,7 @@ static void create_fdt(SiFiveUState *s, const MemMapEntry *memmap,
         (char **)&plic_compat, ARRAY_SIZE(plic_compat));
     qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0);
     qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
-        cells, (ms->smp.cpus * 4 - 2) * sizeof(uint32_t));
+        cells, (machine_topo_get_cpus(ms) * 4 - 2) * sizeof(uint32_t));
     qemu_fdt_setprop_cells(fdt, nodename, "reg",
         0x0, memmap[SIFIVE_U_DEV_PLIC].base,
         0x0, memmap[SIFIVE_U_DEV_PLIC].size);
@@ -797,10 +797,10 @@ static void sifive_u_soc_realize(DeviceState *dev, Error **errp)
     MemoryRegion *mask_rom = g_new(MemoryRegion, 1);
     MemoryRegion *l2lim_mem = g_new(MemoryRegion, 1);
     char *plic_hart_config;
-    int i, j;
+    int i, j, cpus = machine_topo_get_cpus(ms);
     NICInfo *nd = &nd_table[0];
 
-    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
+    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", cpus - 1);
     qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
     qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type", s->cpu_type);
     qdev_prop_set_uint64(DEVICE(&s->u_cpus), "resetvec", 0x1004);
@@ -837,11 +837,11 @@ static void sifive_u_soc_realize(DeviceState *dev, Error **errp)
                                 l2lim_mem);
 
     /* create PLIC hart topology configuration string */
-    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
+    plic_hart_config = riscv_plic_hart_config_string(cpus);
 
     /* MMIO */
     s->plic = sifive_plic_create(memmap[SIFIVE_U_DEV_PLIC].base,
-        plic_hart_config, ms->smp.cpus, 0,
+        plic_hart_config, cpus, 0,
         SIFIVE_U_PLIC_NUM_SOURCES,
         SIFIVE_U_PLIC_NUM_PRIORITIES,
         SIFIVE_U_PLIC_PRIORITY_BASE,
@@ -857,10 +857,10 @@ static void sifive_u_soc_realize(DeviceState *dev, Error **errp)
     sifive_uart_create(system_memory, memmap[SIFIVE_U_DEV_UART1].base,
         serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ));
     riscv_aclint_swi_create(memmap[SIFIVE_U_DEV_CLINT].base, 0,
-        ms->smp.cpus, false);
+        cpus, false);
     riscv_aclint_mtimer_create(memmap[SIFIVE_U_DEV_CLINT].base +
             RISCV_ACLINT_SWI_SIZE,
-        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
+        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
         RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
         CLINT_TIMEBASE_FREQ, false);
 
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index b81081c70b1c..4b0a3dab5ec1 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -506,16 +506,17 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
     char *imsic_name;
     MachineState *ms = MACHINE(s);
     int socket_count = riscv_socket_count(ms);
-    uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
+    uint32_t imsic_max_hart_per_socket, imsic_guest_bits, cpus;
     uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
 
+    cpus = machine_topo_get_cpus(ms);
     *msi_m_phandle = (*phandle)++;
     *msi_s_phandle = (*phandle)++;
-    imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
+    imsic_cells = g_new0(uint32_t, cpus * 2);
     imsic_regs = g_new0(uint32_t, socket_count * 4);
 
     /* M-level IMSIC node */
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    for (cpu = 0; cpu < cpus; cpu++) {
         imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
         imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
     }
@@ -544,7 +545,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
     qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
         NULL, 0);
     qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
+        imsic_cells, cpus * sizeof(uint32_t) * 2);
     qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
         socket_count * sizeof(uint32_t) * 4);
     qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
@@ -562,7 +563,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
     g_free(imsic_name);
 
     /* S-level IMSIC node */
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    for (cpu = 0; cpu < cpus; cpu++) {
         imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
         imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
     }
@@ -593,7 +594,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const MemMapEntry *memmap,
     qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
         NULL, 0);
     qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
-        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
+        imsic_cells, cpus * sizeof(uint32_t) * 2);
     qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
         socket_count * sizeof(uint32_t) * 4);
     qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
@@ -732,7 +733,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
     char *clust_name;
     int socket, phandle_pos;
     MachineState *ms = MACHINE(s);
-    uint32_t msi_m_phandle = 0, msi_s_phandle = 0;
+    uint32_t msi_m_phandle = 0, msi_s_phandle = 0, cpus;
     uint32_t *intc_phandles, xplic_phandles[MAX_NODES];
     int socket_count = riscv_socket_count(ms);
 
@@ -743,9 +744,10 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
     qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
     qemu_fdt_add_subnode(ms->fdt, "/cpus/cpu-map");
 
-    intc_phandles = g_new0(uint32_t, ms->smp.cpus);
+    cpus = machine_topo_get_cpus(ms);
+    intc_phandles = g_new0(uint32_t, cpus);
 
-    phandle_pos = ms->smp.cpus;
+    phandle_pos = cpus;
     for (socket = (socket_count - 1); socket >= 0; socket--) {
         phandle_pos -= s->soc[socket].num_harts;
 
@@ -776,7 +778,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const MemMapEntry *memmap,
         *msi_pcie_phandle = msi_s_phandle;
     }
 
-    phandle_pos = ms->smp.cpus;
+    phandle_pos = cpus;
     for (socket = (socket_count - 1); socket >= 0; socket--) {
         phandle_pos -= s->soc[socket].num_harts;
 
@@ -1113,7 +1115,7 @@ static FWCfgState *create_fw_cfg(const MachineState *ms)
 
     fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16,
                                   &address_space_memory);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine_topo_get_cpus(ms));
 
     return fw_cfg;
 }
-- 
2.34.1

[RFC 23/52] arm: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

Before arm supports hybrid, here we use smp-specific interface to get
"threads per core" and "cores per cluster".

For other cases, it's straightforward to replace topology access with
wrapped generic interfaces.

Cc: Jean-Christophe Dubois <jcd@tribudubois.net>
Cc: Andrey Smirnov <andrew.smirnov@gmail.com>
Cc: Radoslaw Biernacki <rad@semihalf.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Shannon Zhao <shannon.zhaosl@gmail.com>
Cc: Alistair Francis <alistair@alistair23.me>
Cc: Edgar E. Iglesias <edgar.iglesias@gmail.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/arm/fsl-imx6.c        |  4 ++--
 hw/arm/fsl-imx6ul.c      |  4 ++--
 hw/arm/fsl-imx7.c        |  4 ++--
 hw/arm/highbank.c        |  2 +-
 hw/arm/realview.c        |  2 +-
 hw/arm/sbsa-ref.c        |  8 +++----
 hw/arm/vexpress.c        |  2 +-
 hw/arm/virt-acpi-build.c |  4 ++--
 hw/arm/virt.c            | 50 ++++++++++++++++++++++------------------
 hw/arm/xlnx-zynqmp.c     |  6 ++---
 include/hw/arm/virt.h    |  2 +-
 target/arm/cpu.c         |  2 +-
 target/arm/cpu_tcg.c     |  2 +-
 target/arm/kvm.c         |  2 +-
 14 files changed, 50 insertions(+), 44 deletions(-)

diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
index 00dafe3f62de..e94dec5e6c8d 100644
--- a/hw/arm/fsl-imx6.c
+++ b/hw/arm/fsl-imx6.c
@@ -41,7 +41,7 @@ static void fsl_imx6_init(Object *obj)
     char name[NAME_SIZE];
     int i;
 
-    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX6_NUM_CPUS); i++) {
+    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX6_NUM_CPUS); i++) {
         snprintf(name, NAME_SIZE, "cpu%d", i);
         object_initialize_child(obj, name, &s->cpu[i],
                                 ARM_CPU_TYPE_NAME("cortex-a9"));
@@ -108,7 +108,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
     FslIMX6State *s = FSL_IMX6(dev);
     uint16_t i;
     Error *err = NULL;
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
 
     if (smp_cpus > FSL_IMX6_NUM_CPUS) {
         error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
index d88d6cc1c5f9..1216b7ff1a92 100644
--- a/hw/arm/fsl-imx6ul.c
+++ b/hw/arm/fsl-imx6ul.c
@@ -160,9 +160,9 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
     SysBusDevice *sbd;
     DeviceState *d;
 
-    if (ms->smp.cpus > 1) {
+    if (machine_topo_get_cpus(ms) > 1) {
         error_setg(errp, "%s: Only a single CPU is supported (%d requested)",
-                   TYPE_FSL_IMX6UL, ms->smp.cpus);
+                   TYPE_FSL_IMX6UL, machine_topo_get_cpus(ms));
         return;
     }
 
diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
index afc74807990f..f3e569a6ec29 100644
--- a/hw/arm/fsl-imx7.c
+++ b/hw/arm/fsl-imx7.c
@@ -36,7 +36,7 @@ static void fsl_imx7_init(Object *obj)
     char name[NAME_SIZE];
     int i;
 
-    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX7_NUM_CPUS); i++) {
+    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX7_NUM_CPUS); i++) {
         snprintf(name, NAME_SIZE, "cpu%d", i);
         object_initialize_child(obj, name, &s->cpu[i],
                                 ARM_CPU_TYPE_NAME("cortex-a7"));
@@ -148,7 +148,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
     int i;
     qemu_irq irq;
     char name[NAME_SIZE];
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
 
     if (smp_cpus > FSL_IMX7_NUM_CPUS) {
         error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
index f12aacea6b86..22d6987eafe1 100644
--- a/hw/arm/highbank.c
+++ b/hw/arm/highbank.c
@@ -181,7 +181,7 @@ static void calxeda_init(MachineState *machine, enum cxmachines machine_id)
     SysBusDevice *busdev;
     qemu_irq pic[128];
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     qemu_irq cpu_irq[4];
     qemu_irq cpu_fiq[4];
     qemu_irq cpu_virq[4];
diff --git a/hw/arm/realview.c b/hw/arm/realview.c
index a5aa2f046aec..0a2022a34629 100644
--- a/hw/arm/realview.c
+++ b/hw/arm/realview.c
@@ -87,7 +87,7 @@ static void realview_init(MachineState *machine,
     DriveInfo *dinfo;
     I2CBus *i2c;
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     int done_nic = 0;
     qemu_irq cpu_irq[4];
     int is_mpcore = 0;
diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
index f778cb6d0979..35f2b83849d5 100644
--- a/hw/arm/sbsa-ref.c
+++ b/hw/arm/sbsa-ref.c
@@ -394,7 +394,7 @@ static void create_secure_ram(SBSAMachineState *sms,
 
 static void create_gic(SBSAMachineState *sms)
 {
-    unsigned int smp_cpus = MACHINE(sms)->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(MACHINE(sms));
     SysBusDevice *gicbusdev;
     const char *gictype;
     uint32_t redist0_capacity, redist0_count;
@@ -674,8 +674,8 @@ static void create_secure_ec(MemoryRegion *mem)
 
 static void sbsa_ref_init(MachineState *machine)
 {
-    unsigned int smp_cpus = machine->smp.cpus;
-    unsigned int max_cpus = machine->smp.max_cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
+    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
     SBSAMachineState *sms = SBSA_MACHINE(machine);
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     MemoryRegion *sysmem = get_system_memory();
@@ -801,7 +801,7 @@ static void sbsa_ref_init(MachineState *machine)
 
 static const CPUArchIdList *sbsa_ref_possible_cpu_arch_ids(MachineState *ms)
 {
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
     SBSAMachineState *sms = SBSA_MACHINE(ms);
     int n;
 
diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
index 34b012b528b0..5e486da27ee8 100644
--- a/hw/arm/vexpress.c
+++ b/hw/arm/vexpress.c
@@ -207,7 +207,7 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
     DeviceState *dev;
     SysBusDevice *busdev;
     int n;
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
 
     /* Create the actual CPUs */
     for (n = 0; n < smp_cpus; n++) {
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 4156111d49f0..985b945e762d 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -67,7 +67,7 @@ static void acpi_dsdt_add_cpus(Aml *scope, VirtMachineState *vms)
     MachineState *ms = MACHINE(vms);
     uint16_t i;
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
         Aml *dev = aml_device("C%.03X", i);
         aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
         aml_append(dev, aml_name_decl("_UID", aml_int(i)));
@@ -725,7 +725,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     build_append_int_noprefix(table_data, vms->gic_version, 1);
     build_append_int_noprefix(table_data, 0, 3);   /* Reserved */
 
-    for (i = 0; i < MACHINE(vms)->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(MACHINE(vms)); i++) {
         ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(i));
         uint64_t physical_base_address = 0, gich = 0, gicv = 0;
         uint32_t vgic_interrupt = vms->virt ? PPI(ARCH_GIC_MAINT_IRQ) : 0;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 75f28947de07..ae65ba2c929c 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -346,7 +346,7 @@ static void fdt_add_timer_nodes(const VirtMachineState *vms)
     if (vms->gic_version == VIRT_GIC_VERSION_2) {
         irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
                              GIC_FDT_IRQ_PPI_CPU_WIDTH,
-                             (1 << MACHINE(vms)->smp.cpus) - 1);
+                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
     }
 
     qemu_fdt_add_subnode(ms->fdt, "/timer");
@@ -374,7 +374,10 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
     int addr_cells = 1;
     const MachineState *ms = MACHINE(vms);
     const VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
-    int smp_cpus = ms->smp.cpus;
+    int smp_cpus = machine_topo_get_cpus(ms);
+    int smp_clusters = machine_topo_get_clusters(ms);
+    int smp_cores = machine_topo_get_smp_cores(ms);
+    int smp_threads = machine_topo_get_smp_threads(ms);
 
     /*
      * See Linux Documentation/devicetree/bindings/arm/cpus.yaml
@@ -461,19 +464,19 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
             char *cpu_path = g_strdup_printf("/cpus/cpu@%d", cpu);
             char *map_path;
 
-            if (ms->smp.threads > 1) {
+            if (smp_threads > 1) {
                 map_path = g_strdup_printf(
                     "/cpus/cpu-map/socket%d/cluster%d/core%d/thread%d",
-                    cpu / (ms->smp.clusters * ms->smp.cores * ms->smp.threads),
-                    (cpu / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters,
-                    (cpu / ms->smp.threads) % ms->smp.cores,
-                    cpu % ms->smp.threads);
+                    cpu / (smp_clusters * smp_cores * smp_threads),
+                    (cpu / (smp_cores * smp_threads)) % smp_clusters,
+                    (cpu / smp_threads) % smp_cores,
+                    cpu % smp_threads);
             } else {
                 map_path = g_strdup_printf(
                     "/cpus/cpu-map/socket%d/cluster%d/core%d",
-                    cpu / (ms->smp.clusters * ms->smp.cores),
-                    (cpu / ms->smp.cores) % ms->smp.clusters,
-                    cpu % ms->smp.cores);
+                    cpu / (smp_clusters * smp_cores),
+                    (cpu / smp_cores) % smp_clusters,
+                    cpu % smp_cores);
             }
             qemu_fdt_add_path(ms->fdt, map_path);
             qemu_fdt_setprop_phandle(ms->fdt, map_path, "cpu", cpu_path);
@@ -613,7 +616,7 @@ static void fdt_add_pmu_nodes(const VirtMachineState *vms)
     if (vms->gic_version == VIRT_GIC_VERSION_2) {
         irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
                              GIC_FDT_IRQ_PPI_CPU_WIDTH,
-                             (1 << MACHINE(vms)->smp.cpus) - 1);
+                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
     }
 
     qemu_fdt_add_subnode(ms->fdt, "/pmu");
@@ -708,7 +711,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
     SysBusDevice *gicbusdev;
     const char *gictype;
     int i;
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
     uint32_t nb_redist_regions = 0;
     int revision;
 
@@ -1277,7 +1280,7 @@ static FWCfgState *create_fw_cfg(const VirtMachineState *vms, AddressSpace *as)
     char *nodename;
 
     fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16, as);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine_topo_get_cpus(ms));
 
     nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
     qemu_fdt_add_subnode(ms->fdt, nodename);
@@ -1908,7 +1911,7 @@ static VirtGICType finalize_gic_version_do(const char *accel_name,
 static void finalize_gic_version(VirtMachineState *vms)
 {
     const char *accel_name = current_accel_name();
-    unsigned int max_cpus = MACHINE(vms)->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
     int gics_supported = 0;
 
     /* Determine which GIC versions the current environment supports */
@@ -1958,7 +1961,7 @@ static void finalize_gic_version(VirtMachineState *vms)
  */
 static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
 {
-    int max_cpus = MACHINE(vms)->smp.max_cpus;
+    int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
     bool aarch64, pmu, steal_time;
     CPUState *cpu;
 
@@ -2032,8 +2035,8 @@ static void machvirt_init(MachineState *machine)
     bool firmware_loaded;
     bool aarch64 = true;
     bool has_ged = !vmc->no_ged;
-    unsigned int smp_cpus = machine->smp.cpus;
-    unsigned int max_cpus = machine->smp.max_cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
+    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
 
     if (!cpu_type_valid(machine->cpu_type)) {
         error_report("mach-virt: CPU type %s not supported", machine->cpu_type);
@@ -2664,7 +2667,10 @@ static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int idx)
 static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
 {
     int n;
-    unsigned int max_cpus = ms->smp.max_cpus;
+    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
+    unsigned int smp_clusters = machine_topo_get_clusters(ms);
+    unsigned int smp_cores = machine_topo_get_smp_cores(ms);
+    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
     VirtMachineState *vms = VIRT_MACHINE(ms);
     MachineClass *mc = MACHINE_GET_CLASS(vms);
 
@@ -2684,16 +2690,16 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
         assert(!mc->smp_props.dies_supported);
         ms->possible_cpus->cpus[n].props.has_socket_id = true;
         ms->possible_cpus->cpus[n].props.socket_id =
-            n / (ms->smp.clusters * ms->smp.cores * ms->smp.threads);
+            n / (smp_clusters * smp_cores * smp_threads);
         ms->possible_cpus->cpus[n].props.has_cluster_id = true;
         ms->possible_cpus->cpus[n].props.cluster_id =
-            (n / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters;
+            (n / (smp_cores * smp_threads)) % smp_clusters;
         ms->possible_cpus->cpus[n].props.has_core_id = true;
         ms->possible_cpus->cpus[n].props.core_id =
-            (n / ms->smp.threads) % ms->smp.cores;
+            (n / smp_threads) % smp_cores;
         ms->possible_cpus->cpus[n].props.has_thread_id = true;
         ms->possible_cpus->cpus[n].props.thread_id =
-            n % ms->smp.threads;
+            n % smp_threads;
     }
     return ms->possible_cpus;
 }
diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c
index 335cfc417d70..773de7a51680 100644
--- a/hw/arm/xlnx-zynqmp.c
+++ b/hw/arm/xlnx-zynqmp.c
@@ -213,7 +213,7 @@ static void xlnx_zynqmp_create_rpu(MachineState *ms, XlnxZynqMPState *s,
                                    const char *boot_cpu, Error **errp)
 {
     int i;
-    int num_rpus = MIN(ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS,
+    int num_rpus = MIN(machine_topo_get_cpus(ms) - XLNX_ZYNQMP_NUM_APU_CPUS,
                        XLNX_ZYNQMP_NUM_RPU_CPUS);
 
     if (num_rpus <= 0) {
@@ -376,7 +376,7 @@ static void xlnx_zynqmp_init(Object *obj)
     MachineState *ms = MACHINE(qdev_get_machine());
     XlnxZynqMPState *s = XLNX_ZYNQMP(obj);
     int i;
-    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
+    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
 
     object_initialize_child(obj, "apu-cluster", &s->apu_cluster,
                             TYPE_CPU_CLUSTER);
@@ -449,7 +449,7 @@ static void xlnx_zynqmp_realize(DeviceState *dev, Error **errp)
     MemoryRegion *system_memory = get_system_memory();
     uint8_t i;
     uint64_t ram_size;
-    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
+    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
     const char *boot_cpu = s->boot_cpu ? s->boot_cpu : "apu-cpu[0]";
     ram_addr_t ddr_low_size, ddr_high_size;
     qemu_irq gic_spi[GIC_NUM_SPI_INTR];
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index e1ddbea96bea..e046f530990f 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -213,7 +213,7 @@ static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
 
     assert(vms->gic_version != VIRT_GIC_VERSION_2);
 
-    return (MACHINE(vms)->smp.cpus > redist0_capacity &&
+    return (machine_topo_get_cpus(MACHINE(vms)) > redist0_capacity &&
             vms->highmem_redists) ? 2 : 1;
 }
 
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 5f63316dbf22..a7f1d470eed7 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -2079,7 +2079,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
 
 #ifndef CONFIG_USER_ONLY
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
     bool has_secure = cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY);
 
     /*
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index ccde5080eb70..a098d797637c 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -515,7 +515,7 @@ static uint64_t a15_l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
      * Linux wants the number of processors from here.
      * Might as well set the interrupt-controller bit too.
      */
-    return ((ms->smp.cpus - 1) << 24) | (1 << 23);
+    return ((machine_topo_get_cpus(ms) - 1) << 24) | (1 << 23);
 }
 #endif
 
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index f022c644d2ff..eefded5d203b 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -263,7 +263,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
 
     cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
 
-    if (ms->smp.cpus > 256 &&
+    if (machine_topo_get_cpus(ms) > 256 &&
         !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
         error_report("Using more than 256 vcpus requires a host kernel "
                      "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");
-- 
2.34.1

[RFC 24/52] loongarch: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

Before loongarch supports hybrid, here we use smp-specific interface to
get "threads per core" and "cores per cluster".

For other cases, it's straightforward to replace topology access with
wrapped generic interfaces.

Cc: Xiaojuan Yang <yangxiaojuan@loongson.cn>
Cc: Song Gao <gaosong@loongson.cn>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/loongarch/acpi-build.c |  4 ++--
 hw/loongarch/fw_cfg.c     |  4 ++--
 hw/loongarch/virt.c       | 30 ++++++++++++++++--------------
 3 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/hw/loongarch/acpi-build.c b/hw/loongarch/acpi-build.c
index f551296a0e07..87fa5f0836f8 100644
--- a/hw/loongarch/acpi-build.c
+++ b/hw/loongarch/acpi-build.c
@@ -116,7 +116,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, LoongArchMachineState *lams)
     build_append_int_noprefix(table_data, 0, 4);
     build_append_int_noprefix(table_data, 1 /* PCAT_COMPAT */, 4); /* Flags */
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
         /* Processor Core Interrupt Controller Structure */
         build_append_int_noprefix(table_data, 17, 1);    /* Type */
         build_append_int_noprefix(table_data, 15, 1);    /* Length */
@@ -168,7 +168,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
     build_append_int_noprefix(table_data, 1, 4); /* Reserved */
     build_append_int_noprefix(table_data, 0, 8); /* Reserved */
 
-    for (i = 0; i < ms->smp.cpus; ++i) {
+    for (i = 0; i < machine_topo_get_cpus(ms); ++i) {
         /* Processor Local APIC/SAPIC Affinity Structure */
         build_append_int_noprefix(table_data, 0, 1);  /* Type  */
         build_append_int_noprefix(table_data, 16, 1); /* Length */
diff --git a/hw/loongarch/fw_cfg.c b/hw/loongarch/fw_cfg.c
index f15a17416c48..a9c1762a1681 100644
--- a/hw/loongarch/fw_cfg.c
+++ b/hw/loongarch/fw_cfg.c
@@ -20,8 +20,8 @@ static void fw_cfg_boot_set(void *opaque, const char *boot_device,
 FWCfgState *loongarch_fw_cfg_init(ram_addr_t ram_size, MachineState *ms)
 {
     FWCfgState *fw_cfg;
-    int max_cpus = ms->smp.max_cpus;
-    int smp_cpus = ms->smp.cpus;
+    int max_cpus = machine_topo_get_max_cpus(ms);
+    int smp_cpus = machine_topo_get_cpus(ms);
 
     fw_cfg = fw_cfg_init_mem_wide(VIRT_FWCFG_BASE + 8, VIRT_FWCFG_BASE, 8,
                                   VIRT_FWCFG_BASE + 16, &address_space_memory);
diff --git a/hw/loongarch/virt.c b/hw/loongarch/virt.c
index 66be9250684e..475b82d283bf 100644
--- a/hw/loongarch/virt.c
+++ b/hw/loongarch/virt.c
@@ -152,7 +152,9 @@ static void fdt_add_cpu_nodes(const LoongArchMachineState *lams)
 {
     int num;
     const MachineState *ms = MACHINE(lams);
-    int smp_cpus = ms->smp.cpus;
+    int smp_cpus = machine_topo_get_cpus(ms);
+    int smp_cores = machine_topo_get_smp_cores(ms);
+    int smp_threads = machine_topo_get_smp_threads(ms);
 
     qemu_fdt_add_subnode(ms->fdt, "/cpus");
     qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
@@ -180,17 +182,17 @@ static void fdt_add_cpu_nodes(const LoongArchMachineState *lams)
         char *cpu_path = g_strdup_printf("/cpus/cpu@%d", num);
         char *map_path;
 
-        if (ms->smp.threads > 1) {
+        if (smp_threads > 1) {
             map_path = g_strdup_printf(
                 "/cpus/cpu-map/socket%d/core%d/thread%d",
-                num / (ms->smp.cores * ms->smp.threads),
-                (num / ms->smp.threads) % ms->smp.cores,
-                num % ms->smp.threads);
+                num / (smp_cores * smp_threads),
+                (num / smp_threads) % smp_cores,
+                num % smp_threads);
         } else {
             map_path = g_strdup_printf(
                 "/cpus/cpu-map/socket%d/core%d",
-                num / ms->smp.cores,
-                num % ms->smp.cores);
+                num / smp_cores,
+                num % smp_cores);
         }
         qemu_fdt_add_path(ms->fdt, map_path);
         qemu_fdt_setprop_phandle(ms->fdt, map_path, "cpu", cpu_path);
@@ -553,7 +555,7 @@ static void loongarch_irq_init(LoongArchMachineState *lams)
     LoongArchCPU *lacpu;
     CPULoongArchState *env;
     CPUState *cpu_state;
-    int cpu, pin, i, start, num;
+    int cpu, pin, i, start, num, smp_cpus = machine_topo_get_cpus(ms);
 
     ipi = qdev_new(TYPE_LOONGARCH_IPI);
     sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
@@ -582,7 +584,7 @@ static void loongarch_irq_init(LoongArchMachineState *lams)
      * | UARTs  | | Devices | | Devices |
      * +--------+ +---------+ +---------+
      */
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    for (cpu = 0; cpu < smp_cpus; cpu++) {
         cpu_state = qemu_get_cpu(cpu);
         cpudev = DEVICE(cpu_state);
         lacpu = LOONGARCH_CPU(cpu_state);
@@ -607,7 +609,7 @@ static void loongarch_irq_init(LoongArchMachineState *lams)
      * connect ext irq to the cpu irq
      * cpu_pin[9:2] <= intc_pin[7:0]
      */
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    for (cpu = 0; cpu < smp_cpus; cpu++) {
         cpudev = DEVICE(qemu_get_cpu(cpu));
         for (pin = 0; pin < LS3A_INTC_IP; pin++) {
             qdev_connect_gpio_out(extioi, (cpu * 8 + pin),
@@ -736,7 +738,7 @@ static void loongarch_direct_kernel_boot(LoongArchMachineState *lams)
 
     kernel_addr = load_kernel_info();
     if (!machine->firmware) {
-        for (i = 0; i < machine->smp.cpus; i++) {
+        for (i = 0; i < machine_topo_get_cpus(machine); i++) {
             lacpu = LOONGARCH_CPU(qemu_get_cpu(i));
             lacpu->env.load_elf = true;
             lacpu->env.elf_address = kernel_addr;
@@ -753,7 +755,7 @@ static void loongarch_init(MachineState *machine)
     uint64_t highram_size = 0;
     MemoryRegion *address_space_mem = get_system_memory();
     LoongArchMachineState *lams = LOONGARCH_MACHINE(machine);
-    int i;
+    int i, smp_cpus = machine_topo_get_cpus(machine);
     hwaddr fdt_base;
 
     if (!cpu_model) {
@@ -771,7 +773,7 @@ static void loongarch_init(MachineState *machine)
     }
     create_fdt(lams);
     /* Init CPUs */
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < smp_cpus; i++) {
         cpu_create(machine->cpu_type);
     }
     fdt_add_cpu_nodes(lams);
@@ -846,7 +848,7 @@ static void loongarch_init(MachineState *machine)
     }
     fdt_add_flash_node(lams);
     /* register reset function */
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < smp_cpus; i++) {
         lacpu = LOONGARCH_CPU(qemu_get_cpu(i));
         qemu_register_reset(reset_load_elf, lacpu);
     }
-- 
2.34.1

[RFC 25/52] mips: Replace MachineState.smp access with topology helpers

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

For mips, it's straightforward to replace topology access with wrapped
generic interfaces.

Cc: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
Cc: Aurelien Jarno <aurelien@aurel32.net>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/mips/boston.c          |  8 +++++---
 hw/mips/loongson3_bootp.c |  7 ++++---
 hw/mips/loongson3_virt.c  |  8 +++++---
 hw/mips/malta.c           | 10 ++++++----
 4 files changed, 20 insertions(+), 13 deletions(-)

diff --git a/hw/mips/boston.c b/hw/mips/boston.c
index a9d87f34378f..d02c88d769d4 100644
--- a/hw/mips/boston.c
+++ b/hw/mips/boston.c
@@ -514,7 +514,7 @@ static const void *create_fdt(BostonState *s,
                               const MemMapEntry *memmap, int *dt_size)
 {
     void *fdt;
-    int cpu;
+    int cpu, smp_cpus;
     MachineState *ms = s->mach;
     uint32_t platreg_ph, gic_ph, clk_ph;
     char *name, *gic_name, *platreg_name, *stdout_name;
@@ -542,7 +542,8 @@ static const void *create_fdt(BostonState *s,
     qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
     qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
 
-    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
+    smp_cpus = machine_topo_get_cpus(ms);
+    for (cpu = 0; cpu < smp_cpus; cpu++) {
         name = g_strdup_printf("/cpus/cpu@%d", cpu);
         qemu_fdt_add_subnode(fdt, name);
         qemu_fdt_setprop_string(fdt, name, "compatible", "img,mips");
@@ -702,7 +703,8 @@ static void boston_mach_init(MachineState *machine)
     object_initialize_child(OBJECT(machine), "cps", &s->cps, TYPE_MIPS_CPS);
     object_property_set_str(OBJECT(&s->cps), "cpu-type", machine->cpu_type,
                             &error_fatal);
-    object_property_set_int(OBJECT(&s->cps), "num-vp", machine->smp.cpus,
+    object_property_set_int(OBJECT(&s->cps), "num-vp",
+                            machine_topo_get_cpus(machine),
                             &error_fatal);
     qdev_connect_clock_in(DEVICE(&s->cps), "clk-in",
                           qdev_get_clock_out(dev, "cpu-refclk"));
diff --git a/hw/mips/loongson3_bootp.c b/hw/mips/loongson3_bootp.c
index f99af229327a..d9a92825ceae 100644
--- a/hw/mips/loongson3_bootp.c
+++ b/hw/mips/loongson3_bootp.c
@@ -40,9 +40,10 @@ static void init_cpu_info(void *g_cpuinfo, uint64_t cpu_freq)
     }
 
     c->cpu_startup_core_id = cpu_to_le16(0);
-    c->nr_cpus = cpu_to_le32(current_machine->smp.cpus);
-    c->total_node = cpu_to_le32(DIV_ROUND_UP(current_machine->smp.cpus,
-                                             LOONGSON3_CORE_PER_NODE));
+    c->nr_cpus = cpu_to_le32(machine_topo_get_cpus(current_machine));
+    c->total_node =
+        cpu_to_le32(DIV_ROUND_UP(machine_topo_get_cpus(current_machine),
+                                 LOONGSON3_CORE_PER_NODE));
 }
 
 static void init_memory_map(void *g_map, uint64_t ram_size)
diff --git a/hw/mips/loongson3_virt.c b/hw/mips/loongson3_virt.c
index 25534288dd81..c972bb43a1f7 100644
--- a/hw/mips/loongson3_virt.c
+++ b/hw/mips/loongson3_virt.c
@@ -271,8 +271,10 @@ static void fw_conf_init(unsigned long ram_size)
     hwaddr cfg_addr = virt_memmap[VIRT_FW_CFG].base;
 
     fw_cfg = fw_cfg_init_mem_wide(cfg_addr, cfg_addr + 8, 8, 0, NULL);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)current_machine->smp.cpus);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)current_machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
+                   (uint16_t)machine_topo_get_cpus(current_machine));
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
+                   (uint16_t)machine_topo_get_max_cpus(current_machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)ram_size);
     fw_cfg_add_i32(fw_cfg, FW_CFG_MACHINE_VERSION, 1);
     fw_cfg_add_i64(fw_cfg, FW_CFG_CPU_FREQ, get_cpu_freq_hz());
@@ -528,7 +530,7 @@ static void mips_loongson3_virt_init(MachineState *machine)
     cpuclk = clock_new(OBJECT(machine), "cpu-refclk");
     clock_set_hz(cpuclk, DEF_LOONGSON3_FREQ);
 
-    for (i = 0; i < machine->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(machine); i++) {
         int ip;
 
         /* init CPUs */
diff --git a/hw/mips/malta.c b/hw/mips/malta.c
index ec172b111ae4..b3322f74baf5 100644
--- a/hw/mips/malta.c
+++ b/hw/mips/malta.c
@@ -981,7 +981,7 @@ static uint64_t load_kernel(void)
 static void malta_mips_config(MIPSCPU *cpu)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
     CPUMIPSState *env = &cpu->env;
     CPUState *cs = CPU(cpu);
 
@@ -1045,7 +1045,7 @@ static void create_cpu_without_cps(MachineState *ms, MaltaState *s,
     MIPSCPU *cpu;
     int i;
 
-    for (i = 0; i < ms->smp.cpus; i++) {
+    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
         cpu = mips_cpu_create_with_clock(ms->cpu_type, s->cpuclk);
 
         /* Init internal devices */
@@ -1066,7 +1066,8 @@ static void create_cps(MachineState *ms, MaltaState *s,
     object_initialize_child(OBJECT(s), "cps", &s->cps, TYPE_MIPS_CPS);
     object_property_set_str(OBJECT(&s->cps), "cpu-type", ms->cpu_type,
                             &error_fatal);
-    object_property_set_int(OBJECT(&s->cps), "num-vp", ms->smp.cpus,
+    object_property_set_int(OBJECT(&s->cps), "num-vp",
+                            machine_topo_get_cpus(ms),
                             &error_fatal);
     qdev_connect_clock_in(DEVICE(&s->cps), "clk-in", s->cpuclk);
     sysbus_realize(SYS_BUS_DEVICE(&s->cps), &error_fatal);
@@ -1080,7 +1081,8 @@ static void create_cps(MachineState *ms, MaltaState *s,
 static void mips_create_cpu(MachineState *ms, MaltaState *s,
                             qemu_irq *cbus_irq, qemu_irq *i8259_irq)
 {
-    if ((ms->smp.cpus > 1) && cpu_type_supports_cps_smp(ms->cpu_type)) {
+    if ((machine_topo_get_cpus(ms) > 1) &&
+        cpu_type_supports_cps_smp(ms->cpu_type)) {
         create_cps(ms, s, cbus_irq, i8259_irq);
     } else {
         create_cpu_without_cps(ms, s, cbus_irq, i8259_irq);
-- 
2.34.1

[RFC 26/52] hw: Replace MachineState.smp access with topology helpers for all remaining archs

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

When MachineState.topo is introduced, the topology related structures
become complicated. So we wrapped the access to topology fields of
MachineState.topo into some helpers, and we are using these helpers
to replace the use of MachineState.smp.

For alpha, hppa, intc, openrisc, sparc, sparc64, xtensa, since they
don't ask for "threads per core" or "cores per cluster", it's
straightforward to replace topology access with wrapped generic
interfaces.

Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Helge Deller <deller@gmx.de>
Cc: David Gibson <david@gibson.dropbear.id.au>
Cc: Greg Kurz <groug@kaod.org>
Cc: Jia Liu <proljc@gmail.com>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Cc: Artyom Tarasenko <atar4qemu@gmail.com>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/alpha/dp264.c             | 2 +-
 hw/hppa/machine.c            | 6 +++---
 hw/intc/spapr_xive_kvm.c     | 3 ++-
 hw/intc/xics_kvm.c           | 3 ++-
 hw/openrisc/openrisc_sim.c   | 2 +-
 hw/openrisc/virt.c           | 2 +-
 hw/sparc/sun4m.c             | 4 ++--
 hw/sparc64/sun4u.c           | 6 ++++--
 hw/xtensa/sim.c              | 2 +-
 hw/xtensa/xtfpga.c           | 2 +-
 target/openrisc/sys_helper.c | 2 +-
 11 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/hw/alpha/dp264.c b/hw/alpha/dp264.c
index 4161f559a7bc..9d0f74d72ede 100644
--- a/hw/alpha/dp264.c
+++ b/hw/alpha/dp264.c
@@ -60,7 +60,7 @@ static void clipper_init(MachineState *machine)
     char *palcode_filename;
     uint64_t palcode_entry;
     uint64_t kernel_entry, kernel_low;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
 
     /* Create up to 4 cpus.  */
     memset(cpus, 0, sizeof(cpus));
diff --git a/hw/hppa/machine.c b/hw/hppa/machine.c
index 7ac68c943f40..9a3189674ad9 100644
--- a/hw/hppa/machine.c
+++ b/hw/hppa/machine.c
@@ -124,7 +124,7 @@ static FWCfgState *create_fw_cfg(MachineState *ms)
     uint64_t val;
 
     fw_cfg = fw_cfg_init_mem(FW_CFG_IO_BASE, FW_CFG_IO_BASE + 4);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, ms->smp.cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, machine_topo_get_cpus(ms));
     fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, HPPA_MAX_CPUS);
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, ms->ram_size);
 
@@ -188,7 +188,7 @@ static void machine_hppa_init(MachineState *machine)
     MemoryRegion *rom_region;
     MemoryRegion *cpu_region;
     long i;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     SysBusDevice *s;
 
     /* Create CPUs.  */
@@ -410,7 +410,7 @@ static void machine_hppa_init(MachineState *machine)
 
 static void hppa_machine_reset(MachineState *ms, ShutdownCause reason)
 {
-    unsigned int smp_cpus = ms->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(ms);
     int i;
 
     qemu_devices_reset(reason);
diff --git a/hw/intc/spapr_xive_kvm.c b/hw/intc/spapr_xive_kvm.c
index 61fe7bd2d322..bdb021abceb2 100644
--- a/hw/intc/spapr_xive_kvm.c
+++ b/hw/intc/spapr_xive_kvm.c
@@ -174,7 +174,8 @@ int kvmppc_xive_cpu_connect(XiveTCTX *tctx, Error **errp)
                          vcpu_id);
         if (ret == -ENOSPC) {
             error_append_hint(errp, "Try -smp maxcpus=N with N < %u\n",
-                              MACHINE(qdev_get_machine())->smp.max_cpus);
+                              machine_topo_get_max_cpus(
+                                  MACHINE(qdev_get_machine())));
         }
         return ret;
     }
diff --git a/hw/intc/xics_kvm.c b/hw/intc/xics_kvm.c
index 9719d98a179e..f45cc0af8b1e 100644
--- a/hw/intc/xics_kvm.c
+++ b/hw/intc/xics_kvm.c
@@ -169,7 +169,8 @@ void icp_kvm_realize(DeviceState *dev, Error **errp)
                    vcpu_id, strerror(errno));
         if (errno == ENOSPC) {
             error_append_hint(&local_err, "Try -smp maxcpus=N with N < %u\n",
-                              MACHINE(qdev_get_machine())->smp.max_cpus);
+                              machine_topo_get_max_cpus(
+                                  MACHINE(qdev_get_machine())));
         }
         error_propagate(errp, local_err);
         return;
diff --git a/hw/openrisc/openrisc_sim.c b/hw/openrisc/openrisc_sim.c
index 35da123aef4c..98dcbde924a8 100644
--- a/hw/openrisc/openrisc_sim.c
+++ b/hw/openrisc/openrisc_sim.c
@@ -291,7 +291,7 @@ static void openrisc_sim_init(MachineState *machine)
     MemoryRegion *ram;
     hwaddr load_addr;
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
 
     assert(smp_cpus >= 1 && smp_cpus <= OR1KSIM_CPUS_MAX);
     for (n = 0; n < smp_cpus; n++) {
diff --git a/hw/openrisc/virt.c b/hw/openrisc/virt.c
index f8a68a6a6b1f..730013142ca9 100644
--- a/hw/openrisc/virt.c
+++ b/hw/openrisc/virt.c
@@ -476,7 +476,7 @@ static void openrisc_virt_init(MachineState *machine)
     MemoryRegion *ram;
     hwaddr load_addr;
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
     int32_t pic_phandle;
 
     assert(smp_cpus >= 1 && smp_cpus <= VIRT_CPUS_MAX);
diff --git a/hw/sparc/sun4m.c b/hw/sparc/sun4m.c
index d9288326d6ac..09f4beaa01df 100644
--- a/hw/sparc/sun4m.c
+++ b/hw/sparc/sun4m.c
@@ -829,8 +829,8 @@ static void sun4m_hw_init(MachineState *machine)
     FWCfgState *fw_cfg;
     DeviceState *dev, *ms_kb_orgate, *serial_orgate;
     SysBusDevice *s;
-    unsigned int smp_cpus = machine->smp.cpus;
-    unsigned int max_cpus = machine->smp.max_cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
+    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
     HostMemoryBackend *ram_memdev = machine->memdev;
     NICInfo *nd = &nd_table[0];
 
diff --git a/hw/sparc64/sun4u.c b/hw/sparc64/sun4u.c
index 387181ff7762..c0350311e405 100644
--- a/hw/sparc64/sun4u.c
+++ b/hw/sparc64/sun4u.c
@@ -711,8 +711,10 @@ static void sun4uv_init(MemoryRegion *address_space_mem,
                                 &FW_CFG_IO(dev)->comb_iomem);
 
     fw_cfg = FW_CFG(dev);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine->smp.cpus);
-    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)machine->smp.max_cpus);
+    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
+                   (uint16_t)machine_topo_get_cpus(machine));
+    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
+                   (uint16_t)machine_topo_get_max_cpus(machine));
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)machine->ram_size);
     fw_cfg_add_i16(fw_cfg, FW_CFG_MACHINE_ID, hwdef->machine_id);
     fw_cfg_add_i64(fw_cfg, FW_CFG_KERNEL_ADDR, kernel_entry);
diff --git a/hw/xtensa/sim.c b/hw/xtensa/sim.c
index 946c71cb5b5c..dc2f95554311 100644
--- a/hw/xtensa/sim.c
+++ b/hw/xtensa/sim.c
@@ -58,7 +58,7 @@ XtensaCPU *xtensa_sim_common_init(MachineState *machine)
     ram_addr_t ram_size = machine->ram_size;
     int n;
 
-    for (n = 0; n < machine->smp.cpus; n++) {
+    for (n = 0; n < machine_topo_get_cpus(machine); n++) {
         cpu = XTENSA_CPU(cpu_create(machine->cpu_type));
         env = &cpu->env;
 
diff --git a/hw/xtensa/xtfpga.c b/hw/xtensa/xtfpga.c
index 2a5556a35f50..70c308dd1bf6 100644
--- a/hw/xtensa/xtfpga.c
+++ b/hw/xtensa/xtfpga.c
@@ -239,7 +239,7 @@ static void xtfpga_init(const XtfpgaBoardDesc *board, MachineState *machine)
     const unsigned system_io_size = 224 * MiB;
     uint32_t freq = 10000000;
     int n;
-    unsigned int smp_cpus = machine->smp.cpus;
+    unsigned int smp_cpus = machine_topo_get_cpus(machine);
 
     if (smp_cpus > 1) {
         mx_pic = xtensa_mx_pic_init(31);
diff --git a/target/openrisc/sys_helper.c b/target/openrisc/sys_helper.c
index ec145960e3e5..3f3e57291035 100644
--- a/target/openrisc/sys_helper.c
+++ b/target/openrisc/sys_helper.c
@@ -262,7 +262,7 @@ target_ulong HELPER(mfspr)(CPUOpenRISCState *env, target_ulong rd,
         return cpu->parent_obj.cpu_index;
 
     case TO_SPR(0, 129): /* NUMCORES */
-        return ms->smp.max_cpus;
+        return machine_topo_get_max_cpus(ms);
 
     case TO_SPR(0, 1024) ... TO_SPR(0, 1024 + (16 * 32)): /* Shadow GPRs */
         idx = (spr - 1024);
-- 
2.34.1

[RFC 27/52] test/test-smp-parse: Check fields of MachineState.topo.smp

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

MachineState.smp is being replaced by MachineState.topo.smp, so test
MachineState.topo.smp fields parsing for smp command.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 tests/unit/test-smp-parse.c | 62 ++++++++++++++++++++++---------------
 1 file changed, 37 insertions(+), 25 deletions(-)

diff --git a/tests/unit/test-smp-parse.c b/tests/unit/test-smp-parse.c
index fdc39a846ca6..d6816f109084 100644
--- a/tests/unit/test-smp-parse.c
+++ b/tests/unit/test-smp-parse.c
@@ -41,10 +41,13 @@
 #define CPU_TOPOLOGY_GENERIC(a, b, c, d, e)                   \
         {                                                     \
             .cpus     = a,                                    \
-            .sockets  = b,                                    \
-            .cores    = c,                                    \
-            .threads  = d,                                    \
             .max_cpus = e,                                    \
+            .topo_type = CPU_TOPO_TYPE_SMP,                   \
+            .smp = {                                          \
+                .sockets  = b,                                \
+                .cores    = c,                                \
+                .threads  = d,                                \
+            }                                                 \
         }
 
 /*
@@ -406,8 +409,8 @@ static char *cpu_topology_to_string(const CpuTopology *topo)
         "    .threads  = %u,\n"
         "    .max_cpus = %u,\n"
         "}",
-        topo->cpus, topo->sockets, topo->dies, topo->clusters,
-        topo->cores, topo->threads, topo->max_cpus);
+        topo->cpus, topo->smp.sockets, topo->smp.dies, topo->smp.clusters,
+        topo->smp.cores, topo->smp.threads, topo->max_cpus);
 }
 
 static void check_parse(MachineState *ms, const SMPConfiguration *config,
@@ -422,18 +425,19 @@ static void check_parse(MachineState *ms, const SMPConfiguration *config,
     /* call the generic parser */
     machine_parse_smp_config(ms, config, &err);
 
-    output_topo_str = cpu_topology_to_string(&ms->smp);
+    output_topo_str = cpu_topology_to_string(&ms->topo);
 
     /* when the configuration is supposed to be valid */
     if (is_valid) {
         if ((err == NULL) &&
-            (ms->smp.cpus == expect_topo->cpus) &&
-            (ms->smp.sockets == expect_topo->sockets) &&
-            (ms->smp.dies == expect_topo->dies) &&
-            (ms->smp.clusters == expect_topo->clusters) &&
-            (ms->smp.cores == expect_topo->cores) &&
-            (ms->smp.threads == expect_topo->threads) &&
-            (ms->smp.max_cpus == expect_topo->max_cpus)) {
+            (machine_topo_is_smp(ms) == true) &&
+            (machine_topo_get_cpus(ms) == expect_topo->cpus) &&
+            (machine_topo_get_sockets(ms) == expect_topo->smp.sockets) &&
+            (machine_topo_get_dies(ms) == expect_topo->smp.dies) &&
+            (machine_topo_get_clusters(ms) == expect_topo->smp.clusters) &&
+            (machine_topo_get_smp_cores(ms) == expect_topo->smp.cores) &&
+            (machine_topo_get_smp_threads(ms) == expect_topo->smp.threads) &&
+            (machine_topo_get_max_cpus(ms) == expect_topo->max_cpus)) {
             return;
         }
 
@@ -509,13 +513,13 @@ static void smp_parse_test(MachineState *ms, SMPTestData *data, bool is_valid)
 static void unsupported_params_init(const MachineClass *mc, SMPTestData *data)
 {
     if (!mc->smp_props.dies_supported) {
-        data->expect_prefer_sockets.dies = 1;
-        data->expect_prefer_cores.dies = 1;
+        data->expect_prefer_sockets.smp.dies = 1;
+        data->expect_prefer_cores.smp.dies = 1;
     }
 
     if (!mc->smp_props.clusters_supported) {
-        data->expect_prefer_sockets.clusters = 1;
-        data->expect_prefer_cores.clusters = 1;
+        data->expect_prefer_sockets.smp.clusters = 1;
+        data->expect_prefer_cores.smp.clusters = 1;
     }
 }
 
@@ -529,6 +533,13 @@ static void machine_base_class_init(ObjectClass *oc, void *data)
     mc->name = g_strdup(SMP_MACHINE_NAME);
 }
 
+static void machine_smp_topo_initfn(Object *obj)
+{
+    MachineState *ms = MACHINE(obj);
+
+    ms->topo.topo_type = CPU_TOPO_TYPE_SMP;
+}
+
 static void machine_generic_invalid_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -610,8 +621,8 @@ static void test_with_dies(const void *opaque)
         unsupported_params_init(mc, &data);
 
         /* when dies parameter is omitted, it will be set as 1 */
-        data.expect_prefer_sockets.dies = 1;
-        data.expect_prefer_cores.dies = 1;
+        data.expect_prefer_sockets.smp.dies = 1;
+        data.expect_prefer_cores.smp.dies = 1;
 
         smp_parse_test(ms, &data, true);
 
@@ -625,10 +636,10 @@ static void test_with_dies(const void *opaque)
             data.config.maxcpus *= num_dies;
         }
 
-        data.expect_prefer_sockets.dies = num_dies;
+        data.expect_prefer_sockets.smp.dies = num_dies;
         data.expect_prefer_sockets.cpus *= num_dies;
         data.expect_prefer_sockets.max_cpus *= num_dies;
-        data.expect_prefer_cores.dies = num_dies;
+        data.expect_prefer_cores.smp.dies = num_dies;
         data.expect_prefer_cores.cpus *= num_dies;
         data.expect_prefer_cores.max_cpus *= num_dies;
 
@@ -660,8 +671,8 @@ static void test_with_clusters(const void *opaque)
         unsupported_params_init(mc, &data);
 
         /* when clusters parameter is omitted, it will be set as 1 */
-        data.expect_prefer_sockets.clusters = 1;
-        data.expect_prefer_cores.clusters = 1;
+        data.expect_prefer_sockets.smp.clusters = 1;
+        data.expect_prefer_cores.smp.clusters = 1;
 
         smp_parse_test(ms, &data, true);
 
@@ -675,10 +686,10 @@ static void test_with_clusters(const void *opaque)
             data.config.maxcpus *= num_clusters;
         }
 
-        data.expect_prefer_sockets.clusters = num_clusters;
+        data.expect_prefer_sockets.smp.clusters = num_clusters;
         data.expect_prefer_sockets.cpus *= num_clusters;
         data.expect_prefer_sockets.max_cpus *= num_clusters;
-        data.expect_prefer_cores.clusters = num_clusters;
+        data.expect_prefer_cores.smp.clusters = num_clusters;
         data.expect_prefer_cores.cpus *= num_clusters;
         data.expect_prefer_cores.max_cpus *= num_clusters;
 
@@ -704,6 +715,7 @@ static const TypeInfo smp_machine_types[] = {
         .class_init     = machine_base_class_init,
         .class_size     = sizeof(MachineClass),
         .instance_size  = sizeof(MachineState),
+        .instance_init  = machine_smp_topo_initfn,
     }, {
         .name           = MACHINE_TYPE_NAME("smp-generic-valid"),
         .parent         = TYPE_MACHINE,
-- 
2.34.1

[RFC 28/52] hw/core/machine: Remove support of MachineState.smp

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Now the all uses of MachineState.smp are replaced by MachineState.topo.
It's time to completely remove smp field.

And clean up the struct CpuTopology in include/hw/boards.h.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c        |  8 --------
 hw/core/machine.c             |  9 +--------
 include/hw/boards.h           | 23 +----------------------
 include/hw/cpu/cpu-topology.h | 11 +++--------
 4 files changed, 5 insertions(+), 46 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index cdbe65ffd9fd..12c05510c1b5 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -307,14 +307,6 @@ void machine_parse_smp_config(MachineState *ms,
     maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * clusters * cores * threads;
     cpus = cpus > 0 ? cpus : maxcpus;
 
-    ms->smp.cpus = cpus;
-    ms->smp.sockets = sockets;
-    ms->smp.dies = dies;
-    ms->smp.clusters = clusters;
-    ms->smp.cores = cores;
-    ms->smp.threads = threads;
-    ms->smp.max_cpus = maxcpus;
-
     ms->topo.cpus = cpus;
     ms->topo.max_cpus = maxcpus;
     ms->topo.smp.sockets = sockets;
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 56e796c18873..fad990f49b03 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1093,16 +1093,9 @@ static void machine_initfn(Object *obj)
     }
 
     /* default to mc->default_cpus */
-    ms->smp.cpus = mc->default_cpus;
-    ms->smp.max_cpus = mc->default_cpus;
-    ms->smp.sockets = 1;
-    ms->smp.dies = 1;
-    ms->smp.clusters = 1;
-    ms->smp.cores = 1;
-    ms->smp.threads = 1;
-
     ms->topo.cpus = mc->default_cpus;
     ms->topo.max_cpus = mc->default_cpus;
+
     ms->topo.topo_type = CPU_TOPO_TYPE_SMP;
     ms->topo.smp.sockets = 1;
     ms->topo.smp.dies = 1;
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 78e52af38cb1..9364c90d5f1a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -299,26 +299,6 @@ typedef struct DeviceMemoryState {
     MemoryRegion mr;
 } DeviceMemoryState;
 
-/**
- * CpuTopology:
- * @cpus: the number of present logical processors on the machine
- * @sockets: the number of sockets on the machine
- * @dies: the number of dies in one socket
- * @clusters: the number of clusters in one die
- * @cores: the number of cores in one cluster
- * @threads: the number of threads in one core
- * @max_cpus: the maximum number of logical processors on the machine
- */
-typedef struct CpuTopology {
-    unsigned int cpus;
-    unsigned int sockets;
-    unsigned int dies;
-    unsigned int clusters;
-    unsigned int cores;
-    unsigned int threads;
-    unsigned int max_cpus;
-} CpuTopology;
-
 /**
  * MachineState:
  */
@@ -360,8 +340,7 @@ struct MachineState {
     const char *cpu_type;
     AccelState *accelerator;
     CPUArchIdList *possible_cpus;
-    CpuTopology smp;
-    GeneralCpuTopology topo; /* TODO: Completely replace MachineState.smp */
+    CpuTopology topo;
     struct NVDIMMState *nvdimms_state;
     struct NumaState *numa_state;
 };
diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
index 87d832556229..829f98d3a73f 100644
--- a/include/hw/cpu/cpu-topology.h
+++ b/include/hw/cpu/cpu-topology.h
@@ -88,7 +88,7 @@ typedef struct HybridCpuTopology {
 } HybridCpuTopology;
 
 /**
- * GeneralCpuTopology - General cpu topology defination.
+ * CpuTopology - General cpu topology defination.
  *
  * It supports one of two topologies: smp topology or hybrid topology.
  *
@@ -101,7 +101,7 @@ typedef struct HybridCpuTopology {
  * @hybrid: the hybrid cpu topology informantion. Only valid when
  *          topo_type is CPU_TOPO_TYPE_HYBRID.
  */
-typedef struct GeneralCpuTopology {
+typedef struct CpuTopology {
     unsigned int cpus;
     unsigned int max_cpus;
     CpuTopoType topo_type;
@@ -109,11 +109,6 @@ typedef struct GeneralCpuTopology {
         SmpCpuTopology smp;
         HybridCpuTopology hybrid;
     };
-} GeneralCpuTopology; /*
-                       * TODO: This name is temporary, just to distinguish it
-                       * from the CpuTopology in boards.h. When CpuTopology in
-                       * boards.h is merged here, it will be uniformly named as
-                       * CpuTopology.
-                       */
+} CpuTopology;
 
 #endif /* CPU_TOPOLOGY_H */
-- 
2.34.1

[RFC 29/52] hw/core/cpu: Introduce TopologyState in CPUState

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

MachineState has supported very complete topology levels. Although user
emulator doesn't need more levels than core/thread, we'd better to store
and maintain the complete topology information in CPUState for current
CPU to avoid repeated and redundant topology variables scattered
everywhere.

This is beneficial to drop other duplicate topology variables to prevent
confusion.

Meanwhile, rename topology variables (nr_cores/nr_threads) to better
reflect its own meaning.

Based on this, the CPUX86State.nr_dies/nr_modules will be cleaned up, and
X86CPUTopoInfo will be used exclusively to generate APIC ID.

From then on, MachineState.topo maintains the topology information of the
entire machine, while CPUState.topo maintains the topology information
required by the current CPU.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/cpu-common.c                 | 14 ++++++++++++--
 hw/i386/x86.c                        | 10 +++++++---
 hw/mips/malta.c                      | 12 +++++++-----
 include/hw/core/cpu.h                | 26 ++++++++++++++++++++++----
 softmmu/cpus.c                       | 18 +++++++++++++++---
 target/i386/cpu.c                    | 18 +++++++++---------
 target/i386/hvf/x86_emu.c            |  6 ++++--
 target/i386/kvm/kvm.c                |  6 ++++--
 target/i386/tcg/sysemu/misc_helper.c |  2 +-
 target/mips/tcg/sysemu/cp0_helper.c  |  4 ++--
 target/ppc/compat.c                  |  2 +-
 11 files changed, 84 insertions(+), 34 deletions(-)

diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 5ccc3837b678..95a42a9dc65c 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -223,6 +223,17 @@ static void cpu_common_unrealizefn(DeviceState *dev)
     cpu_exec_unrealizefn(cpu);
 }
 
+static void cpu_topo_init(CPUState *cpu)
+{
+    cpu->topo.sockets = 1;
+    cpu->topo.cores_per_socket = 1;
+    cpu->topo.threads_per_socket = 1;
+    cpu->topo.dies_per_socket = 1;
+    cpu->topo.clusters_per_die = 1;
+    cpu->topo.cores_per_cluster = 1;
+    cpu->topo.threads_per_core = 1;
+}
+
 static void cpu_common_initfn(Object *obj)
 {
     CPUState *cpu = CPU(obj);
@@ -233,8 +244,7 @@ static void cpu_common_initfn(Object *obj)
     cpu->gdb_num_regs = cpu->gdb_num_g_regs = cc->gdb_num_core_regs;
     /* *-user doesn't have configurable SMP topology */
     /* the default value is changed by qemu_init_vcpu() for softmmu */
-    cpu->nr_cores = 1;
-    cpu->nr_threads = 1;
+    cpu_topo_init(cpu);
     cpu->cflags_next_tb = -1;
 
     qemu_mutex_init(&cpu->work_mutex);
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 0aa4594455e2..91da2486d99e 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -409,12 +409,16 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         return;
     }
 
-    /* if 'address' properties socket-id/core-id/thread-id are not set, set them
+    /*
+     * if 'address' properties socket-id/core-id/thread-id are not set, set them
      * so that machine_query_hotpluggable_cpus would show correct values
      */
-    /* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
+    /*
+     * TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
      * once -smp refactoring is complete and there will be CPU private
-     * CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
+     * CPUState::topo::cores_per_socket and CPUState::topo::threads_per_core
+     * fields instead of globals
+     */
     x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
     if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
         error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
diff --git a/hw/mips/malta.c b/hw/mips/malta.c
index b3322f74baf5..a894401ac9c5 100644
--- a/hw/mips/malta.c
+++ b/hw/mips/malta.c
@@ -986,11 +986,13 @@ static void malta_mips_config(MIPSCPU *cpu)
     CPUState *cs = CPU(cpu);
 
     if (ase_mt_available(env)) {
-        env->mvp->CP0_MVPConf0 = deposit32(env->mvp->CP0_MVPConf0,
-                                           CP0MVPC0_PTC, 8,
-                                           smp_cpus * cs->nr_threads - 1);
-        env->mvp->CP0_MVPConf0 = deposit32(env->mvp->CP0_MVPConf0,
-                                           CP0MVPC0_PVPE, 4, smp_cpus - 1);
+        env->mvp->CP0_MVPConf0 =
+            deposit32(env->mvp->CP0_MVPConf0,
+                      CP0MVPC0_PTC, 8,
+                      smp_cpus * cs->topo.threads_per_core - 1);
+        env->mvp->CP0_MVPConf0 =
+            deposit32(env->mvp->CP0_MVPConf0,
+                      CP0MVPC0_PVPE, 4, smp_cpus - 1);
     }
 }
 
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 5253e4e839bb..957438718e7e 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -264,6 +264,26 @@ struct qemu_work_item;
 #define CPU_UNSET_NUMA_NODE_ID -1
 #define CPU_TRACE_DSTATE_MAX_EVENTS 32
 
+/**
+ * TopologyState:
+ * @sockets: Number of sockets within this machine.
+ * @cores_per_socket: Number of cores within this CPU socket.
+ * @threads_per_socket: Number of threads within this CPU socket.
+ * @dies_per_socket: Number of dies within this CPU socket.
+ * @clusters_per_die: Number of clusters within this CPU die.
+ * @cores_per_cluster: Number of cores within this CPU cluster.
+ * @threads_per_core: Number of threads within this CPU core.
+ */
+typedef struct TopologyState {
+    int sockets;
+    int cores_per_socket;
+    int threads_per_socket;
+    int dies_per_socket;
+    int clusters_per_die;
+    int cores_per_cluster;
+    int threads_per_core;
+} TopologyState;
+
 /**
  * CPUState:
  * @cpu_index: CPU index (informative).
@@ -273,8 +293,7 @@ struct qemu_work_item;
  *   be the same as the cluster-id property of the CPU object's TYPE_CPU_CLUSTER
  *   QOM parent.
  * @tcg_cflags: Pre-computed cflags for this cpu.
- * @nr_cores: Number of cores within this CPU package.
- * @nr_threads: Number of threads within this CPU core.
+ * @topo: Topology information of this cpu.
  * @running: #true if CPU is currently running (lockless).
  * @has_waiter: #true if a CPU is currently waiting for the cpu_exec_end;
  * valid under cpu_list_lock.
@@ -327,8 +346,7 @@ struct CPUState {
     CPUClass *cc;
     /*< public >*/
 
-    int nr_cores;
-    int nr_threads;
+    TopologyState topo;
 
     struct QemuThread *thread;
 #ifdef _WIN32
diff --git a/softmmu/cpus.c b/softmmu/cpus.c
index 7892da9c82b5..da2e076d51b2 100644
--- a/softmmu/cpus.c
+++ b/softmmu/cpus.c
@@ -626,12 +626,24 @@ const AccelOpsClass *cpus_get_accel(void)
     return cpus_accel;
 }
 
-void qemu_init_vcpu(CPUState *cpu)
+static void qemu_init_vcpu_topo(CPUState *cpu)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
 
-    cpu->nr_cores = machine_topo_get_cores_per_socket(ms);
-    cpu->nr_threads = machine_topo_get_threads_by_idx(ms, cpu->cpu_index);
+    cpu->topo.sockets = machine_topo_get_sockets(ms);
+    cpu->topo.cores_per_socket = machine_topo_get_cores_per_socket(ms);
+    cpu->topo.threads_per_socket = machine_topo_get_threads_per_socket(ms);
+    cpu->topo.dies_per_socket = machine_topo_get_dies(ms);
+    cpu->topo.clusters_per_die = machine_topo_get_clusters(ms);
+    cpu->topo.cores_per_cluster =
+        machine_topo_get_cores_by_idx(ms, cpu->cpu_index);
+    cpu->topo.threads_per_core =
+        machine_topo_get_threads_by_idx(ms, cpu->cpu_index);
+}
+
+void qemu_init_vcpu(CPUState *cpu)
+{
+    qemu_init_vcpu_topo(cpu);
     cpu->stopped = true;
     cpu->random_seed = qemu_guest_random_seed_thread_part1();
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 1aeea0e0ac3f..fcea4ea1a7e2 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5259,13 +5259,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     X86CPUTopoInfo topo_info;
     uint32_t cpus_per_pkg;
 
-    topo_info.dies_per_pkg = env->nr_dies;
-    topo_info.modules_per_die = env->nr_modules;
-    topo_info.cores_per_module = cs->nr_cores / env->nr_dies / env->nr_modules;
-    topo_info.threads_per_core = cs->nr_threads;
+    topo_info.dies_per_pkg = cs->topo.dies_per_socket;
+    topo_info.modules_per_die = cs->topo.clusters_per_die;
+    topo_info.cores_per_module = cs->topo.cores_per_cluster;
+    topo_info.threads_per_core = cs->topo.threads_per_core;
 
-    cpus_per_pkg = topo_info.dies_per_pkg * topo_info.modules_per_die *
-                   topo_info.cores_per_module * topo_info.threads_per_core;
+    cpus_per_pkg = cs->topo.threads_per_socket;
 
     /* Calculate & apply limits for different index ranges */
     if (index >= 0xC0000000) {
@@ -6687,14 +6686,15 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
      * users a warning.
      *
      * NOTE: the following code has to follow qemu_init_vcpu(). Otherwise
-     * cs->nr_threads hasn't be populated yet and the checking is incorrect.
+     * cs->topo.threads_per_core hasn't be populated yet and the checking is
+     * incorrect.
      */
     if (IS_AMD_CPU(env) &&
         !(env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_TOPOEXT) &&
-        cs->nr_threads > 1 && !ht_warned) {
+        cs->topo.threads_per_core > 1 && !ht_warned) {
             warn_report("This family of AMD CPU doesn't support "
                         "hyperthreading(%d)",
-                        cs->nr_threads);
+                        cs->topo.threads_per_core);
             error_printf("Please configure -smp options properly"
                          " or try enabling topoext feature.\n");
             ht_warned = true;
diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c
index f5704f63e8da..f72db54a7265 100644
--- a/target/i386/hvf/x86_emu.c
+++ b/target/i386/hvf/x86_emu.c
@@ -746,8 +746,10 @@ void simulate_rdmsr(struct CPUState *cpu)
         val = env->mtrr_deftype;
         break;
     case MSR_CORE_THREAD_COUNT:
-        val = cs->nr_threads * cs->nr_cores; /* thread count, bits 15..0 */
-        val |= ((uint32_t)cs->nr_cores << 16); /* core count, bits 31..16 */
+        /* thread count, bits 15..0 */
+        val = cs->topo.threads_per_socket;
+        /* core count, bits 31..16 */
+        val |= ((uint32_t)cs->topo.cores_per_socket << 16);
         break;
     default:
         /* fprintf(stderr, "%s: unknown msr 0x%x\n", __func__, msr); */
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 587030199192..77a8c381a73a 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2416,8 +2416,10 @@ static bool kvm_rdmsr_core_thread_count(X86CPU *cpu, uint32_t msr,
 {
     CPUState *cs = CPU(cpu);
 
-    *val = cs->nr_threads * cs->nr_cores; /* thread count, bits 15..0 */
-    *val |= ((uint32_t)cs->nr_cores << 16); /* core count, bits 31..16 */
+    /* thread count, bits 15..0 */
+    *val = cs->topo.threads_per_socket;
+    /* core count, bits 31..16 */
+    *val |= ((uint32_t)cs->topo.cores_per_socket << 16);
 
     return true;
 }
diff --git a/target/i386/tcg/sysemu/misc_helper.c b/target/i386/tcg/sysemu/misc_helper.c
index e1528b7f80be..f359f5869ee1 100644
--- a/target/i386/tcg/sysemu/misc_helper.c
+++ b/target/i386/tcg/sysemu/misc_helper.c
@@ -452,7 +452,7 @@ void helper_rdmsr(CPUX86State *env)
         break;
     case MSR_CORE_THREAD_COUNT: {
         CPUState *cs = CPU(x86_cpu);
-        val = (cs->nr_threads * cs->nr_cores) | (cs->nr_cores << 16);
+        val = (cs->topo.threads_per_socket) | (cs->topo.cores_per_socket << 16);
         break;
     }
     default:
diff --git a/target/mips/tcg/sysemu/cp0_helper.c b/target/mips/tcg/sysemu/cp0_helper.c
index 5da112458928..ee1f4f019ac2 100644
--- a/target/mips/tcg/sysemu/cp0_helper.c
+++ b/target/mips/tcg/sysemu/cp0_helper.c
@@ -124,8 +124,8 @@ static CPUMIPSState *mips_cpu_map_tc(CPUMIPSState *env, int *tc)
     }
 
     cs = env_cpu(env);
-    vpe_idx = tc_idx / cs->nr_threads;
-    *tc = tc_idx % cs->nr_threads;
+    vpe_idx = tc_idx / cs->topo.threads_per_core;
+    *tc = tc_idx % cs->topo.threads_per_core;
     other_cs = qemu_get_cpu(vpe_idx);
     if (other_cs == NULL) {
         return env;
diff --git a/target/ppc/compat.c b/target/ppc/compat.c
index 7949a24f5a15..091654884734 100644
--- a/target/ppc/compat.c
+++ b/target/ppc/compat.c
@@ -232,7 +232,7 @@ int ppc_set_compat_all(uint32_t compat_pvr, Error **errp)
 int ppc_compat_max_vthreads(PowerPCCPU *cpu)
 {
     const CompatInfo *compat = compat_by_pvr(cpu->compat_pvr);
-    int n_threads = CPU(cpu)->nr_threads;
+    int n_threads = CPU(cpu)->topo.threads_per_core;
 
     if (cpu->compat_pvr) {
         g_assert(compat);
-- 
2.34.1

[RFC 30/52] i386: Drop nr_dies and nr_modules CPUX86State

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since CPUState has the complete topology information, there's no need
to keep i386 architecture specific topology.

This avoids the fragmentation of topological information.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c         | 4 ----
 target/i386/cpu.c     | 7 +++----
 target/i386/cpu.h     | 8 --------
 target/i386/kvm/kvm.c | 2 +-
 4 files changed, 4 insertions(+), 17 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 91da2486d99e..52f7a19ceb7c 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -280,7 +280,6 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     CPUArchId *cpu_slot;
     X86CPUTopoIDs topo_ids;
     X86CPU *cpu = X86_CPU(dev);
-    CPUX86State *env = &cpu->env;
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
     X86CPUTopoInfo topo_info;
@@ -304,9 +303,6 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
 
     init_topo_info(&topo_info, x86ms);
 
-    env->nr_dies = machine_topo_get_dies(ms);
-    env->nr_modules = machine_topo_get_clusters(ms);
-
     /*
      * If APIC ID is not set,
      * set it based on socket/die/cluster/core/thread properties.
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index fcea4ea1a7e2..2188097c3ee1 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5499,7 +5499,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0x1F:
         /* V2 Extended Topology Enumeration Leaf */
-        if (env->nr_dies < 2) {
+        if (cs->topo.dies_per_socket < 2) {
             *eax = *ebx = *ecx = *edx = 0;
             break;
         }
@@ -6227,6 +6227,7 @@ static void x86_cpu_enable_xsave_components(X86CPU *cpu)
 void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    CPUState *cs = env_cpu(env);
     FeatureWord w;
     int i;
     GList *l;
@@ -6316,7 +6317,7 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
          * cpu->vendor_cpuid_only has been unset for compatibility with older
          * machine types.
          */
-        if ((env->nr_dies > 1) &&
+        if ((cs->topo.dies_per_socket > 1) &&
             (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
             x86_cpu_adjust_level(cpu, &env->cpuid_min_level, 0x1F);
         }
@@ -6840,8 +6841,6 @@ static void x86_cpu_initfn(Object *obj)
     X86CPUClass *xcc = X86_CPU_GET_CLASS(obj);
     CPUX86State *env = &cpu->env;
 
-    env->nr_dies = 1;
-    env->nr_modules = 1;
     cpu_set_cpustate_pointers(cpu);
 
     object_property_add(obj, "feature-words", "X86CPUFeatureWordInfo",
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index aa7e96c586c7..30b2aa6ab10d 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1825,14 +1825,6 @@ typedef struct CPUArchState {
     uint32_t umwait;
 
     TPRAccess tpr_access_type;
-
-    /* Number of dies per package. */
-    unsigned nr_dies;
-    /*
-     * Number of modules per die. Module level in x86 cpu topology is
-     * corresponding to smp.clusters.
-     */
-    unsigned nr_modules;
 } CPUX86State;
 
 struct kvm_msrs;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 77a8c381a73a..687c56825693 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -1859,7 +1859,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
             break;
         }
         case 0x1f:
-            if (env->nr_dies < 2) {
+            if (cs->topo.dies_per_socket < 2) {
                 break;
             }
             /* fallthrough */
-- 
2.34.1

[RFC 31/52] i386/cpu: Use CPUState.topo to replace X86CPUTopoInfo to get topology info

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

X86CPUTopoInfo is associated with the APIC ID, and strictly speaking, it
represents the topology levels and topology information of the APIC ID.

While CPUState.topo (TopologyState) represents the specific topology
information of the current CPU.

For smp topology, the topology information in X86CPUTopoInfo is same as
CPUState.topo, but it may be different for hybrid topology.

Therefore, X86CPUTopoInfo should be used for APIC ID related work, and
CPUState.topo should be preferred in general CPU topology related use
cases.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 target/i386/cpu.c | 33 +++++++++++++++------------------
 1 file changed, 15 insertions(+), 18 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2188097c3ee1..f626d74639ed 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -387,6 +387,7 @@ static void encode_topo_cpuid8000001e(X86CPU *cpu, X86CPUTopoInfo *topo_info,
                                       uint32_t *eax, uint32_t *ebx,
                                       uint32_t *ecx, uint32_t *edx)
 {
+    CPUState *cs = CPU(cpu);
     X86CPUTopoIDs topo_ids;
 
     x86_topo_ids_from_apicid(cpu->apic_id, topo_info, &topo_ids);
@@ -407,7 +408,7 @@ static void encode_topo_cpuid8000001e(X86CPU *cpu, X86CPUTopoInfo *topo_info,
      *  NOTE: CoreId is already part of apic_id. Just use it. We can
      *  use all the 8 bits to represent the core_id here.
      */
-    *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.core_id & 0xFF);
+    *ebx = ((cs->topo.threads_per_core - 1) << 8) | (topo_ids.core_id & 0xFF);
 
     /*
      * CPUID_Fn8000001E_ECX [Node Identifiers] (NodeId)
@@ -431,7 +432,7 @@ static void encode_topo_cpuid8000001e(X86CPU *cpu, X86CPUTopoInfo *topo_info,
      * NodeId is combination of node and socket_id which is already decoded
      * in apic_id. Just use it by shifting.
      */
-    *ecx = ((topo_info->dies_per_pkg - 1) << 8) |
+    *ecx = ((cs->topo.dies_per_socket - 1) << 8) |
            ((cpu->apic_id >> apicid_die_offset(topo_info)) & 0xFF);
 
     *edx = 0;
@@ -5257,15 +5258,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     uint32_t limit;
     uint32_t signature[3];
     X86CPUTopoInfo topo_info;
-    uint32_t cpus_per_pkg;
 
     topo_info.dies_per_pkg = cs->topo.dies_per_socket;
     topo_info.modules_per_die = cs->topo.clusters_per_die;
     topo_info.cores_per_module = cs->topo.cores_per_cluster;
     topo_info.threads_per_core = cs->topo.threads_per_core;
 
-    cpus_per_pkg = cs->topo.threads_per_socket;
-
     /* Calculate & apply limits for different index ranges */
     if (index >= 0xC0000000) {
         limit = env->cpuid_xlevel2;
@@ -5301,8 +5299,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             *ecx |= CPUID_EXT_OSXSAVE;
         }
         *edx = env->features[FEAT_1_EDX];
-        if (cpus_per_pkg > 1) {
-            *ebx |= cpus_per_pkg << 16;
+        if (cs->topo.threads_per_socket > 1) {
+            *ebx |= cs->topo.threads_per_socket << 16;
             *edx |= CPUID_HT;
         }
         if (!cpu->enable_pmu) {
@@ -5339,9 +5337,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
              */
             if (*eax & 31) {
                 int host_vcpus_per_cache = 1 + ((*eax & 0x3FFC000) >> 14);
-                int vcpus_per_socket = cpus_per_pkg;
-                int cores_per_socket = cpus_per_pkg /
-                                       topo_info.threads_per_core;
+                int vcpus_per_socket = cs->topo.threads_per_socket;
+                int cores_per_socket = cs->topo.cores_per_socket;
                 if (cores_per_socket > 1) {
                     *eax &= ~0xFC000000;
                     *eax |= (pow2ceil(cores_per_socket) - 1) << 26;
@@ -5473,12 +5470,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         switch (count) {
         case 0:
             *eax = apicid_core_offset(&topo_info);
-            *ebx = topo_info.threads_per_core;
+            *ebx = cs->topo.threads_per_core;
             *ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
             break;
         case 1:
             *eax = apicid_pkg_offset(&topo_info);
-            *ebx = cpus_per_pkg;
+            *ebx = cs->topo.threads_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
             break;
         default:
@@ -5509,17 +5506,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         switch (count) {
         case 0:
             *eax = apicid_core_offset(&topo_info);
-            *ebx = topo_info.threads_per_core;
+            *ebx = cs->topo.threads_per_core;
             *ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
             break;
         case 1:
             *eax = apicid_die_offset(&topo_info);
-            *ebx = cpus_per_pkg / topo_info.dies_per_pkg;
+            *ebx = cs->topo.threads_per_socket / cs->topo.dies_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
             break;
         case 2:
             *eax = apicid_pkg_offset(&topo_info);
-            *ebx = cpus_per_pkg;
+            *ebx = cs->topo.threads_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
             break;
         default:
@@ -5744,7 +5741,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
          * discards multiple thread information if it is set.
          * So don't set it here for Intel to make Linux guests happy.
          */
-        if (cpus_per_pkg > 1) {
+        if (cs->topo.threads_per_socket > 1) {
             if (env->cpuid_vendor1 != CPUID_VENDOR_INTEL_1 ||
                 env->cpuid_vendor2 != CPUID_VENDOR_INTEL_2 ||
                 env->cpuid_vendor3 != CPUID_VENDOR_INTEL_3) {
@@ -5806,7 +5803,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
              *eax |= (cpu_x86_virtual_addr_width(env) << 8);
         }
         *ebx = env->features[FEAT_8000_0008_EBX];
-        if (cpus_per_pkg > 1) {
+        if (cs->topo.threads_per_socket > 1) {
             /*
              * Bits 15:12 is "The number of bits in the initial
              * Core::X86::Apic::ApicId[ApicId] value that indicate
@@ -5814,7 +5811,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
              * Bits 7:0 is "The number of threads in the package is NC+1"
              */
             *ecx = (apicid_pkg_offset(&topo_info) << 12) |
-                   (cpus_per_pkg - 1);
+                   (cs->topo.threads_per_socket - 1);
         } else {
             *ecx = 0;
         }
-- 
2.34.1

[RFC 32/52] i386: Rename X86CPUTopoInfo and its members to reflect relationship with APIC ID

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Rename X86CPUTopoInfo to X86ApicidTopoInfo, and also rename its members
to max_{dies, modules, cores, threads} to avoid confusion with
CPUState.topo.

Now the names can better reflect their relationship with APIC ID.

For hybrid CPU topology, X86ApicidTopoInfo will be different with
CPUState.topo and keep the maximum possible number of topology
structures in each topology level other than the actual topology
information of a certain CPU. This is required for the APIC ID under the
hybrid CPU topology.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c                | 18 ++++-----
 include/hw/i386/topology.h   | 72 +++++++++++++++++++++++-------------
 include/hw/i386/x86.h        |  2 +-
 target/i386/cpu.c            | 20 +++++-----
 tests/unit/test-x86-apicid.c | 50 ++++++++++++-------------
 5 files changed, 91 insertions(+), 71 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 52f7a19ceb7c..f20b0c3a5f12 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -65,15 +65,15 @@
 /* Physical Address of PVH entry point read from kernel ELF NOTE */
 static size_t pvh_start_addr;
 
-inline void init_topo_info(X86CPUTopoInfo *topo_info,
+inline void init_topo_info(X86ApicidTopoInfo *topo_info,
                            const X86MachineState *x86ms)
 {
     MachineState *ms = MACHINE(x86ms);
 
-    topo_info->dies_per_pkg = machine_topo_get_dies(ms);
-    topo_info->modules_per_die = machine_topo_get_clusters(ms);
-    topo_info->cores_per_module = machine_topo_get_smp_cores(ms);
-    topo_info->threads_per_core = machine_topo_get_smp_threads(ms);
+    topo_info->max_dies = machine_topo_get_dies(ms);
+    topo_info->max_modules = machine_topo_get_clusters(ms);
+    topo_info->max_cores = machine_topo_get_smp_cores(ms);
+    topo_info->max_threads = machine_topo_get_smp_threads(ms);
 }
 
 /*
@@ -87,7 +87,7 @@ inline void init_topo_info(X86CPUTopoInfo *topo_info,
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
                                     unsigned int cpu_index)
 {
-    X86CPUTopoInfo topo_info;
+    X86ApicidTopoInfo topo_info;
 
     init_topo_info(&topo_info, x86ms);
 
@@ -282,7 +282,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     X86CPU *cpu = X86_CPU(dev);
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
-    X86CPUTopoInfo topo_info;
+    X86ApicidTopoInfo topo_info;
 
     if (!object_dynamic_cast(OBJECT(cpu), ms->cpu_type)) {
         error_setg(errp, "Invalid CPU type, expected cpu type: '%s'",
@@ -481,7 +481,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
 {
    X86CPUTopoIDs topo_ids;
    X86MachineState *x86ms = X86_MACHINE(ms);
-   X86CPUTopoInfo topo_info;
+   X86ApicidTopoInfo topo_info;
 
    init_topo_info(&topo_info, x86ms);
 
@@ -495,7 +495,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
 {
     X86MachineState *x86ms = X86_MACHINE(ms);
     unsigned int max_cpus = machine_topo_get_max_cpus(ms);
-    X86CPUTopoInfo topo_info;
+    X86ApicidTopoInfo topo_info;
     int i;
 
     if (ms->possible_cpus) {
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 3cec97b377f2..45a2ab2a3dfa 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -47,6 +47,15 @@
  */
 typedef uint32_t apic_id_t;
 
+/**
+ * X86CPUTopoIDs - IDs for different topology levels.
+ *
+ * @pkg_id: the ID of package level
+ * @die_id: the ID of die level
+ * @module_id: the ID of module level
+ * @core_id: the ID of core level
+ * @smt_id: the ID of SMT level
+ */
 typedef struct X86CPUTopoIDs {
     unsigned pkg_id;
     unsigned die_id;
@@ -55,12 +64,23 @@ typedef struct X86CPUTopoIDs {
     unsigned smt_id;
 } X86CPUTopoIDs;
 
-typedef struct X86CPUTopoInfo {
-    unsigned dies_per_pkg;
-    unsigned modules_per_die;
-    unsigned cores_per_module;
-    unsigned threads_per_core;
-} X86CPUTopoInfo;
+/**
+ * X86ApicidTopoInfo - Topology information of APIC ID.
+ *
+ * X86ApicidTopoInfo stores the maximum possible number of corresponding
+ * topology structures in each topology level.
+ *
+ * @max_dies: the maximum possible number of dies in one package
+ * @max_modules: the maximum possible number of modules in one die
+ * @max_cores: the maximum possible number of cores in one module
+ * @max_threads: the maximum possible number of threads in one core
+ */
+typedef struct X86ApicidTopoInfo {
+    unsigned max_dies;
+    unsigned max_modules;
+    unsigned max_cores;
+    unsigned max_threads;
+} X86ApicidTopoInfo;
 
 /* Return the bit width needed for 'count' IDs */
 static unsigned apicid_bitwidth_for_count(unsigned count)
@@ -71,49 +91,49 @@ static unsigned apicid_bitwidth_for_count(unsigned count)
 }
 
 /* Bit width of the SMT_ID (thread ID) field on the APIC ID */
-static inline unsigned apicid_smt_width(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_smt_width(X86ApicidTopoInfo *topo_info)
 {
-    return apicid_bitwidth_for_count(topo_info->threads_per_core);
+    return apicid_bitwidth_for_count(topo_info->max_threads);
 }
 
 /* Bit width of the Core_ID field */
-static inline unsigned apicid_core_width(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_core_width(X86ApicidTopoInfo *topo_info)
 {
-    return apicid_bitwidth_for_count(topo_info->cores_per_module);
+    return apicid_bitwidth_for_count(topo_info->max_cores);
 }
 
 /* Bit width of the Module_ID (cluster ID) field */
-static inline unsigned apicid_module_width(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_module_width(X86ApicidTopoInfo *topo_info)
 {
-    return apicid_bitwidth_for_count(topo_info->modules_per_die);
+    return apicid_bitwidth_for_count(topo_info->max_modules);
 }
 
 /* Bit width of the Die_ID field */
-static inline unsigned apicid_die_width(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_die_width(X86ApicidTopoInfo *topo_info)
 {
-    return apicid_bitwidth_for_count(topo_info->dies_per_pkg);
+    return apicid_bitwidth_for_count(topo_info->max_dies);
 }
 
 /* Bit offset of the Core_ID field */
-static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_core_offset(X86ApicidTopoInfo *topo_info)
 {
     return apicid_smt_width(topo_info);
 }
 
 /* Bit offset of the Module_ID (cluster ID) field */
-static inline unsigned apicid_module_offset(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_module_offset(X86ApicidTopoInfo *topo_info)
 {
     return apicid_core_offset(topo_info) + apicid_core_width(topo_info);
 }
 
 /* Bit offset of the Die_ID field */
-static inline unsigned apicid_die_offset(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_die_offset(X86ApicidTopoInfo *topo_info)
 {
     return apicid_module_offset(topo_info) + apicid_module_width(topo_info);
 }
 
 /* Bit offset of the Pkg_ID (socket ID) field */
-static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
+static inline unsigned apicid_pkg_offset(X86ApicidTopoInfo *topo_info)
 {
     return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
 }
@@ -123,7 +143,7 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
  *
  * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
  */
-static inline apic_id_t x86_apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
+static inline apic_id_t x86_apicid_from_topo_ids(X86ApicidTopoInfo *topo_info,
                                                  const X86CPUTopoIDs *topo_ids)
 {
     return (topo_ids->pkg_id  << apicid_pkg_offset(topo_info)) |
@@ -137,14 +157,14 @@ static inline apic_id_t x86_apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
  * Calculate thread/core/package IDs for a specific topology,
  * based on (contiguous) CPU index
  */
-static inline void x86_topo_ids_from_idx(X86CPUTopoInfo *topo_info,
+static inline void x86_topo_ids_from_idx(X86ApicidTopoInfo *topo_info,
                                          unsigned cpu_index,
                                          X86CPUTopoIDs *topo_ids)
 {
-    unsigned nr_dies = topo_info->dies_per_pkg;
-    unsigned nr_modules = topo_info->modules_per_die;
-    unsigned nr_cores = topo_info->cores_per_module;
-    unsigned nr_threads = topo_info->threads_per_core;
+    unsigned nr_dies = topo_info->max_dies;
+    unsigned nr_modules = topo_info->max_modules;
+    unsigned nr_cores = topo_info->max_cores;
+    unsigned nr_threads = topo_info->max_threads;
 
     topo_ids->pkg_id = cpu_index / (nr_dies * nr_modules *
                        nr_cores * nr_threads);
@@ -161,7 +181,7 @@ static inline void x86_topo_ids_from_idx(X86CPUTopoInfo *topo_info,
  * based on APIC ID
  */
 static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
-                                            X86CPUTopoInfo *topo_info,
+                                            X86ApicidTopoInfo *topo_info,
                                             X86CPUTopoIDs *topo_ids)
 {
     topo_ids->smt_id = apicid &
@@ -183,7 +203,7 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
  *
  * 'cpu_index' is a sequential, contiguous ID for the CPU.
  */
-static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info,
+static inline apic_id_t x86_apicid_from_cpu_idx(X86ApicidTopoInfo *topo_info,
                                                 unsigned cpu_index)
 {
     X86CPUTopoIDs topo_ids;
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 62fa5774f849..ac6f1e4a74af 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -98,7 +98,7 @@ struct X86MachineState {
 #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
 OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
 
-void init_topo_info(X86CPUTopoInfo *topo_info, const X86MachineState *x86ms);
+void init_topo_info(X86ApicidTopoInfo *topo_info, const X86MachineState *x86ms);
 
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *pcms,
                                     unsigned int cpu_index);
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index f626d74639ed..4494f01a1635 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -232,7 +232,7 @@ static uint8_t cpuid2_cache_descriptor(CPUCacheInfo *cache)
                        0 /* Invalid value */)
 
 static uint32_t max_processor_ids_for_cache(CPUCacheInfo *cache,
-                                            X86CPUTopoInfo *topo_info)
+                                            X86ApicidTopoInfo *topo_info)
 {
     uint32_t num_ids = 0;
 
@@ -257,7 +257,7 @@ static uint32_t max_processor_ids_for_cache(CPUCacheInfo *cache,
     return num_ids - 1;
 }
 
-static uint32_t max_core_ids_in_package(X86CPUTopoInfo *topo_info)
+static uint32_t max_core_ids_in_package(X86ApicidTopoInfo *topo_info)
 {
     uint32_t num_cores = 1 << (apicid_pkg_offset(topo_info) -
                                apicid_core_offset(topo_info));
@@ -266,7 +266,7 @@ static uint32_t max_core_ids_in_package(X86CPUTopoInfo *topo_info)
 
 /* Encode cache info for CPUID[4] */
 static void encode_cache_cpuid4(CPUCacheInfo *cache,
-                                X86CPUTopoInfo *topo_info,
+                                X86ApicidTopoInfo *topo_info,
                                 uint32_t *eax, uint32_t *ebx,
                                 uint32_t *ecx, uint32_t *edx)
 {
@@ -354,7 +354,7 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
 
 /* Encode cache info for CPUID[8000001D] */
 static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
-                                       X86CPUTopoInfo *topo_info,
+                                       X86ApicidTopoInfo *topo_info,
                                        uint32_t *eax, uint32_t *ebx,
                                        uint32_t *ecx, uint32_t *edx)
 {
@@ -383,7 +383,7 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
 }
 
 /* Encode cache info for CPUID[8000001E] */
-static void encode_topo_cpuid8000001e(X86CPU *cpu, X86CPUTopoInfo *topo_info,
+static void encode_topo_cpuid8000001e(X86CPU *cpu, X86ApicidTopoInfo *topo_info,
                                       uint32_t *eax, uint32_t *ebx,
                                       uint32_t *ecx, uint32_t *edx)
 {
@@ -5257,12 +5257,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     CPUState *cs = env_cpu(env);
     uint32_t limit;
     uint32_t signature[3];
-    X86CPUTopoInfo topo_info;
+    X86ApicidTopoInfo topo_info;
 
-    topo_info.dies_per_pkg = cs->topo.dies_per_socket;
-    topo_info.modules_per_die = cs->topo.clusters_per_die;
-    topo_info.cores_per_module = cs->topo.cores_per_cluster;
-    topo_info.threads_per_core = cs->topo.threads_per_core;
+    topo_info.max_dies = cs->topo.dies_per_socket;
+    topo_info.max_modules = cs->topo.clusters_per_die;
+    topo_info.max_cores = cs->topo.cores_per_cluster;
+    topo_info.max_threads = cs->topo.threads_per_core;
 
     /* Calculate & apply limits for different index ranges */
     if (index >= 0xC0000000) {
diff --git a/tests/unit/test-x86-apicid.c b/tests/unit/test-x86-apicid.c
index 55b731ccae55..225d88bc3262 100644
--- a/tests/unit/test-x86-apicid.c
+++ b/tests/unit/test-x86-apicid.c
@@ -28,19 +28,19 @@
 
 static void test_topo_bits(void)
 {
-    X86CPUTopoInfo topo_info = {0};
+    X86ApicidTopoInfo topo_info = {0};
 
     /*
      * simple tests for 1 thread per core, 1 core per module,
      *                  1 module per die, 1 die per package
      */
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 1};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 1};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 0);
     g_assert_cmpuint(apicid_core_width(&topo_info), ==, 0);
     g_assert_cmpuint(apicid_module_width(&topo_info), ==, 0);
     g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
 
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 1};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 1};
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
@@ -49,48 +49,48 @@ static void test_topo_bits(void)
 
     /* Test field width calculation for multiple values
      */
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 2};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 1);
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 3};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 3};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 4};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 4};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
 
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 14};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 14};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 15};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 15};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 16};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 16};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86CPUTopoInfo) {1, 1, 1, 17};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 17};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 5);
 
 
-    topo_info = (X86CPUTopoInfo) {1, 1, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 30, 2};
     g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86CPUTopoInfo) {1, 1, 31, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 31, 2};
     g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86CPUTopoInfo) {1, 1, 32, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 32, 2};
     g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86CPUTopoInfo) {1, 1, 33, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 33, 2};
     g_assert_cmpuint(apicid_core_width(&topo_info), ==, 6);
 
-    topo_info = (X86CPUTopoInfo) {1, 6, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 6, 30, 2};
     g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86CPUTopoInfo) {1, 7, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 7, 30, 2};
     g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86CPUTopoInfo) {1, 8, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 8, 30, 2};
     g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86CPUTopoInfo) {1, 9, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 9, 30, 2};
     g_assert_cmpuint(apicid_module_width(&topo_info), ==, 4);
 
-    topo_info = (X86CPUTopoInfo) {1, 6, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {1, 6, 30, 2};
     g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
-    topo_info = (X86CPUTopoInfo) {2, 6, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {2, 6, 30, 2};
     g_assert_cmpuint(apicid_die_width(&topo_info), ==, 1);
-    topo_info = (X86CPUTopoInfo) {3, 6, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {3, 6, 30, 2};
     g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
-    topo_info = (X86CPUTopoInfo) {4, 6, 30, 2};
+    topo_info = (X86ApicidTopoInfo) {4, 6, 30, 2};
     g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
 
     /* build a weird topology and see if IDs are calculated correctly
@@ -98,19 +98,19 @@ static void test_topo_bits(void)
 
     /* This will use 2 bits for thread ID and 3 bits for core ID
      */
-    topo_info = (X86CPUTopoInfo) {1, 1, 6, 3};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
     g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
     g_assert_cmpuint(apicid_core_offset(&topo_info), ==, 2);
     g_assert_cmpuint(apicid_module_offset(&topo_info), ==, 5);
     g_assert_cmpuint(apicid_die_offset(&topo_info), ==, 5);
     g_assert_cmpuint(apicid_pkg_offset(&topo_info), ==, 5);
 
-    topo_info = (X86CPUTopoInfo) {1, 1, 6, 3};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
 
-    topo_info = (X86CPUTopoInfo) {1, 1, 6, 3};
+    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 0), ==,
                      (1 << 2) | 0);
     g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 1), ==,
-- 
2.34.1

[RFC 33/52] i386: Rename init_topo_info() to init_apic_topo_info()

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Rename init_topo_info() to init_apic_topo_info() to adapt
X86ApicidTopoInfo.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c         | 12 ++++++------
 include/hw/i386/x86.h |  3 ++-
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index f20b0c3a5f12..a13c931df062 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -65,8 +65,8 @@
 /* Physical Address of PVH entry point read from kernel ELF NOTE */
 static size_t pvh_start_addr;
 
-inline void init_topo_info(X86ApicidTopoInfo *topo_info,
-                           const X86MachineState *x86ms)
+inline void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
+                                  const X86MachineState *x86ms)
 {
     MachineState *ms = MACHINE(x86ms);
 
@@ -89,7 +89,7 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
 {
     X86ApicidTopoInfo topo_info;
 
-    init_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&topo_info, x86ms);
 
     return x86_apicid_from_cpu_idx(&topo_info, cpu_index);
 }
@@ -301,7 +301,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         }
     }
 
-    init_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&topo_info, x86ms);
 
     /*
      * If APIC ID is not set,
@@ -483,7 +483,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
    X86MachineState *x86ms = X86_MACHINE(ms);
    X86ApicidTopoInfo topo_info;
 
-   init_topo_info(&topo_info, x86ms);
+   init_apicid_topo_info(&topo_info, x86ms);
 
    assert(idx < ms->possible_cpus->len);
    x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
@@ -511,7 +511,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
                                   sizeof(CPUArchId) * max_cpus);
     ms->possible_cpus->len = max_cpus;
 
-    init_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&topo_info, x86ms);
 
     for (i = 0; i < ms->possible_cpus->len; i++) {
         X86CPUTopoIDs topo_ids;
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index ac6f1e4a74af..d84f7717900c 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -98,7 +98,8 @@ struct X86MachineState {
 #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
 OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
 
-void init_topo_info(X86ApicidTopoInfo *topo_info, const X86MachineState *x86ms);
+void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
+                           const X86MachineState *x86ms);
 
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *pcms,
                                     unsigned int cpu_index);
-- 
2.34.1

[RFC 34/52] i386: Rename variable topo_info to apicid_topo

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since X86ApicidTopoInfo is only used for APIC ID related work, the
common variable topo_info of X86ApicidTopoInfo type should be also
renamed to better suit its purpose.

Generic topology access should be obtained from MachineState.topo
(for the whole machine) or CPUState.topo (for the current CPU), and
apicid_topo (X86ApicidTopoInfo type) is only used to collaborate with
APIC ID.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c                |  38 ++++-----
 include/hw/i386/topology.h   |  76 ++++++++---------
 include/hw/i386/x86.h        |   2 +-
 target/i386/cpu.c            |  71 ++++++++--------
 tests/unit/test-x86-apicid.c | 158 +++++++++++++++++------------------
 5 files changed, 173 insertions(+), 172 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index a13c931df062..9ee0fcb9a460 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -65,15 +65,15 @@
 /* Physical Address of PVH entry point read from kernel ELF NOTE */
 static size_t pvh_start_addr;
 
-inline void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
+inline void init_apicid_topo_info(X86ApicidTopoInfo *apicid_topo,
                                   const X86MachineState *x86ms)
 {
     MachineState *ms = MACHINE(x86ms);
 
-    topo_info->max_dies = machine_topo_get_dies(ms);
-    topo_info->max_modules = machine_topo_get_clusters(ms);
-    topo_info->max_cores = machine_topo_get_smp_cores(ms);
-    topo_info->max_threads = machine_topo_get_smp_threads(ms);
+    apicid_topo->max_dies = machine_topo_get_dies(ms);
+    apicid_topo->max_modules = machine_topo_get_clusters(ms);
+    apicid_topo->max_cores = machine_topo_get_smp_cores(ms);
+    apicid_topo->max_threads = machine_topo_get_smp_threads(ms);
 }
 
 /*
@@ -87,11 +87,11 @@ inline void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
                                     unsigned int cpu_index)
 {
-    X86ApicidTopoInfo topo_info;
+    X86ApicidTopoInfo apicid_topo;
 
-    init_apicid_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&apicid_topo, x86ms);
 
-    return x86_apicid_from_cpu_idx(&topo_info, cpu_index);
+    return x86_apicid_from_cpu_idx(&apicid_topo, cpu_index);
 }
 
 
@@ -282,7 +282,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     X86CPU *cpu = X86_CPU(dev);
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
-    X86ApicidTopoInfo topo_info;
+    X86ApicidTopoInfo apicid_topo;
 
     if (!object_dynamic_cast(OBJECT(cpu), ms->cpu_type)) {
         error_setg(errp, "Invalid CPU type, expected cpu type: '%s'",
@@ -301,7 +301,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         }
     }
 
-    init_apicid_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&apicid_topo, x86ms);
 
     /*
      * If APIC ID is not set,
@@ -381,14 +381,14 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         topo_ids.module_id = cpu->cluster_id;
         topo_ids.core_id = cpu->core_id;
         topo_ids.smt_id = cpu->thread_id;
-        cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
+        cpu->apic_id = x86_apicid_from_topo_ids(&apicid_topo, &topo_ids);
     }
 
     cpu_slot = x86_find_cpu_slot(MACHINE(x86ms), cpu->apic_id, &idx);
     if (!cpu_slot) {
         MachineState *ms = MACHINE(x86ms);
 
-        x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+        x86_topo_ids_from_apicid(cpu->apic_id, &apicid_topo, &topo_ids);
 
         error_setg(errp,
             "Invalid CPU [socket: %u, die: %u, module: %u, core: %u, thread: %u]"
@@ -415,7 +415,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
      * CPUState::topo::cores_per_socket and CPUState::topo::threads_per_core
      * fields instead of globals
      */
-    x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+    x86_topo_ids_from_apicid(cpu->apic_id, &apicid_topo, &topo_ids);
     if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
         error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
             " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id,
@@ -481,13 +481,13 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
 {
    X86CPUTopoIDs topo_ids;
    X86MachineState *x86ms = X86_MACHINE(ms);
-   X86ApicidTopoInfo topo_info;
+   X86ApicidTopoInfo apicid_topo;
 
-   init_apicid_topo_info(&topo_info, x86ms);
+   init_apicid_topo_info(&apicid_topo, x86ms);
 
    assert(idx < ms->possible_cpus->len);
    x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
-                            &topo_info, &topo_ids);
+                            &apicid_topo, &topo_ids);
    return topo_ids.pkg_id % ms->numa_state->num_nodes;
 }
 
@@ -495,7 +495,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
 {
     X86MachineState *x86ms = X86_MACHINE(ms);
     unsigned int max_cpus = machine_topo_get_max_cpus(ms);
-    X86ApicidTopoInfo topo_info;
+    X86ApicidTopoInfo apicid_topo;
     int i;
 
     if (ms->possible_cpus) {
@@ -511,7 +511,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
                                   sizeof(CPUArchId) * max_cpus);
     ms->possible_cpus->len = max_cpus;
 
-    init_apicid_topo_info(&topo_info, x86ms);
+    init_apicid_topo_info(&apicid_topo, x86ms);
 
     for (i = 0; i < ms->possible_cpus->len; i++) {
         X86CPUTopoIDs topo_ids;
@@ -521,7 +521,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
         ms->possible_cpus->cpus[i].arch_id =
             x86_cpu_apic_id_from_index(x86ms, i);
         x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
-                                 &topo_info, &topo_ids);
+                                 &apicid_topo, &topo_ids);
         ms->possible_cpus->cpus[i].props.has_socket_id = true;
         ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
         if (machine_topo_get_dies(ms) > 1) {
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 45a2ab2a3dfa..5b29c51329f1 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -91,51 +91,51 @@ static unsigned apicid_bitwidth_for_count(unsigned count)
 }
 
 /* Bit width of the SMT_ID (thread ID) field on the APIC ID */
-static inline unsigned apicid_smt_width(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_smt_width(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_bitwidth_for_count(topo_info->max_threads);
+    return apicid_bitwidth_for_count(apicid_topo->max_threads);
 }
 
 /* Bit width of the Core_ID field */
-static inline unsigned apicid_core_width(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_core_width(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_bitwidth_for_count(topo_info->max_cores);
+    return apicid_bitwidth_for_count(apicid_topo->max_cores);
 }
 
 /* Bit width of the Module_ID (cluster ID) field */
-static inline unsigned apicid_module_width(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_module_width(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_bitwidth_for_count(topo_info->max_modules);
+    return apicid_bitwidth_for_count(apicid_topo->max_modules);
 }
 
 /* Bit width of the Die_ID field */
-static inline unsigned apicid_die_width(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_die_width(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_bitwidth_for_count(topo_info->max_dies);
+    return apicid_bitwidth_for_count(apicid_topo->max_dies);
 }
 
 /* Bit offset of the Core_ID field */
-static inline unsigned apicid_core_offset(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_core_offset(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_smt_width(topo_info);
+    return apicid_smt_width(apicid_topo);
 }
 
 /* Bit offset of the Module_ID (cluster ID) field */
-static inline unsigned apicid_module_offset(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_module_offset(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_core_offset(topo_info) + apicid_core_width(topo_info);
+    return apicid_core_offset(apicid_topo) + apicid_core_width(apicid_topo);
 }
 
 /* Bit offset of the Die_ID field */
-static inline unsigned apicid_die_offset(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_die_offset(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_module_offset(topo_info) + apicid_module_width(topo_info);
+    return apicid_module_offset(apicid_topo) + apicid_module_width(apicid_topo);
 }
 
 /* Bit offset of the Pkg_ID (socket ID) field */
-static inline unsigned apicid_pkg_offset(X86ApicidTopoInfo *topo_info)
+static inline unsigned apicid_pkg_offset(X86ApicidTopoInfo *apicid_topo)
 {
-    return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
+    return apicid_die_offset(apicid_topo) + apicid_die_width(apicid_topo);
 }
 
 /*
@@ -143,13 +143,13 @@ static inline unsigned apicid_pkg_offset(X86ApicidTopoInfo *topo_info)
  *
  * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
  */
-static inline apic_id_t x86_apicid_from_topo_ids(X86ApicidTopoInfo *topo_info,
+static inline apic_id_t x86_apicid_from_topo_ids(X86ApicidTopoInfo *apicid_topo,
                                                  const X86CPUTopoIDs *topo_ids)
 {
-    return (topo_ids->pkg_id  << apicid_pkg_offset(topo_info)) |
-           (topo_ids->die_id  << apicid_die_offset(topo_info)) |
-           (topo_ids->module_id << apicid_module_offset(topo_info)) |
-           (topo_ids->core_id << apicid_core_offset(topo_info)) |
+    return (topo_ids->pkg_id  << apicid_pkg_offset(apicid_topo)) |
+           (topo_ids->die_id  << apicid_die_offset(apicid_topo)) |
+           (topo_ids->module_id << apicid_module_offset(apicid_topo)) |
+           (topo_ids->core_id << apicid_core_offset(apicid_topo)) |
            topo_ids->smt_id;
 }
 
@@ -157,14 +157,14 @@ static inline apic_id_t x86_apicid_from_topo_ids(X86ApicidTopoInfo *topo_info,
  * Calculate thread/core/package IDs for a specific topology,
  * based on (contiguous) CPU index
  */
-static inline void x86_topo_ids_from_idx(X86ApicidTopoInfo *topo_info,
+static inline void x86_topo_ids_from_idx(X86ApicidTopoInfo *apicid_topo,
                                          unsigned cpu_index,
                                          X86CPUTopoIDs *topo_ids)
 {
-    unsigned nr_dies = topo_info->max_dies;
-    unsigned nr_modules = topo_info->max_modules;
-    unsigned nr_cores = topo_info->max_cores;
-    unsigned nr_threads = topo_info->max_threads;
+    unsigned nr_dies = apicid_topo->max_dies;
+    unsigned nr_modules = apicid_topo->max_modules;
+    unsigned nr_cores = apicid_topo->max_cores;
+    unsigned nr_threads = apicid_topo->max_threads;
 
     topo_ids->pkg_id = cpu_index / (nr_dies * nr_modules *
                        nr_cores * nr_threads);
@@ -181,21 +181,21 @@ static inline void x86_topo_ids_from_idx(X86ApicidTopoInfo *topo_info,
  * based on APIC ID
  */
 static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
-                                            X86ApicidTopoInfo *topo_info,
+                                            X86ApicidTopoInfo *apicid_topo,
                                             X86CPUTopoIDs *topo_ids)
 {
     topo_ids->smt_id = apicid &
-            ~(0xFFFFFFFFUL << apicid_smt_width(topo_info));
+            ~(0xFFFFFFFFUL << apicid_smt_width(apicid_topo));
     topo_ids->core_id =
-            (apicid >> apicid_core_offset(topo_info)) &
-            ~(0xFFFFFFFFUL << apicid_core_width(topo_info));
+            (apicid >> apicid_core_offset(apicid_topo)) &
+            ~(0xFFFFFFFFUL << apicid_core_width(apicid_topo));
     topo_ids->module_id =
-            (apicid >> apicid_module_offset(topo_info)) &
-            ~(0xFFFFFFFFUL << apicid_module_width(topo_info));
+            (apicid >> apicid_module_offset(apicid_topo)) &
+            ~(0xFFFFFFFFUL << apicid_module_width(apicid_topo));
     topo_ids->die_id =
-            (apicid >> apicid_die_offset(topo_info)) &
-            ~(0xFFFFFFFFUL << apicid_die_width(topo_info));
-    topo_ids->pkg_id = apicid >> apicid_pkg_offset(topo_info);
+            (apicid >> apicid_die_offset(apicid_topo)) &
+            ~(0xFFFFFFFFUL << apicid_die_width(apicid_topo));
+    topo_ids->pkg_id = apicid >> apicid_pkg_offset(apicid_topo);
 }
 
 /*
@@ -203,12 +203,12 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
  *
  * 'cpu_index' is a sequential, contiguous ID for the CPU.
  */
-static inline apic_id_t x86_apicid_from_cpu_idx(X86ApicidTopoInfo *topo_info,
+static inline apic_id_t x86_apicid_from_cpu_idx(X86ApicidTopoInfo *apicid_topo,
                                                 unsigned cpu_index)
 {
     X86CPUTopoIDs topo_ids;
-    x86_topo_ids_from_idx(topo_info, cpu_index, &topo_ids);
-    return x86_apicid_from_topo_ids(topo_info, &topo_ids);
+    x86_topo_ids_from_idx(apicid_topo, cpu_index, &topo_ids);
+    return x86_apicid_from_topo_ids(apicid_topo, &topo_ids);
 }
 
 #endif /* HW_I386_TOPOLOGY_H */
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index d84f7717900c..0d11102599a9 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -98,7 +98,7 @@ struct X86MachineState {
 #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
 OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
 
-void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
+void init_apicid_topo_info(X86ApicidTopoInfo *apicid_topo,
                            const X86MachineState *x86ms);
 
 uint32_t x86_cpu_apic_id_from_index(X86MachineState *pcms,
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 4494f01a1635..844b6df63a4a 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -232,19 +232,19 @@ static uint8_t cpuid2_cache_descriptor(CPUCacheInfo *cache)
                        0 /* Invalid value */)
 
 static uint32_t max_processor_ids_for_cache(CPUCacheInfo *cache,
-                                            X86ApicidTopoInfo *topo_info)
+                                            X86ApicidTopoInfo *apicid_topo)
 {
     uint32_t num_ids = 0;
 
     switch (cache->share_level) {
     case CORE:
-        num_ids = 1 << apicid_core_offset(topo_info);
+        num_ids = 1 << apicid_core_offset(apicid_topo);
         break;
     case MODULE:
-        num_ids = 1 << apicid_module_offset(topo_info);
+        num_ids = 1 << apicid_module_offset(apicid_topo);
         break;
     case DIE:
-        num_ids = 1 << apicid_die_offset(topo_info);
+        num_ids = 1 << apicid_die_offset(apicid_topo);
         break;
     default:
         /*
@@ -257,16 +257,16 @@ static uint32_t max_processor_ids_for_cache(CPUCacheInfo *cache,
     return num_ids - 1;
 }
 
-static uint32_t max_core_ids_in_package(X86ApicidTopoInfo *topo_info)
+static uint32_t max_core_ids_in_package(X86ApicidTopoInfo *apicid_topo)
 {
-    uint32_t num_cores = 1 << (apicid_pkg_offset(topo_info) -
-                               apicid_core_offset(topo_info));
+    uint32_t num_cores = 1 << (apicid_pkg_offset(apicid_topo) -
+                               apicid_core_offset(apicid_topo));
     return num_cores - 1;
 }
 
 /* Encode cache info for CPUID[4] */
 static void encode_cache_cpuid4(CPUCacheInfo *cache,
-                                X86ApicidTopoInfo *topo_info,
+                                X86ApicidTopoInfo *apicid_topo,
                                 uint32_t *eax, uint32_t *ebx,
                                 uint32_t *ecx, uint32_t *edx)
 {
@@ -276,8 +276,8 @@ static void encode_cache_cpuid4(CPUCacheInfo *cache,
     *eax = CACHE_TYPE(cache->type) |
            CACHE_LEVEL(cache->level) |
            (cache->self_init ? CACHE_SELF_INIT_LEVEL : 0) |
-           (max_core_ids_in_package(topo_info) << 26) |
-           (max_processor_ids_for_cache(cache, topo_info) << 14);
+           (max_core_ids_in_package(apicid_topo) << 26) |
+           (max_processor_ids_for_cache(cache, apicid_topo) << 14);
 
     assert(cache->line_size > 0);
     assert(cache->partitions > 0);
@@ -354,7 +354,7 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
 
 /* Encode cache info for CPUID[8000001D] */
 static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
-                                       X86ApicidTopoInfo *topo_info,
+                                       X86ApicidTopoInfo *apicid_topo,
                                        uint32_t *eax, uint32_t *ebx,
                                        uint32_t *ecx, uint32_t *edx)
 {
@@ -363,7 +363,7 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
 
     *eax = CACHE_TYPE(cache->type) | CACHE_LEVEL(cache->level) |
                (cache->self_init ? CACHE_SELF_INIT_LEVEL : 0);
-    *eax |= max_processor_ids_for_cache(cache, topo_info) << 14;
+    *eax |= max_processor_ids_for_cache(cache, apicid_topo) << 14;
 
     assert(cache->line_size > 0);
     assert(cache->partitions > 0);
@@ -383,14 +383,15 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
 }
 
 /* Encode cache info for CPUID[8000001E] */
-static void encode_topo_cpuid8000001e(X86CPU *cpu, X86ApicidTopoInfo *topo_info,
+static void encode_topo_cpuid8000001e(X86CPU *cpu,
+                                      X86ApicidTopoInfo *apicid_topo,
                                       uint32_t *eax, uint32_t *ebx,
                                       uint32_t *ecx, uint32_t *edx)
 {
     CPUState *cs = CPU(cpu);
     X86CPUTopoIDs topo_ids;
 
-    x86_topo_ids_from_apicid(cpu->apic_id, topo_info, &topo_ids);
+    x86_topo_ids_from_apicid(cpu->apic_id, apicid_topo, &topo_ids);
 
     *eax = cpu->apic_id;
 
@@ -433,7 +434,7 @@ static void encode_topo_cpuid8000001e(X86CPU *cpu, X86ApicidTopoInfo *topo_info,
      * in apic_id. Just use it by shifting.
      */
     *ecx = ((cs->topo.dies_per_socket - 1) << 8) |
-           ((cpu->apic_id >> apicid_die_offset(topo_info)) & 0xFF);
+           ((cpu->apic_id >> apicid_die_offset(apicid_topo)) & 0xFF);
 
     *edx = 0;
 }
@@ -5257,12 +5258,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     CPUState *cs = env_cpu(env);
     uint32_t limit;
     uint32_t signature[3];
-    X86ApicidTopoInfo topo_info;
+    X86ApicidTopoInfo apicid_topo;
 
-    topo_info.max_dies = cs->topo.dies_per_socket;
-    topo_info.max_modules = cs->topo.clusters_per_die;
-    topo_info.max_cores = cs->topo.cores_per_cluster;
-    topo_info.max_threads = cs->topo.threads_per_core;
+    apicid_topo.max_dies = cs->topo.dies_per_socket;
+    apicid_topo.max_modules = cs->topo.clusters_per_die;
+    apicid_topo.max_cores = cs->topo.cores_per_cluster;
+    apicid_topo.max_threads = cs->topo.threads_per_core;
 
     /* Calculate & apply limits for different index ranges */
     if (index >= 0xC0000000) {
@@ -5356,23 +5357,23 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             switch (count) {
             case 0: /* L1 dcache info */
                 encode_cache_cpuid4(env->cache_info_cpuid4.l1d_cache,
-                                    &topo_info,
+                                    &apicid_topo,
                                     eax, ebx, ecx, edx);
                 break;
             case 1: /* L1 icache info */
                 encode_cache_cpuid4(env->cache_info_cpuid4.l1i_cache,
-                                    &topo_info,
+                                    &apicid_topo,
                                     eax, ebx, ecx, edx);
                 break;
             case 2: /* L2 cache info */
                 encode_cache_cpuid4(env->cache_info_cpuid4.l2_cache,
-                                    &topo_info,
+                                    &apicid_topo,
                                     eax, ebx, ecx, edx);
                 break;
             case 3: /* L3 cache info */
                 if (cpu->enable_l3_cache) {
                     encode_cache_cpuid4(env->cache_info_cpuid4.l3_cache,
-                                        &topo_info,
+                                        &apicid_topo,
                                         eax, ebx, ecx, edx);
                     break;
                 }
@@ -5469,12 +5470,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
 
         switch (count) {
         case 0:
-            *eax = apicid_core_offset(&topo_info);
+            *eax = apicid_core_offset(&apicid_topo);
             *ebx = cs->topo.threads_per_core;
             *ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
             break;
         case 1:
-            *eax = apicid_pkg_offset(&topo_info);
+            *eax = apicid_pkg_offset(&apicid_topo);
             *ebx = cs->topo.threads_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
             break;
@@ -5505,17 +5506,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         *edx = cpu->apic_id;
         switch (count) {
         case 0:
-            *eax = apicid_core_offset(&topo_info);
+            *eax = apicid_core_offset(&apicid_topo);
             *ebx = cs->topo.threads_per_core;
             *ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
             break;
         case 1:
-            *eax = apicid_die_offset(&topo_info);
+            *eax = apicid_die_offset(&apicid_topo);
             *ebx = cs->topo.threads_per_socket / cs->topo.dies_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
             break;
         case 2:
-            *eax = apicid_pkg_offset(&topo_info);
+            *eax = apicid_pkg_offset(&apicid_topo);
             *ebx = cs->topo.threads_per_socket;
             *ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
             break;
@@ -5810,7 +5811,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
              * thread ID within a package".
              * Bits 7:0 is "The number of threads in the package is NC+1"
              */
-            *ecx = (apicid_pkg_offset(&topo_info) << 12) |
+            *ecx = (apicid_pkg_offset(&apicid_topo) << 12) |
                    (cs->topo.threads_per_socket - 1);
         } else {
             *ecx = 0;
@@ -5839,19 +5840,19 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         switch (count) {
         case 0: /* L1 dcache info */
             encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
-                                       &topo_info, eax, ebx, ecx, edx);
+                                       &apicid_topo, eax, ebx, ecx, edx);
             break;
         case 1: /* L1 icache info */
             encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
-                                       &topo_info, eax, ebx, ecx, edx);
+                                       &apicid_topo, eax, ebx, ecx, edx);
             break;
         case 2: /* L2 cache info */
             encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
-                                       &topo_info, eax, ebx, ecx, edx);
+                                       &apicid_topo, eax, ebx, ecx, edx);
             break;
         case 3: /* L3 cache info */
             encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
-                                       &topo_info, eax, ebx, ecx, edx);
+                                       &apicid_topo, eax, ebx, ecx, edx);
             break;
         default: /* end of info */
             *eax = *ebx = *ecx = *edx = 0;
@@ -5860,7 +5861,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0x8000001E:
         if (cpu->core_id <= 255) {
-            encode_topo_cpuid8000001e(cpu, &topo_info, eax, ebx, ecx, edx);
+            encode_topo_cpuid8000001e(cpu, &apicid_topo, eax, ebx, ecx, edx);
         } else {
             *eax = 0;
             *ebx = 0;
diff --git a/tests/unit/test-x86-apicid.c b/tests/unit/test-x86-apicid.c
index 225d88bc3262..fd76d1775a10 100644
--- a/tests/unit/test-x86-apicid.c
+++ b/tests/unit/test-x86-apicid.c
@@ -28,115 +28,115 @@
 
 static void test_topo_bits(void)
 {
-    X86ApicidTopoInfo topo_info = {0};
+    X86ApicidTopoInfo apicid_topo = {0};
 
     /*
      * simple tests for 1 thread per core, 1 core per module,
      *                  1 module per die, 1 die per package
      */
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 1};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 0);
-    g_assert_cmpuint(apicid_core_width(&topo_info), ==, 0);
-    g_assert_cmpuint(apicid_module_width(&topo_info), ==, 0);
-    g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 1};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 0);
+    g_assert_cmpuint(apicid_core_width(&apicid_topo), ==, 0);
+    g_assert_cmpuint(apicid_module_width(&apicid_topo), ==, 0);
+    g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 0);
 
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 1};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 3), ==, 3);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 1};
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 3), ==, 3);
 
 
     /* Test field width calculation for multiple values
      */
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 2};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 1);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 3};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 4};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
-
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 14};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 15};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 16};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 1, 17};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 5);
-
-
-    topo_info = (X86ApicidTopoInfo) {1, 1, 30, 2};
-    g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 31, 2};
-    g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 32, 2};
-    g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
-    topo_info = (X86ApicidTopoInfo) {1, 1, 33, 2};
-    g_assert_cmpuint(apicid_core_width(&topo_info), ==, 6);
-
-    topo_info = (X86ApicidTopoInfo) {1, 6, 30, 2};
-    g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86ApicidTopoInfo) {1, 7, 30, 2};
-    g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86ApicidTopoInfo) {1, 8, 30, 2};
-    g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3);
-    topo_info = (X86ApicidTopoInfo) {1, 9, 30, 2};
-    g_assert_cmpuint(apicid_module_width(&topo_info), ==, 4);
-
-    topo_info = (X86ApicidTopoInfo) {1, 6, 30, 2};
-    g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
-    topo_info = (X86ApicidTopoInfo) {2, 6, 30, 2};
-    g_assert_cmpuint(apicid_die_width(&topo_info), ==, 1);
-    topo_info = (X86ApicidTopoInfo) {3, 6, 30, 2};
-    g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
-    topo_info = (X86ApicidTopoInfo) {4, 6, 30, 2};
-    g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 2};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 1);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 3};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 2);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 4};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 2);
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 14};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 4);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 15};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 4);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 16};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 4);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 17};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 5);
+
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 30, 2};
+    g_assert_cmpuint(apicid_core_width(&apicid_topo), ==, 5);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 31, 2};
+    g_assert_cmpuint(apicid_core_width(&apicid_topo), ==, 5);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 32, 2};
+    g_assert_cmpuint(apicid_core_width(&apicid_topo), ==, 5);
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 33, 2};
+    g_assert_cmpuint(apicid_core_width(&apicid_topo), ==, 6);
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 6, 30, 2};
+    g_assert_cmpuint(apicid_module_width(&apicid_topo), ==, 3);
+    apicid_topo = (X86ApicidTopoInfo) {1, 7, 30, 2};
+    g_assert_cmpuint(apicid_module_width(&apicid_topo), ==, 3);
+    apicid_topo = (X86ApicidTopoInfo) {1, 8, 30, 2};
+    g_assert_cmpuint(apicid_module_width(&apicid_topo), ==, 3);
+    apicid_topo = (X86ApicidTopoInfo) {1, 9, 30, 2};
+    g_assert_cmpuint(apicid_module_width(&apicid_topo), ==, 4);
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 6, 30, 2};
+    g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 0);
+    apicid_topo = (X86ApicidTopoInfo) {2, 6, 30, 2};
+    g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 1);
+    apicid_topo = (X86ApicidTopoInfo) {3, 6, 30, 2};
+    g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 2);
+    apicid_topo = (X86ApicidTopoInfo) {4, 6, 30, 2};
+    g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 2);
 
     /* build a weird topology and see if IDs are calculated correctly
      */
 
     /* This will use 2 bits for thread ID and 3 bits for core ID
      */
-    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
-    g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
-    g_assert_cmpuint(apicid_core_offset(&topo_info), ==, 2);
-    g_assert_cmpuint(apicid_module_offset(&topo_info), ==, 5);
-    g_assert_cmpuint(apicid_die_offset(&topo_info), ==, 5);
-    g_assert_cmpuint(apicid_pkg_offset(&topo_info), ==, 5);
-
-    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
-
-    topo_info = (X86ApicidTopoInfo) {1, 1, 6, 3};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 0), ==,
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 6, 3};
+    g_assert_cmpuint(apicid_smt_width(&apicid_topo), ==, 2);
+    g_assert_cmpuint(apicid_core_offset(&apicid_topo), ==, 2);
+    g_assert_cmpuint(apicid_module_offset(&apicid_topo), ==, 5);
+    g_assert_cmpuint(apicid_die_offset(&apicid_topo), ==, 5);
+    g_assert_cmpuint(apicid_pkg_offset(&apicid_topo), ==, 5);
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 6, 3};
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
+
+    apicid_topo = (X86ApicidTopoInfo) {1, 1, 6, 3};
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 0), ==,
                      (1 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 1), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 1), ==,
                      (1 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 2), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 2), ==,
                      (1 << 2) | 2);
 
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 0), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 0), ==,
                      (2 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 1), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 1), ==,
                      (2 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 2), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 2), ==,
                      (2 << 2) | 2);
 
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 0), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 0), ==,
                      (5 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 1), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 1), ==,
                      (5 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 2), ==,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 2), ==,
                      (5 << 2) | 2);
 
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
                      1 * 6 * 3 + 0 * 3 + 0), ==, (1 << 5));
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
                      1 * 6 * 3 + 1 * 3 + 1), ==, (1 << 5) | (1 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
+    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
                      3 * 6 * 3 + 5 * 3 + 2), ==, (3 << 5) | (5 << 2) | 2);
 }
 
-- 
2.34.1

[RFC 35/52] i386: Support APIC ID topology for hybrid CPU topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

X86ApicidTopoInfo is used to caculate offsets of topology levels for
APIC ID.

For hybrid CPU topology, X86ApicidTopoInfo should get the maximum
possible number of corresponding topology structures in each topology
level (currently, only for core and thread level).

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 9ee0fcb9a460..1c071f8120cb 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -70,10 +70,37 @@ inline void init_apicid_topo_info(X86ApicidTopoInfo *apicid_topo,
 {
     MachineState *ms = MACHINE(x86ms);
 
+    /*
+     * At present, the packages and dies in the hybrid CPU topology are the
+     * same, so the numbers of dies in one package and the numbers of modules
+     * in the dies are also the same.
+     */
     apicid_topo->max_dies = machine_topo_get_dies(ms);
     apicid_topo->max_modules = machine_topo_get_clusters(ms);
-    apicid_topo->max_cores = machine_topo_get_smp_cores(ms);
-    apicid_topo->max_threads = machine_topo_get_smp_threads(ms);
+
+    if (machine_topo_is_smp(ms)) {
+        apicid_topo->max_cores = machine_topo_get_smp_cores(ms);
+        apicid_topo->max_threads = machine_topo_get_smp_threads(ms);
+    } else {
+        int max_cores = 0;
+        int max_threads = 0;
+        int nr_clusters = machine_topo_get_clusters(ms);
+        int nr_cores, nr_threads;
+
+        for (int i = 0; i < nr_clusters; i++) {
+            nr_cores = machine_topo_get_cores(ms, i);
+            max_cores = max_cores > nr_cores ? max_cores : nr_cores;
+
+            for (int j = 0; j < nr_cores; j++) {
+                nr_threads = machine_topo_get_threads(ms, i, j);
+                max_threads = max_threads > nr_threads ?
+                              max_threads : nr_threads;
+            }
+        }
+
+        apicid_topo->max_cores = max_cores;
+        apicid_topo->max_threads = max_threads;
+    }
 }
 
 /*
-- 
2.34.1

[RFC 36/52] i386: Use init_apicid_topo_info() to initialize APIC ID topology for system emulator

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since APIC ID for hybrid CPU topology needs to consider maximum possible
structures for each topology levels, the APIC IDs for hybrid CPU topology
and smp CPU topology may be different.

User emulator doesn't support hybrid CPU topology so use
init_apicid_topo_info() for system emulator.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 target/i386/cpu.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 844b6df63a4a..7d6722ab3292 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -39,6 +39,7 @@
 #include "exec/address-spaces.h"
 #include "hw/boards.h"
 #include "hw/i386/sgx-epc.h"
+#include "hw/i386/x86.h"
 #endif
 
 #include "disas/capstone.h"
@@ -5260,10 +5261,15 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     uint32_t signature[3];
     X86ApicidTopoInfo apicid_topo;
 
+#ifndef CONFIG_USER_ONLY
+    X86MachineState *x86ms = X86_MACHINE(qdev_get_machine());
+    init_apicid_topo_info(&apicid_topo, x86ms);
+#else
     apicid_topo.max_dies = cs->topo.dies_per_socket;
     apicid_topo.max_modules = cs->topo.clusters_per_die;
     apicid_topo.max_cores = cs->topo.cores_per_cluster;
     apicid_topo.max_threads = cs->topo.threads_per_core;
+#endif
 
     /* Calculate & apply limits for different index ranges */
     if (index >= 0xC0000000) {
-- 
2.34.1

[RFC 37/52] i386: Update X86CPUTopoIDs generating rule for hybrid topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

X86CPUTopoIDs can be parsed form cpu_index or APIC ID. For hybrid CPU
topology, the case, generating form cpu_index, need to update.

In x86_topo_ids_from_idx(), if current topology is hybrid, traverse all
clusters and cores to find the cluster, core and thread that cpu_index
matches.

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c                | 78 ++++++++++++++++++++++++++++++-
 include/hw/i386/topology.h   | 36 --------------
 tests/unit/test-x86-apicid.c | 91 +++++++++++++++++++++++-------------
 3 files changed, 136 insertions(+), 69 deletions(-)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 1c071f8120cb..a09df6d33fff 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -103,6 +103,82 @@ inline void init_apicid_topo_info(X86ApicidTopoInfo *apicid_topo,
     }
 }
 
+/*
+ * Calculate thread/core/package IDs for a specific topology,
+ * based on (contiguous) CPU index
+ */
+static void x86_topo_ids_from_idx(X86MachineState *x86ms,
+                                  unsigned cpu_index,
+                                  X86CPUTopoIDs *topo_ids)
+{
+    MachineState *ms = MACHINE(x86ms);
+    unsigned nr_dies;
+    unsigned nr_modules;
+    unsigned nr_cores;
+    unsigned nr_threads;
+    unsigned cpus_per_pkg;
+    unsigned cpus_per_die;
+
+    nr_dies = machine_topo_get_dies(ms);
+    nr_modules = machine_topo_get_clusters(ms);
+    cpus_per_pkg = machine_topo_get_threads_per_socket(ms);
+    cpus_per_die = cpus_per_pkg / nr_dies;
+
+    topo_ids->pkg_id = cpu_index / cpus_per_pkg;
+    topo_ids->die_id = cpu_index / cpus_per_die % nr_dies;
+
+    if (machine_topo_is_smp(ms)) {
+        nr_cores = machine_topo_get_smp_cores(ms);
+        nr_threads = machine_topo_get_smp_threads(ms);
+
+        topo_ids->module_id = cpu_index / (nr_cores * nr_threads) %
+                              nr_modules;
+        topo_ids->core_id = cpu_index / nr_threads % nr_cores;
+        topo_ids->smt_id = cpu_index % nr_threads;
+    } else {
+        /*
+         * Next we need to traverse in the die, so scale the cpu_index to
+         * the relative index in the die.
+         */
+        int idx = cpu_index % cpus_per_die;
+
+        topo_ids->module_id = 0;
+        topo_ids->core_id = 0;
+        topo_ids->smt_id = 0;
+
+        for (int i = 0; i < nr_modules; i++) {
+            nr_cores = machine_topo_get_cores(ms, i);
+
+            for (int j = 0; j < nr_cores; j++) {
+                nr_threads = machine_topo_get_threads(ms, i, j);
+
+                if (idx < nr_threads) {
+                    topo_ids->module_id = i;
+                    topo_ids->core_id = j;
+                    topo_ids->smt_id = idx;
+                    return;
+                } else {
+                    idx -= nr_threads;
+                }
+            }
+        }
+    }
+}
+
+/*
+ * Make APIC ID for the CPU 'cpu_index'
+ *
+ * 'cpu_index' is a sequential, contiguous ID for the CPU.
+ */
+static inline apic_id_t x86_apicid_from_cpu_idx(X86MachineState *x86ms,
+                                                X86ApicidTopoInfo *apicid_topo,
+                                                unsigned cpu_index)
+{
+    X86CPUTopoIDs topo_ids;
+    x86_topo_ids_from_idx(x86ms, cpu_index, &topo_ids);
+    return x86_apicid_from_topo_ids(apicid_topo, &topo_ids);
+}
+
 /*
  * Calculates initial APIC ID for a specific CPU index
  *
@@ -118,7 +194,7 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
 
     init_apicid_topo_info(&apicid_topo, x86ms);
 
-    return x86_apicid_from_cpu_idx(&apicid_topo, cpu_index);
+    return x86_apicid_from_cpu_idx(x86ms, &apicid_topo, cpu_index);
 }
 
 
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 5b29c51329f1..06403ca77087 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -153,29 +153,6 @@ static inline apic_id_t x86_apicid_from_topo_ids(X86ApicidTopoInfo *apicid_topo,
            topo_ids->smt_id;
 }
 
-/*
- * Calculate thread/core/package IDs for a specific topology,
- * based on (contiguous) CPU index
- */
-static inline void x86_topo_ids_from_idx(X86ApicidTopoInfo *apicid_topo,
-                                         unsigned cpu_index,
-                                         X86CPUTopoIDs *topo_ids)
-{
-    unsigned nr_dies = apicid_topo->max_dies;
-    unsigned nr_modules = apicid_topo->max_modules;
-    unsigned nr_cores = apicid_topo->max_cores;
-    unsigned nr_threads = apicid_topo->max_threads;
-
-    topo_ids->pkg_id = cpu_index / (nr_dies * nr_modules *
-                       nr_cores * nr_threads);
-    topo_ids->die_id = cpu_index / (nr_modules * nr_cores *
-                       nr_threads) % nr_dies;
-    topo_ids->module_id = cpu_index / (nr_cores * nr_threads) %
-                          nr_modules;
-    topo_ids->core_id = cpu_index / nr_threads % nr_cores;
-    topo_ids->smt_id = cpu_index % nr_threads;
-}
-
 /*
  * Calculate thread/core/package IDs for a specific topology,
  * based on APIC ID
@@ -198,17 +175,4 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
     topo_ids->pkg_id = apicid >> apicid_pkg_offset(apicid_topo);
 }
 
-/*
- * Make APIC ID for the CPU 'cpu_index'
- *
- * 'cpu_index' is a sequential, contiguous ID for the CPU.
- */
-static inline apic_id_t x86_apicid_from_cpu_idx(X86ApicidTopoInfo *apicid_topo,
-                                                unsigned cpu_index)
-{
-    X86CPUTopoIDs topo_ids;
-    x86_topo_ids_from_idx(apicid_topo, cpu_index, &topo_ids);
-    return x86_apicid_from_topo_ids(apicid_topo, &topo_ids);
-}
-
 #endif /* HW_I386_TOPOLOGY_H */
diff --git a/tests/unit/test-x86-apicid.c b/tests/unit/test-x86-apicid.c
index fd76d1775a10..917d8c480436 100644
--- a/tests/unit/test-x86-apicid.c
+++ b/tests/unit/test-x86-apicid.c
@@ -23,9 +23,36 @@
  */
 
 #include "qemu/osdep.h"
-
+#include "hw/core/cpu.h"
 #include "hw/i386/topology.h"
 
+static void x86_smp_test_topo_ids_from_idx(X86ApicidTopoInfo *apicid_topo,
+                                           unsigned cpu_index,
+                                           X86CPUTopoIDs *topo_ids)
+{
+    unsigned nr_dies = apicid_topo->max_dies;
+    unsigned nr_modules = apicid_topo->max_modules;
+    unsigned nr_cores = apicid_topo->max_cores;
+    unsigned nr_threads = apicid_topo->max_threads;
+
+    topo_ids->pkg_id = cpu_index / (nr_dies * nr_modules *
+                                    nr_cores * nr_threads);
+    topo_ids->die_id = cpu_index / (nr_modules * nr_cores
+                                    * nr_threads) % nr_dies;
+    topo_ids->module_id = cpu_index / (nr_cores * nr_threads) % nr_modules;
+    topo_ids->core_id = cpu_index / nr_threads % nr_cores;
+    topo_ids->smt_id = cpu_index % nr_threads;
+}
+
+static apic_id_t
+x86_smp_test_apicid_from_cpu_idx(X86ApicidTopoInfo *apicid_topo,
+                                 unsigned cpu_index)
+{
+    X86CPUTopoIDs topo_ids;
+    x86_smp_test_topo_ids_from_idx(apicid_topo, cpu_index, &topo_ids);
+    return x86_apicid_from_topo_ids(apicid_topo, &topo_ids);
+}
+
 static void test_topo_bits(void)
 {
     X86ApicidTopoInfo apicid_topo = {0};
@@ -41,10 +68,10 @@ static void test_topo_bits(void)
     g_assert_cmpuint(apicid_die_width(&apicid_topo), ==, 0);
 
     apicid_topo = (X86ApicidTopoInfo) {1, 1, 1, 1};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 3), ==, 3);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 3), ==, 3);
 
 
     /* Test field width calculation for multiple values
@@ -106,37 +133,37 @@ static void test_topo_bits(void)
     g_assert_cmpuint(apicid_pkg_offset(&apicid_topo), ==, 5);
 
     apicid_topo = (X86ApicidTopoInfo) {1, 1, 6, 3};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 0), ==, 0);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 1), ==, 1);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 2), ==, 2);
 
     apicid_topo = (X86ApicidTopoInfo) {1, 1, 6, 3};
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 0), ==,
-                     (1 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 1), ==,
-                     (1 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 2), ==,
-                     (1 << 2) | 2);
-
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 0), ==,
-                     (2 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 1), ==,
-                     (2 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 2), ==,
-                     (2 << 2) | 2);
-
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 0), ==,
-                     (5 << 2) | 0);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 1), ==,
-                     (5 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 2), ==,
-                     (5 << 2) | 2);
-
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 0),
+                     ==, (1 << 2) | 0);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 1),
+                     ==, (1 << 2) | 1);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 1 * 3 + 2),
+                     ==, (1 << 2) | 2);
+
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 0),
+                     ==, (2 << 2) | 0);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 1),
+                     ==, (2 << 2) | 1);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 2 * 3 + 2),
+                     ==, (2 << 2) | 2);
+
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 0),
+                     ==, (5 << 2) | 0);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 1),
+                     ==, (5 << 2) | 1);
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo, 5 * 3 + 2),
+                     ==, (5 << 2) | 2);
+
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo,
                      1 * 6 * 3 + 0 * 3 + 0), ==, (1 << 5));
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo,
                      1 * 6 * 3 + 1 * 3 + 1), ==, (1 << 5) | (1 << 2) | 1);
-    g_assert_cmpuint(x86_apicid_from_cpu_idx(&apicid_topo,
+    g_assert_cmpuint(x86_smp_test_apicid_from_cpu_idx(&apicid_topo,
                      3 * 6 * 3 + 5 * 3 + 2), ==, (3 << 5) | (5 << 2) | 2);
 }
 
-- 
2.34.1

[RFC 38/52] i386: Introduce hybrid_core_type to CPUX86State

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Currently, x86 only supports two hybrid core types: Intel Core and Intel
Atom [1], so add the core type definations for Intel Core and Intel
Atom, and add hybrid_core_type in CPUX86State.

[1]: SDM, vol.2, ch.3, 3.3 Instructions (A-L), CPUID, CPUID.1AH:EAX[bits
     31-24].

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c     | 4 ++++
 target/i386/cpu.c | 1 +
 target/i386/cpu.h | 8 ++++++++
 3 files changed, 13 insertions(+)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index a09df6d33fff..f381fdc43180 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -383,6 +383,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
     CPUArchId *cpu_slot;
     X86CPUTopoIDs topo_ids;
     X86CPU *cpu = X86_CPU(dev);
+    CPUX86State *env = &cpu->env;
     MachineState *ms = MACHINE(hotplug_dev);
     X86MachineState *x86ms = X86_MACHINE(hotplug_dev);
     X86ApicidTopoInfo apicid_topo;
@@ -564,6 +565,9 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,
         return;
     }
 
+    env->hybrid_core_type =
+        machine_topo_get_hybrid_core_type(ms, cpu->cluster_id, cpu->core_id);
+
     cs = CPU(cpu);
     cs->cpu_index = idx;
 
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 7d6722ab3292..266e981b79e2 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6845,6 +6845,7 @@ static void x86_cpu_initfn(Object *obj)
     X86CPUClass *xcc = X86_CPU_GET_CLASS(obj);
     CPUX86State *env = &cpu->env;
 
+    env->hybrid_core_type = INVALID_HYBRID_TYPE;
     cpu_set_cpustate_pointers(cpu);
 
     object_property_add(obj, "feature-words", "X86CPUFeatureWordInfo",
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 30b2aa6ab10d..bb4e370f9768 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1543,6 +1543,12 @@ typedef struct CPUCaches {
         CPUCacheInfo *l3_cache;
 } CPUCaches;
 
+typedef enum {
+    INVALID_HYBRID_TYPE = 0,
+    INTEL_ATOM_TYPE,
+    INTEL_CORE_TYPE,
+} X86HybridCoreType;
+
 typedef struct HVFX86LazyFlags {
     target_ulong result;
     target_ulong auxbits;
@@ -1825,6 +1831,8 @@ typedef struct CPUArchState {
     uint32_t umwait;
 
     TPRAccess tpr_access_type;
+
+    X86HybridCoreType hybrid_core_type;
 } CPUX86State;
 
 struct kvm_msrs;
-- 
2.34.1

[RFC 39/52] i386/cpu: Add Intel hybrid related CPUID support

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For hybrid cpu topology, Intel exposes these CPUIDs [1]:
1. Set CPUID.07H.0H:EDX.Hybrid[bit 15]. With setting as 1, the processor
   is identified as a hybrid part.
2. Have CPUID.1AH leaf. Set core type and native model ID in
   CPUID.1AH:EAX. Because the native model ID is currently useless for
   the software, no need to emulate.

For hybrid related CPUIDs, especially CPUID.07H.0H:EDX.Hybrid[bit 15],
there's no need to expose this feature in feature_word_info[] to allow
user to set directly, because hybrid features depend on the specific
core type information, and this information needs to be gathered
together with hybrid cpu topology.

[1]: SDM, vol.2, Ch.3, 3.2 Instructions (A-L), CPUID-CPU Identification

Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 target/i386/cpu.c | 43 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h |  2 ++
 2 files changed, 45 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 266e981b79e2..a0aaaf996222 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -620,6 +620,10 @@ static CPUCacheInfo legacy_l3_cache = {
 #define INTEL_AMX_TMUL_MAX_K           0x10
 #define INTEL_AMX_TMUL_MAX_N           0x40
 
+/* CPUID Leaf 0x1A constants: */
+#define INTEL_HYBRID_TYPE_ATOM         0x20
+#define INTEL_HYBRID_TYPE_CORE         0x40
+
 void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
                               uint32_t vendor2, uint32_t vendor3)
 {
@@ -5416,6 +5420,11 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             }
             *edx = env->features[FEAT_7_0_EDX]; /* Feature flags */
 
+            if (env->hybrid_core_type != INVALID_HYBRID_TYPE &&
+                (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
+                *edx |= CPUID_7_0_EDX_HYBRID;
+            }
+
             /*
              * SGX cannot be emulated in software.  If hardware does not
              * support enabling SGX and/or SGX flexible launch control,
@@ -5671,6 +5680,31 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         }
         break;
     }
+    case 0x1A:
+        /* Hybrid Information Enumeration */
+        *eax = 0;
+        *ebx = 0;
+        *ecx = 0;
+        *edx = 0;
+        if (env->hybrid_core_type != INVALID_HYBRID_TYPE &&
+            (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
+            /*
+             * CPUID.1AH:EAX.[bits 23-0] indicates "native model ID of the
+             * core". Since this field currently is useless for software,
+             * no need to emulate.
+             */
+            switch (env->hybrid_core_type) {
+            case INTEL_ATOM_TYPE:
+                *eax = INTEL_HYBRID_TYPE_ATOM << 24;
+                break;
+            case INTEL_CORE_TYPE:
+                *eax = INTEL_HYBRID_TYPE_CORE << 24;
+                break;
+            default:
+                abort();
+            }
+        }
+        break;
     case 0x1D: {
         /* AMX TILE */
         *eax = 0;
@@ -6314,6 +6348,15 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
             }
         }
 
+        /*
+         * Intel CPU topology with hybrid cores support requires CPUID.1AH.
+         * And CPUID.07H.0H:EDX.Hybrid[bit 15] is also required.
+         */
+        if (env->hybrid_core_type != INVALID_HYBRID_TYPE &&
+            (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
+            x86_cpu_adjust_level(cpu, &env->cpuid_min_level, 0x1A);
+        }
+
         /*
          * Intel CPU topology with multi-dies support requires CPUID[0x1F].
          * For AMD Rome/Milan, cpuid level is 0x10, and guest OS should detect
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index bb4e370f9768..8fd7b0c55ae8 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -877,6 +877,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
 #define CPUID_7_0_EDX_AVX512_VP2INTERSECT (1U << 8)
 /* SERIALIZE instruction */
 #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
+/* Hybrid */
+#define CPUID_7_0_EDX_HYBRID            (1U << 15)
 /* TSX Suspend Load Address Tracking instruction */
 #define CPUID_7_0_EDX_TSX_LDTRK         (1U << 16)
 /* Architectural LBRs */
-- 
2.34.1

[RFC 40/52] qapi: Introduce hybrid options

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

For hybrid cpu topology configuration, the original "-smp" lack of
flexibility to expand, and unables to customize different cores.

So we introduce "-hybrid" command:
-hybrid socket,sockets=n
-hybrid die,dies=n
-hybrid cluster,clusters=n
-hybrid core,cores=n,type=core_type[,threads=threads]
        [,clusterid=cluster]

Here, we first define the corresponding qapi options for these 4
topology levels: core, cluster, die and socket.

We doesn't need a thread level since thread doesn't have different
type.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 qapi/machine.json | 90 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)

diff --git a/qapi/machine.json b/qapi/machine.json
index bd7303f34497..931c6dea9819 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -1694,3 +1694,93 @@
 { 'command': 'dumpdtb',
   'data': { 'filename': 'str' },
   'if': 'CONFIG_FDT' }
+
+##
+# @HybridOptionsType:
+#
+# @socket: socket level configuration
+#
+# @die: die level configuration
+#
+# @cluster: cluster level configuration
+#
+# @core: core level configuration
+#
+# Since: 8.0
+##
+{ 'enum': 'HybridOptionsType',
+     'data': [ 'socket', 'die', 'cluster', 'core' ] }
+
+##
+# @HybridOptions:
+#
+# A discriminated record of hybrid options. (for OptsVisitor)
+#
+# Since: 8.0
+##
+{ 'union': 'HybridOptions',
+     'base': { 'type': 'HybridOptionsType' },
+     'discriminator': 'type',
+     'data': {
+       'socket': 'HybridSocketOptions',
+       'die': 'HybridDieOptions',
+       'cluster': 'HybridClusterOptions',
+       'core': 'HybridCoreOptions' }}
+
+##
+# @HybridCoreOptions:
+#
+# Configure core level topology for hybrid cpu topology.
+#
+# @cores: number of current type cores in one die.
+#
+# @coretype: type of current core.
+#
+# @threads: number of threads that current core has.
+#
+# @clusterid: cluster index in one die that current core will be
+#             inserted to.
+#
+# Since: 8.0
+##
+{ 'struct': 'HybridCoreOptions', 'data': {
+     '*cores': 'uint32',
+     '*coretype': 'str',
+     '*threads': 'uint32',
+     '*clusterid': ['uint32']} }
+
+##
+# @HybridClusterOptions:
+#
+# Configure cluster level topology for hybrid cpu topology.
+#
+# @clusters: number of clusters in one socket.
+#
+# Since: 8.0
+##
+{ 'struct': 'HybridClusterOptions', 'data': {
+            '*clusters': 'uint32' } }
+
+##
+# @HybridDieOptions:
+#
+# Configure die level topology for hybrid cpu topology.
+#
+# @dies: number of dies in one socket.
+#
+# Since: 8.0
+##
+{ 'struct': 'HybridDieOptions', 'data': {
+            '*dies': 'uint32' } }
+
+##
+# @HybridSocketOptions:
+#
+# Configure socket level topology for hybrid cpu topology.
+#
+# @sockets: number of sockets in the whole system.
+#
+# Since: 8.0
+##
+{ 'struct': 'HybridSocketOptions', 'data': {
+            '*sockets': 'uint32' } }
-- 
2.34.1

[RFC 41/52] machine: Introduce core_type() hook

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since supported core types are architecture specific, we need this hook
to allow archs define its own parsing or validation method.

As the example, add the x86 core_type() which will be used in "-hybrid"
parameter parsing.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 14 ++++++++++++++
 hw/core/machine.c      |  1 +
 hw/i386/x86.c          | 15 +++++++++++++++
 include/hw/boards.h    |  7 +++++++
 4 files changed, 37 insertions(+)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 12c05510c1b5..f9ab08a1252e 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
         return;
     }
 }
+
+/*
+ * machine_parse_hybrid_core_type: the default hook to parse hybrid core
+ *                                 type corresponding to the coretype
+ *                                 string option.
+ */
+int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
+{
+    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
+        return 0;
+    }
+
+    return -1;
+}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index fad990f49b03..acc32b3be5f6 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -926,6 +926,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
      * On Linux, each node's border has to be 8MB aligned
      */
     mc->numa_mem_align_shift = 23;
+    mc->core_type = machine_parse_hybrid_core_type;
 
     object_class_property_add_str(oc, "kernel",
         machine_get_kernel, machine_set_kernel);
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index f381fdc43180..f58a90359170 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1569,6 +1569,20 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
     qapi_free_SgxEPCList(list);
 }
 
+static int x86_parse_hybrid_core_type(MachineState *ms, const char *coretype)
+{
+    X86HybridCoreType type;
+
+    if (strcmp(coretype, "atom") == 0) {
+        type = INTEL_ATOM_TYPE;
+    } else if (strcmp(coretype, "core") == 0) {
+        type = INTEL_CORE_TYPE;
+    } else {
+        type = INVALID_HYBRID_TYPE;
+    }
+    return type;
+}
+
 static void x86_machine_initfn(Object *obj)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1596,6 +1610,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     x86mc->save_tsc_khz = true;
     x86mc->fwcfg_dma_enabled = true;
     nc->nmi_monitor_handler = x86_nmi;
+    mc->core_type = x86_parse_hybrid_core_type;
 
     object_class_property_add(oc, X86_MACHINE_SMM, "OnOffAuto",
         x86_machine_get_smm, x86_machine_set_smm,
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 9364c90d5f1a..34ec035b5c9f 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
                                Error **errp);
 void machine_parse_smp_config(MachineState *ms,
                               const SMPConfiguration *config, Error **errp);
+int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
 
 /**
  * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
@@ -199,6 +200,11 @@ typedef struct {
  *    Return the type of KVM corresponding to the kvm-type string option or
  *    computed based on other criteria such as the host kernel capabilities.
  *    kvm-type may be NULL if it is not needed.
+ * @core_type:
+ *    Return the type of hybrid cores corresponding to the coretype string
+ *    option. The default hook only accept "none" or "" since the most generic
+ *    core topology should not specify any specific core type. Each arch can
+ *    define its own core_type() hook to override the default one.
  * @numa_mem_supported:
  *    true if '--numa node.mem' option is supported and false otherwise
  * @hotplug_allowed:
@@ -237,6 +243,7 @@ struct MachineClass {
     void (*reset)(MachineState *state, ShutdownCause reason);
     void (*wakeup)(MachineState *state);
     int (*kvm_type)(MachineState *machine, const char *arg);
+    int (*core_type)(MachineState *state, const char *type);
 
     BlockInterfaceType block_default_type;
     int units_per_default_bus;
-- 
2.34.1

[RFC 42/52] hw/machine: Add hybrid_supported in generic topo properties

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Since hybrid cpu topology configuration can benefit not only x86, but
also other architectures/platforms that have supported (in real
machines) or will support hybrid CPU topology, "-hybrid" can be generic.

So add the generic topology property to configure if support hybrid
cpu topology for architectures/platforms in SmpCompatProps.

Also rename SmpCompatProps to TopoCompatProps to make this structure
more generic for both smp topology and hybrid topology.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 include/hw/boards.h | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 34ec035b5c9f..17be3485e823 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -127,19 +127,26 @@ typedef struct {
 } CPUArchIdList;
 
 /**
- * SMPCompatProps:
- * @prefer_sockets - whether sockets are preferred over cores in smp parsing
+ * TopoCompatProps:
+ * @hybrid_support - whether hybrid cpu topology are supported by machine.
+ *                   Note that hybrid cpu topology requires to specify the
+ *                   topology of each core so that there will no longer be
+ *                   a default core topology, thus prefer_sockets won't work
+ *                   when hybrid_support is enabled.
+ * @prefer_sockets - whether sockets are preferred over cores in smp parsing.
+ *                   Not work when hybrid_support is enabled.
  * @dies_supported - whether dies are supported by the machine
  * @clusters_supported - whether clusters are supported by the machine
  * @has_clusters - whether clusters are explicitly specified in the user
  *                 provided SMP configuration
  */
 typedef struct {
+    bool hybrid_supported;
     bool prefer_sockets;
     bool dies_supported;
     bool clusters_supported;
     bool has_clusters;
-} SMPCompatProps;
+} TopoCompatProps;
 
 /**
  * MachineClass:
@@ -281,7 +288,7 @@ struct MachineClass {
     bool nvdimm_supported;
     bool numa_mem_supported;
     bool auto_enable_numa;
-    SMPCompatProps smp_props;
+    TopoCompatProps smp_props;
     const char *default_ram_id;
 
     HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
-- 
2.34.1

[RFC 43/52] hw/machine: Rename MachineClass.smp_props to MachineClass.topo_props

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

MachineClass.topo_props is better to be compatible with both hybrid
topology and smp topology properties.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/acpi/aml-build.c         |  2 +-
 hw/arm/virt.c               |  6 +++---
 hw/core/machine-topo.c      | 12 ++++++------
 hw/i386/pc.c                |  4 ++--
 hw/i386/pc_piix.c           |  2 +-
 hw/i386/pc_q35.c            |  2 +-
 hw/ppc/spapr.c              |  2 +-
 hw/s390x/s390-virtio-ccw.c  |  2 +-
 include/hw/boards.h         |  2 +-
 tests/unit/test-smp-parse.c | 12 ++++++------
 10 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index 693bd8833d10..51634cfd2776 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2030,7 +2030,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
                 0, socket_id, NULL, 0);
         }
 
-        if (mc->smp_props.clusters_supported && mc->smp_props.has_clusters) {
+        if (mc->topo_props.clusters_supported && mc->topo_props.has_clusters) {
             if (cpus->cpus[n].props.cluster_id != cluster_id) {
                 assert(cpus->cpus[n].props.cluster_id > cluster_id);
                 cluster_id = cpus->cpus[n].props.cluster_id;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ae65ba2c929c..5f0517423d8f 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2687,7 +2687,7 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
         ms->possible_cpus->cpus[n].arch_id =
             virt_cpu_mp_affinity(vms, n);
 
-        assert(!mc->smp_props.dies_supported);
+        assert(!mc->topo_props.dies_supported);
         ms->possible_cpus->cpus[n].props.has_socket_id = true;
         ms->possible_cpus->cpus[n].props.socket_id =
             n / (smp_clusters * smp_cores * smp_threads);
@@ -3029,7 +3029,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
     hc->unplug_request = virt_machine_device_unplug_request_cb;
     hc->unplug = virt_machine_device_unplug_cb;
     mc->nvdimm_supported = true;
-    mc->smp_props.clusters_supported = true;
+    mc->topo_props.clusters_supported = true;
     mc->auto_enable_numa_with_memhp = true;
     mc->auto_enable_numa_with_memdev = true;
     mc->default_ram_id = "mach-virt.ram";
@@ -3280,7 +3280,7 @@ static void virt_machine_6_1_options(MachineClass *mc)
 
     virt_machine_6_2_options(mc);
     compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
-    mc->smp_props.prefer_sockets = true;
+    mc->topo_props.prefer_sockets = true;
     vmc->no_cpu_topology = true;
 
     /* qemu ITS was introduced with 6.2 */
diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index f9ab08a1252e..8540b473b8db 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -187,11 +187,11 @@ static char *cpu_hierarchy_to_string(MachineState *ms)
 
     g_string_append_printf(s, "sockets (%u)", machine_topo_get_sockets(ms));
 
-    if (mc->smp_props.dies_supported) {
+    if (mc->topo_props.dies_supported) {
         g_string_append_printf(s, " * dies (%u)", machine_topo_get_dies(ms));
     }
 
-    if (mc->smp_props.clusters_supported) {
+    if (mc->topo_props.clusters_supported) {
         g_string_append_printf(s, " * clusters (%u)",
                                machine_topo_get_clusters(ms));
     }
@@ -256,11 +256,11 @@ void machine_parse_smp_config(MachineState *ms,
      * If not supported by the machine, a topology parameter must be
      * omitted or specified equal to 1.
      */
-    if (!mc->smp_props.dies_supported && dies > 1) {
+    if (!mc->topo_props.dies_supported && dies > 1) {
         error_setg(errp, "dies not supported by this machine's CPU topology");
         return;
     }
-    if (!mc->smp_props.clusters_supported && clusters > 1) {
+    if (!mc->topo_props.clusters_supported && clusters > 1) {
         error_setg(errp, "clusters not supported by this machine's CPU topology");
         return;
     }
@@ -276,7 +276,7 @@ void machine_parse_smp_config(MachineState *ms,
     } else {
         maxcpus = maxcpus > 0 ? maxcpus : cpus;
 
-        if (mc->smp_props.prefer_sockets) {
+        if (mc->topo_props.prefer_sockets) {
             /* prefer sockets over cores before 6.2 */
             if (sockets == 0) {
                 cores = cores > 0 ? cores : 1;
@@ -315,7 +315,7 @@ void machine_parse_smp_config(MachineState *ms,
     ms->topo.smp.cores = cores;
     ms->topo.smp.threads = threads;
 
-    mc->smp_props.has_clusters = config->has_clusters;
+    mc->topo_props.has_clusters = config->has_clusters;
 
     /* sanity-check of the computed topology */
     if (sockets * dies * clusters * cores * threads != maxcpus) {
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index c329df56ebd2..9a7ee0affd87 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1928,8 +1928,8 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     hc->unplug = pc_machine_device_unplug_cb;
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = true;
-    mc->smp_props.dies_supported = true;
-    mc->smp_props.clusters_supported = true;
+    mc->topo_props.dies_supported = true;
+    mc->topo_props.clusters_supported = true;
     mc->default_ram_id = "pc.ram";
 
     object_class_property_add(oc, PC_MACHINE_MAX_RAM_BELOW_4G, "size",
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index df64dd8dcc66..ce725b3d2927 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -513,7 +513,7 @@ static void pc_i440fx_6_1_machine_options(MachineClass *m)
     pc_i440fx_6_2_machine_options(m);
     compat_props_add(m->compat_props, hw_compat_6_1, hw_compat_6_1_len);
     compat_props_add(m->compat_props, pc_compat_6_1, pc_compat_6_1_len);
-    m->smp_props.prefer_sockets = true;
+    m->topo_props.prefer_sockets = true;
 }
 
 DEFINE_I440FX_MACHINE(v6_1, "pc-i440fx-6.1", NULL,
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 66cd718b7081..94352f2b8ea8 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -432,7 +432,7 @@ static void pc_q35_6_1_machine_options(MachineClass *m)
     pc_q35_6_2_machine_options(m);
     compat_props_add(m->compat_props, hw_compat_6_1, hw_compat_6_1_len);
     compat_props_add(m->compat_props, pc_compat_6_1, pc_compat_6_1_len);
-    m->smp_props.prefer_sockets = true;
+    m->topo_props.prefer_sockets = true;
 }
 
 DEFINE_Q35_MACHINE(v6_1, "pc-q35-6.1", NULL,
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 6faad75afd70..b2e2420af870 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -4801,7 +4801,7 @@ static void spapr_machine_6_1_class_options(MachineClass *mc)
     spapr_machine_6_2_class_options(mc);
     compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
     smc->pre_6_2_numa_affinity = true;
-    mc->smp_props.prefer_sockets = true;
+    mc->topo_props.prefer_sockets = true;
 }
 
 DEFINE_SPAPR_MACHINE(6_1, "6.1", false);
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 9b5020a0d395..28cc3c048d78 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -912,7 +912,7 @@ static void ccw_machine_6_1_class_options(MachineClass *mc)
 {
     ccw_machine_6_2_class_options(mc);
     compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
-    mc->smp_props.prefer_sockets = true;
+    mc->topo_props.prefer_sockets = true;
 }
 DEFINE_CCW_MACHINE(6_1, "6.1", false);
 
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 17be3485e823..48444ab7275b 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -288,7 +288,7 @@ struct MachineClass {
     bool nvdimm_supported;
     bool numa_mem_supported;
     bool auto_enable_numa;
-    TopoCompatProps smp_props;
+    TopoCompatProps topo_props;
     const char *default_ram_id;
 
     HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
diff --git a/tests/unit/test-smp-parse.c b/tests/unit/test-smp-parse.c
index d6816f109084..b6dd5fafd3dd 100644
--- a/tests/unit/test-smp-parse.c
+++ b/tests/unit/test-smp-parse.c
@@ -500,11 +500,11 @@ static void smp_parse_test(MachineState *ms, SMPTestData *data, bool is_valid)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
 
-    mc->smp_props.prefer_sockets = true;
+    mc->topo_props.prefer_sockets = true;
     check_parse(ms, &data->config, &data->expect_prefer_sockets,
                 data->expect_error, is_valid);
 
-    mc->smp_props.prefer_sockets = false;
+    mc->topo_props.prefer_sockets = false;
     check_parse(ms, &data->config, &data->expect_prefer_cores,
                 data->expect_error, is_valid);
 }
@@ -512,12 +512,12 @@ static void smp_parse_test(MachineState *ms, SMPTestData *data, bool is_valid)
 /* The parsed results of the unsupported parameters should be 1 */
 static void unsupported_params_init(const MachineClass *mc, SMPTestData *data)
 {
-    if (!mc->smp_props.dies_supported) {
+    if (!mc->topo_props.dies_supported) {
         data->expect_prefer_sockets.smp.dies = 1;
         data->expect_prefer_cores.smp.dies = 1;
     }
 
-    if (!mc->smp_props.clusters_supported) {
+    if (!mc->topo_props.clusters_supported) {
         data->expect_prefer_sockets.smp.clusters = 1;
         data->expect_prefer_cores.smp.clusters = 1;
     }
@@ -553,14 +553,14 @@ static void machine_with_dies_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
 
-    mc->smp_props.dies_supported = true;
+    mc->topo_props.dies_supported = true;
 }
 
 static void machine_with_clusters_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
 
-    mc->smp_props.clusters_supported = true;
+    mc->topo_props.clusters_supported = true;
 }
 
 static void test_generic_valid(const void *opaque)
-- 
2.34.1

[RFC 44/52] machine: Add "-hybrid" parsing rule

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The "-hybrid" format is like:
-hybrid socket,sockets=n
-hybrid die,dies=n
-hybrid cluster,clusters=n
-hybrid core,cores=n,type=core_type[,threads=threads]
        [,clusterid=cluster]

For example:
-hybrid socket,sockets=1
-hybrid die,dies=1
-hybrid cluster,clusters=4
-hybrid core,cores=1,coretype="core",threads=2,clusterid=0-1
-hybrid core,cores=1,coretype="core",threads=2,clusterid=0-1,clusterid=3
-hybrid core,cores=4,coretype="atom",threads=1,clusterid=3

In order to support "-hybrid core" can be inserted into the specified
cluster, introduce HybridCorePack to pack HybridCore and help to parse
"-hybrid core" command. HybridCorePack will be used to build core level
topology later.

Since hybrid cpu topology allocates memory by g_malloc0(), free the
memory when machine finalizes.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c        | 259 ++++++++++++++++++++++++++++++++++
 hw/core/machine.c             |   4 +
 include/hw/boards.h           |   4 +
 include/hw/cpu/cpu-topology.h |  23 +++
 4 files changed, 290 insertions(+)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 8540b473b8db..6e4a9ec1495d 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -366,3 +366,262 @@ int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
 
     return -1;
 }
+
+static void parse_hybrid_socket(MachineState *ms,
+                                const HybridSocketOptions *config,
+                                Error **errp)
+{
+    if (ms->topo.hybrid.sockets != 1) {
+        error_setg(errp, "socket cannot be set repeatedly");
+        return;
+    }
+
+    /*
+     * Since sockets = 0 is deprecated CPU topology for -smp,
+     * now -hybrid returns error directly.
+     */
+    if (!config->sockets) {
+        error_setg(errp, "Invalid sockets. sockets mustn't be zero");
+        return;
+    }
+
+    ms->topo.hybrid.sockets = config->sockets;
+
+    /*
+     * Even though we require that the "socket" must be set before the "core",
+     * in order to simplify the code logic, we do not check the order of the
+     * "socket" subcommand. Instead, multiply the "sockets" for the cpus and
+     * max_cpus here as a compensation measure.
+     */
+    ms->topo.cpus *= ms->topo.hybrid.sockets;
+    ms->topo.max_cpus *= ms->topo.hybrid.sockets;
+}
+
+static void parse_hybrid_die(MachineState *ms,
+                             const HybridDieOptions *config, Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (ms->topo.hybrid.dies != 1) {
+        error_setg(errp, "die cannot be set repeatedly");
+        return;
+    }
+
+    /*
+     * Since dies = 0 is deprecated CPU topology for -smp,
+     * now -hybrid returns error directly.
+     */
+    if (!config->dies) {
+        error_setg(errp, "Invalid dies. dies mustn't be zero");
+        return;
+    }
+    if (!mc->topo_props.dies_supported && config->dies > 1) {
+        error_setg(errp, "dies not supported by this machine's CPU topology");
+        return;
+    }
+
+    ms->topo.hybrid.dies = config->dies;
+
+    /*
+     * Even though we require that the "die" must be set before the "core",
+     * in order to simplify the code logic, we do not check the order of the
+     * "die" subcommand. Instead, multiply the "dies" for the cpus and
+     * max_cpus here as a compensation measure.
+     */
+    ms->topo.cpus *= ms->topo.hybrid.dies;
+    ms->topo.max_cpus *= ms->topo.hybrid.dies;
+}
+
+static void parse_hybrid_cluster(MachineState *ms,
+                                 const HybridClusterOptions *config,
+                                 Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (ms->topo.hybrid.cluster_list != NULL) {
+        error_setg(errp, "The cluster configuration "
+                   "clusters=%" PRIu32 " should be provided before"
+                   "core configuration", config->clusters);
+        return;
+    }
+
+    /*
+     * Since clusters = 0 is deprecated CPU topology for -smp,
+     * now -hybrid returns error directly.
+     */
+    if (!config->clusters) {
+        error_setg(errp, "Invalid clusters. clusters mustn't be zero");
+        return;
+    }
+
+    if (!mc->topo_props.clusters_supported && config->clusters > 1) {
+        error_setg(errp, "clusters not supported by this "
+                   "machine's CPU topology");
+        return;
+    }
+
+    mc->topo_props.has_clusters = true;
+
+    ms->topo.hybrid.clusters = config->clusters;
+    ms->topo.hybrid.cluster_list = g_malloc0(sizeof(HybridCluster) *
+                                             ms->topo.hybrid.clusters);
+
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        QSLIST_INIT(&ms->topo.hybrid.cluster_list[i].core_pack_list);
+    }
+}
+
+static void insert_core_into_cluster(MachineState *ms,
+                                     HybridCluster *cluster,
+                                     const HybridCoreOptions *config,
+                                     Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    HybridCorePack *core_pack;
+    int ret;
+
+    /*
+     * Since cores = 0 or threads = 0 is deprecated CPU topology for
+     * -smp, now -hybrid return error directly.
+     */
+    if (!config->cores) {
+        error_setg(errp, "Invalid cores. cores=%u mustn't be zero",
+                   config->cores);
+        return;
+    }
+    if (config->has_threads && !config->threads) {
+        error_setg(errp, "Invalid threads. threads=%u mustn't be zero",
+                   config->threads);
+        return;
+    }
+
+    core_pack = g_malloc0(sizeof(*core_pack));
+    core_pack->core_num = config->cores;
+    core_pack->core.threads = config->has_threads ? config->threads : 1;
+
+    ret = mc->core_type(ms, config->coretype);
+    if (!ret) {
+        error_setg(errp, "Invalid coretype=%s", config->coretype);
+    }
+    core_pack->core.core_type = ret;
+
+    QSLIST_INSERT_HEAD(&cluster->core_pack_list, core_pack, node);
+    cluster->cores += core_pack->core_num;
+
+    /*
+     * Note here we just multiply by "dies" and "sockets" other that
+     * "clusters".
+     *
+     * This function caculates cluster's topology one by one, and different
+     * cluster in the same die may have different topology. So we can't
+     * multiply by "clusters". But for current hybrid topology support, we
+     * suppose sockets and dies are same. So "sockets" and "dies" should be
+     * considerred.
+     */
+    ms->topo.cpus += core_pack->core.threads * core_pack->core_num *
+                     ms->topo.hybrid.dies * ms->topo.hybrid.sockets;
+    /*
+     * Because of the "-hybrid" format limitation, before all the cores are
+     * configured, the cpus and max_cpus cannot be obtained. At the same
+     * time, because of heterogeneous, if user wants to specify the online
+     * cpus, user must subdivide it into a specific core. So currently,
+     * temporarily limit max_cpus to be consistent with cpus.
+     *
+     * TODO: Consider adding more complete online cpus configuration support
+     * in the future.
+     */
+    ms->topo.max_cpus = ms->topo.cpus;
+}
+
+static void parse_hybrid_core(MachineState *ms,
+                              const HybridCoreOptions *config, Error **errp)
+{
+    if (!ms->topo.hybrid.cluster_list) {
+        if (config->has_clusterid) {
+            error_setg(errp, "The core configuration clusterid "
+                       "should be provided after"
+                       "cluster configuration");
+            return;
+        }
+        /* Set 1 cluster per die by default. */
+        ms->topo.hybrid.clusters = 1;
+        ms->topo.hybrid.cluster_list = g_malloc0(sizeof(HybridCluster) *
+                                                 ms->topo.hybrid.clusters);
+        QSLIST_INIT(&ms->topo.hybrid.cluster_list[0].core_pack_list);
+    }
+
+    if (config->has_clusterid) {
+        for (uint32List *clusterid = config->clusterid; clusterid;
+             clusterid = clusterid->next) {
+            if (clusterid->value >= ms->topo.hybrid.clusters) {
+                error_setg(errp, "Invalid clusterid. "
+                           "clusterid=%" PRIu32 " is out of range "
+                           "(only %" PRIu32 "clusters)",
+                           clusterid->value, ms->topo.hybrid.clusters);
+                return;
+            }
+            insert_core_into_cluster(ms,
+                &ms->topo.hybrid.cluster_list[clusterid->value], config, errp);
+        }
+    } else {
+        for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+            insert_core_into_cluster(ms,
+                &ms->topo.hybrid.cluster_list[i], config, errp);
+        }
+    }
+}
+
+void set_hybrid_options(MachineState *ms,
+                        const HybridOptions *config,
+                        Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (!mc->topo_props.hybrid_supported) {
+        error_setg(errp, "hybrid topology not supported "
+                   "by this machine's CPU");
+        return;
+    }
+
+    switch (config->type) {
+    case HYBRID_OPTIONS_TYPE_SOCKET:
+        parse_hybrid_socket(ms, &config->u.socket, errp);
+        break;
+    case HYBRID_OPTIONS_TYPE_DIE:
+        parse_hybrid_die(ms, &config->u.die, errp);
+        break;
+    case HYBRID_OPTIONS_TYPE_CLUSTER:
+        parse_hybrid_cluster(ms, &config->u.cluster, errp);
+        break;
+    case HYBRID_OPTIONS_TYPE_CORE:
+        parse_hybrid_core(ms, &config->u.core, errp);
+        break;
+    default:
+        abort();
+    }
+}
+
+void machine_free_hybrid_topology(MachineState *ms)
+{
+    HybridCluster *cluster;
+    HybridCorePack *core_pack;
+    HybridCorePack *tmp;
+
+    if (ms->topo.hybrid.clusters) {
+        for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+            cluster = &ms->topo.hybrid.cluster_list[i];
+
+            /*
+             * TODO: Temporarily free core_pack_list here. When the
+             * building of core_list array is supported, it will be
+             * freeed there.
+             */
+            QSLIST_FOREACH_SAFE(core_pack, &cluster->core_pack_list,
+                                node, tmp) {
+                QSLIST_REMOVE_HEAD(&cluster->core_pack_list, node);
+                g_free(core_pack);
+            }
+        }
+        g_free(ms->topo.hybrid.cluster_list);
+    }
+}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index acc32b3be5f6..f2c6aac4ef94 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1122,6 +1122,10 @@ static void machine_finalize(Object *obj)
     g_free(ms->device_memory);
     g_free(ms->nvdimms_state);
     g_free(ms->numa_state);
+
+    if (!machine_topo_is_smp(ms)) {
+        machine_free_hybrid_topology(ms);
+    }
 }
 
 bool machine_usb(MachineState *machine)
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 48444ab7275b..09b93c17a245 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -37,6 +37,10 @@ void machine_set_cpu_numa_node(MachineState *machine,
 void machine_parse_smp_config(MachineState *ms,
                               const SMPConfiguration *config, Error **errp);
 int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
+void set_hybrid_options(MachineState *ms,
+                        const HybridOptions *config,
+                        Error **errp);
+void machine_free_hybrid_topology(MachineState *ms);
 
 /**
  * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
index 829f98d3a73f..eef752d5168d 100644
--- a/include/hw/cpu/cpu-topology.h
+++ b/include/hw/cpu/cpu-topology.h
@@ -52,15 +52,38 @@ typedef struct HybridCore {
     unsigned int core_type;
 } HybridCore;
 
+/**
+ * HybridCorePack - Wrapper of HybridCore to pack the same
+ *                  cores in one cluster.
+ *
+ * HybridCorePack is used for `-hybrid` parsing and will help to build
+ * core_list array.
+ *
+ * @core_num: the number of threads in one core.
+ * @core: the specific core information of current core pack.
+ * @node: the singly-linked list node of current core pack. This node
+ *        is added to the core_pack_list of a cluster which the current
+ *        core pack belongs to.
+ */
+typedef struct HybridCorePack {
+    unsigned int core_num;
+    HybridCore core;
+    QSLIST_ENTRY(HybridCorePack) node;
+} HybridCorePack;
+
 /**
  * HybridCluster - hybrid cluster topology defination:
  * @cores: the number of cores in current cluster.
  * @core_list: the array includes all the cores that belong to current
  *             cluster.
+ * @core_pack_list: the list that links all the core packs that belong to
+ *                  current cluster. It is used for `-hybrid` parsing and
+ *                  will help to build core_list array.
  */
 typedef struct HybridCluster {
     unsigned int cores;
     HybridCore *core_list;
+    QSLIST_HEAD(, HybridCorePack) core_pack_list;
 } HybridCluster;
 
 /**
-- 
2.34.1

[RFC 45/52] hw/machine: Add hybrid cpu topology validation

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Because of the "-hybrid" format, in the process of parsing the command
line, we cannot know exactly the complete topology details until all the
hybrid commands have been parsed, so we cannot verify the hybrid
topology during the parsing process.

Thus validate the hybrid topology before machine running. At this moment,
the hybrid topology must have been parsed.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 126 +++++++++++++++++++++++++++++++++++++++++
 hw/core/machine.c      |   4 ++
 include/hw/boards.h    |   1 +
 3 files changed, 131 insertions(+)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 6e4a9ec1495d..2cf71cc466aa 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -625,3 +625,129 @@ void machine_free_hybrid_topology(MachineState *ms)
         g_free(ms->topo.hybrid.cluster_list);
     }
 }
+
+/*
+ * Report information of a machine's supported CPU hybrid topology
+ * hierarchy. Topology members will be ordered in the string as:
+ * sockets (%u) * dies (%u) * ( cluster0:[ core0:[ cores (type: %s, %u) *
+ * threads (%u) ] + ... + core_n:[ cores (type: %s, %u) * threads (%u) ] ] +
+ * ... + cluster_m:[ core0:[ cores (type: %s, %u) * threads (%u) ] + ... +
+ * core_m:[ cores (type: %s, %u) * threads (%u)) ] ].
+ */
+static char *hybrid_cpu_hierarchy_to_string(MachineState *ms)
+{
+    HybridCluster *cluster;
+    HybridCorePack *core_pack;
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    GString *s = g_string_new(NULL);
+
+    g_string_append_printf(s, "sockets (%u)", ms->topo.hybrid.sockets);
+
+    if (mc->topo_props.dies_supported) {
+        g_string_append_printf(s, " * dies (%u)", ms->topo.hybrid.dies);
+    }
+
+    if (ms->topo.hybrid.cluster_list != NULL) {
+        g_string_append_printf(s, " * ( ");
+        for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+            if (i) {
+                g_string_append_printf(s, " + ");
+            }
+            if (mc->topo_props.clusters_supported) {
+                g_string_append_printf(s, "cluster%u ", i);
+            }
+            if (ms->topo.hybrid.cluster_list[i].cores) {
+                int core_idx = 0;
+                cluster = &ms->topo.hybrid.cluster_list[i];
+
+                QSLIST_FOREACH(core_pack, &cluster->core_pack_list, node) {
+                    if (!core_idx) {
+                        g_string_append_printf(
+                            s, "%s ",
+                            mc->topo_props.clusters_supported ?
+                            ":[" : "[");
+                    } else {
+                        g_string_append_printf(s, " + ");
+                    }
+                    g_string_append_printf(
+                        s, "core%u:[ cores (type: %u, "
+                        "num: %u) * threads (%u) ]",
+                        core_idx++, core_pack->core.core_type,
+                        core_pack->core_num, core_pack->core.threads);
+                }
+                if (mc->topo_props.clusters_supported) {
+                    g_string_append_printf(s, " ] ");
+                }
+            }
+        }
+        g_string_append_printf(s, " )");
+    }
+
+    return g_string_free(s, false);
+}
+
+void machine_validate_hybrid_topology(MachineState *ms, Error **errp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+    HybridCluster *cluster;
+
+    if (ms->topo.max_cpus == 0) {
+        /*
+         * Cores are not set, and maybe clusters are also not set.
+         * At present, since the coretype cannot be omitted, the core
+         * level must be specified.
+         *
+         * TODO: Support coretype and core level can be omitted.
+         */
+        g_autofree char *topo_msg = hybrid_cpu_hierarchy_to_string(ms);
+        error_setg(errp, "Invalid hybrid CPU topology: Lack "
+                   "of core configuration, unable to generate "
+                   "valid topology: %s", topo_msg);
+        exit(1);
+    } else {
+        /*
+         * Though currently max_cpus equals to cpus, still check cpus here
+         * as the complete code logic.
+         *
+         * TODO: Consider adding more complete online cpus configuration
+         * support in the future.
+         *
+         * Since cpus = 0 is deprecated CPU topology for smp, now -hybrid
+         * returns error directly.
+         */
+        if (ms->topo.cpus == 0) {
+            error_setg(errp, "Must set at least one cpu online");
+            exit(1);
+        }
+    }
+
+    /* Each cluster need at least one core. */
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        cluster = &ms->topo.hybrid.cluster_list[i];
+
+        if (!cluster->cores) {
+            g_autofree char *topo_msg = hybrid_cpu_hierarchy_to_string(ms);
+            error_setg(errp, "Invalid hybrid CPU topology: no core is "
+                       "specified for cluster (id: %d), unable to "
+                       "generate valid topology: %s",
+                       i, topo_msg);
+            exit(1);
+        }
+    }
+
+    if (ms->topo.cpus < mc->min_cpus) {
+        g_autofree char *topo_msg = hybrid_cpu_hierarchy_to_string(ms);
+        error_setg(errp, "Invalid Hybrid CPUs %d. The min CPUs "
+                   "supported by machine '%s' is %d. Topology: %s",
+                   ms->topo.cpus, mc->name, mc->min_cpus, topo_msg);
+        exit(1);
+    }
+
+    if (ms->topo.max_cpus > mc->max_cpus) {
+        g_autofree char *topo_msg = hybrid_cpu_hierarchy_to_string(ms);
+        error_setg(errp, "Invalid Hybrid CPUs %d. The max CPUs "
+                   "supported by machine '%s' is %d. Topology: %s",
+                   ms->topo.max_cpus, mc->name, mc->max_cpus, topo_msg);
+        exit(1);
+    }
+}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index f2c6aac4ef94..630934317e3c 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1327,6 +1327,10 @@ void machine_run_board_init(MachineState *machine, const char *mem_path, Error *
         }
     }
 
+    if (!machine_topo_is_smp(machine)) {
+        machine_validate_hybrid_topology(machine, errp);
+    }
+
     if (machine->memdev) {
         ram_addr_t backend_size = object_property_get_uint(OBJECT(machine->memdev),
                                                            "size",  &error_abort);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 09b93c17a245..9156982e4de6 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -41,6 +41,7 @@ void set_hybrid_options(MachineState *ms,
                         const HybridOptions *config,
                         Error **errp);
 void machine_free_hybrid_topology(MachineState *ms);
+void machine_validate_hybrid_topology(MachineState *ms, Error **errp);
 
 /**
  * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
-- 
2.34.1

[RFC 46/52] hw/machine: build core level hybrid topology form HybridCorePack

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

After verifying the hybrid topology, we need to convert the topology
structure of the core level from a single linked list to an array,
so that we can make full use of the core-id to quickly access the
corresponding core.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 48 ++++++++++++++++++++++++++++++++----------
 hw/core/machine.c      |  1 +
 include/hw/boards.h    |  1 +
 3 files changed, 39 insertions(+), 11 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 2cf71cc466aa..ebd2c40396a2 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -604,22 +604,13 @@ void set_hybrid_options(MachineState *ms,
 void machine_free_hybrid_topology(MachineState *ms)
 {
     HybridCluster *cluster;
-    HybridCorePack *core_pack;
-    HybridCorePack *tmp;
 
     if (ms->topo.hybrid.clusters) {
         for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
             cluster = &ms->topo.hybrid.cluster_list[i];
 
-            /*
-             * TODO: Temporarily free core_pack_list here. When the
-             * building of core_list array is supported, it will be
-             * freeed there.
-             */
-            QSLIST_FOREACH_SAFE(core_pack, &cluster->core_pack_list,
-                                node, tmp) {
-                QSLIST_REMOVE_HEAD(&cluster->core_pack_list, node);
-                g_free(core_pack);
+            if (cluster->core_list != NULL) {
+                g_free(cluster->core_list);
             }
         }
         g_free(ms->topo.hybrid.cluster_list);
@@ -751,3 +742,38 @@ void machine_validate_hybrid_topology(MachineState *ms, Error **errp)
         exit(1);
     }
 }
+
+void machine_consolidate_hybrid_topology(MachineState *ms)
+{
+    HybridCluster *cluster;
+    HybridCorePack *core_pack;
+    HybridCorePack *tmp;
+    int core_num;
+
+    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
+        cluster = &ms->topo.hybrid.cluster_list[i];
+        cluster->core_list = g_malloc0(sizeof(HybridCore) * cluster->cores);
+
+        core_num = cluster->cores;
+        QSLIST_FOREACH_SAFE(core_pack, &cluster->core_pack_list, node, tmp) {
+            QSLIST_REMOVE_HEAD(&cluster->core_pack_list, node);
+
+            /*
+             * Here we add cores in "reverse order" because core_pack_list uses
+             * "QSLIST_INSERT_HEAD()" to collect the cores in the "-hybrid"
+             * command line in reverse order.
+             *
+             * The "reverse order" here can ensure that the growth of core-id
+             * is consistent with the order of adding cores in "-hybrid".
+             */
+            for (int j = 0; j < core_pack->core_num; j++) {
+                cluster->core_list[core_num - j - 1].core_type =
+                    core_pack->core.core_type;
+                cluster->core_list[core_num - j - 1].threads =
+                    core_pack->core.threads;
+            }
+            core_num -= core_pack->core_num;
+            g_free(core_pack);
+        }
+    }
+}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 630934317e3c..08a0c117ce1b 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1329,6 +1329,7 @@ void machine_run_board_init(MachineState *machine, const char *mem_path, Error *
 
     if (!machine_topo_is_smp(machine)) {
         machine_validate_hybrid_topology(machine, errp);
+        machine_consolidate_hybrid_topology(machine);
     }
 
     if (machine->memdev) {
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 9156982e4de6..0f865c21e2a8 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -42,6 +42,7 @@ void set_hybrid_options(MachineState *ms,
                         Error **errp);
 void machine_free_hybrid_topology(MachineState *ms);
 void machine_validate_hybrid_topology(MachineState *ms, Error **errp);
+void machine_consolidate_hybrid_topology(MachineState *ms);
 
 /**
  * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
-- 
2.34.1

[RFC 47/52] hw/machine: Use opts_visitor to parse hybrid topo

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Because of the "-hybrid" format, it cannot be defined as the machine
property to use input_visitor to parse as what "-smp" did.

So here we use opts_visitor to parse hybrid topology.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 36 +++++++++++++++++++++++++++++++++---
 include/hw/boards.h    |  4 +---
 2 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index ebd2c40396a2..9e37de04ce75 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -20,6 +20,8 @@
 #include "qemu/osdep.h"
 #include "hw/boards.h"
 #include "qapi/error.h"
+#include "qapi/opts-visitor.h"
+#include "qapi/qapi-visit-machine.h"
 
 unsigned int machine_topo_get_sockets(const MachineState *ms)
 {
@@ -571,9 +573,9 @@ static void parse_hybrid_core(MachineState *ms,
     }
 }
 
-void set_hybrid_options(MachineState *ms,
-                        const HybridOptions *config,
-                        Error **errp)
+static void set_hybrid_options(MachineState *ms,
+                               const HybridOptions *config,
+                               Error **errp)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
 
@@ -601,6 +603,34 @@ void set_hybrid_options(MachineState *ms,
     }
 }
 
+static int parse_hybrid(void *opaque, QemuOpts *opts, Error **errp)
+{
+    g_autoptr(HybridOptions) config = NULL;
+    MachineState *ms = MACHINE(opaque);
+    Error *err = NULL;
+    Visitor *v = opts_visitor_new(opts);
+
+    visit_type_HybridOptions(v, NULL, &config, errp);
+    visit_free(v);
+    if (!config) {
+        return -1;
+    }
+
+    set_hybrid_options(ms, config, &err);
+
+    if (err) {
+        error_propagate(errp, err);
+        return -1;
+    }
+
+    return 0;
+}
+
+void parse_hybrid_opts(MachineState *ms)
+{
+    qemu_opts_foreach(qemu_find_opts("hybrid"), parse_hybrid, ms, &error_fatal);
+}
+
 void machine_free_hybrid_topology(MachineState *ms)
 {
     HybridCluster *cluster;
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 0f865c21e2a8..0395990139bc 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -37,9 +37,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
 void machine_parse_smp_config(MachineState *ms,
                               const SMPConfiguration *config, Error **errp);
 int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
-void set_hybrid_options(MachineState *ms,
-                        const HybridOptions *config,
-                        Error **errp);
+void parse_hybrid_opts(MachineState *ms);
 void machine_free_hybrid_topology(MachineState *ms);
 void machine_validate_hybrid_topology(MachineState *ms, Error **errp);
 void machine_consolidate_hybrid_topology(MachineState *ms);
-- 
2.34.1

[RFC 48/52] machine: Support "-hybrid" command

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Add "-hybrid" command parsing.

And now we have reason to move MachineState.topo initialization to its
original place [1], because we can't know whcih topologies should be
initialized before collecting all commands.

If "-hybrid" is set, initialize MachineState.topo as hybrid topology.
Otherwise, use smp topology as default.

[1]: 8b0e484 (machine: move SMP initialization from vl.c)

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-topo.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 hw/core/machine.c      | 11 -----------
 include/hw/boards.h    |  1 +
 qemu-options.hx        |  7 +++++++
 softmmu/vl.c           | 33 +++++++++++++++++++++++++++++++++
 5 files changed, 81 insertions(+), 12 deletions(-)

diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index 9e37de04ce75..f38b8c683026 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -177,6 +177,33 @@ unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
     return ms->topo.max_cpus / sockets;
 }
 
+void machine_init_topology_default(MachineState *ms, bool smp)
+{
+    MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (smp) {
+        /* default to mc->default_cpus */
+        ms->topo.cpus = mc->default_cpus;
+        ms->topo.max_cpus = mc->default_cpus;
+
+        ms->topo.topo_type = CPU_TOPO_TYPE_SMP;
+        ms->topo.smp.sockets = 1;
+        ms->topo.smp.dies = 1;
+        ms->topo.smp.clusters = 1;
+        ms->topo.smp.cores = 1;
+        ms->topo.smp.threads = 1;
+    } else {
+        ms->topo.cpus = 0;
+        ms->topo.max_cpus = 0;
+
+        ms->topo.topo_type = CPU_TOPO_TYPE_HYBRID;
+        ms->topo.hybrid.sockets = 1;
+        ms->topo.hybrid.dies = 1;
+        ms->topo.hybrid.clusters = 1;
+        ms->topo.hybrid.cluster_list = NULL;
+    }
+}
+
 /*
  * Report information of a machine's supported CPU topology hierarchy.
  * Topology members will be ordered from the largest to the smallest
@@ -231,6 +258,12 @@ void machine_parse_smp_config(MachineState *ms,
                               const SMPConfiguration *config, Error **errp)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
+
+    if (!machine_topo_is_smp(ms)) {
+        error_setg(errp, "Cannot set smp and hybrid at the same time");
+        return;
+    }
+
     unsigned cpus    = config->has_cpus ? config->cpus : 0;
     unsigned sockets = config->has_sockets ? config->sockets : 0;
     unsigned dies    = config->has_dies ? config->dies : 0;
@@ -608,8 +641,14 @@ static int parse_hybrid(void *opaque, QemuOpts *opts, Error **errp)
     g_autoptr(HybridOptions) config = NULL;
     MachineState *ms = MACHINE(opaque);
     Error *err = NULL;
-    Visitor *v = opts_visitor_new(opts);
+    Visitor *v;
+
+    if (machine_topo_is_smp(ms)) {
+        error_setg(errp, "Cannot set hybrid and smp at the same time");
+        return -1;
+    }
 
+    v = opts_visitor_new(opts);
     visit_type_HybridOptions(v, NULL, &config, errp);
     visit_free(v);
     if (!config) {
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 08a0c117ce1b..212749f984d6 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1093,17 +1093,6 @@ static void machine_initfn(Object *obj)
                                         "Table (HMAT)");
     }
 
-    /* default to mc->default_cpus */
-    ms->topo.cpus = mc->default_cpus;
-    ms->topo.max_cpus = mc->default_cpus;
-
-    ms->topo.topo_type = CPU_TOPO_TYPE_SMP;
-    ms->topo.smp.sockets = 1;
-    ms->topo.smp.dies = 1;
-    ms->topo.smp.clusters = 1;
-    ms->topo.smp.cores = 1;
-    ms->topo.smp.threads = 1;
-
     machine_copy_boot_config(ms, &(BootConfiguration){ 0 });
 }
 
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 0395990139bc..c93bb1206244 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -41,6 +41,7 @@ void parse_hybrid_opts(MachineState *ms);
 void machine_free_hybrid_topology(MachineState *ms);
 void machine_validate_hybrid_topology(MachineState *ms, Error **errp);
 void machine_consolidate_hybrid_topology(MachineState *ms);
+void machine_init_topology_default(MachineState *ms, bool smp);
 
 /**
  * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
diff --git a/qemu-options.hx b/qemu-options.hx
index 3caf9da4c3af..8987972a8d5f 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -5605,6 +5605,13 @@ SRST
             (qemu) qom-set /objects/iothread1 poll-max-ns 100000
 ERST
 
+DEF("hybrid", HAS_ARG, QEMU_OPTION_hybrid,
+    "-hybrid socket,sockets=n"
+    "-hybrid die,dies=n"
+    "-hybrid cluster,clusters=n"
+    "-hybrid core,cores=n,coretype=core_type[,threads=threads][,clusterid=cluster]",
+    QEMU_ARCH_ALL)
+
 
 HXCOMM This is the last statement. Insert new options before this line!
 
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 0547ad390f52..ce5e021006f8 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -500,6 +500,13 @@ static QemuOptsList qemu_action_opts = {
     },
 };
 
+static QemuOptsList qemu_hybrid_opts = {
+    .name = "hybrid",
+    .implied_opt_name = "type",
+    .head = QTAILQ_HEAD_INITIALIZER(qemu_hybrid_opts.head),
+    .desc = { { 0 } } /* validated with OptsVisitor */
+};
+
 const char *qemu_get_vm_name(void)
 {
     return qemu_name;
@@ -2010,6 +2017,17 @@ static void parse_memory_options(void)
     loc_pop(&loc);
 }
 
+static void qemu_machine_init_topology(MachineState *machine)
+{
+    bool is_smp = true;
+    QemuOptsList *list = qemu_find_opts("hybrid");
+
+    if (!QTAILQ_EMPTY(&list->head)) {
+        is_smp = false;
+    }
+    machine_init_topology_default(machine, is_smp);
+}
+
 static void qemu_create_machine(QDict *qdict)
 {
     MachineClass *machine_class = select_machine(qdict, &error_fatal);
@@ -2038,6 +2056,12 @@ static void qemu_create_machine(QDict *qdict)
         qemu_set_hw_version(machine_class->hw_version);
     }
 
+    /*
+     * Initialize cpu topology. If hybrid is set, initialize as hybrid
+     * topology. Otherwise, initialize as smp topology.
+     */
+    qemu_machine_init_topology(current_machine);
+
     /*
      * Get the default machine options from the machine if it is not already
      * specified either by the configuration file or by the command line.
@@ -2667,6 +2691,7 @@ void qemu_init(int argc, char **argv)
     qemu_add_opts(&qemu_semihosting_config_opts);
     qemu_add_opts(&qemu_fw_cfg_opts);
     qemu_add_opts(&qemu_action_opts);
+    qemu_add_opts(&qemu_hybrid_opts);
     module_call_init(MODULE_INIT_OPTS);
 
     error_init(argv[0]);
@@ -3489,6 +3514,13 @@ void qemu_init(int argc, char **argv)
             case QEMU_OPTION_nouserconfig:
                 /* Nothing to be parsed here. Especially, do not error out below. */
                 break;
+            case QEMU_OPTION_hybrid:
+                opts = qemu_opts_parse_noisily(qemu_find_opts("hybrid"),
+                                               optarg, true);
+                if (!opts) {
+                    exit(1);
+                }
+                break;
             default:
                 if (os_parse_cmd_args(popt->index, optarg)) {
                     error_report("Option not supported in this build");
@@ -3598,6 +3630,7 @@ void qemu_init(int argc, char **argv)
 
     qemu_resolve_machine_memdev();
     parse_numa_opts(current_machine);
+    parse_hybrid_opts(current_machine);
 
     if (vmstate_dump_file) {
         /* dump and exit */
-- 
2.34.1

[RFC 49/52] i386/pc: Support hybrid cpu topology

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Allow i386/pc to set hybrid cpu topology with "-hybrid" commands.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/pc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 9a7ee0affd87..5cd45921205c 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1928,6 +1928,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     hc->unplug = pc_machine_device_unplug_cb;
     mc->default_cpu_type = TARGET_DEFAULT_CPU_TYPE;
     mc->nvdimm_supported = true;
+    mc->topo_props.hybrid_supported = true;
     mc->topo_props.dies_supported = true;
     mc->topo_props.clusters_supported = true;
     mc->default_ram_id = "pc.ram";
-- 
2.34.1

[RFC 50/52] qemu-options: Add the document of hybrid command

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

Added the detailed introduction to the four hybrid options with specific
examples.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 qemu-options.hx | 77 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 77 insertions(+)

diff --git a/qemu-options.hx b/qemu-options.hx
index 8987972a8d5f..2bc7a5ed51ed 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -5611,7 +5611,84 @@ DEF("hybrid", HAS_ARG, QEMU_OPTION_hybrid,
     "-hybrid cluster,clusters=n"
     "-hybrid core,cores=n,coretype=core_type[,threads=threads][,clusterid=cluster]",
     QEMU_ARCH_ALL)
+SRST
+``-hybrid socket,sockets=n``
+  \
+``-hybrid die,dies=n``
+  \
+``-hybrid cluster,clusters=n``
+  \
+``-hybrid core,cores=n,coretype=core_type[,threads=threads][,clusterid=cluster]``
+    Define a hybrid cpu topology. Currently, only i386 PC machine supports this command.
+
+    There're 4 levels need to be configured: socket, die, cluster and core. And the
+    thread level is included in core level configuration.
+
+    Hybrid and smp cannot be set at the same time.
+
+    As for now, hybrid cpu topology only supports differences between the core and cluster
+    levels (please note the differences between clusters is caused by containing different
+    cores), and it assumes threads, dies and sockets are the same.
+
+    These four level options must be written in the order of "socket, die, cluster, core".
+
+    For example, the following options assigns 12 VCPUs which contains 8 Intel Atom cores
+    and 2 Intel Core cores.
+
+    ::
+
+        -hybrid socket,sockets=1    \
+        -hybrid die,dies=1          \
+        -hybrid cluster,clusters=4  \
+        -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-1    \
+        -hybrid core,cores=4,coretype="atom",threads=1,clusterid=2-3
+
+    '\ ``socket``\ ' option is used to configure how many sockets in the machine. Here
+    '\ ``sockets``\ ' means the number of sockets in the machine. If this option is not
+    set, the default is only 1 socket.
+
+    For example:
+
+    ::
+
+        -hybrid socket,sockets=1
 
+    '\ ``die``\ ' option is used to configure how many dies in one socket. Here
+    '\ ``dies``\ ' means the number of dies in one socket. If this option is not
+    set, the default is only 1 die per socket.
+
+    For example:
+
+    ::
+
+        -hybrid die,dies=1
+
+    '\ ``cluster``\ ' option is used to configure how many clusters in one die. Here
+    '\ ``clusters``\ ' means the number of clusters in one die. If this option is not
+    set, the default is only 1 cluster per die.
+
+    For example:
+
+    ::
+
+        -hybrid cluster,clusters=1
+
+    '\ ``core``\ ' option is used to configure hybrid core topology. Here '\ ``cores``\ '
+    means the number of current type cores in one cluster. '\ ``coretype``\ ' means the
+    type of the current core. '\ ``threads``\ ' means how many threads in one current type
+    core (the default is 1 thread per current core). And '\ ``clusterid``\ ' means the cluster
+    index list that the current type core belongs to (the default is all clusters have current
+    type cores). This option cannot be omitted.
+
+    For i386 PC machine, the supported '\ ``coretype``\ ' are "atom" and "core".
+
+    For example (clusters=4):
+
+    ::
+
+        -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-1,clusterid=3    \
+        -hybrid core,cores=4,coretype="atom",threads=1,clusterid=2
+ERST
 
 HXCOMM This is the last statement. Insert new options before this line!
 
-- 
2.34.1

[RFC 51/52] qapi: Expose CPU topology info in query_cpus_fast

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The topology type and core type are useful to users, so expose them
in query_cpus_fast.

Since the hybrid core is pre-set and does not support hotplug for
the entire core, the topology type and core type are not exposed in
query-hotpluggable-cpus.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/core/machine-qmp-cmds.c | 11 +++++++++++
 hw/core/machine-topo.c     |  5 +++++
 include/hw/boards.h        |  5 +++++
 qapi/machine.json          | 19 +++++++++++++++++--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
index 44b5da888024..6be7148a8374 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -67,6 +67,17 @@ CpuInfoFastList *qmp_query_cpus_fast(Error **errp)
             value->props = props;
         }
 
+        if (mc->cpu_index_to_core_type) {
+            CpuTopoInfo *topo;
+            const char *core_type;
+
+            topo = g_malloc0(sizeof(CpuTopoInfo));
+            topo->topo_type = machine_topo_get_type(ms);
+            core_type = mc->cpu_index_to_core_type(ms, cpu->cpu_index);
+            topo->core_type = g_strdup(core_type);
+            value->topo = topo;
+        }
+
         value->target = target;
         if (target == SYS_EMU_TARGET_S390X) {
             cpustate_to_cpuinfo_s390(&value->u.s390x, cpu);
diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
index f38b8c683026..f0ff550a2519 100644
--- a/hw/core/machine-topo.c
+++ b/hw/core/machine-topo.c
@@ -177,6 +177,11 @@ unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
     return ms->topo.max_cpus / sockets;
 }
 
+CpuTopoType machine_topo_get_type(const MachineState *ms)
+{
+    return ms->topo.topo_type;
+}
+
 void machine_init_topology_default(MachineState *ms, bool smp)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index c93bb1206244..d925e8af9ed6 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -239,6 +239,8 @@ typedef struct {
  *    purposes only.
  *    Applies only to default memory backend, i.e., explicit memory backend
  *    wasn't used.
+ * @cpu_index_to_core_type:
+ *    Return the core_type of specified CPU with @cpu_index.
  */
 struct MachineClass {
     /*< private >*/
@@ -305,6 +307,8 @@ struct MachineClass {
     const CPUArchIdList *(*possible_cpu_arch_ids)(MachineState *machine);
     int64_t (*get_default_cpu_node_id)(const MachineState *ms, int idx);
     ram_addr_t (*fixup_ram_size)(ram_addr_t size);
+    const char *(*cpu_index_to_core_type)(MachineState *machine,
+                                          unsigned cpu_index);
 };
 
 /**
@@ -496,5 +500,6 @@ unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
                                            unsigned int cpu_index);
 unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
 unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
+CpuTopoType machine_topo_get_type(const MachineState *ms);
 
 #endif
diff --git a/qapi/machine.json b/qapi/machine.json
index 931c6dea9819..e86778b7a777 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -40,7 +40,6 @@
 # @CpuTopoType:
 #
 # An enumeration of cpu topology type
-# TODO: Expose topology type in query-cpus-fast
 #
 # Since: 8.0
 ##
@@ -48,6 +47,21 @@
   'prefix': 'CPU_TOPO_TYPE',
   'data': [ 'smp', 'hybrid' ] }
 
+##
+# @CpuTopoInfo:
+#
+# Additional information about basic CPU topology
+#
+# @topo-type: the type of CPU topology
+#
+# @core-type: the type of CPU core
+#
+# Since: 8.0
+##
+{ 'struct' : 'CpuTopoInfo',
+  'data'   : { 'topo-type': 'CpuTopoType',
+               'core-type': 'str'} }
+
 ##
 # @CpuS390State:
 #
@@ -95,7 +109,8 @@
                       'qom-path'     : 'str',
                       'thread-id'    : 'int',
                       '*props'       : 'CpuInstanceProperties',
-                      'target'       : 'SysEmuTarget' },
+                      'target'       : 'SysEmuTarget',
+                      '*topo'         : 'CpuTopoInfo' },
   'discriminator' : 'target',
   'data'          : { 's390x'        : 'CpuInfoS390' } }
 
-- 
2.34.1

[RFC 52/52] i386: Support cpu_index_to_core_type() for x86

[Zhao Liu]

From: Zhao Liu <zhao1.liu@intel.com>

The cpu_index_to_core_type() of MachineClass is implemented in x86 to
obtain the string name of core type, so that users can get the basic
topology information (topology type and core type) of x86 machine
(currently, only PC machine) in query_cpus_fast.

Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
---
 hw/i386/x86.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index f58a90359170..67318a527c6c 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1583,6 +1583,38 @@ static int x86_parse_hybrid_core_type(MachineState *ms, const char *coretype)
     return type;
 }
 
+static const char *x86_cpu_index_to_hybrid_core_type(MachineState *machine,
+                                                     unsigned cpu_index)
+{
+    CPUState *cs;
+
+    if (machine_topo_is_smp(machine)) {
+        return NULL;
+    }
+
+    CPU_FOREACH(cs) {
+        X86CPU *cpu = X86_CPU(cs);
+        CPUX86State *env = &cpu->env;
+
+        if (cs->cpu_index == cpu_index) {
+            const char *core_type;
+
+            switch (env->hybrid_core_type) {
+            case INTEL_ATOM_TYPE:
+                core_type = "atom";
+                break;
+            case INTEL_CORE_TYPE:
+                core_type = "core";
+                break;
+            default:
+                abort();
+            }
+            return core_type;
+        }
+    }
+    return NULL;
+}
+
 static void x86_machine_initfn(Object *obj)
 {
     X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1611,6 +1643,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
     x86mc->fwcfg_dma_enabled = true;
     nc->nmi_monitor_handler = x86_nmi;
     mc->core_type = x86_parse_hybrid_core_type;
+    mc->cpu_index_to_core_type = x86_cpu_index_to_hybrid_core_type;
 
     object_class_property_add(oc, X86_MACHINE_SMM, "OnOffAuto",
         x86_machine_get_smm, x86_machine_set_smm,
-- 
2.34.1

Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name and helper

[Juan Quintela]

Zhao Liu <zhao1.liu@linux.intel.com> wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> As the generic code, here we should respect the different topologies:
> smp or hybrid.
>
> So rename PostcopyBlocktimeContext.smp_cpus_down to
> PostcopyBlocktimeContext.cpus_down, and also rename other local
> variables from smp_cpus to cpus_num, to decouple with smp topology.
>
> And use generic topology helpers to get topology information.
>
> Cc: Juan Quintela <quintela@redhat.com>
> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

but if you ever have to rebase.

> ---
>  migration/postcopy-ram.c | 24 ++++++++++++------------
>  1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> index 53299b7a5ebd..1e861e313258 100644
> --- a/migration/postcopy-ram.c
> +++ b/migration/postcopy-ram.c
> @@ -122,7 +122,7 @@ typedef struct PostcopyBlocktimeContext {
>      /* point in time when last page fault was initiated */
>      uint32_t last_begin;
>      /* number of vCPU are suspended */
> -    int smp_cpus_down;
> +    int cpus_down;

Put the rename of the variable in a single patch.  Trivial to review.

> +    unsigned int cpus_num = machine_topo_get_cpus(ms);

Put the meat in another patch.  I think you call this function in two
places instead of the old one.


Later, Juan.

Re: [RFC 00/52] Introduce hybrid CPU topology

[Alex Bennée]

Zhao Liu <zhao1.liu@linux.intel.com> writes:

> From: Zhao Liu <zhao1.liu@intel.com>
>
> Hi list,
>
> This RFC series is to introduce hybrid topology in QEMU, and is based
> on our previous patches about cluster support for x86 [1] ([PATCH RESEND
> 00/18] Support smp.clusters for x86).
>
> Though our cluster support patches [1] has not yet accepted, we are
> still posting the hybrid topology support patches in advance, hoping to
> get discussion and feedbacks.
>
> In this RFC series, we mainly have the following work:
> * Introduce the new cpu topology under heterogeneous architecture, which
>   we called "hybrid" cpu topology.
>       - since the name "hmp" is already used in QEMU and "hybrid" is
>         also a synonym for heterogeneous.
> * Reorganized the relevant topology information in qemu to adapt to the
>   hybrid topology.
> * Introduce a new command "hybrid", which allows user to set a custom
>   hybrid topology.
>       - currently limited to custom cluster and custom core
> * Support hybrid topology in i386 PC machine.
> * Expose basic topology information (topology type and core type) in
>   query_cpus_fast.
>
> We hope that the hybrid topology can be general enough to be compatible
> with the needs of hybrids with various architectures in the future.
>
> Welcome your feedbacks!
>
>
> # 1. Background
>
> Heterogeneous computing refers to systems that use more than one kind of
> processor or cores [2]. Now there are more and more heterogeneous
> computing architectures, typically including arm's big.LITTLE [3] and
> intel hybrid architecture [4].
>
> The development of heterogeneous computing brings new challenges to
> virtualization. The first problem we face is how to support the vCPU
> topology under the heterogeneous architecture.
>
> Here, to distinguish it from "smp" cpu topology, we call it "hybrid" cpu
> topology.
>
>
> # 2. Motivation
>
> In a typical hybrid system, different types of core may have different
> capabilities. This difference may be reflected in different capacities,
> performance or power efficiencies, and even different instruction-per-
> cycle (IPC) capabilities. The direct consequence of these differences is
> to affect the performance of the workload.

This elides a use case for heterogeneous computing under emulation where
different cores can have totally different ISA's. We already support
emulation models where we have varying classes of the same ISA, for
example xlnx-zyncmp supports 4 A profile and 2 R profile Arm chips.

I've glanced over the patch set and I don't think there is any direct
clash with that as this is mainly about representing the topology for
x86 non-emulation accelerators but I just want ensure we don't end up
stepping on each others toes.

> For a virtual machines, vCPUs are just threads, although we can bind
> vCPUs to different cores to benefit from the different advantages of
> these physical cores, it's obviously not enough since CPU topology (and
> cache topology) will still have a significant impact on scheduling
> performance.
>
<snip>
> ## 3.3. "-hybrid" command
>
> For hybrid cpu topology configuration, the original "-smp" lack of
> flexibility to expand, and unables to customize different cores.
>
> So we introduce "-hybrid" command and implement it as the multi-
> line command. The multi-line command format is more complex than the
> single-line one, but it can bring stronger scalability and
> intuitiveness. In the future, it can also be easily extended to more
> heterogeneous topology levels.
>
> "-hybrid" command is as follows:
>
> -hybrid socket,sockets=n
> -hybrid die,dies=n
> -hybrid cluster,clusters=n
> -hybrid core,cores=n,type=core_type[,threads=threads]
>         [,clusterid=cluster]
>
> Here, we first define the corresponding qapi options for these 4
> topology levels: core, cluster, die and socket.
>
> We doesn't need a thread level since thread doesn't have different
> type.
>
> For example:
>
> -hybrid socket,sockets=1
> -hybrid die,dies=1
> -hybrid cluster,clusters=4
> -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
> -hybrid core,cores=4,coretype="atom",threads=1
>
> Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
> socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
> core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
> cores with 2 threads.
>
> Please note this example is not an actual machine topology, but it shows
> the powerful flexibility of "hybrid" command.

Is there any intention to apply this command to emulation or will it
only be for hardware virtualisation? For example the x86 translator
should be capable of emulating two different classes of x86 with
different subsets of instructions. It requires ensuring each CPU is
assigned to the correct TYPE_CPU_CLUSTER so we don't re-use TBs from
the "wrong" CPU.

<snip>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name and helper

[Juan Quintela]

Zhao Liu <zhao1.liu@linux.intel.com> wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> As the generic code, here we should respect the different topologies:
> smp or hybrid.
>
> So rename PostcopyBlocktimeContext.smp_cpus_down to
> PostcopyBlocktimeContext.cpus_down, and also rename other local
> variables from smp_cpus to cpus_num, to decouple with smp topology.
>
> And use generic topology helpers to get topology information.
>
> Cc: Juan Quintela <quintela@redhat.com>
> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>

I expect that this will go with your other changes through another tree,
or do you want me to take it through migration thread?

Later, Juan.

Re: [RFC 05/52] hw/core/machine: Rename machine-smp.c to machine-topo.c

[wangyanan (Y) via]

Hi Zhao，
在 2023/2/13 17:49, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> Hybrid topology support will also be added to this file.
>
> In order to maintain the semantic consistency of the file name and
> content, the file name is changed to "machine-topo.c".
>
> This file will handle all cpu topology related things, thus also change
> the file description comment.
>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   MAINTAINERS                               | 2 +-
>   hw/core/{machine-smp.c => machine-topo.c} | 2 +-
>   hw/core/meson.build                       | 2 +-
>   tests/unit/meson.build                    | 2 +-
>   4 files changed, 4 insertions(+), 4 deletions(-)
>   rename hw/core/{machine-smp.c => machine-topo.c} (99%)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 71c1bc24371b..58794885ced3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1733,7 +1733,7 @@ F: cpu.c
>   F: hw/core/cpu.c
>   F: hw/core/machine-qmp-cmds.c
>   F: hw/core/machine.c
> -F: hw/core/machine-smp.c
> +F: hw/core/machine-topo.c
>   F: hw/core/null-machine.c
>   F: hw/core/numa.c
>   F: hw/cpu/cluster.c
> diff --git a/hw/core/machine-smp.c b/hw/core/machine-topo.c
> similarity index 99%
> rename from hw/core/machine-smp.c
> rename to hw/core/machine-topo.c
Better named as cpu-topology.c? So that it's consistent with the
header "cpu-topology.h" in a following patch.

Thanks,
Yanan
> index 3fd9e641efde..8066d2c46bef 100644
> --- a/hw/core/machine-smp.c
> +++ b/hw/core/machine-topo.c
> @@ -1,5 +1,5 @@
>   /*
> - * QEMU Machine core (related to -smp parsing)
> + * QEMU Machine core (related to CPU topology)
>    *
>    * Copyright (c) 2021 Huawei Technologies Co., Ltd
>    *
> diff --git a/hw/core/meson.build b/hw/core/meson.build
> index 7a4d02b6c050..71d523dc6826 100644
> --- a/hw/core/meson.build
> +++ b/hw/core/meson.build
> @@ -23,7 +23,7 @@ else
>   endif
>   
>   common_ss.add(files('cpu-common.c'))
> -common_ss.add(files('machine-smp.c'))
> +common_ss.add(files('machine-topo.c'))
>   softmmu_ss.add(when: 'CONFIG_FITLOADER', if_true: files('loader-fit.c'))
>   softmmu_ss.add(when: 'CONFIG_GENERIC_LOADER', if_true: files('generic-loader.c'))
>   softmmu_ss.add(when: ['CONFIG_GUEST_LOADER', fdt], if_true: files('guest-loader.c'))
> diff --git a/tests/unit/meson.build b/tests/unit/meson.build
> index a9df2843e92e..1720a769f1a2 100644
> --- a/tests/unit/meson.build
> +++ b/tests/unit/meson.build
> @@ -46,7 +46,7 @@ tests = {
>     'test-uuid': [],
>     'ptimer-test': ['ptimer-test-stubs.c', meson.project_source_root() / 'hw/core/ptimer.c'],
>     'test-qapi-util': [],
> -  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c'],
> +  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-topo.c'],
>     'test-interval-tree': [],
>   }
>   

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Philippe Mathieu-Daudé]

On 13/2/23 10:49, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> For smp systems, the parts in one topology level are the same. But now
> there are more and more systems with hybrid architectures. Different
> parts of the same topology level may have differences. For example,
> Intel's Alder Lake series CPU has two types of cores, so the CPU
> topology is no longer symmetrical.
> 
> The hybrid topology is compatible with the smp topology type, that is,
> different parts on the same level of the hybrid topology can set to be
> the same, but the hybrid topology will introduce more complexity (need
> to allocate more memory, organized with array or linked-list), so the
> original smp topology support is retained while introducing the hybrid
> topology, and the hybrid topology is only built when the hybrid is
> explicitly required.
> 
> Therefore, we introduce the definition support of hybrid cpu topology
> here. At the same time, in order to unify with the original smp, we
> introduce a new cpu topology structure that can support smp topology
> or hybrid topology. This structure will replace the CpuTopology type (in
> include/hw/boards.h) used by MachineState.smp.
> 
> As for now, we only support two hybrid topology levels: core and
> cluster.
> 
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   MAINTAINERS                   |   1 +
>   include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
>   qapi/machine.json             |  12 ++++
>   3 files changed, 130 insertions(+)
>   create mode 100644 include/hw/cpu/cpu-topology.h
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 58794885ced3..918a9418d98e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
>   F: include/hw/boards.h
>   F: include/hw/core/cpu.h
>   F: include/hw/cpu/cluster.h
> +F: include/hw/cpu/cpu-topology.h
>   F: include/sysemu/numa.h
>   F: tests/unit/test-smp-parse.c
>   T: git https://gitlab.com/ehabkost/qemu.git machine-next

Worth splitting this section in 2, machine VS numa/clusters?

Re: [RFC 09/52] hw/machine: Introduce core type for hybrid topology

[Philippe Mathieu-Daudé]

On 13/2/23 10:49, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Under the hybrid cpu topology, some CPUs care about the core type.
> For example, Intel's Alder Lake series CPU contains two types of cores:
> Intel Core and Intel Atom. The type information of these two types is
> exposed in 1A leaf of CPUID.
> 
> Core type should also be part of the hybrid topology, and
> MachineState.cpu_type cannot provide different type information for
> different cpus in the same machine, so add a type field for the core
> level in the hybrid cpu topology.
> 
> Additionally, add a helper to get core type information from
> MachineState. Though core_type is only used in hybrid case, don't
> use assert since it may be used to initialize some generic fields.
> 
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/core/machine-topo.c        | 12 ++++++++++++
>   include/hw/boards.h           |  3 +++
>   include/hw/cpu/cpu-topology.h |  2 ++
>   3 files changed, 17 insertions(+)
> 
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> index b20160479629..e0ec07b53d42 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -51,6 +51,18 @@ unsigned int machine_topo_get_smp_threads(const MachineState *ms)
>       return ms->topo.smp.threads;
>   }
>   
> +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> +                                               unsigned int cluster_id,
> +                                               unsigned int core_id)
> +{
> +    if (!machine_topo_is_smp(ms)) {
> +        return ms->topo.hybrid.cluster_list[cluster_id]
> +                       .core_list[core_id].core_type;
> +    } else {
> +        return 0;

Is '0' an invalid type?

> +    }
> +}

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[wangyanan (Y) via]

Hi Zhao,

在 2023/2/13 17:49, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> For smp systems, the parts in one topology level are the same. But now
> there are more and more systems with hybrid architectures. Different
> parts of the same topology level may have differences. For example,
> Intel's Alder Lake series CPU has two types of cores, so the CPU
> topology is no longer symmetrical.
>
> The hybrid topology is compatible with the smp topology type, that is,
> different parts on the same level of the hybrid topology can set to be
> the same, but the hybrid topology will introduce more complexity (need
> to allocate more memory, organized with array or linked-list), so the
> original smp topology support is retained while introducing the hybrid
> topology, and the hybrid topology is only built when the hybrid is
> explicitly required.
>
> Therefore, we introduce the definition support of hybrid cpu topology
> here. At the same time, in order to unify with the original smp, we
> introduce a new cpu topology structure that can support smp topology
> or hybrid topology. This structure will replace the CpuTopology type (in
> include/hw/boards.h) used by MachineState.smp.
>
> As for now, we only support two hybrid topology levels: core and
> cluster.
>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   MAINTAINERS                   |   1 +
>   include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
>   qapi/machine.json             |  12 ++++
>   3 files changed, 130 insertions(+)
>   create mode 100644 include/hw/cpu/cpu-topology.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 58794885ced3..918a9418d98e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
>   F: include/hw/boards.h
>   F: include/hw/core/cpu.h
>   F: include/hw/cpu/cluster.h
> +F: include/hw/cpu/cpu-topology.h
Should't it be in include/hw/core/* directory？
>   F: include/sysemu/numa.h
>   F: tests/unit/test-smp-parse.c
>   T: git https://gitlab.com/ehabkost/qemu.git machine-next
> diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
> new file mode 100644
> index 000000000000..8268ea3a8569
> --- /dev/null
> +++ b/include/hw/cpu/cpu-topology.h
> @@ -0,0 +1,117 @@
> +/*
> + * CPU topology defination for Machine core
> + *
> + * Copyright (c) 2023 Intel Corporation
> + * Author: Zhao Liu <zhao1.liu@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License,
> + * or (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef CPU_TOPOLOGY_H
> +#define CPU_TOPOLOGY_H
> +
> +#include "qemu/queue.h"
> +
> +/**
> + * SmpCpuTopology - smp cpu topology defination.
> + *
> + * For smp system, the parts in one topology level are the same.
> + *
> + * @sockets: the number of sockets on the machine
> + * @dies: the number of dies in one socket
> + * @clusters: the number of clusters in one die
> + * @cores: the number of cores in one cluster
> + * @threads: the number of threads in one core
> + */
> +typedef struct SmpCpuTopology {
> +    unsigned int sockets;
> +    unsigned int dies;
> +    unsigned int clusters;
> +    unsigned int cores;
> +    unsigned int threads;
> +} SmpCpuTopology;
> +
> +/**
> + * HybridCore - hybrid core topology defination:
> + * @threads: the number of threads in one core.
> + */
> +typedef struct HybridCore {
> +    unsigned int threads;
> +} HybridCore;
> +
> +/**
> + * HybridCluster - hybrid cluster topology defination:
> + * @cores: the number of cores in current cluster.
> + * @core_list: the array includes all the cores that belong to current
> + *             cluster.
> + */
> +typedef struct HybridCluster {
> +    unsigned int cores;
> +    HybridCore *core_list;
> +} HybridCluster;
> +
> +/**
> + * HybridCpuTopology - hybrid cpu topology defination.
> + *
> + * At present we only support two heterogeneous topology levels: core
> + * and cluster. For heterogeneous levels, we need additional structs
> + * to define their custom internal topology. So here we defines
> + * symmetric topology levels, and use a list to point to heterogeneous
> + * levels.
> + *
> + * @sockets: the number of sockets on the machine. All sockets are the
> + *           same.
> + * @dies: the number of dies in one socket. All dies are the same.
> + * @clusters: the number of clusters in one die. Cluster may be
> + *            different. This field indicates the length of
> + *            cluster_list.
> + * @cluster_list: the array includes all the clusters in one die.
> + */
> +typedef struct HybridCpuTopology {
> +    unsigned int sockets;
> +    unsigned int dies;
> +    unsigned int clusters;
> +    HybridCluster *cluster_list;
> +} HybridCpuTopology;
> +
> +/**
> + * GeneralCpuTopology - General cpu topology defination.
> + *
> + * It supports one of two topologies: smp topology or hybrid topology.
> + *
> + * @cpus: the number of present logical processors on the machine
> + * @max_cpus: the maximum number of logical processors on the machine
> + * @topo_type: the topology type of the machine and this decides which
> + *             member of the union to visit: smp or hybrid.
> + * @smp: the smp cpu topology informantion. Only valid when topo_type is
> + *       CPU_TOPO_TYPE_SMP.
> + * @hybrid: the hybrid cpu topology informantion. Only valid when
> + *          topo_type is CPU_TOPO_TYPE_HYBRID.
> + */
> +typedef struct GeneralCpuTopology {
> +    unsigned int cpus;
> +    unsigned int max_cpus;
> +    CpuTopoType topo_type;
> +    union {
> +        SmpCpuTopology smp;
> +        HybridCpuTopology hybrid;
> +    };
> +} GeneralCpuTopology; /*
> +                       * TODO: This name is temporary, just to distinguish it
> +                       * from the CpuTopology in boards.h. When CpuTopology in
> +                       * boards.h is merged here, it will be uniformly named as
> +                       * CpuTopology.
> +                       */
> +
A suggestion:
1、Move definition of CpuTopology from boards.h to cpu-topology.h
and re-structure it to include SmpCpuTopology, being a generic cpu
topology structure.
2、Rename "CpuTopology smp" in MachineState to a generic name
"CpuTopology topo".
3、Adapt all the code in QEMU to above change.

If you can pack above into a single patch, and then add the hybird
topology extansion in a next patch, we will not need the temporary
thing "GeneralCpuTopology" and the TODO comments, which makes
code clearer.

Thanks,
Yanan
> +#endif /* CPU_TOPOLOGY_H */
> diff --git a/qapi/machine.json b/qapi/machine.json
> index b9228a5e4616..bd7303f34497 100644
> --- a/qapi/machine.json
> +++ b/qapi/machine.json
> @@ -36,6 +36,18 @@
>                'sh4eb', 'sparc', 'sparc64', 'tricore',
>                'x86_64', 'xtensa', 'xtensaeb' ] }
>   
> +##
> +# @CpuTopoType:
> +#
> +# An enumeration of cpu topology type
> +# TODO: Expose topology type in query-cpus-fast
> +#
> +# Since: 8.0
> +##
> +{ 'enum': 'CpuTopoType',
> +  'prefix': 'CPU_TOPO_TYPE',
> +  'data': [ 'smp', 'hybrid' ] }
> +
>   ##
>   # @CpuS390State:
>   #

Re: [RFC 30/52] i386: Drop nr_dies and nr_modules CPUX86State

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Since CPUState has the complete topology information, there's no need
> to keep i386 architecture specific topology.
> 
> This avoids the fragmentation of topological information.
> 
> Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/i386/x86.c         | 4 ----
>   target/i386/cpu.c     | 7 +++----
>   target/i386/cpu.h     | 8 --------
>   target/i386/kvm/kvm.c | 2 +-
>   4 files changed, 4 insertions(+), 17 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC 32/52] i386: Rename X86CPUTopoInfo and its members to reflect relationship with APIC ID

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Rename X86CPUTopoInfo to X86ApicidTopoInfo, and also rename its members
> to max_{dies, modules, cores, threads} to avoid confusion with
> CPUState.topo.
> 
> Now the names can better reflect their relationship with APIC ID.
> 
> For hybrid CPU topology, X86ApicidTopoInfo will be different with
> CPUState.topo and keep the maximum possible number of topology
> structures in each topology level other than the actual topology
> information of a certain CPU. This is required for the APIC ID under the
> hybrid CPU topology.
> 
> Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/i386/x86.c                | 18 ++++-----
>   include/hw/i386/topology.h   | 72 +++++++++++++++++++++++-------------
>   include/hw/i386/x86.h        |  2 +-
>   target/i386/cpu.c            | 20 +++++-----
>   tests/unit/test-x86-apicid.c | 50 ++++++++++++-------------
>   5 files changed, 91 insertions(+), 71 deletions(-)

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC 33/52] i386: Rename init_topo_info() to init_apic_topo_info()

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Rename init_topo_info() to init_apic_topo_info() to adapt
> X86ApicidTopoInfo.
> 
> Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/i386/x86.c         | 12 ++++++------
>   include/hw/i386/x86.h |  3 ++-
>   2 files changed, 8 insertions(+), 7 deletions(-)


> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> index ac6f1e4a74af..d84f7717900c 100644
> --- a/include/hw/i386/x86.h
> +++ b/include/hw/i386/x86.h
> @@ -98,7 +98,8 @@ struct X86MachineState {
>   #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
>   OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
>   
> -void init_topo_info(X86ApicidTopoInfo *topo_info, const X86MachineState *x86ms);
> +void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
> +                           const X86MachineState *x86ms);

Maybe s/init_apicid_topo_info/init_apic_topo_info/?

Otherwise,

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC 34/52] i386: Rename variable topo_info to apicid_topo

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Since X86ApicidTopoInfo is only used for APIC ID related work, the
> common variable topo_info of X86ApicidTopoInfo type should be also
> renamed to better suit its purpose.
> 
> Generic topology access should be obtained from MachineState.topo
> (for the whole machine) or CPUState.topo (for the current CPU), and
> apicid_topo (X86ApicidTopoInfo type) is only used to collaborate with
> APIC ID.
> 
> Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/i386/x86.c                |  38 ++++-----
>   include/hw/i386/topology.h   |  76 ++++++++---------
>   include/hw/i386/x86.h        |   2 +-
>   target/i386/cpu.c            |  71 ++++++++--------
>   tests/unit/test-x86-apicid.c | 158 +++++++++++++++++------------------
>   5 files changed, 173 insertions(+), 172 deletions(-)

The 'id' suffix doesn't add a lot of value IMO. Anyway,

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Re: [RFC 41/52] machine: Introduce core_type() hook

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Since supported core types are architecture specific, we need this hook
> to allow archs define its own parsing or validation method.
> 
> As the example, add the x86 core_type() which will be used in "-hybrid"
> parameter parsing.

What would be a "core type" for other archs?

> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/core/machine-topo.c | 14 ++++++++++++++
>   hw/core/machine.c      |  1 +
>   hw/i386/x86.c          | 15 +++++++++++++++
>   include/hw/boards.h    |  7 +++++++
>   4 files changed, 37 insertions(+)


> index 9364c90d5f1a..34ec035b5c9f 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
>                                  Error **errp);
>   void machine_parse_smp_config(MachineState *ms,
>                                 const SMPConfiguration *config, Error **errp);
> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
>   
>   /**
>    * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
> @@ -199,6 +200,11 @@ typedef struct {
>    *    Return the type of KVM corresponding to the kvm-type string option or
>    *    computed based on other criteria such as the host kernel capabilities.
>    *    kvm-type may be NULL if it is not needed.
> + * @core_type:
> + *    Return the type of hybrid cores corresponding to the coretype string
> + *    option. The default hook only accept "none" or "" since the most generic
> + *    core topology should not specify any specific core type. Each arch can
> + *    define its own core_type() hook to override the default one.
>    * @numa_mem_supported:
>    *    true if '--numa node.mem' option is supported and false otherwise
>    * @hotplug_allowed:
> @@ -237,6 +243,7 @@ struct MachineClass {
>       void (*reset)(MachineState *state, ShutdownCause reason);
>       void (*wakeup)(MachineState *state);
>       int (*kvm_type)(MachineState *machine, const char *arg);
> +    int (*core_type)(MachineState *state, const char *type);
>   
>       BlockInterfaceType block_default_type;
>       int units_per_default_bus;

Re: [RFC 41/52] machine: Introduce core_type() hook

[Philippe Mathieu-Daudé]

On 13/2/23 10:50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Since supported core types are architecture specific, we need this hook
> to allow archs define its own parsing or validation method.
> 
> As the example, add the x86 core_type() which will be used in "-hybrid"
> parameter parsing.
> 
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/core/machine-topo.c | 14 ++++++++++++++
>   hw/core/machine.c      |  1 +
>   hw/i386/x86.c          | 15 +++++++++++++++
>   include/hw/boards.h    |  7 +++++++
>   4 files changed, 37 insertions(+)
> 
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> index 12c05510c1b5..f9ab08a1252e 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
>           return;
>       }
>   }
> +
> +/*
> + * machine_parse_hybrid_core_type: the default hook to parse hybrid core
> + *                                 type corresponding to the coretype
> + *                                 string option.
> + */
> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> +{
> +    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
> +        return 0;
> +    }
> +
> +    return -1;

Shouldn't this use mc->core_type()? Where is it used?

> +}

Re: [RFC 00/52] Introduce hybrid CPU topology

[Daniel P. Berrangé]

On Mon, Feb 13, 2023 at 05:49:43PM +0800, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> ## 3.3. "-hybrid" command
> 
> For hybrid cpu topology configuration, the original "-smp" lack of
> flexibility to expand, and unables to customize different cores.
> 
> So we introduce "-hybrid" command and implement it as the multi-
> line command. The multi-line command format is more complex than the
> single-line one, but it can bring stronger scalability and
> intuitiveness. In the future, it can also be easily extended to more
> heterogeneous topology levels.
> 
> "-hybrid" command is as follows:
> 
> -hybrid socket,sockets=n
> -hybrid die,dies=n
> -hybrid cluster,clusters=n
> -hybrid core,cores=n,type=core_type[,threads=threads]
>         [,clusterid=cluster]
> 
> Here, we first define the corresponding qapi options for these 4
> topology levels: core, cluster, die and socket.
> 
> We doesn't need a thread level since thread doesn't have different
> type.
> 
> For example:
> 
> -hybrid socket,sockets=1
> -hybrid die,dies=1
> -hybrid cluster,clusters=4
> -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
> -hybrid core,cores=4,coretype="atom",threads=1
> 
> Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
> socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
> core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
> cores with 2 threads.

How will this interact with the -cpu  parameter. Presumably we now
need two distinct -cpu parameters, one for the 'core' CPU model and
one for the 'atom' CPU model ?

> Please note this example is not an actual machine topology, but it shows
> the powerful flexibility of "hybrid" command.

IIUC the functionality offered by -hybrid should be a superset
of the -smp functionality. IOW, -smp ought to be possible to
re-implement -smp as an alias for -hybrid, such that internally
code only ever has to deal with the modern approach. Having to
keep support for both -smp and -hybrid throughout the code is
undesirable IMHO. Keeping the compat at the CLI parsing level
limits the burden.


As a more general thought, rather than introducing a new top level
command line argument -hybrid, I'm thinking we should possibly just
define this all using QOM and thus re-use the existing -object
argument. 

I'm also finding the above example command lines quite difficult
to understand, as there is alot of implicit linkage and expansion
between the different levels. With devices we're much more
explicitly with the parent/child relationships, and have to
express everything with no automatic expansion, linking it all
together via the id=/bus= properties.  This is quite a bit more
verbose, but it is also very effective at letting us express
arbitrarily complex relationships.

I think it would be worth exploring that approach for the CPU
topology expression too.

If we followed the more explicit device approach to modelling
then instead of:

 -cpu core,...
 -cpu atom,...
 -hybrid socket,sockets=1
 -hybrid die,dies=1
 -hybrid cluster,clusters=4
 -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
 -hybrid core,cores=4,coretype="atom",threads=1

we would end up with something like

  -object cpu-socket,id=sock0
  -object cpu-die,id=die0,parent=sock0
  -object cpu-cluster,id=cluster0,parent=die0
  -object cpu-cluster,id=cluster1,parent=die0
  -object cpu-cluster,id=cluster2,parent=die0
  -object cpu-cluster,id=cluster3,parent=die0
  -object x86-cpu-model-atom,id=cpu0,parent=cluster0
  -object x86-cpu-model-atom,id=cpu1,parent=cluster0
  -object x86-cpu-model-atom,id=cpu2,parent=cluster0
  -object x86-cpu-model-atom,id=cpu3,parent=cluster0
  -object x86-cpu-model-core,id=cpu4,parent=cluster0,threads=2
  -object x86-cpu-model-atom,id=cpu5,parent=cluster1
  -object x86-cpu-model-atom,id=cpu6,parent=cluster1
  -object x86-cpu-model-atom,id=cpu7,parent=cluster1
  -object x86-cpu-model-atom,id=cpu8,parent=cluster1
  -object x86-cpu-model-core,id=cpu9,parent=cluster1,threads=2
  -object x86-cpu-model-atom,id=cpu10,parent=cluster2
  -object x86-cpu-model-atom,id=cpu11,parent=cluster2
  -object x86-cpu-model-atom,id=cpu12,parent=cluster2
  -object x86-cpu-model-atom,id=cpu13,parent=cluster2
  -object x86-cpu-model-core,id=cpu14,parent=cluster2,threads=2
  -object x86-cpu-model-atom,id=cpu15,parent=cluster3
  -object x86-cpu-model-atom,id=cpu16,parent=cluster3
  -object x86-cpu-model-atom,id=cpu17,parent=cluster3
  -object x86-cpu-model-atom,id=cpu18,parent=cluster3
  -object x86-cpu-model-core,id=cpu19,parent=cluster3,threads=2

The really obvious downside is that it is much more verbose.

This example only has 20 CPUs. For a VM with say 1000 CPUs
this will be very big, but that doesn't neccesarily make it
wrong.

On the flipside

 * It is really clear exactly how many CPUs I've added

 * The relationship between the topology levels is clear

 * Every CPU has a unique ID given that can be used in
   later QMP commands

 * Whether or not 'threads' are permitted is now a property
   of the specific CPU model implementation, not the global
   config. IOW we can express that some CPU models allowing
   for threads, and some don't.

 * The -cpu arg is also obsoleted, replaced by the
   -object x86-cpu-model-core. This might facilitate the
   modelling of machines with CPUs from different architectures.


We could potentially compress the leaf node level by expressing
how many instances of an object we want. it we want. ie, define
a more convenient shorthand syntax to creating many instances of
an object. so eg

  -object-set $TYPE,$PROPS,idbase=foo,count=4

would be functionally identical to

  -object $TYPE,$PROPS,id=foo.0
  -object $TYPE,$PROPS,id=foo.1
  -object $TYPE,$PROPS,id=foo.2
  -object $TYPE,$PROPS,id=foo.3

QEMU just expands it and creates all the objects internally.

So the huge example I have above for 20 cpus would become much
shorter: e.g.

  -object cpu-socket,id=sock0
  -object cpu-die,id=die0,parent=sock0
  -object cpu-cluster,id=cluster0,parent=die0
  -object cpu-cluster,id=cluster1,parent=die0
  -object cpu-cluster,id=cluster2,parent=die0
  -object cpu-cluster,id=cluster3,parent=die0
  -object-set x86-cpu-core-atom,idbase=cpu0,parent=cluster0,count=4
  -object-set x86-cpu-core-core,id=cpu1,parent=cluster0,threads=2,count=1
  -object-set x86-cpu-core-atom,idbase=cpu2,parent=cluster1,count=4
  -object-set x86-cpu-core-core,id=cpu3,parent=cluster1,threads=2,count=1
  -object-set x86-cpu-core-atom,idbase=cpu4,parent=cluster2,count=4
  -object-set x86-cpu-core-core,id=cpu5,parent=cluster2,threads=2,count=1
  -object-set x86-cpu-core-atom,idbase=cpu6,parent=cluster3,count=4
  -object-set x86-cpu-core-core,id=cpu7,parent=cluster3,threads=2,count=1

IOW, the size of the CLI config only depends on the number of elements
in the hierarchy, and is independant of the number of leaf CPU cores.

Obviously in describing all of the above, I've ignored any complexity
of dealing with our existing code implementation and pain of getting
it converted to the new model.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [RFC 04/52] i386/WHPX: Fix error message when fail to set ProcessorCount

[Daniel P. Berrangé]

On Mon, Feb 13, 2023 at 05:49:47PM +0800, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> 003f230 (machine: Tweak the order of topology members in struct
> CpuTopology) changes the meaning of MachineState.smp.cores from "the
> number of cores in one package" to "the number of cores in one die"
> and doesn't fix other uses of MachineState.smp.cores. And because of
> the introduction of cluster, now smp.cores just means "the number of
> cores in one cluster". This clearly does not fit the semantics here.
> 
> And before this error message, WHvSetPartitionProperty() is called to
> set prop.ProcessorCount.
> 
> So the error message should show the prop.ProcessorCount other than
> "cores per cluster" or "cores per package".
> 
> Cc: Sunil Muthuswamy <sunilmut@microsoft.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>  target/i386/whpx/whpx-all.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

This patch and the 3 patches before it all look like basic bug
fixes to current code, not really part of the new hybrid topology
feature work.

As such I'd suggest sending these first four patches as a separate
series, so the bug fixes can be merged fairly quickly. I expect the
rest of the hybrid topology series is going to take a long time to
get agreement on, so no need to delay the easy bug fixes.

> 
> diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
> index e738d83e8191..fc349f887e47 100644
> --- a/target/i386/whpx/whpx-all.c
> +++ b/target/i386/whpx/whpx-all.c
> @@ -2613,8 +2613,8 @@ static int whpx_accel_init(MachineState *ms)
>          sizeof(WHV_PARTITION_PROPERTY));
>  
>      if (FAILED(hr)) {
> -        error_report("WHPX: Failed to set partition core count to %d,"
> -                     " hr=%08lx", ms->smp.cores, hr);
> +        error_report("WHPX: Failed to set partition processor count to %d,"
> +                     " hr=%08lx", prop.ProcessorCount, hr);
>          ret = -EINVAL;
>          goto error;
>      }
> -- 
> 2.34.1
> 
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RE: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[Mi, Dapeng1]

> From: Zhao Liu <zhao1.liu@linux.intel.com>
> Sent: Monday, February 13, 2023 5:50 PM
> To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daudé <philmd@linaro.org>;
> Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>
> Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> Christopherson,, Sean <seanjc@google.com>; Like Xu
> <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>
> Subject: [RFC 08/52] machine: Add helpers to get cpu topology info from
> MachineState.topo
> 
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> When MachineState.topo is introduced, the topology related structures become
> complicated. In the general case (hybrid or smp topology), accessing the
> topology information needs to determine whether it is currently smp or hybrid
> topology, and then access the corresponding MachineState.topo.smp or
> MachineState.topo.hybrid.
> 
> The best way to do this is to wrap the access to the topology to avoid having to
> check each time it is accessed.
> 
> The following helpers are provided here:
> 
> - General interfaces - no need to worry about whether the underlying
>   topology is smp or hybrid:
> 
> * machine_topo_get_cpus()
> * machine_topo_get_max_cpus()
> * machine_topo_is_smp()
> * machine_topo_get_sockets()
> * machine_topo_get_dies()
> * machine_topo_get_clusters()
> * machine_topo_get_threads();
> * machine_topo_get_cores();
> * machine_topo_get_threads_by_idx()
> * machine_topo_get_cores_by_idx()
> * machine_topo_get_cores_per_socket()
> * machine_topo_get_threads_per_socket()
> 
> - SMP-specific interfaces - provided for the cases that are clearly known to be
> smp topology:
> 
> * machine_topo_get_smp_cores()
> * machine_topo_get_smp_threads()
> 
> Since for hybrid topology, each core may has different threads, if someone
> wants "cpus per core", the cpu_index is need to target a specific core
> (machine_topo_get_threads_by_idx()). But for smp, there is no need to be so
> troublesome, so for this case, we provide smp-specific interfaces.
> 
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>  hw/core/machine-topo.c | 142
> +++++++++++++++++++++++++++++++++++++++++
>  include/hw/boards.h    |  35 ++++++++++
>  2 files changed, 177 insertions(+)
> 
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c index
> 7223f73f99b0..b20160479629 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -21,6 +21,148 @@
>  #include "hw/boards.h"
>  #include "qapi/error.h"
> 
> +unsigned int machine_topo_get_sockets(const MachineState *ms) {
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> +                                     ms->topo.hybrid.sockets; }
> +
> +unsigned int machine_topo_get_dies(const MachineState *ms) {
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
> +                                     ms->topo.hybrid.dies; }
> +
> +unsigned int machine_topo_get_clusters(const MachineState *ms) {
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
> +                                     ms->topo.hybrid.clusters; }
> +
> +unsigned int machine_topo_get_smp_cores(const MachineState *ms) {
> +    g_assert(machine_topo_is_smp(ms));
> +    return ms->topo.smp.cores;
> +}
> +
> +unsigned int machine_topo_get_smp_threads(const MachineState *ms) {
> +    g_assert(machine_topo_is_smp(ms));
> +    return ms->topo.smp.threads;
> +}
> +
> +unsigned int machine_topo_get_threads(const MachineState *ms,
> +                                      unsigned int cluster_id,
> +                                      unsigned int core_id) {
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.threads;
> +    } else {
> +        return ms->topo.hybrid.cluster_list[cluster_id]
> +                   .core_list[core_id].threads;
> +    }
> +
> +    return 0;
> +}
> +
> +unsigned int machine_topo_get_cores(const MachineState *ms,
> +                                    unsigned int cluster_id) {
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores;
> +    } else {
> +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
> +    }
> +}
> +
> +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> +                                             unsigned int cpu_index) {
> +    unsigned cpus_per_die;
> +    unsigned tmp_idx;
> +    HybridCluster *cluster;
> +    HybridCore *core;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.threads;
> +    }
> +
> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> +                            ms->topo.hybrid.dies);
> +    tmp_idx = cpu_index % cpus_per_die;
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cluster = &ms->topo.hybrid.cluster_list[i];
> +
> +        for (int j = 0; j < cluster->cores; j++) {
> +            core = &cluster->core_list[j];
> +
> +            if (tmp_idx < core->threads) {
> +                return core->threads;
> +            } else {
> +                tmp_idx -= core->threads;
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> +                                           unsigned int cpu_index) {
> +    unsigned cpus_per_die;
> +    unsigned tmp_idx;
> +    HybridCluster *cluster;
> +    HybridCore *core;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores;
> +    }
> +
> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> +                            ms->topo.hybrid.dies);
> +    tmp_idx = cpu_index % cpus_per_die;
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cluster = &ms->topo.hybrid.cluster_list[i];
> +
> +        for (int j = 0; j < cluster->cores; j++) {
> +            core = &cluster->core_list[j];
> +
> +            if (tmp_idx < core->threads) {
> +                return cluster->cores;
> +            } else {
> +                tmp_idx -= core->threads;
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +

It looks most code of this function is same with previous function. Could we extract
the same part code and define a low level function?

> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
> +{
> +    unsigned int cores_per_die = 0;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
> +    }
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
> +    }
> +
> +    return cores_per_die * ms->topo.hybrid.dies; }
> +
> +unsigned int machine_topo_get_threads_per_socket(const MachineState
> +*ms) {
> +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> +                           ms->topo.hybrid.sockets;
> +    return ms->topo.max_cpus / sockets; }
> +
>  /*
>   * Report information of a machine's supported CPU topology hierarchy.
>   * Topology members will be ordered from the largest to the smallest diff --git
> a/include/hw/boards.h b/include/hw/boards.h index
> 0a61855499e3..34b64b012022 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;  extern
> GlobalProperty hw_compat_2_1[];  extern const size_t hw_compat_2_1_len;
> 
> +static inline
> +unsigned int machine_topo_get_cpus(const MachineState *ms) {
> +    return ms->topo.cpus;
> +}
> +
> +static inline
> +unsigned int machine_topo_get_max_cpus(const MachineState *ms) {
> +    return ms->topo.max_cpus;
> +}
> +
> +static inline
> +bool machine_topo_is_smp(const MachineState *ms) {
> +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP; }
> +
> +unsigned int machine_topo_get_sockets(const MachineState *ms); unsigned
> +int machine_topo_get_dies(const MachineState *ms); unsigned int
> +machine_topo_get_clusters(const MachineState *ms); unsigned int
> +machine_topo_get_smp_cores(const MachineState *ms); unsigned int
> +machine_topo_get_smp_threads(const MachineState *ms); unsigned int
> +machine_topo_get_threads(const MachineState *ms,
> +                                      unsigned int cluster_id,
> +                                      unsigned int core_id); unsigned
> +int machine_topo_get_cores(const MachineState *ms,
> +                                    unsigned int cluster_id); unsigned
> +int machine_topo_get_threads_by_idx(const MachineState *ms,
> +                                             unsigned int cpu_index);
> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> +                                           unsigned int cpu_index);
> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
> +unsigned int machine_topo_get_threads_per_socket(const MachineState
> +*ms);
> +
>  #endif
> --
> 2.34.1

RE: [RFC 09/52] hw/machine: Introduce core type for hybrid topology

[Mi, Dapeng1]

> From: Zhao Liu <zhao1.liu@linux.intel.com>
> Sent: Monday, February 13, 2023 5:50 PM
> To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daudé <philmd@linaro.org>;
> Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>
> Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> Christopherson,, Sean <seanjc@google.com>; Like Xu
> <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>
> Subject: [RFC 09/52] hw/machine: Introduce core type for hybrid topology
> 
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> Under the hybrid cpu topology, some CPUs care about the core type.
> For example, Intel's Alder Lake series CPU contains two types of cores:
> Intel Core and Intel Atom. The type information of these two types is exposed in
> 1A leaf of CPUID.
> 
> Core type should also be part of the hybrid topology, and
> MachineState.cpu_type cannot provide different type information for different
> cpus in the same machine, so add a type field for the core level in the hybrid cpu
> topology.
> 
> Additionally, add a helper to get core type information from MachineState.
> Though core_type is only used in hybrid case, don't use assert since it may be
> used to initialize some generic fields.
> 
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>  hw/core/machine-topo.c        | 12 ++++++++++++
>  include/hw/boards.h           |  3 +++
>  include/hw/cpu/cpu-topology.h |  2 ++
>  3 files changed, 17 insertions(+)
> 
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c index
> b20160479629..e0ec07b53d42 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -51,6 +51,18 @@ unsigned int machine_topo_get_smp_threads(const
> MachineState *ms)
>      return ms->topo.smp.threads;
>  }
> 
> +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> +                                               unsigned int cluster_id,
> +                                               unsigned int core_id) {
> +    if (!machine_topo_is_smp(ms)) {
> +        return ms->topo.hybrid.cluster_list[cluster_id]
> +                       .core_list[core_id].core_type;
> +    } else {
> +        return 0;
> +    }
> +}
> +

We'd better not to return the hard-coded '0'. Suggest to define a 'enum'
data structure to represent the core_type. This makes the code look more intuitively.

>  unsigned int machine_topo_get_threads(const MachineState *ms,
>                                        unsigned int cluster_id,
>                                        unsigned int core_id) diff --git a/include/hw/boards.h
> b/include/hw/boards.h index 34b64b012022..78e52af38cb1 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -484,6 +484,9 @@ unsigned int machine_topo_get_dies(const
> MachineState *ms);  unsigned int machine_topo_get_clusters(const
> MachineState *ms);  unsigned int machine_topo_get_smp_cores(const
> MachineState *ms);  unsigned int machine_topo_get_smp_threads(const
> MachineState *ms);
> +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> +                                               unsigned int cluster_id,
> +                                               unsigned int core_id);
>  unsigned int machine_topo_get_threads(const MachineState *ms,
>                                        unsigned int cluster_id,
>                                        unsigned int core_id); diff --git a/include/hw/cpu/cpu-
> topology.h b/include/hw/cpu/cpu-topology.h index
> 8268ea3a8569..87d832556229 100644
> --- a/include/hw/cpu/cpu-topology.h
> +++ b/include/hw/cpu/cpu-topology.h
> @@ -45,9 +45,11 @@ typedef struct SmpCpuTopology {
>  /**
>   * HybridCore - hybrid core topology defination:
>   * @threads: the number of threads in one core.
> + * @core_type: the type of current core.
>   */
>  typedef struct HybridCore {
>      unsigned int threads;
> +    unsigned int core_type;
>  } HybridCore;
> 
>  /**
> --
> 2.34.1

Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo properties

[wangyanan (Y) via]

Hi Zhao,

在 2023/2/13 17:50, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> Since hybrid cpu topology configuration can benefit not only x86, but
> also other architectures/platforms that have supported (in real
> machines) or will support hybrid CPU topology, "-hybrid" can be generic.
>
> So add the generic topology property to configure if support hybrid
> cpu topology for architectures/platforms in SmpCompatProps.
>
> Also rename SmpCompatProps to TopoCompatProps to make this structure
> more generic for both smp topology and hybrid topology.
>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   include/hw/boards.h | 15 +++++++++++----
>   1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 34ec035b5c9f..17be3485e823 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -127,19 +127,26 @@ typedef struct {
>   } CPUArchIdList;
>   
>   /**
> - * SMPCompatProps:
> - * @prefer_sockets - whether sockets are preferred over cores in smp parsing
> + * TopoCompatProps:
> + * @hybrid_support - whether hybrid cpu topology are supported by machine.
inconsistent with the name in the definition below.
> + *                   Note that hybrid cpu topology requires to specify the
> + *                   topology of each core so that there will no longer be
> + *                   a default core topology, thus prefer_sockets won't work
> + *                   when hybrid_support is enabled.
> + * @prefer_sockets - whether sockets are preferred over cores in smp parsing.
> + *                   Not work when hybrid_support is enabled.
>    * @dies_supported - whether dies are supported by the machine
>    * @clusters_supported - whether clusters are supported by the machine
>    * @has_clusters - whether clusters are explicitly specified in the user
>    *                 provided SMP configuration
>    */
>   typedef struct {
> +    bool hybrid_supported;
>       bool prefer_sockets;
>       bool dies_supported;
>       bool clusters_supported;
>       bool has_clusters;
> -} SMPCompatProps;
> +} TopoCompatProps;
>   
Also here. "Rename SMPCompatProps to TopoCompatProps and
move it to cpu-topology.h and adapt the code" should be organized
in one or more separate patches, being pre-patches together with
the conversion of CpuTopology before. And put the "hybrid_supported"
extension into another patch. Would this make it easier to review?

Thanks,
Yanan
>   /**
>    * MachineClass:
> @@ -281,7 +288,7 @@ struct MachineClass {
>       bool nvdimm_supported;
>       bool numa_mem_supported;
>       bool auto_enable_numa;
> -    SMPCompatProps smp_props;
> +    TopoCompatProps smp_props;
>       const char *default_ram_id;
>   
>       HotplugHandler *(*get_hotplug_handler)(MachineState *machine,

RE: [RFC 22/52] riscv: Replace MachineState.smp access with topology helpers

[Mi, Dapeng1]

> From: Zhao Liu <zhao1.liu@linux.intel.com>
> Sent: Monday, February 13, 2023 5:50 PM
> To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daudé <philmd@linaro.org>;
> Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>
> Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> Christopherson,, Sean <seanjc@google.com>; Like Xu
> <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>; Meng, Bin
> <bin.meng@windriver.com>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair
> Francis <alistair.francis@wdc.com>; Vijai Kumar K <vijai@behindbytes.com>
> Subject: [RFC 22/52] riscv: Replace MachineState.smp access with topology
> helpers
> 
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> When MachineState.topo is introduced, the topology related structures
> become complicated. So we wrapped the access to topology fields of
> MachineState.topo into some helpers, and we are using these helpers
> to replace the use of MachineState.smp.
> 
> In the codes of riscv, it's straightforward to replace topology access
> with wrapped generic interfaces.
> 
> Cc: Bin Meng <bin.meng@windriver.com>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: Alistair Francis <alistair.francis@wdc.com>
> CC: Vijai Kumar K <vijai@behindbytes.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>  hw/riscv/microchip_pfsoc.c | 11 ++++++-----
>  hw/riscv/numa.c            | 21 +++++++++++----------
>  hw/riscv/opentitan.c       |  8 ++++----
>  hw/riscv/shakti_c.c        |  2 +-
>  hw/riscv/sifive_e.c        | 10 ++++++----
>  hw/riscv/sifive_u.c        | 28 ++++++++++++++--------------
>  hw/riscv/virt.c            | 24 +++++++++++++-----------
>  7 files changed, 55 insertions(+), 49 deletions(-)
> 
> diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c
> index 2b91e49561f1..30295cce17e7 100644
> --- a/hw/riscv/microchip_pfsoc.c
> +++ b/hw/riscv/microchip_pfsoc.c
> @@ -164,7 +164,8 @@ static void microchip_pfsoc_soc_instance_init(Object
> *obj)
> 
>      object_initialize_child(OBJECT(&s->u_cluster), "u-cpus", &s->u_cpus,
>                              TYPE_RISCV_HART_ARRAY);
> -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
> +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts",
> +                         machine_topo_get_cpus(ms) - 1);
>      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
>      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type",
>                           TYPE_RISCV_CPU_SIFIVE_U54);
> @@ -249,10 +250,10 @@ static void microchip_pfsoc_soc_realize(DeviceState
> *dev, Error **errp)
> 
>      /* CLINT */
>      riscv_aclint_swi_create(memmap[MICROCHIP_PFSOC_CLINT].base,
> -        0, ms->smp.cpus, false);
> +        0, machine_topo_get_cpus(ms), false);
>      riscv_aclint_mtimer_create(
>          memmap[MICROCHIP_PFSOC_CLINT].base + RISCV_ACLINT_SWI_SIZE,
> -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, machine_topo_get_cpus(ms),
>          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>          CLINT_TIMEBASE_FREQ, false);
> 
> @@ -276,11 +277,11 @@ static void microchip_pfsoc_soc_realize(DeviceState
> *dev, Error **errp)
>                                  l2lim_mem);
> 
>      /* create PLIC hart topology configuration string */
> -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
> +    plic_hart_config =
> riscv_plic_hart_config_string(machine_topo_get_cpus(ms));
> 
>      /* PLIC */
>      s->plic = sifive_plic_create(memmap[MICROCHIP_PFSOC_PLIC].base,
> -        plic_hart_config, ms->smp.cpus, 0,
> +        plic_hart_config, machine_topo_get_cpus(ms), 0,
>          MICROCHIP_PFSOC_PLIC_NUM_SOURCES,
>          MICROCHIP_PFSOC_PLIC_NUM_PRIORITIES,
>          MICROCHIP_PFSOC_PLIC_PRIORITY_BASE,
> diff --git a/hw/riscv/numa.c b/hw/riscv/numa.c
> index 472010256183..1fabdc42e767 100644
> --- a/hw/riscv/numa.c
> +++ b/hw/riscv/numa.c
> @@ -37,13 +37,14 @@ int riscv_socket_count(const MachineState *ms)
> 
>  int riscv_socket_first_hartid(const MachineState *ms, int socket_id)
>  {
> -    int i, first_hartid = ms->smp.cpus;
> +    int i, first_hartid, cpus = machine_topo_get_cpus(ms);
> 
> +    first_hartid = cpus;
>      if (!numa_enabled(ms)) {
>          return (!socket_id) ? 0 : -1;
>      }
> 
> -    for (i = 0; i < ms->smp.cpus; i++) {
> +    for (i = 0; i < cpus; i++) {
>          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
>              continue;
>          }
> @@ -52,18 +53,18 @@ int riscv_socket_first_hartid(const MachineState *ms,
> int socket_id)
>          }
>      }
> 
> -    return (first_hartid < ms->smp.cpus) ? first_hartid : -1;
> +    return (first_hartid < cpus) ? first_hartid : -1;
>  }
> 
>  int riscv_socket_last_hartid(const MachineState *ms, int socket_id)
>  {
> -    int i, last_hartid = -1;
> +    int i, last_hartid = -1, cpus = machine_topo_get_cpus(ms);
> 
>      if (!numa_enabled(ms)) {
> -        return (!socket_id) ? ms->smp.cpus - 1 : -1;
> +        return (!socket_id) ? cpus - 1 : -1;
>      }
> 
> -    for (i = 0; i < ms->smp.cpus; i++) {
> +    for (i = 0; i < cpus; i++) {
>          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
>              continue;
>          }
> @@ -72,7 +73,7 @@ int riscv_socket_last_hartid(const MachineState *ms, int
> socket_id)
>          }
>      }
> 
> -    return (last_hartid < ms->smp.cpus) ? last_hartid : -1;
> +    return (last_hartid < cpus) ? last_hartid : -1;
>  }
> 
>  int riscv_socket_hart_count(const MachineState *ms, int socket_id)
> @@ -80,7 +81,7 @@ int riscv_socket_hart_count(const MachineState *ms, int
> socket_id)
>      int first_hartid, last_hartid;
> 
>      if (!numa_enabled(ms)) {
> -        return (!socket_id) ? ms->smp.cpus : -1;
> +        return (!socket_id) ? machine_topo_get_cpus(ms) : -1;
>      }
> 
>      first_hartid = riscv_socket_first_hartid(ms, socket_id);
> @@ -208,7 +209,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
> MachineState *ms, int idx)
>      int64_t nidx = 0;
> 
>      if (ms->numa_state->num_nodes) {
> -        nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes);
> +        nidx = idx / (machine_topo_get_cpus(ms) / ms->numa_state->num_nodes);
>          if (ms->numa_state->num_nodes <= nidx) {
>              nidx = ms->numa_state->num_nodes - 1;
>          }
> @@ -220,7 +221,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
> MachineState *ms, int idx)
>  const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms)
>  {
>      int n;
> -    unsigned int max_cpus = ms->smp.max_cpus;
> +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> 
>      if (ms->possible_cpus) {
>          assert(ms->possible_cpus->len == max_cpus);
> diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
> index 353f030d80c1..fb59c43dcc08 100644
> --- a/hw/riscv/opentitan.c
> +++ b/hw/riscv/opentitan.c
> @@ -143,11 +143,11 @@ static void lowrisc_ibex_soc_realize(DeviceState
> *dev_soc, Error **errp)
>      MachineState *ms = MACHINE(qdev_get_machine());
>      LowRISCIbexSoCState *s = RISCV_IBEX_SOC(dev_soc);
>      MemoryRegion *sys_mem = get_system_memory();
> -    int i;
> +    int i, cpus = machine_topo_get_cpus(ms);
> 
>      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
>                              &error_abort);
> -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
> +    object_property_set_int(OBJECT(&s->cpus), "num-harts", cpus,
>                              &error_abort);
>      object_property_set_int(OBJECT(&s->cpus), "resetvec", s->resetvec,
>                              &error_abort);
> @@ -186,10 +186,10 @@ static void lowrisc_ibex_soc_realize(DeviceState
> *dev_soc, Error **errp)
>      }
>      sysbus_mmio_map(SYS_BUS_DEVICE(&s->plic), 0,
> memmap[IBEX_DEV_PLIC].base);
> 
> -    for (i = 0; i < ms->smp.cpus; i++) {
> +    for (i = 0; i < cpus; i++) {
>          CPUState *cpu = qemu_get_cpu(i);
> 
> -        qdev_connect_gpio_out(DEVICE(&s->plic), ms->smp.cpus + i,
> +        qdev_connect_gpio_out(DEVICE(&s->plic), cpus + i,
>                                qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
>      }
> 
> diff --git a/hw/riscv/shakti_c.c b/hw/riscv/shakti_c.c
> index e43cc9445cb1..34499510e2f0 100644
> --- a/hw/riscv/shakti_c.c
> +++ b/hw/riscv/shakti_c.c
> @@ -110,7 +110,7 @@ static void shakti_c_soc_state_realize(DeviceState *dev,
> Error **errp)
>      sysbus_realize(SYS_BUS_DEVICE(&sss->cpus), &error_abort);
> 
>      sss->plic = sifive_plic_create(shakti_c_memmap[SHAKTI_C_PLIC].base,
> -        (char *)SHAKTI_C_PLIC_HART_CONFIG, ms->smp.cpus, 0,
> +        (char *)SHAKTI_C_PLIC_HART_CONFIG, machine_topo_get_cpus(ms), 0,
>          SHAKTI_C_PLIC_NUM_SOURCES,
>          SHAKTI_C_PLIC_NUM_PRIORITIES,
>          SHAKTI_C_PLIC_PRIORITY_BASE,
> diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c
> index 3e3f4b0088ca..ca3a7a0877cd 100644
> --- a/hw/riscv/sifive_e.c
> +++ b/hw/riscv/sifive_e.c
> @@ -178,7 +178,8 @@ static void sifive_e_soc_init(Object *obj)
>      SiFiveESoCState *s = RISCV_E_SOC(obj);
> 
>      object_initialize_child(obj, "cpus", &s->cpus, TYPE_RISCV_HART_ARRAY);
> -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
> +    object_property_set_int(OBJECT(&s->cpus), "num-harts",
> +                            machine_topo_get_cpus(ms),
>                              &error_abort);

With this change, we don't need to start a new line for '&error_abort);', just move
it to previous line.

>      object_property_set_int(OBJECT(&s->cpus), "resetvec", 0x1004,
> &error_abort);
>      object_initialize_child(obj, "riscv.sifive.e.gpio0", &s->gpio,
> @@ -191,6 +192,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
> **errp)
>      const MemMapEntry *memmap = sifive_e_memmap;
>      SiFiveESoCState *s = RISCV_E_SOC(dev);
>      MemoryRegion *sys_mem = get_system_memory();
> +    unsigned int cpus = machine_topo_get_cpus(ms);
> 
>      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
>                              &error_abort);
> @@ -204,7 +206,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
> **errp)
> 
>      /* MMIO */
>      s->plic = sifive_plic_create(memmap[SIFIVE_E_DEV_PLIC].base,
> -        (char *)SIFIVE_E_PLIC_HART_CONFIG, ms->smp.cpus, 0,
> +        (char *)SIFIVE_E_PLIC_HART_CONFIG, cpus, 0,
>          SIFIVE_E_PLIC_NUM_SOURCES,
>          SIFIVE_E_PLIC_NUM_PRIORITIES,
>          SIFIVE_E_PLIC_PRIORITY_BASE,
> @@ -215,10 +217,10 @@ static void sifive_e_soc_realize(DeviceState *dev,
> Error **errp)
>          SIFIVE_E_PLIC_CONTEXT_STRIDE,
>          memmap[SIFIVE_E_DEV_PLIC].size);
>      riscv_aclint_swi_create(memmap[SIFIVE_E_DEV_CLINT].base,
> -        0, ms->smp.cpus, false);
> +        0, cpus, false);
>      riscv_aclint_mtimer_create(memmap[SIFIVE_E_DEV_CLINT].base +
>              RISCV_ACLINT_SWI_SIZE,
> -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
>          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>          RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, false);
>      create_unimplemented_device("riscv.sifive.e.aon",
> diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> index d3ab7a9cda5d..c7f969510209 100644
> --- a/hw/riscv/sifive_u.c
> +++ b/hw/riscv/sifive_u.c
> @@ -99,7 +99,7 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>      MachineState *ms = MACHINE(s);
>      uint64_t mem_size = ms->ram_size;
>      void *fdt;
> -    int cpu, fdt_size;
> +    int cpu, fdt_size, num_cpus = machine_topo_get_cpus(ms);
>      uint32_t *cells;
>      char *nodename;
>      uint32_t plic_phandle, prci_phandle, gpio_phandle, phandle = 1;
> @@ -175,7 +175,7 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>      qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
>      qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
> 
> -    for (cpu = ms->smp.cpus - 1; cpu >= 0; cpu--) {
> +    for (cpu = num_cpus - 1; cpu >= 0; cpu--) {
>          int cpu_phandle = phandle++;
>          nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
>          char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
> @@ -207,8 +207,8 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>          g_free(nodename);
>      }
> 
> -    cells =  g_new0(uint32_t, ms->smp.cpus * 4);
> -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> +    cells =  g_new0(uint32_t, num_cpus * 4);
> +    for (cpu = 0; cpu < num_cpus; cpu++) {
>          nodename =
>              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
>          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
> @@ -227,7 +227,7 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>          0x0, memmap[SIFIVE_U_DEV_CLINT].base,
>          0x0, memmap[SIFIVE_U_DEV_CLINT].size);
>      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
> -        cells, ms->smp.cpus * sizeof(uint32_t) * 4);
> +        cells, num_cpus * sizeof(uint32_t) * 4);
>      g_free(cells);
>      g_free(nodename);
> 
> @@ -258,8 +258,8 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>      g_free(nodename);
> 
>      plic_phandle = phandle++;
> -    cells =  g_new0(uint32_t, ms->smp.cpus * 4 - 2);
> -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> +    cells =  g_new0(uint32_t, machine_topo_get_cpus(ms) * 4 - 2);
> +    for (cpu = 0; cpu < machine_topo_get_cpus(ms); cpu++) {
>          nodename =
>              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
>          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
> @@ -283,7 +283,7 @@ static void create_fdt(SiFiveUState *s, const
> MemMapEntry *memmap,
>          (char **)&plic_compat, ARRAY_SIZE(plic_compat));
>      qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0);
>      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
> -        cells, (ms->smp.cpus * 4 - 2) * sizeof(uint32_t));
> +        cells, (machine_topo_get_cpus(ms) * 4 - 2) * sizeof(uint32_t));
>      qemu_fdt_setprop_cells(fdt, nodename, "reg",
>          0x0, memmap[SIFIVE_U_DEV_PLIC].base,
>          0x0, memmap[SIFIVE_U_DEV_PLIC].size);
> @@ -797,10 +797,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
> Error **errp)
>      MemoryRegion *mask_rom = g_new(MemoryRegion, 1);
>      MemoryRegion *l2lim_mem = g_new(MemoryRegion, 1);
>      char *plic_hart_config;
> -    int i, j;
> +    int i, j, cpus = machine_topo_get_cpus(ms);
>      NICInfo *nd = &nd_table[0];
> 
> -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
> +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", cpus - 1);
>      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
>      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type", s->cpu_type);
>      qdev_prop_set_uint64(DEVICE(&s->u_cpus), "resetvec", 0x1004);
> @@ -837,11 +837,11 @@ static void sifive_u_soc_realize(DeviceState *dev,
> Error **errp)
>                                  l2lim_mem);
> 
>      /* create PLIC hart topology configuration string */
> -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
> +    plic_hart_config = riscv_plic_hart_config_string(cpus);
> 
>      /* MMIO */
>      s->plic = sifive_plic_create(memmap[SIFIVE_U_DEV_PLIC].base,
> -        plic_hart_config, ms->smp.cpus, 0,
> +        plic_hart_config, cpus, 0,
>          SIFIVE_U_PLIC_NUM_SOURCES,
>          SIFIVE_U_PLIC_NUM_PRIORITIES,
>          SIFIVE_U_PLIC_PRIORITY_BASE,
> @@ -857,10 +857,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
> Error **errp)
>      sifive_uart_create(system_memory, memmap[SIFIVE_U_DEV_UART1].base,
>          serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ));
>      riscv_aclint_swi_create(memmap[SIFIVE_U_DEV_CLINT].base, 0,
> -        ms->smp.cpus, false);
> +        cpus, false);
>      riscv_aclint_mtimer_create(memmap[SIFIVE_U_DEV_CLINT].base +
>              RISCV_ACLINT_SWI_SIZE,
> -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
>          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>          CLINT_TIMEBASE_FREQ, false);
> 
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index b81081c70b1c..4b0a3dab5ec1 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -506,16 +506,17 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      char *imsic_name;
>      MachineState *ms = MACHINE(s);
>      int socket_count = riscv_socket_count(ms);
> -    uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
> +    uint32_t imsic_max_hart_per_socket, imsic_guest_bits, cpus;
>      uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
> 
> +    cpus = machine_topo_get_cpus(ms);
>      *msi_m_phandle = (*phandle)++;
>      *msi_s_phandle = (*phandle)++;
> -    imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
> +    imsic_cells = g_new0(uint32_t, cpus * 2);
>      imsic_regs = g_new0(uint32_t, socket_count * 4);
> 
>      /* M-level IMSIC node */
> -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> +    for (cpu = 0; cpu < cpus; cpu++) {
>          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
>          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
>      }
> @@ -544,7 +545,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
>          NULL, 0);
>      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
> -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
> +        imsic_cells, cpus * sizeof(uint32_t) * 2);
>      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
>          socket_count * sizeof(uint32_t) * 4);
>      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
> @@ -562,7 +563,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      g_free(imsic_name);
> 
>      /* S-level IMSIC node */
> -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> +    for (cpu = 0; cpu < cpus; cpu++) {
>          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
>          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
>      }
> @@ -593,7 +594,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
>          NULL, 0);
>      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
> -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
> +        imsic_cells, cpus * sizeof(uint32_t) * 2);
>      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
>          socket_count * sizeof(uint32_t) * 4);
>      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
> @@ -732,7 +733,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      char *clust_name;
>      int socket, phandle_pos;
>      MachineState *ms = MACHINE(s);
> -    uint32_t msi_m_phandle = 0, msi_s_phandle = 0;
> +    uint32_t msi_m_phandle = 0, msi_s_phandle = 0, cpus;
>      uint32_t *intc_phandles, xplic_phandles[MAX_NODES];
>      int socket_count = riscv_socket_count(ms);
> 
> @@ -743,9 +744,10 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> MemMapEntry *memmap,
>      qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
>      qemu_fdt_add_subnode(ms->fdt, "/cpus/cpu-map");
> 
> -    intc_phandles = g_new0(uint32_t, ms->smp.cpus);
> +    cpus = machine_topo_get_cpus(ms);
> +    intc_phandles = g_new0(uint32_t, cpus);
> 
> -    phandle_pos = ms->smp.cpus;
> +    phandle_pos = cpus;
>      for (socket = (socket_count - 1); socket >= 0; socket--) {
>          phandle_pos -= s->soc[socket].num_harts;
> 
> @@ -776,7 +778,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> MemMapEntry *memmap,
>          *msi_pcie_phandle = msi_s_phandle;
>      }
> 
> -    phandle_pos = ms->smp.cpus;
> +    phandle_pos = cpus;
>      for (socket = (socket_count - 1); socket >= 0; socket--) {
>          phandle_pos -= s->soc[socket].num_harts;
> 
> @@ -1113,7 +1115,7 @@ static FWCfgState *create_fw_cfg(const
> MachineState *ms)
> 
>      fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16,
>                                    &address_space_memory);
> -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
> +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
> (uint16_t)machine_topo_get_cpus(ms));
> 
>      return fw_cfg;
>  }
> --
> 2.34.1

RE: [RFC 25/52] mips: Replace MachineState.smp access with topology helpers

[Mi, Dapeng1]

> From: Zhao Liu <zhao1.liu@linux.intel.com>
> Sent: Monday, February 13, 2023 5:50 PM
> To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daudé <philmd@linaro.org>;
> Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> Armbruster <armbru@redhat.com>
> Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> Christopherson,, Sean <seanjc@google.com>; Like Xu
> <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>; Aleksandar
> Rikalo <aleksandar.rikalo@syrmia.com>; Huacai Chen
> <chenhuacai@kernel.org>; Jiaxun Yang <jiaxun.yang@flygoat.com>; Aurelien
> Jarno <aurelien@aurel32.net>
> Subject: [RFC 25/52] mips: Replace MachineState.smp access with topology
> helpers
> 
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> When MachineState.topo is introduced, the topology related structures
> become complicated. So we wrapped the access to topology fields of
> MachineState.topo into some helpers, and we are using these helpers
> to replace the use of MachineState.smp.
> 
> For mips, it's straightforward to replace topology access with wrapped
> generic interfaces.
> 
> Cc: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>
> Cc: Huacai Chen <chenhuacai@kernel.org>
> Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
> Cc: Aurelien Jarno <aurelien@aurel32.net>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>  hw/mips/boston.c          |  8 +++++---
>  hw/mips/loongson3_bootp.c |  7 ++++---
>  hw/mips/loongson3_virt.c  |  8 +++++---
>  hw/mips/malta.c           | 10 ++++++----
>  4 files changed, 20 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/mips/boston.c b/hw/mips/boston.c
> index a9d87f34378f..d02c88d769d4 100644
> --- a/hw/mips/boston.c
> +++ b/hw/mips/boston.c
> @@ -514,7 +514,7 @@ static const void *create_fdt(BostonState *s,
>                                const MemMapEntry *memmap, int *dt_size)
>  {
>      void *fdt;
> -    int cpu;
> +    int cpu, smp_cpus;
>      MachineState *ms = s->mach;
>      uint32_t platreg_ph, gic_ph, clk_ph;
>      char *name, *gic_name, *platreg_name, *stdout_name;
> @@ -542,7 +542,8 @@ static const void *create_fdt(BostonState *s,
>      qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
>      qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
> 
> -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> +    smp_cpus = machine_topo_get_cpus(ms);
> +    for (cpu = 0; cpu < smp_cpus; cpu++) {
>          name = g_strdup_printf("/cpus/cpu@%d", cpu);
>          qemu_fdt_add_subnode(fdt, name);
>          qemu_fdt_setprop_string(fdt, name, "compatible", "img,mips");
> @@ -702,7 +703,8 @@ static void boston_mach_init(MachineState *machine)
>      object_initialize_child(OBJECT(machine), "cps", &s->cps, TYPE_MIPS_CPS);
>      object_property_set_str(OBJECT(&s->cps), "cpu-type", machine->cpu_type,
>                              &error_fatal);
> -    object_property_set_int(OBJECT(&s->cps), "num-vp", machine->smp.cpus,
> +    object_property_set_int(OBJECT(&s->cps), "num-vp",
> +                            machine_topo_get_cpus(machine),
>                              &error_fatal);

It's better move "&error_fatal);" to previous line.

>      qdev_connect_clock_in(DEVICE(&s->cps), "clk-in",
>                            qdev_get_clock_out(dev, "cpu-refclk"));
> diff --git a/hw/mips/loongson3_bootp.c b/hw/mips/loongson3_bootp.c
> index f99af229327a..d9a92825ceae 100644
> --- a/hw/mips/loongson3_bootp.c
> +++ b/hw/mips/loongson3_bootp.c
> @@ -40,9 +40,10 @@ static void init_cpu_info(void *g_cpuinfo, uint64_t
> cpu_freq)
>      }
> 
>      c->cpu_startup_core_id = cpu_to_le16(0);
> -    c->nr_cpus = cpu_to_le32(current_machine->smp.cpus);
> -    c->total_node = cpu_to_le32(DIV_ROUND_UP(current_machine->smp.cpus,
> -                                             LOONGSON3_CORE_PER_NODE));
> +    c->nr_cpus = cpu_to_le32(machine_topo_get_cpus(current_machine));
> +    c->total_node =
> +        cpu_to_le32(DIV_ROUND_UP(machine_topo_get_cpus(current_machine),
> +                                 LOONGSON3_CORE_PER_NODE));
>  }
> 
>  static void init_memory_map(void *g_map, uint64_t ram_size)
> diff --git a/hw/mips/loongson3_virt.c b/hw/mips/loongson3_virt.c
> index 25534288dd81..c972bb43a1f7 100644
> --- a/hw/mips/loongson3_virt.c
> +++ b/hw/mips/loongson3_virt.c
> @@ -271,8 +271,10 @@ static void fw_conf_init(unsigned long ram_size)
>      hwaddr cfg_addr = virt_memmap[VIRT_FW_CFG].base;
> 
>      fw_cfg = fw_cfg_init_mem_wide(cfg_addr, cfg_addr + 8, 8, 0, NULL);
> -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)current_machine-
> >smp.cpus);
> -    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)current_machine-
> >smp.max_cpus);
> +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
> +                   (uint16_t)machine_topo_get_cpus(current_machine));
> +    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
> +                   (uint16_t)machine_topo_get_max_cpus(current_machine));
>      fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)ram_size);
>      fw_cfg_add_i32(fw_cfg, FW_CFG_MACHINE_VERSION, 1);
>      fw_cfg_add_i64(fw_cfg, FW_CFG_CPU_FREQ, get_cpu_freq_hz());
> @@ -528,7 +530,7 @@ static void mips_loongson3_virt_init(MachineState
> *machine)
>      cpuclk = clock_new(OBJECT(machine), "cpu-refclk");
>      clock_set_hz(cpuclk, DEF_LOONGSON3_FREQ);
> 
> -    for (i = 0; i < machine->smp.cpus; i++) {
> +    for (i = 0; i < machine_topo_get_cpus(machine); i++) {
>          int ip;
> 
>          /* init CPUs */
> diff --git a/hw/mips/malta.c b/hw/mips/malta.c
> index ec172b111ae4..b3322f74baf5 100644
> --- a/hw/mips/malta.c
> +++ b/hw/mips/malta.c
> @@ -981,7 +981,7 @@ static uint64_t load_kernel(void)
>  static void malta_mips_config(MIPSCPU *cpu)
>  {
>      MachineState *ms = MACHINE(qdev_get_machine());
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>      CPUMIPSState *env = &cpu->env;
>      CPUState *cs = CPU(cpu);
> 
> @@ -1045,7 +1045,7 @@ static void create_cpu_without_cps(MachineState
> *ms, MaltaState *s,
>      MIPSCPU *cpu;
>      int i;
> 
> -    for (i = 0; i < ms->smp.cpus; i++) {
> +    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
>          cpu = mips_cpu_create_with_clock(ms->cpu_type, s->cpuclk);
> 
>          /* Init internal devices */
> @@ -1066,7 +1066,8 @@ static void create_cps(MachineState *ms, MaltaState
> *s,
>      object_initialize_child(OBJECT(s), "cps", &s->cps, TYPE_MIPS_CPS);
>      object_property_set_str(OBJECT(&s->cps), "cpu-type", ms->cpu_type,
>                              &error_fatal);
> -    object_property_set_int(OBJECT(&s->cps), "num-vp", ms->smp.cpus,
> +    object_property_set_int(OBJECT(&s->cps), "num-vp",
> +                            machine_topo_get_cpus(ms),
>                              &error_fatal);
>      qdev_connect_clock_in(DEVICE(&s->cps), "clk-in", s->cpuclk);
>      sysbus_realize(SYS_BUS_DEVICE(&s->cps), &error_fatal);
> @@ -1080,7 +1081,8 @@ static void create_cps(MachineState *ms, MaltaState
> *s,
>  static void mips_create_cpu(MachineState *ms, MaltaState *s,
>                              qemu_irq *cbus_irq, qemu_irq *i8259_irq)
>  {
> -    if ((ms->smp.cpus > 1) && cpu_type_supports_cps_smp(ms->cpu_type)) {
> +    if ((machine_topo_get_cpus(ms) > 1) &&
> +        cpu_type_supports_cps_smp(ms->cpu_type)) {
>          create_cps(ms, s, cbus_irq, i8259_irq);
>      } else {
>          create_cpu_without_cps(ms, s, cbus_irq, i8259_irq);
> --
> 2.34.1

Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name and helper

[Zhao Liu]

On Mon, Feb 13, 2023 at 11:07:33AM +0100, Juan Quintela wrote:
> Date: Mon, 13 Feb 2023 11:07:33 +0100
> From: Juan Quintela <quintela@redhat.com>
> Subject: Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name
>  and helper
> 
> Zhao Liu <zhao1.liu@linux.intel.com> wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> >
> > As the generic code, here we should respect the different topologies:
> > smp or hybrid.
> >
> > So rename PostcopyBlocktimeContext.smp_cpus_down to
> > PostcopyBlocktimeContext.cpus_down, and also rename other local
> > variables from smp_cpus to cpus_num, to decouple with smp topology.
> >
> > And use generic topology helpers to get topology information.
> >
> > Cc: Juan Quintela <quintela@redhat.com>
> > Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> 
> Reviewed-by: Juan Quintela <quintela@redhat.com>
> 
> but if you ever have to rebase.
> 
> > ---
> >  migration/postcopy-ram.c | 24 ++++++++++++------------
> >  1 file changed, 12 insertions(+), 12 deletions(-)
> >
> > diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c
> > index 53299b7a5ebd..1e861e313258 100644
> > --- a/migration/postcopy-ram.c
> > +++ b/migration/postcopy-ram.c
> > @@ -122,7 +122,7 @@ typedef struct PostcopyBlocktimeContext {
> >      /* point in time when last page fault was initiated */
> >      uint32_t last_begin;
> >      /* number of vCPU are suspended */
> > -    int smp_cpus_down;
> > +    int cpus_down;
> 
> Put the rename of the variable in a single patch.  Trivial to review.
> 
> > +    unsigned int cpus_num = machine_topo_get_cpus(ms);
> 
> Put the meat in another patch.  I think you call this function in two
> places instead of the old one.
> 

Thanks Juan! On the next send, I'll do the split.

> 
> Later, Juan.
> 

Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name and helper

[Zhao Liu]

On Mon, Feb 13, 2023 at 11:16:13AM +0100, Juan Quintela wrote:
> Date: Mon, 13 Feb 2023 11:16:13 +0100
> From: Juan Quintela <quintela@redhat.com>
> Subject: Re: [RFC 15/52] migration/postcopy-ram: Use generic topology name
>  and helper
> 
> Zhao Liu <zhao1.liu@linux.intel.com> wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> >
> > As the generic code, here we should respect the different topologies:
> > smp or hybrid.
> >
> > So rename PostcopyBlocktimeContext.smp_cpus_down to
> > PostcopyBlocktimeContext.cpus_down, and also rename other local
> > variables from smp_cpus to cpus_num, to decouple with smp topology.
> >
> > And use generic topology helpers to get topology information.
> >
> > Cc: Juan Quintela <quintela@redhat.com>
> > Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> 
> I expect that this will go with your other changes through another tree,

Yes, this one is based on my another patch series. It is currently sent
out for preview in advance. I think it may take a long time before the
entire hybrid patchset is accepted. Thanks for your review!

Zhao

> or do you want me to take it through migration thread?
> 
> Later, Juan.
> 

Re: [RFC 00/52] Introduce hybrid CPU topology

[Zhao Liu]

On Mon, Feb 13, 2023 at 10:14:22AM +0000, Alex Bennée wrote:
> Date: Mon, 13 Feb 2023 10:14:22 +0000
> From: Alex Bennée <alex.bennee@linaro.org>
> Subject: Re: [RFC 00/52] Introduce hybrid CPU topology
> 
> 
> Zhao Liu <zhao1.liu@linux.intel.com> writes:
> 
> > From: Zhao Liu <zhao1.liu@intel.com>
> >
> > Hi list,
> >
> > This RFC series is to introduce hybrid topology in QEMU, and is based
> > on our previous patches about cluster support for x86 [1] ([PATCH RESEND
> > 00/18] Support smp.clusters for x86).
> >
> > Though our cluster support patches [1] has not yet accepted, we are
> > still posting the hybrid topology support patches in advance, hoping to
> > get discussion and feedbacks.
> >
> > In this RFC series, we mainly have the following work:
> > * Introduce the new cpu topology under heterogeneous architecture, which
> >   we called "hybrid" cpu topology.
> >       - since the name "hmp" is already used in QEMU and "hybrid" is
> >         also a synonym for heterogeneous.
> > * Reorganized the relevant topology information in qemu to adapt to the
> >   hybrid topology.
> > * Introduce a new command "hybrid", which allows user to set a custom
> >   hybrid topology.
> >       - currently limited to custom cluster and custom core
> > * Support hybrid topology in i386 PC machine.
> > * Expose basic topology information (topology type and core type) in
> >   query_cpus_fast.
> >
> > We hope that the hybrid topology can be general enough to be compatible
> > with the needs of hybrids with various architectures in the future.
> >
> > Welcome your feedbacks!
> >
> >
> > # 1. Background
> >
> > Heterogeneous computing refers to systems that use more than one kind of
> > processor or cores [2]. Now there are more and more heterogeneous
> > computing architectures, typically including arm's big.LITTLE [3] and
> > intel hybrid architecture [4].
> >
> > The development of heterogeneous computing brings new challenges to
> > virtualization. The first problem we face is how to support the vCPU
> > topology under the heterogeneous architecture.
> >
> > Here, to distinguish it from "smp" cpu topology, we call it "hybrid" cpu
> > topology.
> >
> >
> > # 2. Motivation
> >
> > In a typical hybrid system, different types of core may have different
> > capabilities. This difference may be reflected in different capacities,
> > performance or power efficiencies, and even different instruction-per-
> > cycle (IPC) capabilities. The direct consequence of these differences is
> > to affect the performance of the workload.
> 
> This elides a use case for heterogeneous computing under emulation where
> different cores can have totally different ISA's. We already support
> emulation models where we have varying classes of the same ISA, for
> example xlnx-zyncmp supports 4 A profile and 2 R profile Arm chips.

This is emulation mode, so you also need to define heterogeneous CPU
topology in this case, right?

> 
> I've glanced over the patch set and I don't think there is any direct
> clash with that as this is mainly about representing the topology for
> x86 non-emulation accelerators but I just want ensure we don't end up
> stepping on each others toes.

Yes, this is for non-emulation case.

> 
> > For a virtual machines, vCPUs are just threads, although we can bind
> > vCPUs to different cores to benefit from the different advantages of
> > these physical cores, it's obviously not enough since CPU topology (and
> > cache topology) will still have a significant impact on scheduling
> > performance.
> >
> <snip>
> > ## 3.3. "-hybrid" command
> >
> > For hybrid cpu topology configuration, the original "-smp" lack of
> > flexibility to expand, and unables to customize different cores.
> >
> > So we introduce "-hybrid" command and implement it as the multi-
> > line command. The multi-line command format is more complex than the
> > single-line one, but it can bring stronger scalability and
> > intuitiveness. In the future, it can also be easily extended to more
> > heterogeneous topology levels.
> >
> > "-hybrid" command is as follows:
> >
> > -hybrid socket,sockets=n
> > -hybrid die,dies=n
> > -hybrid cluster,clusters=n
> > -hybrid core,cores=n,type=core_type[,threads=threads]
> >         [,clusterid=cluster]
> >
> > Here, we first define the corresponding qapi options for these 4
> > topology levels: core, cluster, die and socket.
> >
> > We doesn't need a thread level since thread doesn't have different
> > type.
> >
> > For example:
> >
> > -hybrid socket,sockets=1
> > -hybrid die,dies=1
> > -hybrid cluster,clusters=4
> > -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
> > -hybrid core,cores=4,coretype="atom",threads=1
> >
> > Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
> > socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
> > core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
> > cores with 2 threads.
> >
> > Please note this example is not an actual machine topology, but it shows
> > the powerful flexibility of "hybrid" command.
> 
> Is there any intention to apply this command to emulation or will it
> only be for hardware virtualisation? For example the x86 translator
> should be capable of emulating two different classes of x86 with
> different subsets of instructions.

We have not considered the hybrid case in emulation mode, and
the current hybrid CPUs of intel are same-ISA.

> It requires ensuring each CPU is
> assigned to the correct TYPE_CPU_CLUSTER so we don't re-use TBs from
> the "wrong" CPU.

From my understanding, this also seems like the topology of heterogeneous
core and heterogeneous cluster.

Zhao

> 
> <snip>
> 
> -- 
> Alex Bennée
> Virtualisation Tech Lead @ Linaro

Re: [RFC 05/52] hw/core/machine: Rename machine-smp.c to machine-topo.c

[Zhao Liu]

On Mon, Feb 13, 2023 at 08:52:13PM +0800, wangyanan (Y) wrote:
> Date: Mon, 13 Feb 2023 20:52:13 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 05/52] hw/core/machine: Rename machine-smp.c to
>  machine-topo.c
> 
> Hi Zhao，
> 在 2023/2/13 17:49, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Hybrid topology support will also be added to this file.
> > 
> > In order to maintain the semantic consistency of the file name and
> > content, the file name is changed to "machine-topo.c".
> > 
> > This file will handle all cpu topology related things, thus also change
> > the file description comment.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   MAINTAINERS                               | 2 +-
> >   hw/core/{machine-smp.c => machine-topo.c} | 2 +-
> >   hw/core/meson.build                       | 2 +-
> >   tests/unit/meson.build                    | 2 +-
> >   4 files changed, 4 insertions(+), 4 deletions(-)
> >   rename hw/core/{machine-smp.c => machine-topo.c} (99%)
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 71c1bc24371b..58794885ced3 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1733,7 +1733,7 @@ F: cpu.c
> >   F: hw/core/cpu.c
> >   F: hw/core/machine-qmp-cmds.c
> >   F: hw/core/machine.c
> > -F: hw/core/machine-smp.c
> > +F: hw/core/machine-topo.c
> >   F: hw/core/null-machine.c
> >   F: hw/core/numa.c
> >   F: hw/cpu/cluster.c
> > diff --git a/hw/core/machine-smp.c b/hw/core/machine-topo.c
> > similarity index 99%
> > rename from hw/core/machine-smp.c
> > rename to hw/core/machine-topo.c
> Better named as cpu-topology.c? So that it's consistent with the
> header "cpu-topology.h" in a following patch.

Nice, I'll rename it.

Zhao

> 
> Thanks,
> Yanan
> > index 3fd9e641efde..8066d2c46bef 100644
> > --- a/hw/core/machine-smp.c
> > +++ b/hw/core/machine-topo.c
> > @@ -1,5 +1,5 @@
> >   /*
> > - * QEMU Machine core (related to -smp parsing)
> > + * QEMU Machine core (related to CPU topology)
> >    *
> >    * Copyright (c) 2021 Huawei Technologies Co., Ltd
> >    *
> > diff --git a/hw/core/meson.build b/hw/core/meson.build
> > index 7a4d02b6c050..71d523dc6826 100644
> > --- a/hw/core/meson.build
> > +++ b/hw/core/meson.build
> > @@ -23,7 +23,7 @@ else
> >   endif
> >   common_ss.add(files('cpu-common.c'))
> > -common_ss.add(files('machine-smp.c'))
> > +common_ss.add(files('machine-topo.c'))
> >   softmmu_ss.add(when: 'CONFIG_FITLOADER', if_true: files('loader-fit.c'))
> >   softmmu_ss.add(when: 'CONFIG_GENERIC_LOADER', if_true: files('generic-loader.c'))
> >   softmmu_ss.add(when: ['CONFIG_GUEST_LOADER', fdt], if_true: files('guest-loader.c'))
> > diff --git a/tests/unit/meson.build b/tests/unit/meson.build
> > index a9df2843e92e..1720a769f1a2 100644
> > --- a/tests/unit/meson.build
> > +++ b/tests/unit/meson.build
> > @@ -46,7 +46,7 @@ tests = {
> >     'test-uuid': [],
> >     'ptimer-test': ['ptimer-test-stubs.c', meson.project_source_root() / 'hw/core/ptimer.c'],
> >     'test-qapi-util': [],
> > -  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-smp.c'],
> > +  'test-smp-parse': [qom, meson.project_source_root() / 'hw/core/machine-topo.c'],
> >     'test-interval-tree': [],
> >   }
> 

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Philippe Mathieu-Daudé]

On 14/2/23 10:30, Zhao Liu wrote:
> On Mon, Feb 13, 2023 at 02:10:17PM +0100, Philippe Mathieu-Daudé wrote:
>> Date: Mon, 13 Feb 2023 14:10:17 +0100
>> From: Philippe Mathieu-Daudé <philmd@linaro.org>
>> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
>>
>> On 13/2/23 10:49, Zhao Liu wrote:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> For smp systems, the parts in one topology level are the same. But now
>>> there are more and more systems with hybrid architectures. Different
>>> parts of the same topology level may have differences. For example,
>>> Intel's Alder Lake series CPU has two types of cores, so the CPU
>>> topology is no longer symmetrical.
>>>
>>> The hybrid topology is compatible with the smp topology type, that is,
>>> different parts on the same level of the hybrid topology can set to be
>>> the same, but the hybrid topology will introduce more complexity (need
>>> to allocate more memory, organized with array or linked-list), so the
>>> original smp topology support is retained while introducing the hybrid
>>> topology, and the hybrid topology is only built when the hybrid is
>>> explicitly required.
>>>
>>> Therefore, we introduce the definition support of hybrid cpu topology
>>> here. At the same time, in order to unify with the original smp, we
>>> introduce a new cpu topology structure that can support smp topology
>>> or hybrid topology. This structure will replace the CpuTopology type (in
>>> include/hw/boards.h) used by MachineState.smp.
>>>
>>> As for now, we only support two hybrid topology levels: core and
>>> cluster.
>>>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    MAINTAINERS                   |   1 +
>>>    include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
>>>    qapi/machine.json             |  12 ++++
>>>    3 files changed, 130 insertions(+)
>>>    create mode 100644 include/hw/cpu/cpu-topology.h
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 58794885ced3..918a9418d98e 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
>>>    F: include/hw/boards.h
>>>    F: include/hw/core/cpu.h
>>>    F: include/hw/cpu/cluster.h
>>> +F: include/hw/cpu/cpu-topology.h
>>>    F: include/sysemu/numa.h
>>>    F: tests/unit/test-smp-parse.c
>>>    T: git https://gitlab.com/ehabkost/qemu.git machine-next
>>
>> Worth splitting this section in 2, machine VS numa/clusters?
> 
> Do you mean splitting this header file into numa.h and cluster.h?

No, I meant the MAINTAINERS 'Machine' section.

> It seems that the cpu topology is not related to the content of
> numa.h,

And you seem to agree ;)

> and it may be possible to merge some content into cluster.h,
> but the definition in cluster.h is to use the cluster as a device
> (if I understand correctly...) , and cpu_topology.h is just about
> topology-related things.
> 
> I have not included the contents of cluster.h/cluster.c into hybrid
> considerations, but only modified the topology defined by smp.
> 
> Zhao

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:10:17PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:10:17 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> 
> On 13/2/23 10:49, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > For smp systems, the parts in one topology level are the same. But now
> > there are more and more systems with hybrid architectures. Different
> > parts of the same topology level may have differences. For example,
> > Intel's Alder Lake series CPU has two types of cores, so the CPU
> > topology is no longer symmetrical.
> > 
> > The hybrid topology is compatible with the smp topology type, that is,
> > different parts on the same level of the hybrid topology can set to be
> > the same, but the hybrid topology will introduce more complexity (need
> > to allocate more memory, organized with array or linked-list), so the
> > original smp topology support is retained while introducing the hybrid
> > topology, and the hybrid topology is only built when the hybrid is
> > explicitly required.
> > 
> > Therefore, we introduce the definition support of hybrid cpu topology
> > here. At the same time, in order to unify with the original smp, we
> > introduce a new cpu topology structure that can support smp topology
> > or hybrid topology. This structure will replace the CpuTopology type (in
> > include/hw/boards.h) used by MachineState.smp.
> > 
> > As for now, we only support two hybrid topology levels: core and
> > cluster.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   MAINTAINERS                   |   1 +
> >   include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
> >   qapi/machine.json             |  12 ++++
> >   3 files changed, 130 insertions(+)
> >   create mode 100644 include/hw/cpu/cpu-topology.h
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 58794885ced3..918a9418d98e 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
> >   F: include/hw/boards.h
> >   F: include/hw/core/cpu.h
> >   F: include/hw/cpu/cluster.h
> > +F: include/hw/cpu/cpu-topology.h
> >   F: include/sysemu/numa.h
> >   F: tests/unit/test-smp-parse.c
> >   T: git https://gitlab.com/ehabkost/qemu.git machine-next
> 
> Worth splitting this section in 2, machine VS numa/clusters?

Do you mean splitting this header file into numa.h and cluster.h?

It seems that the cpu topology is not related to the content of
numa.h, and it may be possible to merge some content into cluster.h,
but the definition in cluster.h is to use the cluster as a device
(if I understand correctly...) , and cpu_topology.h is just about
topology-related things.

I have not included the contents of cluster.h/cluster.c into hybrid
considerations, but only modified the topology defined by smp.

Zhao

Re: [RFC 09/52] hw/machine: Introduce core type for hybrid topology

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:13:40PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:13:40 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 09/52] hw/machine: Introduce core type for hybrid topology
> 
> On 13/2/23 10:49, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Under the hybrid cpu topology, some CPUs care about the core type.
> > For example, Intel's Alder Lake series CPU contains two types of cores:
> > Intel Core and Intel Atom. The type information of these two types is
> > exposed in 1A leaf of CPUID.
> > 
> > Core type should also be part of the hybrid topology, and
> > MachineState.cpu_type cannot provide different type information for
> > different cpus in the same machine, so add a type field for the core
> > level in the hybrid cpu topology.
> > 
> > Additionally, add a helper to get core type information from
> > MachineState. Though core_type is only used in hybrid case, don't
> > use assert since it may be used to initialize some generic fields.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/core/machine-topo.c        | 12 ++++++++++++
> >   include/hw/boards.h           |  3 +++
> >   include/hw/cpu/cpu-topology.h |  2 ++
> >   3 files changed, 17 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> > index b20160479629..e0ec07b53d42 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -51,6 +51,18 @@ unsigned int machine_topo_get_smp_threads(const MachineState *ms)
> >       return ms->topo.smp.threads;
> >   }
> > +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> > +                                               unsigned int cluster_id,
> > +                                               unsigned int core_id)
> > +{
> > +    if (!machine_topo_is_smp(ms)) {
> > +        return ms->topo.hybrid.cluster_list[cluster_id]
> > +                       .core_list[core_id].core_type;
> > +    } else {
> > +        return 0;
> 
> Is '0' an invalid type?

My intention is to use 0 as the default type under smp (it can be
regarded as invalid under hybrid, this should depend on the check
of the architecture itself), I'll use a macro instead of hardcode.

> 
> > +    }
> > +}
> 

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Zhao Liu]

On Mon, Feb 13, 2023 at 09:18:05PM +0800, wangyanan (Y) wrote:
> Date: Mon, 13 Feb 2023 21:18:05 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> 
> Hi Zhao,
> 
> 在 2023/2/13 17:49, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > For smp systems, the parts in one topology level are the same. But now
> > there are more and more systems with hybrid architectures. Different
> > parts of the same topology level may have differences. For example,
> > Intel's Alder Lake series CPU has two types of cores, so the CPU
> > topology is no longer symmetrical.
> > 
> > The hybrid topology is compatible with the smp topology type, that is,
> > different parts on the same level of the hybrid topology can set to be
> > the same, but the hybrid topology will introduce more complexity (need
> > to allocate more memory, organized with array or linked-list), so the
> > original smp topology support is retained while introducing the hybrid
> > topology, and the hybrid topology is only built when the hybrid is
> > explicitly required.
> > 
> > Therefore, we introduce the definition support of hybrid cpu topology
> > here. At the same time, in order to unify with the original smp, we
> > introduce a new cpu topology structure that can support smp topology
> > or hybrid topology. This structure will replace the CpuTopology type (in
> > include/hw/boards.h) used by MachineState.smp.
> > 
> > As for now, we only support two hybrid topology levels: core and
> > cluster.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   MAINTAINERS                   |   1 +
> >   include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
> >   qapi/machine.json             |  12 ++++
> >   3 files changed, 130 insertions(+)
> >   create mode 100644 include/hw/cpu/cpu-topology.h
> > 
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 58794885ced3..918a9418d98e 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
> >   F: include/hw/boards.h
> >   F: include/hw/core/cpu.h
> >   F: include/hw/cpu/cluster.h
> > +F: include/hw/cpu/cpu-topology.h
> Should't it be in include/hw/core/* directory？

Yes, I'll move it to the correct place.

> >   F: include/sysemu/numa.h
> >   F: tests/unit/test-smp-parse.c
> >   T: git https://gitlab.com/ehabkost/qemu.git machine-next
> > diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
> > new file mode 100644
> > index 000000000000..8268ea3a8569
> > --- /dev/null
> > +++ b/include/hw/cpu/cpu-topology.h
> > @@ -0,0 +1,117 @@
> > +/*
> > + * CPU topology defination for Machine core
> > + *
> > + * Copyright (c) 2023 Intel Corporation
> > + * Author: Zhao Liu <zhao1.liu@intel.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License as published by
> > + * the Free Software Foundation; either version 2 of the License,
> > + * or (at your option) any later version.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > + * GNU General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public License
> > + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#ifndef CPU_TOPOLOGY_H
> > +#define CPU_TOPOLOGY_H
> > +
> > +#include "qemu/queue.h"
> > +
> > +/**
> > + * SmpCpuTopology - smp cpu topology defination.
> > + *
> > + * For smp system, the parts in one topology level are the same.
> > + *
> > + * @sockets: the number of sockets on the machine
> > + * @dies: the number of dies in one socket
> > + * @clusters: the number of clusters in one die
> > + * @cores: the number of cores in one cluster
> > + * @threads: the number of threads in one core
> > + */
> > +typedef struct SmpCpuTopology {
> > +    unsigned int sockets;
> > +    unsigned int dies;
> > +    unsigned int clusters;
> > +    unsigned int cores;
> > +    unsigned int threads;
> > +} SmpCpuTopology;
> > +
> > +/**
> > + * HybridCore - hybrid core topology defination:
> > + * @threads: the number of threads in one core.
> > + */
> > +typedef struct HybridCore {
> > +    unsigned int threads;
> > +} HybridCore;
> > +
> > +/**
> > + * HybridCluster - hybrid cluster topology defination:
> > + * @cores: the number of cores in current cluster.
> > + * @core_list: the array includes all the cores that belong to current
> > + *             cluster.
> > + */
> > +typedef struct HybridCluster {
> > +    unsigned int cores;
> > +    HybridCore *core_list;
> > +} HybridCluster;
> > +
> > +/**
> > + * HybridCpuTopology - hybrid cpu topology defination.
> > + *
> > + * At present we only support two heterogeneous topology levels: core
> > + * and cluster. For heterogeneous levels, we need additional structs
> > + * to define their custom internal topology. So here we defines
> > + * symmetric topology levels, and use a list to point to heterogeneous
> > + * levels.
> > + *
> > + * @sockets: the number of sockets on the machine. All sockets are the
> > + *           same.
> > + * @dies: the number of dies in one socket. All dies are the same.
> > + * @clusters: the number of clusters in one die. Cluster may be
> > + *            different. This field indicates the length of
> > + *            cluster_list.
> > + * @cluster_list: the array includes all the clusters in one die.
> > + */
> > +typedef struct HybridCpuTopology {
> > +    unsigned int sockets;
> > +    unsigned int dies;
> > +    unsigned int clusters;
> > +    HybridCluster *cluster_list;
> > +} HybridCpuTopology;
> > +
> > +/**
> > + * GeneralCpuTopology - General cpu topology defination.
> > + *
> > + * It supports one of two topologies: smp topology or hybrid topology.
> > + *
> > + * @cpus: the number of present logical processors on the machine
> > + * @max_cpus: the maximum number of logical processors on the machine
> > + * @topo_type: the topology type of the machine and this decides which
> > + *             member of the union to visit: smp or hybrid.
> > + * @smp: the smp cpu topology informantion. Only valid when topo_type is
> > + *       CPU_TOPO_TYPE_SMP.
> > + * @hybrid: the hybrid cpu topology informantion. Only valid when
> > + *          topo_type is CPU_TOPO_TYPE_HYBRID.
> > + */
> > +typedef struct GeneralCpuTopology {
> > +    unsigned int cpus;
> > +    unsigned int max_cpus;
> > +    CpuTopoType topo_type;
> > +    union {
> > +        SmpCpuTopology smp;
> > +        HybridCpuTopology hybrid;
> > +    };
> > +} GeneralCpuTopology; /*
> > +                       * TODO: This name is temporary, just to distinguish it
> > +                       * from the CpuTopology in boards.h. When CpuTopology in
> > +                       * boards.h is merged here, it will be uniformly named as
> > +                       * CpuTopology.
> > +                       */
> > +
> A suggestion:
> 1、Move definition of CpuTopology from boards.h to cpu-topology.h
> and re-structure it to include SmpCpuTopology, being a generic cpu
> topology structure.
> 2、Rename "CpuTopology smp" in MachineState to a generic name
> "CpuTopology topo".

Here we need to change the access to MachineState.smp to
MachineState.topo.smp in other modules.

If replacement of MachineState.topo is in a single patch, do we also
need to include the modification of access to MachineState.topo.smp
in other modules? Otherwise, it will break the compilation.

In this way, the patch seems be too large.


Thanks,
Zhao

> 3、Adapt all the code in QEMU to above change.
> 
> If you can pack above into a single patch, and then add the hybird
> topology extansion in a next patch, we will not need the temporary
> thing "GeneralCpuTopology" and the TODO comments, which makes
> code clearer.
> 
> Thanks,
> Yanan
> > +#endif /* CPU_TOPOLOGY_H */
> > diff --git a/qapi/machine.json b/qapi/machine.json
> > index b9228a5e4616..bd7303f34497 100644
> > --- a/qapi/machine.json
> > +++ b/qapi/machine.json
> > @@ -36,6 +36,18 @@
> >                'sh4eb', 'sparc', 'sparc64', 'tricore',
> >                'x86_64', 'xtensa', 'xtensaeb' ] }
> > +##
> > +# @CpuTopoType:
> > +#
> > +# An enumeration of cpu topology type
> > +# TODO: Expose topology type in query-cpus-fast
> > +#
> > +# Since: 8.0
> > +##
> > +{ 'enum': 'CpuTopoType',
> > +  'prefix': 'CPU_TOPO_TYPE',
> > +  'data': [ 'smp', 'hybrid' ] }
> > +
> >   ##
> >   # @CpuS390State:
> >   #
> 

Re: [RFC 34/52] i386: Rename variable topo_info to apicid_topo

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:28:54PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:28:54 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 34/52] i386: Rename variable topo_info to apicid_topo
> 
> On 13/2/23 10:50, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Since X86ApicidTopoInfo is only used for APIC ID related work, the
> > common variable topo_info of X86ApicidTopoInfo type should be also
> > renamed to better suit its purpose.
> > 
> > Generic topology access should be obtained from MachineState.topo
> > (for the whole machine) or CPUState.topo (for the current CPU), and
> > apicid_topo (X86ApicidTopoInfo type) is only used to collaborate with
> > APIC ID.
> > 
> > Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> > Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/i386/x86.c                |  38 ++++-----
> >   include/hw/i386/topology.h   |  76 ++++++++---------
> >   include/hw/i386/x86.h        |   2 +-
> >   target/i386/cpu.c            |  71 ++++++++--------
> >   tests/unit/test-x86-apicid.c | 158 +++++++++++++++++------------------
> >   5 files changed, 173 insertions(+), 172 deletions(-)
> 
> The 'id' suffix doesn't add a lot of value IMO. Anyway,
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>

Thanks, it makes sense. I'll remove the id suffix.

Zhao

Re: [RFC 33/52] i386: Rename init_topo_info() to init_apic_topo_info()

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:27:18PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:27:18 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 33/52] i386: Rename init_topo_info() to
>  init_apic_topo_info()
> 
> On 13/2/23 10:50, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Rename init_topo_info() to init_apic_topo_info() to adapt
> > X86ApicidTopoInfo.
> > 
> > Co-Developed-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> > Signed-off-by: Zhuocheng Ding <zhuocheng.ding@intel.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/i386/x86.c         | 12 ++++++------
> >   include/hw/i386/x86.h |  3 ++-
> >   2 files changed, 8 insertions(+), 7 deletions(-)
> 
> 
> > diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> > index ac6f1e4a74af..d84f7717900c 100644
> > --- a/include/hw/i386/x86.h
> > +++ b/include/hw/i386/x86.h
> > @@ -98,7 +98,8 @@ struct X86MachineState {
> >   #define TYPE_X86_MACHINE   MACHINE_TYPE_NAME("x86")
> >   OBJECT_DECLARE_TYPE(X86MachineState, X86MachineClass, X86_MACHINE)
> > -void init_topo_info(X86ApicidTopoInfo *topo_info, const X86MachineState *x86ms);
> > +void init_apicid_topo_info(X86ApicidTopoInfo *topo_info,
> > +                           const X86MachineState *x86ms);
> 
> Maybe s/init_apicid_topo_info/init_apic_topo_info/?
> 
> Otherwise,
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> 
Will remove the "id" suffix. Thanks!

Zhao

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[wangyanan (Y) via]

在 2023/2/14 18:16, Zhao Liu 写道:
> On Mon, Feb 13, 2023 at 09:18:05PM +0800, wangyanan (Y) wrote:
>> Date: Mon, 13 Feb 2023 21:18:05 +0800
>> From: "wangyanan (Y)" <wangyanan55@huawei.com>
>> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
>>
>> Hi Zhao,
>>
>> 在 2023/2/13 17:49, Zhao Liu 写道:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> For smp systems, the parts in one topology level are the same. But now
>>> there are more and more systems with hybrid architectures. Different
>>> parts of the same topology level may have differences. For example,
>>> Intel's Alder Lake series CPU has two types of cores, so the CPU
>>> topology is no longer symmetrical.
>>>
>>> The hybrid topology is compatible with the smp topology type, that is,
>>> different parts on the same level of the hybrid topology can set to be
>>> the same, but the hybrid topology will introduce more complexity (need
>>> to allocate more memory, organized with array or linked-list), so the
>>> original smp topology support is retained while introducing the hybrid
>>> topology, and the hybrid topology is only built when the hybrid is
>>> explicitly required.
>>>
>>> Therefore, we introduce the definition support of hybrid cpu topology
>>> here. At the same time, in order to unify with the original smp, we
>>> introduce a new cpu topology structure that can support smp topology
>>> or hybrid topology. This structure will replace the CpuTopology type (in
>>> include/hw/boards.h) used by MachineState.smp.
>>>
>>> As for now, we only support two hybrid topology levels: core and
>>> cluster.
>>>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    MAINTAINERS                   |   1 +
>>>    include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
>>>    qapi/machine.json             |  12 ++++
>>>    3 files changed, 130 insertions(+)
>>>    create mode 100644 include/hw/cpu/cpu-topology.h
>>>
>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>> index 58794885ced3..918a9418d98e 100644
>>> --- a/MAINTAINERS
>>> +++ b/MAINTAINERS
>>> @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
>>>    F: include/hw/boards.h
>>>    F: include/hw/core/cpu.h
>>>    F: include/hw/cpu/cluster.h
>>> +F: include/hw/cpu/cpu-topology.h
>> Should't it be in include/hw/core/* directory？
> Yes, I'll move it to the correct place.
>
>>>    F: include/sysemu/numa.h
>>>    F: tests/unit/test-smp-parse.c
>>>    T: git https://gitlab.com/ehabkost/qemu.git machine-next
>>> diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
>>> new file mode 100644
>>> index 000000000000..8268ea3a8569
>>> --- /dev/null
>>> +++ b/include/hw/cpu/cpu-topology.h
>>> @@ -0,0 +1,117 @@
>>> +/*
>>> + * CPU topology defination for Machine core
>>> + *
>>> + * Copyright (c) 2023 Intel Corporation
>>> + * Author: Zhao Liu <zhao1.liu@intel.com>
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify
>>> + * it under the terms of the GNU General Public License as published by
>>> + * the Free Software Foundation; either version 2 of the License,
>>> + * or (at your option) any later version.
>>> + *
>>> + * This program is distributed in the hope that it will be useful,
>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> + * GNU General Public License for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License
>>> + * along with this program; if not, see <http://www.gnu.org/licenses/>.
>>> + */
>>> +
>>> +#ifndef CPU_TOPOLOGY_H
>>> +#define CPU_TOPOLOGY_H
>>> +
>>> +#include "qemu/queue.h"
>>> +
>>> +/**
>>> + * SmpCpuTopology - smp cpu topology defination.
>>> + *
>>> + * For smp system, the parts in one topology level are the same.
>>> + *
>>> + * @sockets: the number of sockets on the machine
>>> + * @dies: the number of dies in one socket
>>> + * @clusters: the number of clusters in one die
>>> + * @cores: the number of cores in one cluster
>>> + * @threads: the number of threads in one core
>>> + */
>>> +typedef struct SmpCpuTopology {
>>> +    unsigned int sockets;
>>> +    unsigned int dies;
>>> +    unsigned int clusters;
>>> +    unsigned int cores;
>>> +    unsigned int threads;
>>> +} SmpCpuTopology;
>>> +
>>> +/**
>>> + * HybridCore - hybrid core topology defination:
>>> + * @threads: the number of threads in one core.
>>> + */
>>> +typedef struct HybridCore {
>>> +    unsigned int threads;
>>> +} HybridCore;
>>> +
>>> +/**
>>> + * HybridCluster - hybrid cluster topology defination:
>>> + * @cores: the number of cores in current cluster.
>>> + * @core_list: the array includes all the cores that belong to current
>>> + *             cluster.
>>> + */
>>> +typedef struct HybridCluster {
>>> +    unsigned int cores;
>>> +    HybridCore *core_list;
>>> +} HybridCluster;
>>> +
>>> +/**
>>> + * HybridCpuTopology - hybrid cpu topology defination.
>>> + *
>>> + * At present we only support two heterogeneous topology levels: core
>>> + * and cluster. For heterogeneous levels, we need additional structs
>>> + * to define their custom internal topology. So here we defines
>>> + * symmetric topology levels, and use a list to point to heterogeneous
>>> + * levels.
>>> + *
>>> + * @sockets: the number of sockets on the machine. All sockets are the
>>> + *           same.
>>> + * @dies: the number of dies in one socket. All dies are the same.
>>> + * @clusters: the number of clusters in one die. Cluster may be
>>> + *            different. This field indicates the length of
>>> + *            cluster_list.
>>> + * @cluster_list: the array includes all the clusters in one die.
>>> + */
>>> +typedef struct HybridCpuTopology {
>>> +    unsigned int sockets;
>>> +    unsigned int dies;
>>> +    unsigned int clusters;
>>> +    HybridCluster *cluster_list;
>>> +} HybridCpuTopology;
>>> +
>>> +/**
>>> + * GeneralCpuTopology - General cpu topology defination.
>>> + *
>>> + * It supports one of two topologies: smp topology or hybrid topology.
>>> + *
>>> + * @cpus: the number of present logical processors on the machine
>>> + * @max_cpus: the maximum number of logical processors on the machine
>>> + * @topo_type: the topology type of the machine and this decides which
>>> + *             member of the union to visit: smp or hybrid.
>>> + * @smp: the smp cpu topology informantion. Only valid when topo_type is
>>> + *       CPU_TOPO_TYPE_SMP.
>>> + * @hybrid: the hybrid cpu topology informantion. Only valid when
>>> + *          topo_type is CPU_TOPO_TYPE_HYBRID.
>>> + */
>>> +typedef struct GeneralCpuTopology {
>>> +    unsigned int cpus;
>>> +    unsigned int max_cpus;
>>> +    CpuTopoType topo_type;
>>> +    union {
>>> +        SmpCpuTopology smp;
>>> +        HybridCpuTopology hybrid;
>>> +    };
>>> +} GeneralCpuTopology; /*
>>> +                       * TODO: This name is temporary, just to distinguish it
>>> +                       * from the CpuTopology in boards.h. When CpuTopology in
>>> +                       * boards.h is merged here, it will be uniformly named as
>>> +                       * CpuTopology.
>>> +                       */
>>> +
>> A suggestion:
>> 1、Move definition of CpuTopology from boards.h to cpu-topology.h
>> and re-structure it to include SmpCpuTopology, being a generic cpu
>> topology structure.
>> 2、Rename "CpuTopology smp" in MachineState to a generic name
>> "CpuTopology topo".
> Here we need to change the access to MachineState.smp to
> MachineState.topo.smp in other modules.
>
> If replacement of MachineState.topo is in a single patch, do we also
> need to include the modification of access to MachineState.topo.smp
> in other modules? Otherwise, it will break the compilation.
>
> In this way, the patch seems be too large.
Oh, I understand. The temporary "GeneralCpuTopology" seems necessary.
With this patch and help of the introduced generic helpers, we can replace
access to MachineState.smp gradually. And that's what you are doing in
this series.

But is it possible to strip the whole hybrid related code from the re-work
of smp. After MachineState.smp is re-worked/generalized completely,
then we add the hybrid extansion? Now the re-work code of smp and
hybrid code looks coupled all the way.

Thanks,
Yanan
>
> Thanks,
> Zhao
>
>> 3、Adapt all the code in QEMU to above change.
>>
>> If you can pack above into a single patch, and then add the hybird
>> topology extansion in a next patch, we will not need the temporary
>> thing "GeneralCpuTopology" and the TODO comments, which makes
>> code clearer.
>>
>> Thanks,
>> Yanan
>>> +#endif /* CPU_TOPOLOGY_H */
>>> diff --git a/qapi/machine.json b/qapi/machine.json
>>> index b9228a5e4616..bd7303f34497 100644
>>> --- a/qapi/machine.json
>>> +++ b/qapi/machine.json
>>> @@ -36,6 +36,18 @@
>>>                 'sh4eb', 'sparc', 'sparc64', 'tricore',
>>>                 'x86_64', 'xtensa', 'xtensaeb' ] }
>>> +##
>>> +# @CpuTopoType:
>>> +#
>>> +# An enumeration of cpu topology type
>>> +# TODO: Expose topology type in query-cpus-fast
>>> +#
>>> +# Since: 8.0
>>> +##
>>> +{ 'enum': 'CpuTopoType',
>>> +  'prefix': 'CPU_TOPO_TYPE',
>>> +  'data': [ 'smp', 'hybrid' ] }
>>> +
>>>    ##
>>>    # @CpuS390State:
>>>    #

Re: [RFC 41/52] machine: Introduce core_type() hook

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:33:22PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:33:22 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 41/52] machine: Introduce core_type() hook
> 
> On 13/2/23 10:50, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Since supported core types are architecture specific, we need this hook
> > to allow archs define its own parsing or validation method.
> > 
> > As the example, add the x86 core_type() which will be used in "-hybrid"
> > parameter parsing.
> 
> What would be a "core type" for other archs?

For intel, core type refers to the CPUID.1AH, and it is different with
CPU model. For example, Alder Lake is a CPU model, it can be marked as
"Core" or "Atom".

But for other architectures, the core type may be considered "cpu type"
or "cpu model".

Therefore, whether "core type" is used as a mark of a big or small core,
or as a different cpu model/cpu type, each architecture needs to define
this by itself.

I do have limited understanding of other architectures. Do you think
whether this explanation and handling is generic enough?

Thanks,
Zhao

> 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/core/machine-topo.c | 14 ++++++++++++++
> >   hw/core/machine.c      |  1 +
> >   hw/i386/x86.c          | 15 +++++++++++++++
> >   include/hw/boards.h    |  7 +++++++
> >   4 files changed, 37 insertions(+)
> 
> 
> > index 9364c90d5f1a..34ec035b5c9f 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
> >                                  Error **errp);
> >   void machine_parse_smp_config(MachineState *ms,
> >                                 const SMPConfiguration *config, Error **errp);
> > +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
> >   /**
> >    * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
> > @@ -199,6 +200,11 @@ typedef struct {
> >    *    Return the type of KVM corresponding to the kvm-type string option or
> >    *    computed based on other criteria such as the host kernel capabilities.
> >    *    kvm-type may be NULL if it is not needed.
> > + * @core_type:
> > + *    Return the type of hybrid cores corresponding to the coretype string
> > + *    option. The default hook only accept "none" or "" since the most generic
> > + *    core topology should not specify any specific core type. Each arch can
> > + *    define its own core_type() hook to override the default one.
> >    * @numa_mem_supported:
> >    *    true if '--numa node.mem' option is supported and false otherwise
> >    * @hotplug_allowed:
> > @@ -237,6 +243,7 @@ struct MachineClass {
> >       void (*reset)(MachineState *state, ShutdownCause reason);
> >       void (*wakeup)(MachineState *state);
> >       int (*kvm_type)(MachineState *machine, const char *arg);
> > +    int (*core_type)(MachineState *state, const char *type);
> >       BlockInterfaceType block_default_type;
> >       int units_per_default_bus;
> 

Re: [RFC 41/52] machine: Introduce core_type() hook

[Zhao Liu]

On Mon, Feb 13, 2023 at 02:35:24PM +0100, Philippe Mathieu-Daudé wrote:
> Date: Mon, 13 Feb 2023 14:35:24 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 41/52] machine: Introduce core_type() hook
> 
> On 13/2/23 10:50, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Since supported core types are architecture specific, we need this hook
> > to allow archs define its own parsing or validation method.
> > 
> > As the example, add the x86 core_type() which will be used in "-hybrid"
> > parameter parsing.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/core/machine-topo.c | 14 ++++++++++++++
> >   hw/core/machine.c      |  1 +
> >   hw/i386/x86.c          | 15 +++++++++++++++
> >   include/hw/boards.h    |  7 +++++++
> >   4 files changed, 37 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> > index 12c05510c1b5..f9ab08a1252e 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
> >           return;
> >       }
> >   }
> > +
> > +/*
> > + * machine_parse_hybrid_core_type: the default hook to parse hybrid core
> > + *                                 type corresponding to the coretype
> > + *                                 string option.
> > + */
> > +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> > +{
> > +    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
> > +        return 0;
> > +    }
> > +
> > +    return -1;
> 
> Shouldn't this use mc->core_type()? Where is it used?

This interface is used in "[RFC 44/52] machine: Add "-hybrid" parsing
rule"[1] to check the "coretype" info passed from command line:

static void insert_core_into_cluster(MachineState *ms,
                                     HybridCluster *cluster,
                                     const HybridCoreOptions *config,
                                     Error **errp)
{
    ...
    ret = mc->core_type(ms, config->coretype);
    if (!ret) {
        error_setg(errp, "Invalid coretype=%s", config->coretype);
    }
    core_pack->core.core_type = ret;
    ...
}

I use that machine_parse_hybrid_core_type() as the default mc->core_type()
implementation. Other arch can override this default core_type() if
necessary, for example x86 has its own core_type implementation:
x86_parse_hybrid_core_type().

This default core_type() avoids useless and wrong core_type information
from the command line so that I can expose core type info in query_cpus_fast.

[1]: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03250.html


Zhao

> 
> > +}

Re: [RFC 00/52] Introduce hybrid CPU topology

[Zhao Liu]

On Mon, Feb 13, 2023 at 01:38:28PM +0000, Daniel P. Berrangé wrote:
> Date: Mon, 13 Feb 2023 13:38:28 +0000
> From: "Daniel P. Berrangé" <berrange@redhat.com>
> Subject: Re: [RFC 00/52] Introduce hybrid CPU topology
> 
> On Mon, Feb 13, 2023 at 05:49:43PM +0800, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > ## 3.3. "-hybrid" command
> > 
> > For hybrid cpu topology configuration, the original "-smp" lack of
> > flexibility to expand, and unables to customize different cores.
> > 
> > So we introduce "-hybrid" command and implement it as the multi-
> > line command. The multi-line command format is more complex than the
> > single-line one, but it can bring stronger scalability and
> > intuitiveness. In the future, it can also be easily extended to more
> > heterogeneous topology levels.
> > 
> > "-hybrid" command is as follows:
> > 
> > -hybrid socket,sockets=n
> > -hybrid die,dies=n
> > -hybrid cluster,clusters=n
> > -hybrid core,cores=n,type=core_type[,threads=threads]
> >         [,clusterid=cluster]
> > 
> > Here, we first define the corresponding qapi options for these 4
> > topology levels: core, cluster, die and socket.
> > 
> > We doesn't need a thread level since thread doesn't have different
> > type.
> > 
> > For example:
> > 
> > -hybrid socket,sockets=1
> > -hybrid die,dies=1
> > -hybrid cluster,clusters=4
> > -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
> > -hybrid core,cores=4,coretype="atom",threads=1
> > 
> > Here we can build a hybrid cpu topology, which has 1 socket, 1 die per
> > socket, 4 clusters per die. And in each die, every clusters has 4 "atom"
> > core with 1 threads, and cluster0, cluster1 and cluster2 have 1 "core"
> > cores with 2 threads.
> 
> How will this interact with the -cpu  parameter. Presumably we now
> need two distinct -cpu parameters, one for the 'core' CPU model and
> one for the 'atom' CPU model ?

(About this explanation for core type, also CC Philippe again)

For intel's hybrid, the core type is not a specific cpu model, and it
refers CPUID.1AH, sounds like a flag to distinguish between "core" and
"atom". For example, Alder Lake (not defined in QEMU yet) is a CPU model,
the Alder Lake CPU can be "core" or "atom".

Although the real "core" and "atom" have different microarchitectures,
they belong to the same CPU model and have the same ISA, and the
difference that the software layer can recognize is whether the encode
in CPUID.1AH is "core" type or "atom" type.

I think this is also the difference between intel hybrid and other
heterogeneous architectures. IIUC the heterogeneity of arm contains
different CPU models, while intel hybrid only marks the cpu as "core"
or "atom" in the same CPU model.

I think this difference between intel hybrid and other heterogeneous can
be distinguished by different handling methods of mc->core_type() which
can be defined by other arch to check "coretype" field [1].

For x86, mc->core_type() asks "coretype" as "core" or "atom" and treat
it as a hybrid "flag". But for other arch, who contains different CPU
models in heterogeneous (maybe arm), its mc->coretype asks for the
correct cpu model.

In this sense, x86 does not need a different "-cpu" to correspond to
core/atom, and even "-cpu" can be set to "host" (ie, -cpu host). But
other arch, who contains different CPU models in heterogeneous, needs
to support distinct -cpu parameters.

Do you think it is appropriate to use arch-specific mc->coretype to
handle different requirements for CPU model/core type?

[1]: [RFC 41/52] machine: Introduce core_type() hook,
     https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03244.html

> 
> > Please note this example is not an actual machine topology, but it shows
> > the powerful flexibility of "hybrid" command.
> 
> IIUC the functionality offered by -hybrid should be a superset
> of the -smp functionality. IOW, -smp ought to be possible to
> re-implement -smp as an alias for -hybrid, such that internally
> code only ever has to deal with the modern approach.

Yes, I agree.

> Having to
> keep support for both -smp and -hybrid throughout the code is
> undesirable IMHO. Keeping the compat at the CLI parsing level
> limits the burden.
>


"-hybrid" is introduced as a completely different implementation from
"-smp", for these 2 reasons:

1. Single-line commands can be very complex, and it is more difficult
to identify the relationship between different levels.

2. smp is implemented as the property of machine [2] and I don't have a
good way to define "hybrid" also as a property since I need to use
opts_visitor for multi-line commands instead of input_visitor.

[2]: fe68090: machine: add smp compound property.

> 
> As a more general thought, rather than introducing a new top level
> command line argument -hybrid, I'm thinking we should possibly just
> define this all using QOM and thus re-use the existing -object
> argument. 
> 
> I'm also finding the above example command lines quite difficult
> to understand, as there is alot of implicit linkage and expansion
> between the different levels. With devices we're much more
> explicitly with the parent/child relationships, and have to
> express everything with no automatic expansion, linking it all
> together via the id=/bus= properties.  This is quite a bit more
> verbose, but it is also very effective at letting us express
> arbitrarily complex relationships.
> 
> I think it would be worth exploring that approach for the CPU
> topology expression too.
> 
> If we followed the more explicit device approach to modelling
> then instead of:
> 
>  -cpu core,...
>  -cpu atom,...
>  -hybrid socket,sockets=1
>  -hybrid die,dies=1
>  -hybrid cluster,clusters=4
>  -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
>  -hybrid core,cores=4,coretype="atom",threads=1
> 
> we would end up with something like
> 
>   -object cpu-socket,id=sock0
>   -object cpu-die,id=die0,parent=sock0
>   -object cpu-cluster,id=cluster0,parent=die0
>   -object cpu-cluster,id=cluster1,parent=die0
>   -object cpu-cluster,id=cluster2,parent=die0
>   -object cpu-cluster,id=cluster3,parent=die0
>   -object x86-cpu-model-atom,id=cpu0,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu1,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu2,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu3,parent=cluster0
>   -object x86-cpu-model-core,id=cpu4,parent=cluster0,threads=2
>   -object x86-cpu-model-atom,id=cpu5,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu6,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu7,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu8,parent=cluster1
>   -object x86-cpu-model-core,id=cpu9,parent=cluster1,threads=2
>   -object x86-cpu-model-atom,id=cpu10,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu11,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu12,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu13,parent=cluster2
>   -object x86-cpu-model-core,id=cpu14,parent=cluster2,threads=2
>   -object x86-cpu-model-atom,id=cpu15,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu16,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu17,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu18,parent=cluster3
>   -object x86-cpu-model-core,id=cpu19,parent=cluster3,threads=2


Given that x86 hybrid currently has the same cpu model and "core"/"atom"
is not CPU model, so the -object still needs a coretype fields to specify
"core" or "atom", right?

Given an example of "-cpu host":

-cpu host,...
-hybrid socket,sockets=1
-hybrid die,dies=1
-hybrid cluster,clusters=4
-hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
-hybrid core,cores=4,coretype="atom",threads=1


It occurred to me that there may be two conversion formats for qom.

The one with "coretype" field like:

-object x86-cpu-model-host,coretype="atom",id=cpu0,parent=cluster0
-object x86-cpu-model-host,coretype="core",id=cpu1,parent=cluster0

or another format like "x86-cpu-model-[real model name]-[core type]":

-object x86-cpu-model-host-atom,id=cpu0,parent=cluster0
-object x86-cpu-model-host-core,id=cpu1,parent=cluster0

Can qom support the parsing of the second naming format?

At the same time, I understand that for other architectures, there is
no need to consider the issue of core type, just specify different
heterogeneous CPU models.

> 
> The really obvious downside is that it is much more verbose.
> 
> This example only has 20 CPUs. For a VM with say 1000 CPUs
> this will be very big, but that doesn't neccesarily make it
> wrong.
> 
> On the flipside
> 
>  * It is really clear exactly how many CPUs I've added
> 
>  * The relationship between the topology levels is clear
> 
>  * Every CPU has a unique ID given that can be used in
>    later QMP commands
> 
>  * Whether or not 'threads' are permitted is now a property
>    of the specific CPU model implementation, not the global
>    config. IOW we can express that some CPU models allowing
>    for threads, and some don't.

Will this way break the smp property of machine (the above mentioned [2])?

If so, IIUC we still need keep smp code to support this smp format:

"-machine smp.{cores,threads,cpu,...}",

but we can mark this smp format as deprecated and at some point completely
remove the smp code and replace it with the qom implementation, right?

> 
>  * The -cpu arg is also obsoleted, replaced by the
>    -object x86-cpu-model-core. This might facilitate the
>    modelling of machines with CPUs from different architectures.
> 
> 
> We could potentially compress the leaf node level by expressing
> how many instances of an object we want. it we want. ie, define
> a more convenient shorthand syntax to creating many instances of
> an object. so eg
> 
>   -object-set $TYPE,$PROPS,idbase=foo,count=4
> 
> would be functionally identical to
> 
>   -object $TYPE,$PROPS,id=foo.0
>   -object $TYPE,$PROPS,id=foo.1
>   -object $TYPE,$PROPS,id=foo.2
>   -object $TYPE,$PROPS,id=foo.3
> 
> QEMU just expands it and creates all the objects internally.
> 
> So the huge example I have above for 20 cpus would become much
> shorter: e.g.
> 
>   -object cpu-socket,id=sock0
>   -object cpu-die,id=die0,parent=sock0
>   -object cpu-cluster,id=cluster0,parent=die0
>   -object cpu-cluster,id=cluster1,parent=die0
>   -object cpu-cluster,id=cluster2,parent=die0
>   -object cpu-cluster,id=cluster3,parent=die0
>   -object-set x86-cpu-core-atom,idbase=cpu0,parent=cluster0,count=4
>   -object-set x86-cpu-core-core,id=cpu1,parent=cluster0,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu2,parent=cluster1,count=4
>   -object-set x86-cpu-core-core,id=cpu3,parent=cluster1,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu4,parent=cluster2,count=4
>   -object-set x86-cpu-core-core,id=cpu5,parent=cluster2,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu6,parent=cluster3,count=4
>   -object-set x86-cpu-core-core,id=cpu7,parent=cluster3,threads=2,count=1

What's difference between "id" and "idbase"?

> 
> IOW, the size of the CLI config only depends on the number of elements
> in the hierarchy, and is independant of the number of leaf CPU cores.
> 
> Obviously in describing all of the above, I've ignored any complexity
> of dealing with our existing code implementation and pain of getting
> it converted to the new model.

Thanks and sounds cool! I think I need some time to study this way.


Zhao

> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 

Re: [RFC 04/52] i386/WHPX: Fix error message when fail to set ProcessorCount

[Zhao Liu]

On Mon, Feb 13, 2023 at 01:41:19PM +0000, Daniel P. Berrangé wrote:
> Date: Mon, 13 Feb 2023 13:41:19 +0000
> From: "Daniel P. Berrangé" <berrange@redhat.com>
> Subject: Re: [RFC 04/52] i386/WHPX: Fix error message when fail to set
>  ProcessorCount
> 
> On Mon, Feb 13, 2023 at 05:49:47PM +0800, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > 003f230 (machine: Tweak the order of topology members in struct
> > CpuTopology) changes the meaning of MachineState.smp.cores from "the
> > number of cores in one package" to "the number of cores in one die"
> > and doesn't fix other uses of MachineState.smp.cores. And because of
> > the introduction of cluster, now smp.cores just means "the number of
> > cores in one cluster". This clearly does not fit the semantics here.
> > 
> > And before this error message, WHvSetPartitionProperty() is called to
> > set prop.ProcessorCount.
> > 
> > So the error message should show the prop.ProcessorCount other than
> > "cores per cluster" or "cores per package".
> > 
> > Cc: Sunil Muthuswamy <sunilmut@microsoft.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >  target/i386/whpx/whpx-all.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> This patch and the 3 patches before it all look like basic bug
> fixes to current code, not really part of the new hybrid topology
> feature work.
> 
> As such I'd suggest sending these first four patches as a separate
> series, so the bug fixes can be merged fairly quickly. I expect the
> rest of the hybrid topology series is going to take a long time to
> get agreement on, so no need to delay the easy bug fixes.

Yes, I'll split out the first four. Thanks!

Zhao

> 
> > 
> > diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
> > index e738d83e8191..fc349f887e47 100644
> > --- a/target/i386/whpx/whpx-all.c
> > +++ b/target/i386/whpx/whpx-all.c
> > @@ -2613,8 +2613,8 @@ static int whpx_accel_init(MachineState *ms)
> >          sizeof(WHV_PARTITION_PROPERTY));
> >  
> >      if (FAILED(hr)) {
> > -        error_report("WHPX: Failed to set partition core count to %d,"
> > -                     " hr=%08lx", ms->smp.cores, hr);
> > +        error_report("WHPX: Failed to set partition processor count to %d,"
> > +                     " hr=%08lx", prop.ProcessorCount, hr);
> >          ret = -EINVAL;
> >          goto error;
> >      }
> > -- 
> > 2.34.1
> > 
> > 
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 

Re: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[Zhao Liu]

On Tue, Feb 14, 2023 at 09:12:11AM +0800, Mi, Dapeng1 wrote:
> Date: Tue, 14 Feb 2023 09:12:11 +0800
> From: "Mi, Dapeng1" <dapeng1.mi@intel.com>
> Subject: RE: [RFC 08/52] machine: Add helpers to get cpu topology info from
>  MachineState.topo
> 
> > From: Zhao Liu <zhao1.liu@linux.intel.com>
> > Sent: Monday, February 13, 2023 5:50 PM
> > To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> > <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daud? <philmd@linaro.org>;
> > Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> > <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> > Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>
> > Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> > Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> > <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> > Christopherson,, Sean <seanjc@google.com>; Like Xu
> > <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>
> > Subject: [RFC 08/52] machine: Add helpers to get cpu topology info from
> > MachineState.topo
> > 
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures become
> > complicated. In the general case (hybrid or smp topology), accessing the
> > topology information needs to determine whether it is currently smp or hybrid
> > topology, and then access the corresponding MachineState.topo.smp or
> > MachineState.topo.hybrid.
> > 
> > The best way to do this is to wrap the access to the topology to avoid having to
> > check each time it is accessed.
> > 
> > The following helpers are provided here:
> > 
> > - General interfaces - no need to worry about whether the underlying
> >   topology is smp or hybrid:
> > 
> > * machine_topo_get_cpus()
> > * machine_topo_get_max_cpus()
> > * machine_topo_is_smp()
> > * machine_topo_get_sockets()
> > * machine_topo_get_dies()
> > * machine_topo_get_clusters()
> > * machine_topo_get_threads();
> > * machine_topo_get_cores();
> > * machine_topo_get_threads_by_idx()
> > * machine_topo_get_cores_by_idx()
> > * machine_topo_get_cores_per_socket()
> > * machine_topo_get_threads_per_socket()
> > 
> > - SMP-specific interfaces - provided for the cases that are clearly known to be
> > smp topology:
> > 
> > * machine_topo_get_smp_cores()
> > * machine_topo_get_smp_threads()
> > 
> > Since for hybrid topology, each core may has different threads, if someone
> > wants "cpus per core", the cpu_index is need to target a specific core
> > (machine_topo_get_threads_by_idx()). But for smp, there is no need to be so
> > troublesome, so for this case, we provide smp-specific interfaces.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >  hw/core/machine-topo.c | 142
> > +++++++++++++++++++++++++++++++++++++++++
> >  include/hw/boards.h    |  35 ++++++++++
> >  2 files changed, 177 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c index
> > 7223f73f99b0..b20160479629 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -21,6 +21,148 @@
> >  #include "hw/boards.h"
> >  #include "qapi/error.h"
> > 
> > +unsigned int machine_topo_get_sockets(const MachineState *ms) {
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > +                                     ms->topo.hybrid.sockets; }
> > +
> > +unsigned int machine_topo_get_dies(const MachineState *ms) {
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
> > +                                     ms->topo.hybrid.dies; }
> > +
> > +unsigned int machine_topo_get_clusters(const MachineState *ms) {
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
> > +                                     ms->topo.hybrid.clusters; }
> > +
> > +unsigned int machine_topo_get_smp_cores(const MachineState *ms) {
> > +    g_assert(machine_topo_is_smp(ms));
> > +    return ms->topo.smp.cores;
> > +}
> > +
> > +unsigned int machine_topo_get_smp_threads(const MachineState *ms) {
> > +    g_assert(machine_topo_is_smp(ms));
> > +    return ms->topo.smp.threads;
> > +}
> > +
> > +unsigned int machine_topo_get_threads(const MachineState *ms,
> > +                                      unsigned int cluster_id,
> > +                                      unsigned int core_id) {
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.threads;
> > +    } else {
> > +        return ms->topo.hybrid.cluster_list[cluster_id]
> > +                   .core_list[core_id].threads;
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +unsigned int machine_topo_get_cores(const MachineState *ms,
> > +                                    unsigned int cluster_id) {
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores;
> > +    } else {
> > +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
> > +    }
> > +}
> > +
> > +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> > +                                             unsigned int cpu_index) {
> > +    unsigned cpus_per_die;
> > +    unsigned tmp_idx;
> > +    HybridCluster *cluster;
> > +    HybridCore *core;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.threads;
> > +    }
> > +
> > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > +                            ms->topo.hybrid.dies);
> > +    tmp_idx = cpu_index % cpus_per_die;
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > +
> > +        for (int j = 0; j < cluster->cores; j++) {
> > +            core = &cluster->core_list[j];
> > +
> > +            if (tmp_idx < core->threads) {
> > +                return core->threads;
> > +            } else {
> > +                tmp_idx -= core->threads;
> > +            }
> > +        }
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > +                                           unsigned int cpu_index) {
> > +    unsigned cpus_per_die;
> > +    unsigned tmp_idx;
> > +    HybridCluster *cluster;
> > +    HybridCore *core;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores;
> > +    }
> > +
> > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > +                            ms->topo.hybrid.dies);
> > +    tmp_idx = cpu_index % cpus_per_die;
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > +
> > +        for (int j = 0; j < cluster->cores; j++) {
> > +            core = &cluster->core_list[j];
> > +
> > +            if (tmp_idx < core->threads) {
> > +                return cluster->cores;
> > +            } else {
> > +                tmp_idx -= core->threads;
> > +            }
> > +        }
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> 
> It looks most code of this function is same with previous function. Could we extract
> the same part code and define a low level function?

Yes, I can try it.

Thanks,
Zhao

> 
> > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
> > +{
> > +    unsigned int cores_per_die = 0;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
> > +    }
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
> > +    }
> > +
> > +    return cores_per_die * ms->topo.hybrid.dies; }
> > +
> > +unsigned int machine_topo_get_threads_per_socket(const MachineState
> > +*ms) {
> > +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > +                           ms->topo.hybrid.sockets;
> > +    return ms->topo.max_cpus / sockets; }
> > +
> >  /*
> >   * Report information of a machine's supported CPU topology hierarchy.
> >   * Topology members will be ordered from the largest to the smallest diff --git
> > a/include/hw/boards.h b/include/hw/boards.h index
> > 0a61855499e3..34b64b012022 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;  extern
> > GlobalProperty hw_compat_2_1[];  extern const size_t hw_compat_2_1_len;
> > 
> > +static inline
> > +unsigned int machine_topo_get_cpus(const MachineState *ms) {
> > +    return ms->topo.cpus;
> > +}
> > +
> > +static inline
> > +unsigned int machine_topo_get_max_cpus(const MachineState *ms) {
> > +    return ms->topo.max_cpus;
> > +}
> > +
> > +static inline
> > +bool machine_topo_is_smp(const MachineState *ms) {
> > +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP; }
> > +
> > +unsigned int machine_topo_get_sockets(const MachineState *ms); unsigned
> > +int machine_topo_get_dies(const MachineState *ms); unsigned int
> > +machine_topo_get_clusters(const MachineState *ms); unsigned int
> > +machine_topo_get_smp_cores(const MachineState *ms); unsigned int
> > +machine_topo_get_smp_threads(const MachineState *ms); unsigned int
> > +machine_topo_get_threads(const MachineState *ms,
> > +                                      unsigned int cluster_id,
> > +                                      unsigned int core_id); unsigned
> > +int machine_topo_get_cores(const MachineState *ms,
> > +                                    unsigned int cluster_id); unsigned
> > +int machine_topo_get_threads_by_idx(const MachineState *ms,
> > +                                             unsigned int cpu_index);
> > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > +                                           unsigned int cpu_index);
> > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
> > +unsigned int machine_topo_get_threads_per_socket(const MachineState
> > +*ms);
> > +
> >  #endif
> > --
> > 2.34.1
> 

Re: [RFC 09/52] hw/machine: Introduce core type for hybrid topology

[Zhao Liu]

On Tue, Feb 14, 2023 at 09:14:13AM +0800, Mi, Dapeng1 wrote:
> Date: Tue, 14 Feb 2023 09:14:13 +0800
> From: "Mi, Dapeng1" <dapeng1.mi@intel.com>
> Subject: RE: [RFC 09/52] hw/machine: Introduce core type for hybrid topology
> 
> > From: Zhao Liu <zhao1.liu@linux.intel.com>
> > Sent: Monday, February 13, 2023 5:50 PM
> > To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> > <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daud? <philmd@linaro.org>;
> > Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> > <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> > Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>
> > Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> > Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> > <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> > Christopherson,, Sean <seanjc@google.com>; Like Xu
> > <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>
> > Subject: [RFC 09/52] hw/machine: Introduce core type for hybrid topology
> > 
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Under the hybrid cpu topology, some CPUs care about the core type.
> > For example, Intel's Alder Lake series CPU contains two types of cores:
> > Intel Core and Intel Atom. The type information of these two types is exposed in
> > 1A leaf of CPUID.
> > 
> > Core type should also be part of the hybrid topology, and
> > MachineState.cpu_type cannot provide different type information for different
> > cpus in the same machine, so add a type field for the core level in the hybrid cpu
> > topology.
> > 
> > Additionally, add a helper to get core type information from MachineState.
> > Though core_type is only used in hybrid case, don't use assert since it may be
> > used to initialize some generic fields.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >  hw/core/machine-topo.c        | 12 ++++++++++++
> >  include/hw/boards.h           |  3 +++
> >  include/hw/cpu/cpu-topology.h |  2 ++
> >  3 files changed, 17 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c index
> > b20160479629..e0ec07b53d42 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -51,6 +51,18 @@ unsigned int machine_topo_get_smp_threads(const
> > MachineState *ms)
> >      return ms->topo.smp.threads;
> >  }
> > 
> > +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> > +                                               unsigned int cluster_id,
> > +                                               unsigned int core_id) {
> > +    if (!machine_topo_is_smp(ms)) {
> > +        return ms->topo.hybrid.cluster_list[cluster_id]
> > +                       .core_list[core_id].core_type;
> > +    } else {
> > +        return 0;
> > +    }
> > +}
> > +
> 
> We'd better not to return the hard-coded '0'. Suggest to define a 'enum'
> data structure to represent the core_type. This makes the code look more intuitively.

Yes. I defined a core type 'enum' in x86 code, Here I can use a macro to
avoid hardcoding.

Zhao

> 
> >  unsigned int machine_topo_get_threads(const MachineState *ms,
> >                                        unsigned int cluster_id,
> >                                        unsigned int core_id) diff --git a/include/hw/boards.h
> > b/include/hw/boards.h index 34b64b012022..78e52af38cb1 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -484,6 +484,9 @@ unsigned int machine_topo_get_dies(const
> > MachineState *ms);  unsigned int machine_topo_get_clusters(const
> > MachineState *ms);  unsigned int machine_topo_get_smp_cores(const
> > MachineState *ms);  unsigned int machine_topo_get_smp_threads(const
> > MachineState *ms);
> > +unsigned int machine_topo_get_hybrid_core_type(const MachineState *ms,
> > +                                               unsigned int cluster_id,
> > +                                               unsigned int core_id);
> >  unsigned int machine_topo_get_threads(const MachineState *ms,
> >                                        unsigned int cluster_id,
> >                                        unsigned int core_id); diff --git a/include/hw/cpu/cpu-
> > topology.h b/include/hw/cpu/cpu-topology.h index
> > 8268ea3a8569..87d832556229 100644
> > --- a/include/hw/cpu/cpu-topology.h
> > +++ b/include/hw/cpu/cpu-topology.h
> > @@ -45,9 +45,11 @@ typedef struct SmpCpuTopology {
> >  /**
> >   * HybridCore - hybrid core topology defination:
> >   * @threads: the number of threads in one core.
> > + * @core_type: the type of current core.
> >   */
> >  typedef struct HybridCore {
> >      unsigned int threads;
> > +    unsigned int core_type;
> >  } HybridCore;
> > 
> >  /**
> > --
> > 2.34.1
> 

Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo properties

[Zhao Liu]

On Tue, Feb 14, 2023 at 09:46:50AM +0800, wangyanan (Y) wrote:
> Date: Tue, 14 Feb 2023 09:46:50 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo
>  properties
> 
> Hi Zhao,
> 
> 在 2023/2/13 17:50, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Since hybrid cpu topology configuration can benefit not only x86, but
> > also other architectures/platforms that have supported (in real
> > machines) or will support hybrid CPU topology, "-hybrid" can be generic.
> > 
> > So add the generic topology property to configure if support hybrid
> > cpu topology for architectures/platforms in SmpCompatProps.
> > 
> > Also rename SmpCompatProps to TopoCompatProps to make this structure
> > more generic for both smp topology and hybrid topology.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   include/hw/boards.h | 15 +++++++++++----
> >   1 file changed, 11 insertions(+), 4 deletions(-)
> > 
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 34ec035b5c9f..17be3485e823 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -127,19 +127,26 @@ typedef struct {
> >   } CPUArchIdList;
> >   /**
> > - * SMPCompatProps:
> > - * @prefer_sockets - whether sockets are preferred over cores in smp parsing
> > + * TopoCompatProps:
> > + * @hybrid_support - whether hybrid cpu topology are supported by machine.
> inconsistent with the name in the definition below.

Thanks! Will fix.

> > + *                   Note that hybrid cpu topology requires to specify the
> > + *                   topology of each core so that there will no longer be
> > + *                   a default core topology, thus prefer_sockets won't work
> > + *                   when hybrid_support is enabled.
> > + * @prefer_sockets - whether sockets are preferred over cores in smp parsing.
> > + *                   Not work when hybrid_support is enabled.
> >    * @dies_supported - whether dies are supported by the machine
> >    * @clusters_supported - whether clusters are supported by the machine
> >    * @has_clusters - whether clusters are explicitly specified in the user
> >    *                 provided SMP configuration
> >    */
> >   typedef struct {
> > +    bool hybrid_supported;
> >       bool prefer_sockets;
> >       bool dies_supported;
> >       bool clusters_supported;
> >       bool has_clusters;
> > -} SMPCompatProps;
> > +} TopoCompatProps;
> Also here. "Rename SMPCompatProps to TopoCompatProps and
> move it to cpu-topology.h and adapt the code" should be organized
> in one or more separate patches, being pre-patches together with
> the conversion of CpuTopology before. 

Do you think TopoCompatProps/SMPCompatProps should also be moved
into cpu-topology.h? It seems that SMPCompatProps is a collection
of properties of MachineClass.

> And put the "hybrid_supported"
> extension into another patch. Would this make it easier to review?

Yes, I agree. Thanks!

Zhao

> 
> Thanks,
> Yanan
> >   /**
> >    * MachineClass:
> > @@ -281,7 +288,7 @@ struct MachineClass {
> >       bool nvdimm_supported;
> >       bool numa_mem_supported;
> >       bool auto_enable_numa;
> > -    SMPCompatProps smp_props;
> > +    TopoCompatProps smp_props;
> >       const char *default_ram_id;
> >       HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> 

Re: [RFC 22/52] riscv: Replace MachineState.smp access with topology helpers

[Zhao Liu]

On Tue, Feb 14, 2023 at 10:17:45AM +0800, Mi, Dapeng1 wrote:
> Date: Tue, 14 Feb 2023 10:17:45 +0800
> From: "Mi, Dapeng1" <dapeng1.mi@intel.com>
> Subject: RE: [RFC 22/52] riscv: Replace MachineState.smp access with
>  topology helpers
> 
> > From: Zhao Liu <zhao1.liu@linux.intel.com>
> > Sent: Monday, February 13, 2023 5:50 PM
> > To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> > <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daud? <philmd@linaro.org>;
> > Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> > <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> > Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>
> > Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> > Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> > <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> > Christopherson,, Sean <seanjc@google.com>; Like Xu
> > <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>; Meng, Bin
> > <bin.meng@windriver.com>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair
> > Francis <alistair.francis@wdc.com>; Vijai Kumar K <vijai@behindbytes.com>
> > Subject: [RFC 22/52] riscv: Replace MachineState.smp access with topology
> > helpers
> > 
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures
> > become complicated. So we wrapped the access to topology fields of
> > MachineState.topo into some helpers, and we are using these helpers
> > to replace the use of MachineState.smp.
> > 
> > In the codes of riscv, it's straightforward to replace topology access
> > with wrapped generic interfaces.
> > 
> > Cc: Bin Meng <bin.meng@windriver.com>
> > Cc: Palmer Dabbelt <palmer@dabbelt.com>
> > Cc: Alistair Francis <alistair.francis@wdc.com>
> > CC: Vijai Kumar K <vijai@behindbytes.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >  hw/riscv/microchip_pfsoc.c | 11 ++++++-----
> >  hw/riscv/numa.c            | 21 +++++++++++----------
> >  hw/riscv/opentitan.c       |  8 ++++----
> >  hw/riscv/shakti_c.c        |  2 +-
> >  hw/riscv/sifive_e.c        | 10 ++++++----
> >  hw/riscv/sifive_u.c        | 28 ++++++++++++++--------------
> >  hw/riscv/virt.c            | 24 +++++++++++++-----------
> >  7 files changed, 55 insertions(+), 49 deletions(-)
> > 
> > diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c
> > index 2b91e49561f1..30295cce17e7 100644
> > --- a/hw/riscv/microchip_pfsoc.c
> > +++ b/hw/riscv/microchip_pfsoc.c
> > @@ -164,7 +164,8 @@ static void microchip_pfsoc_soc_instance_init(Object
> > *obj)
> > 
> >      object_initialize_child(OBJECT(&s->u_cluster), "u-cpus", &s->u_cpus,
> >                              TYPE_RISCV_HART_ARRAY);
> > -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
> > +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts",
> > +                         machine_topo_get_cpus(ms) - 1);
> >      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
> >      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type",
> >                           TYPE_RISCV_CPU_SIFIVE_U54);
> > @@ -249,10 +250,10 @@ static void microchip_pfsoc_soc_realize(DeviceState
> > *dev, Error **errp)
> > 
> >      /* CLINT */
> >      riscv_aclint_swi_create(memmap[MICROCHIP_PFSOC_CLINT].base,
> > -        0, ms->smp.cpus, false);
> > +        0, machine_topo_get_cpus(ms), false);
> >      riscv_aclint_mtimer_create(
> >          memmap[MICROCHIP_PFSOC_CLINT].base + RISCV_ACLINT_SWI_SIZE,
> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, machine_topo_get_cpus(ms),
> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
> >          CLINT_TIMEBASE_FREQ, false);
> > 
> > @@ -276,11 +277,11 @@ static void microchip_pfsoc_soc_realize(DeviceState
> > *dev, Error **errp)
> >                                  l2lim_mem);
> > 
> >      /* create PLIC hart topology configuration string */
> > -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
> > +    plic_hart_config =
> > riscv_plic_hart_config_string(machine_topo_get_cpus(ms));
> > 
> >      /* PLIC */
> >      s->plic = sifive_plic_create(memmap[MICROCHIP_PFSOC_PLIC].base,
> > -        plic_hart_config, ms->smp.cpus, 0,
> > +        plic_hart_config, machine_topo_get_cpus(ms), 0,
> >          MICROCHIP_PFSOC_PLIC_NUM_SOURCES,
> >          MICROCHIP_PFSOC_PLIC_NUM_PRIORITIES,
> >          MICROCHIP_PFSOC_PLIC_PRIORITY_BASE,
> > diff --git a/hw/riscv/numa.c b/hw/riscv/numa.c
> > index 472010256183..1fabdc42e767 100644
> > --- a/hw/riscv/numa.c
> > +++ b/hw/riscv/numa.c
> > @@ -37,13 +37,14 @@ int riscv_socket_count(const MachineState *ms)
> > 
> >  int riscv_socket_first_hartid(const MachineState *ms, int socket_id)
> >  {
> > -    int i, first_hartid = ms->smp.cpus;
> > +    int i, first_hartid, cpus = machine_topo_get_cpus(ms);
> > 
> > +    first_hartid = cpus;
> >      if (!numa_enabled(ms)) {
> >          return (!socket_id) ? 0 : -1;
> >      }
> > 
> > -    for (i = 0; i < ms->smp.cpus; i++) {
> > +    for (i = 0; i < cpus; i++) {
> >          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
> >              continue;
> >          }
> > @@ -52,18 +53,18 @@ int riscv_socket_first_hartid(const MachineState *ms,
> > int socket_id)
> >          }
> >      }
> > 
> > -    return (first_hartid < ms->smp.cpus) ? first_hartid : -1;
> > +    return (first_hartid < cpus) ? first_hartid : -1;
> >  }
> > 
> >  int riscv_socket_last_hartid(const MachineState *ms, int socket_id)
> >  {
> > -    int i, last_hartid = -1;
> > +    int i, last_hartid = -1, cpus = machine_topo_get_cpus(ms);
> > 
> >      if (!numa_enabled(ms)) {
> > -        return (!socket_id) ? ms->smp.cpus - 1 : -1;
> > +        return (!socket_id) ? cpus - 1 : -1;
> >      }
> > 
> > -    for (i = 0; i < ms->smp.cpus; i++) {
> > +    for (i = 0; i < cpus; i++) {
> >          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
> >              continue;
> >          }
> > @@ -72,7 +73,7 @@ int riscv_socket_last_hartid(const MachineState *ms, int
> > socket_id)
> >          }
> >      }
> > 
> > -    return (last_hartid < ms->smp.cpus) ? last_hartid : -1;
> > +    return (last_hartid < cpus) ? last_hartid : -1;
> >  }
> > 
> >  int riscv_socket_hart_count(const MachineState *ms, int socket_id)
> > @@ -80,7 +81,7 @@ int riscv_socket_hart_count(const MachineState *ms, int
> > socket_id)
> >      int first_hartid, last_hartid;
> > 
> >      if (!numa_enabled(ms)) {
> > -        return (!socket_id) ? ms->smp.cpus : -1;
> > +        return (!socket_id) ? machine_topo_get_cpus(ms) : -1;
> >      }
> > 
> >      first_hartid = riscv_socket_first_hartid(ms, socket_id);
> > @@ -208,7 +209,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
> > MachineState *ms, int idx)
> >      int64_t nidx = 0;
> > 
> >      if (ms->numa_state->num_nodes) {
> > -        nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes);
> > +        nidx = idx / (machine_topo_get_cpus(ms) / ms->numa_state->num_nodes);
> >          if (ms->numa_state->num_nodes <= nidx) {
> >              nidx = ms->numa_state->num_nodes - 1;
> >          }
> > @@ -220,7 +221,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
> > MachineState *ms, int idx)
> >  const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms)
> >  {
> >      int n;
> > -    unsigned int max_cpus = ms->smp.max_cpus;
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> > 
> >      if (ms->possible_cpus) {
> >          assert(ms->possible_cpus->len == max_cpus);
> > diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
> > index 353f030d80c1..fb59c43dcc08 100644
> > --- a/hw/riscv/opentitan.c
> > +++ b/hw/riscv/opentitan.c
> > @@ -143,11 +143,11 @@ static void lowrisc_ibex_soc_realize(DeviceState
> > *dev_soc, Error **errp)
> >      MachineState *ms = MACHINE(qdev_get_machine());
> >      LowRISCIbexSoCState *s = RISCV_IBEX_SOC(dev_soc);
> >      MemoryRegion *sys_mem = get_system_memory();
> > -    int i;
> > +    int i, cpus = machine_topo_get_cpus(ms);
> > 
> >      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
> >                              &error_abort);
> > -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
> > +    object_property_set_int(OBJECT(&s->cpus), "num-harts", cpus,
> >                              &error_abort);
> >      object_property_set_int(OBJECT(&s->cpus), "resetvec", s->resetvec,
> >                              &error_abort);
> > @@ -186,10 +186,10 @@ static void lowrisc_ibex_soc_realize(DeviceState
> > *dev_soc, Error **errp)
> >      }
> >      sysbus_mmio_map(SYS_BUS_DEVICE(&s->plic), 0,
> > memmap[IBEX_DEV_PLIC].base);
> > 
> > -    for (i = 0; i < ms->smp.cpus; i++) {
> > +    for (i = 0; i < cpus; i++) {
> >          CPUState *cpu = qemu_get_cpu(i);
> > 
> > -        qdev_connect_gpio_out(DEVICE(&s->plic), ms->smp.cpus + i,
> > +        qdev_connect_gpio_out(DEVICE(&s->plic), cpus + i,
> >                                qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
> >      }
> > 
> > diff --git a/hw/riscv/shakti_c.c b/hw/riscv/shakti_c.c
> > index e43cc9445cb1..34499510e2f0 100644
> > --- a/hw/riscv/shakti_c.c
> > +++ b/hw/riscv/shakti_c.c
> > @@ -110,7 +110,7 @@ static void shakti_c_soc_state_realize(DeviceState *dev,
> > Error **errp)
> >      sysbus_realize(SYS_BUS_DEVICE(&sss->cpus), &error_abort);
> > 
> >      sss->plic = sifive_plic_create(shakti_c_memmap[SHAKTI_C_PLIC].base,
> > -        (char *)SHAKTI_C_PLIC_HART_CONFIG, ms->smp.cpus, 0,
> > +        (char *)SHAKTI_C_PLIC_HART_CONFIG, machine_topo_get_cpus(ms), 0,
> >          SHAKTI_C_PLIC_NUM_SOURCES,
> >          SHAKTI_C_PLIC_NUM_PRIORITIES,
> >          SHAKTI_C_PLIC_PRIORITY_BASE,
> > diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c
> > index 3e3f4b0088ca..ca3a7a0877cd 100644
> > --- a/hw/riscv/sifive_e.c
> > +++ b/hw/riscv/sifive_e.c
> > @@ -178,7 +178,8 @@ static void sifive_e_soc_init(Object *obj)
> >      SiFiveESoCState *s = RISCV_E_SOC(obj);
> > 
> >      object_initialize_child(obj, "cpus", &s->cpus, TYPE_RISCV_HART_ARRAY);
> > -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
> > +    object_property_set_int(OBJECT(&s->cpus), "num-harts",
> > +                            machine_topo_get_cpus(ms),
> >                              &error_abort);
> 
> With this change, we don't need to start a new line for '&error_abort);', just move
> it to previous line.

Yes, this is no more than 80 characters.

> 
> >      object_property_set_int(OBJECT(&s->cpus), "resetvec", 0x1004,
> > &error_abort);
> >      object_initialize_child(obj, "riscv.sifive.e.gpio0", &s->gpio,
> > @@ -191,6 +192,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
> > **errp)
> >      const MemMapEntry *memmap = sifive_e_memmap;
> >      SiFiveESoCState *s = RISCV_E_SOC(dev);
> >      MemoryRegion *sys_mem = get_system_memory();
> > +    unsigned int cpus = machine_topo_get_cpus(ms);
> > 
> >      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
> >                              &error_abort);
> > @@ -204,7 +206,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
> > **errp)
> > 
> >      /* MMIO */
> >      s->plic = sifive_plic_create(memmap[SIFIVE_E_DEV_PLIC].base,
> > -        (char *)SIFIVE_E_PLIC_HART_CONFIG, ms->smp.cpus, 0,
> > +        (char *)SIFIVE_E_PLIC_HART_CONFIG, cpus, 0,
> >          SIFIVE_E_PLIC_NUM_SOURCES,
> >          SIFIVE_E_PLIC_NUM_PRIORITIES,
> >          SIFIVE_E_PLIC_PRIORITY_BASE,
> > @@ -215,10 +217,10 @@ static void sifive_e_soc_realize(DeviceState *dev,
> > Error **errp)
> >          SIFIVE_E_PLIC_CONTEXT_STRIDE,
> >          memmap[SIFIVE_E_DEV_PLIC].size);
> >      riscv_aclint_swi_create(memmap[SIFIVE_E_DEV_CLINT].base,
> > -        0, ms->smp.cpus, false);
> > +        0, cpus, false);
> >      riscv_aclint_mtimer_create(memmap[SIFIVE_E_DEV_CLINT].base +
> >              RISCV_ACLINT_SWI_SIZE,
> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
> >          RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, false);
> >      create_unimplemented_device("riscv.sifive.e.aon",
> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
> > index d3ab7a9cda5d..c7f969510209 100644
> > --- a/hw/riscv/sifive_u.c
> > +++ b/hw/riscv/sifive_u.c
> > @@ -99,7 +99,7 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >      MachineState *ms = MACHINE(s);
> >      uint64_t mem_size = ms->ram_size;
> >      void *fdt;
> > -    int cpu, fdt_size;
> > +    int cpu, fdt_size, num_cpus = machine_topo_get_cpus(ms);
> >      uint32_t *cells;
> >      char *nodename;
> >      uint32_t plic_phandle, prci_phandle, gpio_phandle, phandle = 1;
> > @@ -175,7 +175,7 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
> > 
> > -    for (cpu = ms->smp.cpus - 1; cpu >= 0; cpu--) {
> > +    for (cpu = num_cpus - 1; cpu >= 0; cpu--) {
> >          int cpu_phandle = phandle++;
> >          nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
> >          char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
> > @@ -207,8 +207,8 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >          g_free(nodename);
> >      }
> > 
> > -    cells =  g_new0(uint32_t, ms->smp.cpus * 4);
> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +    cells =  g_new0(uint32_t, num_cpus * 4);
> > +    for (cpu = 0; cpu < num_cpus; cpu++) {
> >          nodename =
> >              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
> >          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
> > @@ -227,7 +227,7 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >          0x0, memmap[SIFIVE_U_DEV_CLINT].base,
> >          0x0, memmap[SIFIVE_U_DEV_CLINT].size);
> >      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
> > -        cells, ms->smp.cpus * sizeof(uint32_t) * 4);
> > +        cells, num_cpus * sizeof(uint32_t) * 4);
> >      g_free(cells);
> >      g_free(nodename);
> > 
> > @@ -258,8 +258,8 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >      g_free(nodename);
> > 
> >      plic_phandle = phandle++;
> > -    cells =  g_new0(uint32_t, ms->smp.cpus * 4 - 2);
> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +    cells =  g_new0(uint32_t, machine_topo_get_cpus(ms) * 4 - 2);
> > +    for (cpu = 0; cpu < machine_topo_get_cpus(ms); cpu++) {
> >          nodename =
> >              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
> >          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
> > @@ -283,7 +283,7 @@ static void create_fdt(SiFiveUState *s, const
> > MemMapEntry *memmap,
> >          (char **)&plic_compat, ARRAY_SIZE(plic_compat));
> >      qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0);
> >      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
> > -        cells, (ms->smp.cpus * 4 - 2) * sizeof(uint32_t));
> > +        cells, (machine_topo_get_cpus(ms) * 4 - 2) * sizeof(uint32_t));
> >      qemu_fdt_setprop_cells(fdt, nodename, "reg",
> >          0x0, memmap[SIFIVE_U_DEV_PLIC].base,
> >          0x0, memmap[SIFIVE_U_DEV_PLIC].size);
> > @@ -797,10 +797,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
> > Error **errp)
> >      MemoryRegion *mask_rom = g_new(MemoryRegion, 1);
> >      MemoryRegion *l2lim_mem = g_new(MemoryRegion, 1);
> >      char *plic_hart_config;
> > -    int i, j;
> > +    int i, j, cpus = machine_topo_get_cpus(ms);
> >      NICInfo *nd = &nd_table[0];
> > 
> > -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
> > +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", cpus - 1);
> >      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
> >      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type", s->cpu_type);
> >      qdev_prop_set_uint64(DEVICE(&s->u_cpus), "resetvec", 0x1004);
> > @@ -837,11 +837,11 @@ static void sifive_u_soc_realize(DeviceState *dev,
> > Error **errp)
> >                                  l2lim_mem);
> > 
> >      /* create PLIC hart topology configuration string */
> > -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
> > +    plic_hart_config = riscv_plic_hart_config_string(cpus);
> > 
> >      /* MMIO */
> >      s->plic = sifive_plic_create(memmap[SIFIVE_U_DEV_PLIC].base,
> > -        plic_hart_config, ms->smp.cpus, 0,
> > +        plic_hart_config, cpus, 0,
> >          SIFIVE_U_PLIC_NUM_SOURCES,
> >          SIFIVE_U_PLIC_NUM_PRIORITIES,
> >          SIFIVE_U_PLIC_PRIORITY_BASE,
> > @@ -857,10 +857,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
> > Error **errp)
> >      sifive_uart_create(system_memory, memmap[SIFIVE_U_DEV_UART1].base,
> >          serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ));
> >      riscv_aclint_swi_create(memmap[SIFIVE_U_DEV_CLINT].base, 0,
> > -        ms->smp.cpus, false);
> > +        cpus, false);
> >      riscv_aclint_mtimer_create(memmap[SIFIVE_U_DEV_CLINT].base +
> >              RISCV_ACLINT_SWI_SIZE,
> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
> >          CLINT_TIMEBASE_FREQ, false);
> > 
> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> > index b81081c70b1c..4b0a3dab5ec1 100644
> > --- a/hw/riscv/virt.c
> > +++ b/hw/riscv/virt.c
> > @@ -506,16 +506,17 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      char *imsic_name;
> >      MachineState *ms = MACHINE(s);
> >      int socket_count = riscv_socket_count(ms);
> > -    uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
> > +    uint32_t imsic_max_hart_per_socket, imsic_guest_bits, cpus;
> >      uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
> > 
> > +    cpus = machine_topo_get_cpus(ms);
> >      *msi_m_phandle = (*phandle)++;
> >      *msi_s_phandle = (*phandle)++;
> > -    imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
> > +    imsic_cells = g_new0(uint32_t, cpus * 2);
> >      imsic_regs = g_new0(uint32_t, socket_count * 4);
> > 
> >      /* M-level IMSIC node */
> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +    for (cpu = 0; cpu < cpus; cpu++) {
> >          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
> >          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
> >      }
> > @@ -544,7 +545,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
> >          NULL, 0);
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
> > -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
> > +        imsic_cells, cpus * sizeof(uint32_t) * 2);
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
> >          socket_count * sizeof(uint32_t) * 4);
> >      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
> > @@ -562,7 +563,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      g_free(imsic_name);
> > 
> >      /* S-level IMSIC node */
> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +    for (cpu = 0; cpu < cpus; cpu++) {
> >          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
> >          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
> >      }
> > @@ -593,7 +594,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
> >          NULL, 0);
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
> > -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
> > +        imsic_cells, cpus * sizeof(uint32_t) * 2);
> >      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
> >          socket_count * sizeof(uint32_t) * 4);
> >      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
> > @@ -732,7 +733,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      char *clust_name;
> >      int socket, phandle_pos;
> >      MachineState *ms = MACHINE(s);
> > -    uint32_t msi_m_phandle = 0, msi_s_phandle = 0;
> > +    uint32_t msi_m_phandle = 0, msi_s_phandle = 0, cpus;
> >      uint32_t *intc_phandles, xplic_phandles[MAX_NODES];
> >      int socket_count = riscv_socket_count(ms);
> > 
> > @@ -743,9 +744,10 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >      qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
> >      qemu_fdt_add_subnode(ms->fdt, "/cpus/cpu-map");
> > 
> > -    intc_phandles = g_new0(uint32_t, ms->smp.cpus);
> > +    cpus = machine_topo_get_cpus(ms);
> > +    intc_phandles = g_new0(uint32_t, cpus);
> > 
> > -    phandle_pos = ms->smp.cpus;
> > +    phandle_pos = cpus;
> >      for (socket = (socket_count - 1); socket >= 0; socket--) {
> >          phandle_pos -= s->soc[socket].num_harts;
> > 
> > @@ -776,7 +778,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
> > MemMapEntry *memmap,
> >          *msi_pcie_phandle = msi_s_phandle;
> >      }
> > 
> > -    phandle_pos = ms->smp.cpus;
> > +    phandle_pos = cpus;
> >      for (socket = (socket_count - 1); socket >= 0; socket--) {
> >          phandle_pos -= s->soc[socket].num_harts;
> > 
> > @@ -1113,7 +1115,7 @@ static FWCfgState *create_fw_cfg(const
> > MachineState *ms)
> > 
> >      fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16,
> >                                    &address_space_memory);
> > -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
> > +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
> > (uint16_t)machine_topo_get_cpus(ms));
> > 
> >      return fw_cfg;
> >  }
> > --
> > 2.34.1
> 

Re: [RFC 25/52] mips: Replace MachineState.smp access with topology helpers

[Zhao Liu]

On Tue, Feb 14, 2023 at 11:40:14AM +0800, Mi, Dapeng1 wrote:
> Date: Tue, 14 Feb 2023 11:40:14 +0800
> From: "Mi, Dapeng1" <dapeng1.mi@intel.com>
> Subject: RE: [RFC 25/52] mips: Replace MachineState.smp access with
>  topology helpers
> 
> > From: Zhao Liu <zhao1.liu@linux.intel.com>
> > Sent: Monday, February 13, 2023 5:50 PM
> > To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
> > <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daud? <philmd@linaro.org>;
> > Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
> > <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
> > Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
> > Armbruster <armbru@redhat.com>
> > Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
> > Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
> > <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
> > Christopherson,, Sean <seanjc@google.com>; Like Xu
> > <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>; Aleksandar
> > Rikalo <aleksandar.rikalo@syrmia.com>; Huacai Chen
> > <chenhuacai@kernel.org>; Jiaxun Yang <jiaxun.yang@flygoat.com>; Aurelien
> > Jarno <aurelien@aurel32.net>
> > Subject: [RFC 25/52] mips: Replace MachineState.smp access with topology
> > helpers
> > 
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures
> > become complicated. So we wrapped the access to topology fields of
> > MachineState.topo into some helpers, and we are using these helpers
> > to replace the use of MachineState.smp.
> > 
> > For mips, it's straightforward to replace topology access with wrapped
> > generic interfaces.
> > 
> > Cc: Aleksandar Rikalo <aleksandar.rikalo@syrmia.com>
> > Cc: Huacai Chen <chenhuacai@kernel.org>
> > Cc: Jiaxun Yang <jiaxun.yang@flygoat.com>
> > Cc: Aurelien Jarno <aurelien@aurel32.net>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >  hw/mips/boston.c          |  8 +++++---
> >  hw/mips/loongson3_bootp.c |  7 ++++---
> >  hw/mips/loongson3_virt.c  |  8 +++++---
> >  hw/mips/malta.c           | 10 ++++++----
> >  4 files changed, 20 insertions(+), 13 deletions(-)
> > 
> > diff --git a/hw/mips/boston.c b/hw/mips/boston.c
> > index a9d87f34378f..d02c88d769d4 100644
> > --- a/hw/mips/boston.c
> > +++ b/hw/mips/boston.c
> > @@ -514,7 +514,7 @@ static const void *create_fdt(BostonState *s,
> >                                const MemMapEntry *memmap, int *dt_size)
> >  {
> >      void *fdt;
> > -    int cpu;
> > +    int cpu, smp_cpus;
> >      MachineState *ms = s->mach;
> >      uint32_t platreg_ph, gic_ph, clk_ph;
> >      char *name, *gic_name, *platreg_name, *stdout_name;
> > @@ -542,7 +542,8 @@ static const void *create_fdt(BostonState *s,
> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
> > 
> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
> > +    smp_cpus = machine_topo_get_cpus(ms);
> > +    for (cpu = 0; cpu < smp_cpus; cpu++) {
> >          name = g_strdup_printf("/cpus/cpu@%d", cpu);
> >          qemu_fdt_add_subnode(fdt, name);
> >          qemu_fdt_setprop_string(fdt, name, "compatible", "img,mips");
> > @@ -702,7 +703,8 @@ static void boston_mach_init(MachineState *machine)
> >      object_initialize_child(OBJECT(machine), "cps", &s->cps, TYPE_MIPS_CPS);
> >      object_property_set_str(OBJECT(&s->cps), "cpu-type", machine->cpu_type,
> >                              &error_fatal);
> > -    object_property_set_int(OBJECT(&s->cps), "num-vp", machine->smp.cpus,
> > +    object_property_set_int(OBJECT(&s->cps), "num-vp",
> > +                            machine_topo_get_cpus(machine),
> >                              &error_fatal);
> 
> It's better move "&error_fatal);" to previous line.

Yes, this is no more than 80 characters.

> 
> >      qdev_connect_clock_in(DEVICE(&s->cps), "clk-in",
> >                            qdev_get_clock_out(dev, "cpu-refclk"));
> > diff --git a/hw/mips/loongson3_bootp.c b/hw/mips/loongson3_bootp.c
> > index f99af229327a..d9a92825ceae 100644
> > --- a/hw/mips/loongson3_bootp.c
> > +++ b/hw/mips/loongson3_bootp.c
> > @@ -40,9 +40,10 @@ static void init_cpu_info(void *g_cpuinfo, uint64_t
> > cpu_freq)
> >      }
> > 
> >      c->cpu_startup_core_id = cpu_to_le16(0);
> > -    c->nr_cpus = cpu_to_le32(current_machine->smp.cpus);
> > -    c->total_node = cpu_to_le32(DIV_ROUND_UP(current_machine->smp.cpus,
> > -                                             LOONGSON3_CORE_PER_NODE));
> > +    c->nr_cpus = cpu_to_le32(machine_topo_get_cpus(current_machine));
> > +    c->total_node =
> > +        cpu_to_le32(DIV_ROUND_UP(machine_topo_get_cpus(current_machine),
> > +                                 LOONGSON3_CORE_PER_NODE));
> >  }
> > 
> >  static void init_memory_map(void *g_map, uint64_t ram_size)
> > diff --git a/hw/mips/loongson3_virt.c b/hw/mips/loongson3_virt.c
> > index 25534288dd81..c972bb43a1f7 100644
> > --- a/hw/mips/loongson3_virt.c
> > +++ b/hw/mips/loongson3_virt.c
> > @@ -271,8 +271,10 @@ static void fw_conf_init(unsigned long ram_size)
> >      hwaddr cfg_addr = virt_memmap[VIRT_FW_CFG].base;
> > 
> >      fw_cfg = fw_cfg_init_mem_wide(cfg_addr, cfg_addr + 8, 8, 0, NULL);
> > -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)current_machine-
> > >smp.cpus);
> > -    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)current_machine-
> > >smp.max_cpus);
> > +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
> > +                   (uint16_t)machine_topo_get_cpus(current_machine));
> > +    fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS,
> > +                   (uint16_t)machine_topo_get_max_cpus(current_machine));
> >      fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)ram_size);
> >      fw_cfg_add_i32(fw_cfg, FW_CFG_MACHINE_VERSION, 1);
> >      fw_cfg_add_i64(fw_cfg, FW_CFG_CPU_FREQ, get_cpu_freq_hz());
> > @@ -528,7 +530,7 @@ static void mips_loongson3_virt_init(MachineState
> > *machine)
> >      cpuclk = clock_new(OBJECT(machine), "cpu-refclk");
> >      clock_set_hz(cpuclk, DEF_LOONGSON3_FREQ);
> > 
> > -    for (i = 0; i < machine->smp.cpus; i++) {
> > +    for (i = 0; i < machine_topo_get_cpus(machine); i++) {
> >          int ip;
> > 
> >          /* init CPUs */
> > diff --git a/hw/mips/malta.c b/hw/mips/malta.c
> > index ec172b111ae4..b3322f74baf5 100644
> > --- a/hw/mips/malta.c
> > +++ b/hw/mips/malta.c
> > @@ -981,7 +981,7 @@ static uint64_t load_kernel(void)
> >  static void malta_mips_config(MIPSCPU *cpu)
> >  {
> >      MachineState *ms = MACHINE(qdev_get_machine());
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >      CPUMIPSState *env = &cpu->env;
> >      CPUState *cs = CPU(cpu);
> > 
> > @@ -1045,7 +1045,7 @@ static void create_cpu_without_cps(MachineState
> > *ms, MaltaState *s,
> >      MIPSCPU *cpu;
> >      int i;
> > 
> > -    for (i = 0; i < ms->smp.cpus; i++) {
> > +    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
> >          cpu = mips_cpu_create_with_clock(ms->cpu_type, s->cpuclk);
> > 
> >          /* Init internal devices */
> > @@ -1066,7 +1066,8 @@ static void create_cps(MachineState *ms, MaltaState
> > *s,
> >      object_initialize_child(OBJECT(s), "cps", &s->cps, TYPE_MIPS_CPS);
> >      object_property_set_str(OBJECT(&s->cps), "cpu-type", ms->cpu_type,
> >                              &error_fatal);
> > -    object_property_set_int(OBJECT(&s->cps), "num-vp", ms->smp.cpus,
> > +    object_property_set_int(OBJECT(&s->cps), "num-vp",
> > +                            machine_topo_get_cpus(ms),
> >                              &error_fatal);
> >      qdev_connect_clock_in(DEVICE(&s->cps), "clk-in", s->cpuclk);
> >      sysbus_realize(SYS_BUS_DEVICE(&s->cps), &error_fatal);
> > @@ -1080,7 +1081,8 @@ static void create_cps(MachineState *ms, MaltaState
> > *s,
> >  static void mips_create_cpu(MachineState *ms, MaltaState *s,
> >                              qemu_irq *cbus_irq, qemu_irq *i8259_irq)
> >  {
> > -    if ((ms->smp.cpus > 1) && cpu_type_supports_cps_smp(ms->cpu_type)) {
> > +    if ((machine_topo_get_cpus(ms) > 1) &&
> > +        cpu_type_supports_cps_smp(ms->cpu_type)) {
> >          create_cps(ms, s, cbus_irq, i8259_irq);
> >      } else {
> >          create_cpu_without_cps(ms, s, cbus_irq, i8259_irq);
> > --
> > 2.34.1
> 

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Zhao Liu]

On Tue, Feb 14, 2023 at 10:27:53AM +0100, Philippe Mathieu-Daudé wrote:
> Date: Tue, 14 Feb 2023 10:27:53 +0100
> From: Philippe Mathieu-Daudé <philmd@linaro.org>
> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> 
> On 14/2/23 10:30, Zhao Liu wrote:
> > On Mon, Feb 13, 2023 at 02:10:17PM +0100, Philippe Mathieu-Daudé wrote:
> > > Date: Mon, 13 Feb 2023 14:10:17 +0100
> > > From: Philippe Mathieu-Daudé <philmd@linaro.org>
> > > Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> > > 
> > > On 13/2/23 10:49, Zhao Liu wrote:
> > > > From: Zhao Liu <zhao1.liu@intel.com>
> > > > 
> > > > For smp systems, the parts in one topology level are the same. But now
> > > > there are more and more systems with hybrid architectures. Different
> > > > parts of the same topology level may have differences. For example,
> > > > Intel's Alder Lake series CPU has two types of cores, so the CPU
> > > > topology is no longer symmetrical.
> > > > 
> > > > The hybrid topology is compatible with the smp topology type, that is,
> > > > different parts on the same level of the hybrid topology can set to be
> > > > the same, but the hybrid topology will introduce more complexity (need
> > > > to allocate more memory, organized with array or linked-list), so the
> > > > original smp topology support is retained while introducing the hybrid
> > > > topology, and the hybrid topology is only built when the hybrid is
> > > > explicitly required.
> > > > 
> > > > Therefore, we introduce the definition support of hybrid cpu topology
> > > > here. At the same time, in order to unify with the original smp, we
> > > > introduce a new cpu topology structure that can support smp topology
> > > > or hybrid topology. This structure will replace the CpuTopology type (in
> > > > include/hw/boards.h) used by MachineState.smp.
> > > > 
> > > > As for now, we only support two hybrid topology levels: core and
> > > > cluster.
> > > > 
> > > > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > > > ---
> > > >    MAINTAINERS                   |   1 +
> > > >    include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
> > > >    qapi/machine.json             |  12 ++++
> > > >    3 files changed, 130 insertions(+)
> > > >    create mode 100644 include/hw/cpu/cpu-topology.h
> > > > 
> > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > index 58794885ced3..918a9418d98e 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
> > > >    F: include/hw/boards.h
> > > >    F: include/hw/core/cpu.h
> > > >    F: include/hw/cpu/cluster.h
> > > > +F: include/hw/cpu/cpu-topology.h
> > > >    F: include/sysemu/numa.h
> > > >    F: tests/unit/test-smp-parse.c
> > > >    T: git https://gitlab.com/ehabkost/qemu.git machine-next
> > > 
> > > Worth splitting this section in 2, machine VS numa/clusters?
> > 
> > Do you mean splitting this header file into numa.h and cluster.h?
> 
> No, I meant the MAINTAINERS 'Machine' section.

I see. I will slit this.

> 
> > It seems that the cpu topology is not related to the content of
> > numa.h,
> 
> And you seem to agree ;)
> 
> > and it may be possible to merge some content into cluster.h,
> > but the definition in cluster.h is to use the cluster as a device
> > (if I understand correctly...) , and cpu_topology.h is just about
> > topology-related things.
> > 
> > I have not included the contents of cluster.h/cluster.c into hybrid
> > considerations, but only modified the topology defined by smp.
> > 
> > Zhao
> 

Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology

[Zhao Liu]

On Tue, Feb 14, 2023 at 07:23:37PM +0800, wangyanan (Y) wrote:
> Date: Tue, 14 Feb 2023 19:23:37 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> 
> 在 2023/2/14 18:16, Zhao Liu 写道:
> > On Mon, Feb 13, 2023 at 09:18:05PM +0800, wangyanan (Y) wrote:
> > > Date: Mon, 13 Feb 2023 21:18:05 +0800
> > > From: "wangyanan (Y)" <wangyanan55@huawei.com>
> > > Subject: Re: [RFC 06/52] hw/cpu: Introduce hybrid CPU topology
> > > 
> > > Hi Zhao,
> > > 
> > > 在 2023/2/13 17:49, Zhao Liu 写道:
> > > > From: Zhao Liu <zhao1.liu@intel.com>
> > > > 
> > > > For smp systems, the parts in one topology level are the same. But now
> > > > there are more and more systems with hybrid architectures. Different
> > > > parts of the same topology level may have differences. For example,
> > > > Intel's Alder Lake series CPU has two types of cores, so the CPU
> > > > topology is no longer symmetrical.
> > > > 
> > > > The hybrid topology is compatible with the smp topology type, that is,
> > > > different parts on the same level of the hybrid topology can set to be
> > > > the same, but the hybrid topology will introduce more complexity (need
> > > > to allocate more memory, organized with array or linked-list), so the
> > > > original smp topology support is retained while introducing the hybrid
> > > > topology, and the hybrid topology is only built when the hybrid is
> > > > explicitly required.
> > > > 
> > > > Therefore, we introduce the definition support of hybrid cpu topology
> > > > here. At the same time, in order to unify with the original smp, we
> > > > introduce a new cpu topology structure that can support smp topology
> > > > or hybrid topology. This structure will replace the CpuTopology type (in
> > > > include/hw/boards.h) used by MachineState.smp.
> > > > 
> > > > As for now, we only support two hybrid topology levels: core and
> > > > cluster.
> > > > 
> > > > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > > > ---
> > > >    MAINTAINERS                   |   1 +
> > > >    include/hw/cpu/cpu-topology.h | 117 ++++++++++++++++++++++++++++++++++
> > > >    qapi/machine.json             |  12 ++++
> > > >    3 files changed, 130 insertions(+)
> > > >    create mode 100644 include/hw/cpu/cpu-topology.h
> > > > 
> > > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > > index 58794885ced3..918a9418d98e 100644
> > > > --- a/MAINTAINERS
> > > > +++ b/MAINTAINERS
> > > > @@ -1742,6 +1742,7 @@ F: qapi/machine-target.json
> > > >    F: include/hw/boards.h
> > > >    F: include/hw/core/cpu.h
> > > >    F: include/hw/cpu/cluster.h
> > > > +F: include/hw/cpu/cpu-topology.h
> > > Should't it be in include/hw/core/* directory？
> > Yes, I'll move it to the correct place.
> > 
> > > >    F: include/sysemu/numa.h
> > > >    F: tests/unit/test-smp-parse.c
> > > >    T: git https://gitlab.com/ehabkost/qemu.git machine-next
> > > > diff --git a/include/hw/cpu/cpu-topology.h b/include/hw/cpu/cpu-topology.h
> > > > new file mode 100644
> > > > index 000000000000..8268ea3a8569
> > > > --- /dev/null
> > > > +++ b/include/hw/cpu/cpu-topology.h
> > > > @@ -0,0 +1,117 @@
> > > > +/*
> > > > + * CPU topology defination for Machine core
> > > > + *
> > > > + * Copyright (c) 2023 Intel Corporation
> > > > + * Author: Zhao Liu <zhao1.liu@intel.com>
> > > > + *
> > > > + * This program is free software; you can redistribute it and/or modify
> > > > + * it under the terms of the GNU General Public License as published by
> > > > + * the Free Software Foundation; either version 2 of the License,
> > > > + * or (at your option) any later version.
> > > > + *
> > > > + * This program is distributed in the hope that it will be useful,
> > > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > > > + * GNU General Public License for more details.
> > > > + *
> > > > + * You should have received a copy of the GNU General Public License
> > > > + * along with this program; if not, see <http://www.gnu.org/licenses/>.
> > > > + */
> > > > +
> > > > +#ifndef CPU_TOPOLOGY_H
> > > > +#define CPU_TOPOLOGY_H
> > > > +
> > > > +#include "qemu/queue.h"
> > > > +
> > > > +/**
> > > > + * SmpCpuTopology - smp cpu topology defination.
> > > > + *
> > > > + * For smp system, the parts in one topology level are the same.
> > > > + *
> > > > + * @sockets: the number of sockets on the machine
> > > > + * @dies: the number of dies in one socket
> > > > + * @clusters: the number of clusters in one die
> > > > + * @cores: the number of cores in one cluster
> > > > + * @threads: the number of threads in one core
> > > > + */
> > > > +typedef struct SmpCpuTopology {
> > > > +    unsigned int sockets;
> > > > +    unsigned int dies;
> > > > +    unsigned int clusters;
> > > > +    unsigned int cores;
> > > > +    unsigned int threads;
> > > > +} SmpCpuTopology;
> > > > +
> > > > +/**
> > > > + * HybridCore - hybrid core topology defination:
> > > > + * @threads: the number of threads in one core.
> > > > + */
> > > > +typedef struct HybridCore {
> > > > +    unsigned int threads;
> > > > +} HybridCore;
> > > > +
> > > > +/**
> > > > + * HybridCluster - hybrid cluster topology defination:
> > > > + * @cores: the number of cores in current cluster.
> > > > + * @core_list: the array includes all the cores that belong to current
> > > > + *             cluster.
> > > > + */
> > > > +typedef struct HybridCluster {
> > > > +    unsigned int cores;
> > > > +    HybridCore *core_list;
> > > > +} HybridCluster;
> > > > +
> > > > +/**
> > > > + * HybridCpuTopology - hybrid cpu topology defination.
> > > > + *
> > > > + * At present we only support two heterogeneous topology levels: core
> > > > + * and cluster. For heterogeneous levels, we need additional structs
> > > > + * to define their custom internal topology. So here we defines
> > > > + * symmetric topology levels, and use a list to point to heterogeneous
> > > > + * levels.
> > > > + *
> > > > + * @sockets: the number of sockets on the machine. All sockets are the
> > > > + *           same.
> > > > + * @dies: the number of dies in one socket. All dies are the same.
> > > > + * @clusters: the number of clusters in one die. Cluster may be
> > > > + *            different. This field indicates the length of
> > > > + *            cluster_list.
> > > > + * @cluster_list: the array includes all the clusters in one die.
> > > > + */
> > > > +typedef struct HybridCpuTopology {
> > > > +    unsigned int sockets;
> > > > +    unsigned int dies;
> > > > +    unsigned int clusters;
> > > > +    HybridCluster *cluster_list;
> > > > +} HybridCpuTopology;
> > > > +
> > > > +/**
> > > > + * GeneralCpuTopology - General cpu topology defination.
> > > > + *
> > > > + * It supports one of two topologies: smp topology or hybrid topology.
> > > > + *
> > > > + * @cpus: the number of present logical processors on the machine
> > > > + * @max_cpus: the maximum number of logical processors on the machine
> > > > + * @topo_type: the topology type of the machine and this decides which
> > > > + *             member of the union to visit: smp or hybrid.
> > > > + * @smp: the smp cpu topology informantion. Only valid when topo_type is
> > > > + *       CPU_TOPO_TYPE_SMP.
> > > > + * @hybrid: the hybrid cpu topology informantion. Only valid when
> > > > + *          topo_type is CPU_TOPO_TYPE_HYBRID.
> > > > + */
> > > > +typedef struct GeneralCpuTopology {
> > > > +    unsigned int cpus;
> > > > +    unsigned int max_cpus;
> > > > +    CpuTopoType topo_type;
> > > > +    union {
> > > > +        SmpCpuTopology smp;
> > > > +        HybridCpuTopology hybrid;
> > > > +    };
> > > > +} GeneralCpuTopology; /*
> > > > +                       * TODO: This name is temporary, just to distinguish it
> > > > +                       * from the CpuTopology in boards.h. When CpuTopology in
> > > > +                       * boards.h is merged here, it will be uniformly named as
> > > > +                       * CpuTopology.
> > > > +                       */
> > > > +
> > > A suggestion:
> > > 1、Move definition of CpuTopology from boards.h to cpu-topology.h
> > > and re-structure it to include SmpCpuTopology, being a generic cpu
> > > topology structure.
> > > 2、Rename "CpuTopology smp" in MachineState to a generic name
> > > "CpuTopology topo".
> > Here we need to change the access to MachineState.smp to
> > MachineState.topo.smp in other modules.
> > 
> > If replacement of MachineState.topo is in a single patch, do we also
> > need to include the modification of access to MachineState.topo.smp
> > in other modules? Otherwise, it will break the compilation.
> > 
> > In this way, the patch seems be too large.
> Oh, I understand. The temporary "GeneralCpuTopology" seems necessary.
> With this patch and help of the introduced generic helpers, we can replace
> access to MachineState.smp gradually. And that's what you are doing in
> this series.
> 
> But is it possible to strip the whole hybrid related code from the re-work
> of smp. After MachineState.smp is re-worked/generalized completely,
> then we add the hybrid extansion? Now the re-work code of smp and
> hybrid code looks coupled all the way.

I agree. This is more clear.

Thanks,
Zhao

> 
> Thanks,
> Yanan
> > 
> > Thanks,
> > Zhao
> > 
> > > 3、Adapt all the code in QEMU to above change.
> > > 
> > > If you can pack above into a single patch, and then add the hybird
> > > topology extansion in a next patch, we will not need the temporary
> > > thing "GeneralCpuTopology" and the TODO comments, which makes
> > > code clearer.
> > > 
> > > Thanks,
> > > Yanan
> > > > +#endif /* CPU_TOPOLOGY_H */
> > > > diff --git a/qapi/machine.json b/qapi/machine.json
> > > > index b9228a5e4616..bd7303f34497 100644
> > > > --- a/qapi/machine.json
> > > > +++ b/qapi/machine.json
> > > > @@ -36,6 +36,18 @@
> > > >                 'sh4eb', 'sparc', 'sparc64', 'tricore',
> > > >                 'x86_64', 'xtensa', 'xtensaeb' ] }
> > > > +##
> > > > +# @CpuTopoType:
> > > > +#
> > > > +# An enumeration of cpu topology type
> > > > +# TODO: Expose topology type in query-cpus-fast
> > > > +#
> > > > +# Since: 8.0
> > > > +##
> > > > +{ 'enum': 'CpuTopoType',
> > > > +  'prefix': 'CPU_TOPO_TYPE',
> > > > +  'data': [ 'smp', 'hybrid' ] }
> > > > +
> > > >    ##
> > > >    # @CpuS390State:
> > > >    #
> 

Re: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[wangyanan (Y) via]

Hi Zhao,

在 2023/2/13 17:49, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> When MachineState.topo is introduced, the topology related structures
> become complicated. In the general case (hybrid or smp topology),
> accessing the topology information needs to determine whether it is
> currently smp or hybrid topology, and then access the corresponding
> MachineState.topo.smp or MachineState.topo.hybrid.
>
> The best way to do this is to wrap the access to the topology to
> avoid having to check each time it is accessed.
>
> The following helpers are provided here:
>
> - General interfaces - no need to worry about whether the underlying
>    topology is smp or hybrid:
>
> * machine_topo_get_cpus()
> * machine_topo_get_max_cpus()
> * machine_topo_is_smp()
> * machine_topo_get_sockets()
> * machine_topo_get_dies()
> * machine_topo_get_clusters()
> * machine_topo_get_threads();
> * machine_topo_get_cores();
> * machine_topo_get_threads_by_idx()
> * machine_topo_get_cores_by_idx()
> * machine_topo_get_cores_per_socket()
> * machine_topo_get_threads_per_socket()
>
> - SMP-specific interfaces - provided for the cases that are clearly
> known to be smp topology:
>
> * machine_topo_get_smp_cores()
> * machine_topo_get_smp_threads()
>
> Since for hybrid topology, each core may has different threads, if
> someone wants "cpus per core", the cpu_index is need to target a
> specific core (machine_topo_get_threads_by_idx()). But for smp, there is
> no need to be so troublesome, so for this case, we provide smp-specific
> interfaces.
>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/core/machine-topo.c | 142 +++++++++++++++++++++++++++++++++++++++++
>   include/hw/boards.h    |  35 ++++++++++
>   2 files changed, 177 insertions(+)
>
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> index 7223f73f99b0..b20160479629 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -21,6 +21,148 @@
>   #include "hw/boards.h"
>   #include "qapi/error.h"
>   
> +unsigned int machine_topo_get_sockets(const MachineState *ms)
> +{
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> +                                     ms->topo.hybrid.sockets;
> +}
> +
> +unsigned int machine_topo_get_dies(const MachineState *ms)
> +{
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
> +                                     ms->topo.hybrid.dies;
> +}
> +
> +unsigned int machine_topo_get_clusters(const MachineState *ms)
> +{
> +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
> +                                     ms->topo.hybrid.clusters;
> +}
> +
> +unsigned int machine_topo_get_smp_cores(const MachineState *ms)
> +{
> +    g_assert(machine_topo_is_smp(ms));
> +    return ms->topo.smp.cores;
> +}
> +
> +unsigned int machine_topo_get_smp_threads(const MachineState *ms)
> +{
> +    g_assert(machine_topo_is_smp(ms));
> +    return ms->topo.smp.threads;
> +}
> +
> +unsigned int machine_topo_get_threads(const MachineState *ms,
> +                                      unsigned int cluster_id,
> +                                      unsigned int core_id)
> +{
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.threads;
> +    } else {
> +        return ms->topo.hybrid.cluster_list[cluster_id]
> +                   .core_list[core_id].threads;
> +    }
> +
> +    return 0;
> +}
> +
> +unsigned int machine_topo_get_cores(const MachineState *ms,
> +                                    unsigned int cluster_id)
> +{
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores;
> +    } else {
> +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
> +    }
> +}
Is it possible to use variadic function so that those two smp specific
helpers can be avoided? It's a bit wired that we have the generic
machine_topo_get_threads but also need machine_topo_get_smp_threads
at the same time.
> +
> +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> +                                             unsigned int cpu_index)
> +{
> +    unsigned cpus_per_die;
> +    unsigned tmp_idx;
> +    HybridCluster *cluster;
> +    HybridCore *core;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.threads;
> +    }
> +
> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> +                            ms->topo.hybrid.dies);
> +    tmp_idx = cpu_index % cpus_per_die;
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cluster = &ms->topo.hybrid.cluster_list[i];
> +
> +        for (int j = 0; j < cluster->cores; j++) {
> +            core = &cluster->core_list[j];
> +
> +            if (tmp_idx < core->threads) {
> +                return core->threads;
> +            } else {
> +                tmp_idx -= core->threads;
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> +                                           unsigned int cpu_index)
> +{
> +    unsigned cpus_per_die;
> +    unsigned tmp_idx;
> +    HybridCluster *cluster;
> +    HybridCore *core;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores;
> +    }
> +
> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> +                            ms->topo.hybrid.dies);
> +    tmp_idx = cpu_index % cpus_per_die;
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cluster = &ms->topo.hybrid.cluster_list[i];
> +
> +        for (int j = 0; j < cluster->cores; j++) {
> +            core = &cluster->core_list[j];
> +
> +            if (tmp_idx < core->threads) {
> +                return cluster->cores;
> +            } else {
> +                tmp_idx -= core->threads;
> +            }
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
> +{
> +    unsigned int cores_per_die = 0;
> +
> +    if (machine_topo_is_smp(ms)) {
> +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
> +    }
> +
> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
> +    }
> +
> +    return cores_per_die * ms->topo.hybrid.dies;
> +}
> +
> +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
> +{
> +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> +                           ms->topo.hybrid.sockets;
> +    return ms->topo.max_cpus / sockets;
> +}
> +
>   /*
>    * Report information of a machine's supported CPU topology hierarchy.
>    * Topology members will be ordered from the largest to the smallest
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 0a61855499e3..34b64b012022 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;
>   extern GlobalProperty hw_compat_2_1[];
>   extern const size_t hw_compat_2_1_len;
>   
> +static inline
> +unsigned int machine_topo_get_cpus(const MachineState *ms)
> +{
> +    return ms->topo.cpus;
> +}
> +
> +static inline
> +unsigned int machine_topo_get_max_cpus(const MachineState *ms)
> +{
> +    return ms->topo.max_cpus;
> +}
> +
> +static inline
> +bool machine_topo_is_smp(const MachineState *ms)
> +{
> +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP;
> +}
> +
> +unsigned int machine_topo_get_sockets(const MachineState *ms);
> +unsigned int machine_topo_get_dies(const MachineState *ms);
> +unsigned int machine_topo_get_clusters(const MachineState *ms);
> +unsigned int machine_topo_get_smp_cores(const MachineState *ms);
> +unsigned int machine_topo_get_smp_threads(const MachineState *ms);
> +unsigned int machine_topo_get_threads(const MachineState *ms,
> +                                      unsigned int cluster_id,
> +                                      unsigned int core_id);
> +unsigned int machine_topo_get_cores(const MachineState *ms,
> +                                    unsigned int cluster_id);
> +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> +                                             unsigned int cpu_index);
> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> +                                           unsigned int cpu_index);
> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
> +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
> +
>   #endif
I think it's necessary to document the ablity for each helper.
For example, at a flance, I cant figure out what 
machine_topo_get_threads_idx
does. Add some something like:
/*
  * Get number of threads within the CPU core where a processor locates,
  * according to the processor index.
  *
  * @param: ...
  */
will be friendly to future users.

Thanks,
Yanan

Re: [RFC 12/52] hw/acpi: Replace MachineState.smp access with topology helpers

[wangyanan (Y) via]

Hi Zhao,

在 2023/2/13 17:49, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> At present, in QEMU only arm needs PPTT table to build cpu topology.
>
> Before QEMU's arm supports hybrid architectures, it's enough to limit
> the cpu topology of PPTT to smp type through the explicit smp interface
> (machine_topo_get_smp_threads()).
>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Igor Mammedov <imammedo@redhat.com>
> Cc: Ani Sinha <ani@anisinha.ca>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/acpi/aml-build.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> index ea331a20d131..693bd8833d10 100644
> --- a/hw/acpi/aml-build.c
> +++ b/hw/acpi/aml-build.c
> @@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
>               cluster_offset = socket_offset;
>           }
>   
> -        if (ms->smp.threads == 1) {
> +        if (machine_topo_get_smp_threads(ms) == 1) {
>               build_processor_hierarchy_node(table_data,
>                   (1 << 1) | /* ACPI Processor ID valid */
>                   (1 << 3),  /* Node is a Leaf */
ACPI PPTT table is designed to also support the hybrid CPU topology
case where nodes on the same CPU topology level can have different
number of child nodes.

So to be general, the diff should be:
diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
index ea331a20d1..dfded95bbc 100644
--- a/hw/acpi/aml-build.c
+++ b/hw/acpi/aml-build.c
@@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker 
*linker, MachineState *ms,
              cluster_offset = socket_offset;
          }

-        if (ms->smp.threads == 1) {
+        if (machine_topo_get_threads_by_idx(n) == 1) {
              build_processor_hierarchy_node(table_data,
                  (1 << 1) | /* ACPI Processor ID valid */
                  (1 << 3),  /* Node is a Leaf */

Actually I'm recently working on ARM hmp virtualization which relys on
PPTT for topology representation, so we will also need PPTT to be general
for hybrid case anyway.

Thanks,
Yanan

Re: [RFC 23/52] arm: Replace MachineState.smp access with topology helpers

[wangyanan (Y) via]

在 2023/2/13 17:50, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> When MachineState.topo is introduced, the topology related structures
> become complicated. So we wrapped the access to topology fields of
> MachineState.topo into some helpers, and we are using these helpers
> to replace the use of MachineState.smp.
>
> Before arm supports hybrid, here we use smp-specific interface to get
> "threads per core" and "cores per cluster".
>
> For other cases, it's straightforward to replace topology access with
> wrapped generic interfaces.
Sorry. I have not yet understand the necessity of the mixed use
of generic helpers and smp specific helpers😉. For a machine, the
topo type is either smp or hybrid. So far the ARM virt machine's
topo type is always smp, I don't see the difference between
machine_topo_get_cores and machine_topo_get_smp_cores.

When we want to support hybrid for ARM, change the naming
of variables will be enough.

Thanks,
Yanan
> Cc: Jean-Christophe Dubois <jcd@tribudubois.net>
> Cc: Andrey Smirnov <andrew.smirnov@gmail.com>
> Cc: Radoslaw Biernacki <rad@semihalf.com>
> Cc: Leif Lindholm <quic_llindhol@quicinc.com>
> Cc: Shannon Zhao <shannon.zhaosl@gmail.com>
> Cc: Alistair Francis <alistair@alistair23.me>
> Cc: Edgar E. Iglesias <edgar.iglesias@gmail.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/arm/fsl-imx6.c        |  4 ++--
>   hw/arm/fsl-imx6ul.c      |  4 ++--
>   hw/arm/fsl-imx7.c        |  4 ++--
>   hw/arm/highbank.c        |  2 +-
>   hw/arm/realview.c        |  2 +-
>   hw/arm/sbsa-ref.c        |  8 +++----
>   hw/arm/vexpress.c        |  2 +-
>   hw/arm/virt-acpi-build.c |  4 ++--
>   hw/arm/virt.c            | 50 ++++++++++++++++++++++------------------
>   hw/arm/xlnx-zynqmp.c     |  6 ++---
>   include/hw/arm/virt.h    |  2 +-
>   target/arm/cpu.c         |  2 +-
>   target/arm/cpu_tcg.c     |  2 +-
>   target/arm/kvm.c         |  2 +-
>   14 files changed, 50 insertions(+), 44 deletions(-)
>
> diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
> index 00dafe3f62de..e94dec5e6c8d 100644
> --- a/hw/arm/fsl-imx6.c
> +++ b/hw/arm/fsl-imx6.c
> @@ -41,7 +41,7 @@ static void fsl_imx6_init(Object *obj)
>       char name[NAME_SIZE];
>       int i;
>   
> -    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX6_NUM_CPUS); i++) {
> +    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX6_NUM_CPUS); i++) {
>           snprintf(name, NAME_SIZE, "cpu%d", i);
>           object_initialize_child(obj, name, &s->cpu[i],
>                                   ARM_CPU_TYPE_NAME("cortex-a9"));
> @@ -108,7 +108,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
>       FslIMX6State *s = FSL_IMX6(dev);
>       uint16_t i;
>       Error *err = NULL;
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>   
>       if (smp_cpus > FSL_IMX6_NUM_CPUS) {
>           error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
> diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
> index d88d6cc1c5f9..1216b7ff1a92 100644
> --- a/hw/arm/fsl-imx6ul.c
> +++ b/hw/arm/fsl-imx6ul.c
> @@ -160,9 +160,9 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
>       SysBusDevice *sbd;
>       DeviceState *d;
>   
> -    if (ms->smp.cpus > 1) {
> +    if (machine_topo_get_cpus(ms) > 1) {
>           error_setg(errp, "%s: Only a single CPU is supported (%d requested)",
> -                   TYPE_FSL_IMX6UL, ms->smp.cpus);
> +                   TYPE_FSL_IMX6UL, machine_topo_get_cpus(ms));
>           return;
>       }
>   
> diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
> index afc74807990f..f3e569a6ec29 100644
> --- a/hw/arm/fsl-imx7.c
> +++ b/hw/arm/fsl-imx7.c
> @@ -36,7 +36,7 @@ static void fsl_imx7_init(Object *obj)
>       char name[NAME_SIZE];
>       int i;
>   
> -    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX7_NUM_CPUS); i++) {
> +    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX7_NUM_CPUS); i++) {
>           snprintf(name, NAME_SIZE, "cpu%d", i);
>           object_initialize_child(obj, name, &s->cpu[i],
>                                   ARM_CPU_TYPE_NAME("cortex-a7"));
> @@ -148,7 +148,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
>       int i;
>       qemu_irq irq;
>       char name[NAME_SIZE];
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>   
>       if (smp_cpus > FSL_IMX7_NUM_CPUS) {
>           error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
> diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
> index f12aacea6b86..22d6987eafe1 100644
> --- a/hw/arm/highbank.c
> +++ b/hw/arm/highbank.c
> @@ -181,7 +181,7 @@ static void calxeda_init(MachineState *machine, enum cxmachines machine_id)
>       SysBusDevice *busdev;
>       qemu_irq pic[128];
>       int n;
> -    unsigned int smp_cpus = machine->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
>       qemu_irq cpu_irq[4];
>       qemu_irq cpu_fiq[4];
>       qemu_irq cpu_virq[4];
> diff --git a/hw/arm/realview.c b/hw/arm/realview.c
> index a5aa2f046aec..0a2022a34629 100644
> --- a/hw/arm/realview.c
> +++ b/hw/arm/realview.c
> @@ -87,7 +87,7 @@ static void realview_init(MachineState *machine,
>       DriveInfo *dinfo;
>       I2CBus *i2c;
>       int n;
> -    unsigned int smp_cpus = machine->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
>       int done_nic = 0;
>       qemu_irq cpu_irq[4];
>       int is_mpcore = 0;
> diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
> index f778cb6d0979..35f2b83849d5 100644
> --- a/hw/arm/sbsa-ref.c
> +++ b/hw/arm/sbsa-ref.c
> @@ -394,7 +394,7 @@ static void create_secure_ram(SBSAMachineState *sms,
>   
>   static void create_gic(SBSAMachineState *sms)
>   {
> -    unsigned int smp_cpus = MACHINE(sms)->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(MACHINE(sms));
>       SysBusDevice *gicbusdev;
>       const char *gictype;
>       uint32_t redist0_capacity, redist0_count;
> @@ -674,8 +674,8 @@ static void create_secure_ec(MemoryRegion *mem)
>   
>   static void sbsa_ref_init(MachineState *machine)
>   {
> -    unsigned int smp_cpus = machine->smp.cpus;
> -    unsigned int max_cpus = machine->smp.max_cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> +    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
>       SBSAMachineState *sms = SBSA_MACHINE(machine);
>       MachineClass *mc = MACHINE_GET_CLASS(machine);
>       MemoryRegion *sysmem = get_system_memory();
> @@ -801,7 +801,7 @@ static void sbsa_ref_init(MachineState *machine)
>   
>   static const CPUArchIdList *sbsa_ref_possible_cpu_arch_ids(MachineState *ms)
>   {
> -    unsigned int max_cpus = ms->smp.max_cpus;
> +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
>       SBSAMachineState *sms = SBSA_MACHINE(ms);
>       int n;
>   
> diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
> index 34b012b528b0..5e486da27ee8 100644
> --- a/hw/arm/vexpress.c
> +++ b/hw/arm/vexpress.c
> @@ -207,7 +207,7 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
>       DeviceState *dev;
>       SysBusDevice *busdev;
>       int n;
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>   
>       /* Create the actual CPUs */
>       for (n = 0; n < smp_cpus; n++) {
> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> index 4156111d49f0..985b945e762d 100644
> --- a/hw/arm/virt-acpi-build.c
> +++ b/hw/arm/virt-acpi-build.c
> @@ -67,7 +67,7 @@ static void acpi_dsdt_add_cpus(Aml *scope, VirtMachineState *vms)
>       MachineState *ms = MACHINE(vms);
>       uint16_t i;
>   
> -    for (i = 0; i < ms->smp.cpus; i++) {
> +    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
>           Aml *dev = aml_device("C%.03X", i);
>           aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
>           aml_append(dev, aml_name_decl("_UID", aml_int(i)));
> @@ -725,7 +725,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
>       build_append_int_noprefix(table_data, vms->gic_version, 1);
>       build_append_int_noprefix(table_data, 0, 3);   /* Reserved */
>   
> -    for (i = 0; i < MACHINE(vms)->smp.cpus; i++) {
> +    for (i = 0; i < machine_topo_get_cpus(MACHINE(vms)); i++) {
>           ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(i));
>           uint64_t physical_base_address = 0, gich = 0, gicv = 0;
>           uint32_t vgic_interrupt = vms->virt ? PPI(ARCH_GIC_MAINT_IRQ) : 0;
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 75f28947de07..ae65ba2c929c 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -346,7 +346,7 @@ static void fdt_add_timer_nodes(const VirtMachineState *vms)
>       if (vms->gic_version == VIRT_GIC_VERSION_2) {
>           irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
>                                GIC_FDT_IRQ_PPI_CPU_WIDTH,
> -                             (1 << MACHINE(vms)->smp.cpus) - 1);
> +                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
>       }
>   
>       qemu_fdt_add_subnode(ms->fdt, "/timer");
> @@ -374,7 +374,10 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
>       int addr_cells = 1;
>       const MachineState *ms = MACHINE(vms);
>       const VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
> -    int smp_cpus = ms->smp.cpus;
> +    int smp_cpus = machine_topo_get_cpus(ms);
> +    int smp_clusters = machine_topo_get_clusters(ms);
> +    int smp_cores = machine_topo_get_smp_cores(ms);
> +    int smp_threads = machine_topo_get_smp_threads(ms);
>   
>       /*
>        * See Linux Documentation/devicetree/bindings/arm/cpus.yaml
> @@ -461,19 +464,19 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
>               char *cpu_path = g_strdup_printf("/cpus/cpu@%d", cpu);
>               char *map_path;
>   
> -            if (ms->smp.threads > 1) {
> +            if (smp_threads > 1) {
>                   map_path = g_strdup_printf(
>                       "/cpus/cpu-map/socket%d/cluster%d/core%d/thread%d",
> -                    cpu / (ms->smp.clusters * ms->smp.cores * ms->smp.threads),
> -                    (cpu / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters,
> -                    (cpu / ms->smp.threads) % ms->smp.cores,
> -                    cpu % ms->smp.threads);
> +                    cpu / (smp_clusters * smp_cores * smp_threads),
> +                    (cpu / (smp_cores * smp_threads)) % smp_clusters,
> +                    (cpu / smp_threads) % smp_cores,
> +                    cpu % smp_threads);
>               } else {
>                   map_path = g_strdup_printf(
>                       "/cpus/cpu-map/socket%d/cluster%d/core%d",
> -                    cpu / (ms->smp.clusters * ms->smp.cores),
> -                    (cpu / ms->smp.cores) % ms->smp.clusters,
> -                    cpu % ms->smp.cores);
> +                    cpu / (smp_clusters * smp_cores),
> +                    (cpu / smp_cores) % smp_clusters,
> +                    cpu % smp_cores);
>               }
>               qemu_fdt_add_path(ms->fdt, map_path);
>               qemu_fdt_setprop_phandle(ms->fdt, map_path, "cpu", cpu_path);
> @@ -613,7 +616,7 @@ static void fdt_add_pmu_nodes(const VirtMachineState *vms)
>       if (vms->gic_version == VIRT_GIC_VERSION_2) {
>           irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
>                                GIC_FDT_IRQ_PPI_CPU_WIDTH,
> -                             (1 << MACHINE(vms)->smp.cpus) - 1);
> +                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
>       }
>   
>       qemu_fdt_add_subnode(ms->fdt, "/pmu");
> @@ -708,7 +711,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
>       SysBusDevice *gicbusdev;
>       const char *gictype;
>       int i;
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>       uint32_t nb_redist_regions = 0;
>       int revision;
>   
> @@ -1277,7 +1280,7 @@ static FWCfgState *create_fw_cfg(const VirtMachineState *vms, AddressSpace *as)
>       char *nodename;
>   
>       fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16, as);
> -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
> +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine_topo_get_cpus(ms));
>   
>       nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
>       qemu_fdt_add_subnode(ms->fdt, nodename);
> @@ -1908,7 +1911,7 @@ static VirtGICType finalize_gic_version_do(const char *accel_name,
>   static void finalize_gic_version(VirtMachineState *vms)
>   {
>       const char *accel_name = current_accel_name();
> -    unsigned int max_cpus = MACHINE(vms)->smp.max_cpus;
> +    unsigned int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
>       int gics_supported = 0;
>   
>       /* Determine which GIC versions the current environment supports */
> @@ -1958,7 +1961,7 @@ static void finalize_gic_version(VirtMachineState *vms)
>    */
>   static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
>   {
> -    int max_cpus = MACHINE(vms)->smp.max_cpus;
> +    int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
>       bool aarch64, pmu, steal_time;
>       CPUState *cpu;
>   
> @@ -2032,8 +2035,8 @@ static void machvirt_init(MachineState *machine)
>       bool firmware_loaded;
>       bool aarch64 = true;
>       bool has_ged = !vmc->no_ged;
> -    unsigned int smp_cpus = machine->smp.cpus;
> -    unsigned int max_cpus = machine->smp.max_cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> +    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
>   
>       if (!cpu_type_valid(machine->cpu_type)) {
>           error_report("mach-virt: CPU type %s not supported", machine->cpu_type);
> @@ -2664,7 +2667,10 @@ static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int idx)
>   static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
>   {
>       int n;
> -    unsigned int max_cpus = ms->smp.max_cpus;
> +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> +    unsigned int smp_clusters = machine_topo_get_clusters(ms);
> +    unsigned int smp_cores = machine_topo_get_smp_cores(ms);
> +    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
>       VirtMachineState *vms = VIRT_MACHINE(ms);
>       MachineClass *mc = MACHINE_GET_CLASS(vms);
>   
> @@ -2684,16 +2690,16 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
>           assert(!mc->smp_props.dies_supported);
>           ms->possible_cpus->cpus[n].props.has_socket_id = true;
>           ms->possible_cpus->cpus[n].props.socket_id =
> -            n / (ms->smp.clusters * ms->smp.cores * ms->smp.threads);
> +            n / (smp_clusters * smp_cores * smp_threads);
>           ms->possible_cpus->cpus[n].props.has_cluster_id = true;
>           ms->possible_cpus->cpus[n].props.cluster_id =
> -            (n / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters;
> +            (n / (smp_cores * smp_threads)) % smp_clusters;
>           ms->possible_cpus->cpus[n].props.has_core_id = true;
>           ms->possible_cpus->cpus[n].props.core_id =
> -            (n / ms->smp.threads) % ms->smp.cores;
> +            (n / smp_threads) % smp_cores;
>           ms->possible_cpus->cpus[n].props.has_thread_id = true;
>           ms->possible_cpus->cpus[n].props.thread_id =
> -            n % ms->smp.threads;
> +            n % smp_threads;
>       }
>       return ms->possible_cpus;
>   }
> diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c
> index 335cfc417d70..773de7a51680 100644
> --- a/hw/arm/xlnx-zynqmp.c
> +++ b/hw/arm/xlnx-zynqmp.c
> @@ -213,7 +213,7 @@ static void xlnx_zynqmp_create_rpu(MachineState *ms, XlnxZynqMPState *s,
>                                      const char *boot_cpu, Error **errp)
>   {
>       int i;
> -    int num_rpus = MIN(ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS,
> +    int num_rpus = MIN(machine_topo_get_cpus(ms) - XLNX_ZYNQMP_NUM_APU_CPUS,
>                          XLNX_ZYNQMP_NUM_RPU_CPUS);
>   
>       if (num_rpus <= 0) {
> @@ -376,7 +376,7 @@ static void xlnx_zynqmp_init(Object *obj)
>       MachineState *ms = MACHINE(qdev_get_machine());
>       XlnxZynqMPState *s = XLNX_ZYNQMP(obj);
>       int i;
> -    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
> +    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
>   
>       object_initialize_child(obj, "apu-cluster", &s->apu_cluster,
>                               TYPE_CPU_CLUSTER);
> @@ -449,7 +449,7 @@ static void xlnx_zynqmp_realize(DeviceState *dev, Error **errp)
>       MemoryRegion *system_memory = get_system_memory();
>       uint8_t i;
>       uint64_t ram_size;
> -    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
> +    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
>       const char *boot_cpu = s->boot_cpu ? s->boot_cpu : "apu-cpu[0]";
>       ram_addr_t ddr_low_size, ddr_high_size;
>       qemu_irq gic_spi[GIC_NUM_SPI_INTR];
> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
> index e1ddbea96bea..e046f530990f 100644
> --- a/include/hw/arm/virt.h
> +++ b/include/hw/arm/virt.h
> @@ -213,7 +213,7 @@ static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
>   
>       assert(vms->gic_version != VIRT_GIC_VERSION_2);
>   
> -    return (MACHINE(vms)->smp.cpus > redist0_capacity &&
> +    return (machine_topo_get_cpus(MACHINE(vms)) > redist0_capacity &&
>               vms->highmem_redists) ? 2 : 1;
>   }
>   
> diff --git a/target/arm/cpu.c b/target/arm/cpu.c
> index 5f63316dbf22..a7f1d470eed7 100644
> --- a/target/arm/cpu.c
> +++ b/target/arm/cpu.c
> @@ -2079,7 +2079,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
>   
>   #ifndef CONFIG_USER_ONLY
>       MachineState *ms = MACHINE(qdev_get_machine());
> -    unsigned int smp_cpus = ms->smp.cpus;
> +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
>       bool has_secure = cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY);
>   
>       /*
> diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
> index ccde5080eb70..a098d797637c 100644
> --- a/target/arm/cpu_tcg.c
> +++ b/target/arm/cpu_tcg.c
> @@ -515,7 +515,7 @@ static uint64_t a15_l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
>        * Linux wants the number of processors from here.
>        * Might as well set the interrupt-controller bit too.
>        */
> -    return ((ms->smp.cpus - 1) << 24) | (1 << 23);
> +    return ((machine_topo_get_cpus(ms) - 1) << 24) | (1 << 23);
>   }
>   #endif
>   
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index f022c644d2ff..eefded5d203b 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -263,7 +263,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>   
>       cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
>   
> -    if (ms->smp.cpus > 256 &&
> +    if (machine_topo_get_cpus(ms) > 256 &&
>           !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
>           error_report("Using more than 256 vcpus requires a host kernel "
>                        "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");

Re: [RFC 41/52] machine: Introduce core_type() hook

[wangyanan (Y) via]

Hi Zhao,

在 2023/2/13 17:50, Zhao Liu 写道:
> From: Zhao Liu <zhao1.liu@intel.com>
>
> Since supported core types are architecture specific, we need this hook
> to allow archs define its own parsing or validation method.
>
> As the example, add the x86 core_type() which will be used in "-hybrid"
> parameter parsing.
>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
>   hw/core/machine-topo.c | 14 ++++++++++++++
>   hw/core/machine.c      |  1 +
>   hw/i386/x86.c          | 15 +++++++++++++++
>   include/hw/boards.h    |  7 +++++++
>   4 files changed, 37 insertions(+)
>
> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> index 12c05510c1b5..f9ab08a1252e 100644
> --- a/hw/core/machine-topo.c
> +++ b/hw/core/machine-topo.c
> @@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
>           return;
>       }
>   }
> +
> +/*
> + * machine_parse_hybrid_core_type: the default hook to parse hybrid core
> + *                                 type corresponding to the coretype
> + *                                 string option.
> + */
> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> +{
> +    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
> +        return 0;
> +    }
> +
> +    return -1;
> +}
Is it possible that coretype can be NULL?
What would *coretype be if the users don't explicitly specify coretype
in the command line?
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index fad990f49b03..acc32b3be5f6 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -926,6 +926,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
>        * On Linux, each node's border has to be 8MB aligned
>        */
>       mc->numa_mem_align_shift = 23;
> +    mc->core_type = machine_parse_hybrid_core_type;
>   
>       object_class_property_add_str(oc, "kernel",
>           machine_get_kernel, machine_set_kernel);
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index f381fdc43180..f58a90359170 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -1569,6 +1569,20 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
>       qapi_free_SgxEPCList(list);
>   }
>   
> +static int x86_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> +{
> +    X86HybridCoreType type;
> +
> +    if (strcmp(coretype, "atom") == 0) {
> +        type = INTEL_ATOM_TYPE;
> +    } else if (strcmp(coretype, "core") == 0) {
> +        type = INTEL_CORE_TYPE;
> +    } else {
> +        type = INVALID_HYBRID_TYPE;
> +    }
What about:
INTEL_CORE_TYPE_ATOM
INTEL_CORE_TYPE_CORE
X86_CORE_TYPE_UNKNOWN ?
just a suggestion.

Thanks,
Yanan
> +    return type;
> +}
> +
>   static void x86_machine_initfn(Object *obj)
>   {
>       X86MachineState *x86ms = X86_MACHINE(obj);
> @@ -1596,6 +1610,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
>       x86mc->save_tsc_khz = true;
>       x86mc->fwcfg_dma_enabled = true;
>       nc->nmi_monitor_handler = x86_nmi;
> +    mc->core_type = x86_parse_hybrid_core_type;
>   
>       object_class_property_add(oc, X86_MACHINE_SMM, "OnOffAuto",
>           x86_machine_get_smm, x86_machine_set_smm,
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 9364c90d5f1a..34ec035b5c9f 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
>                                  Error **errp);
>   void machine_parse_smp_config(MachineState *ms,
>                                 const SMPConfiguration *config, Error **errp);
> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
>   
>   /**
>    * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
> @@ -199,6 +200,11 @@ typedef struct {
>    *    Return the type of KVM corresponding to the kvm-type string option or
>    *    computed based on other criteria such as the host kernel capabilities.
>    *    kvm-type may be NULL if it is not needed.
> + * @core_type:
> + *    Return the type of hybrid cores corresponding to the coretype string
> + *    option. The default hook only accept "none" or "" since the most generic
> + *    core topology should not specify any specific core type. Each arch can
> + *    define its own core_type() hook to override the default one.
>    * @numa_mem_supported:
>    *    true if '--numa node.mem' option is supported and false otherwise
>    * @hotplug_allowed:
> @@ -237,6 +243,7 @@ struct MachineClass {
>       void (*reset)(MachineState *state, ShutdownCause reason);
>       void (*wakeup)(MachineState *state);
>       int (*kvm_type)(MachineState *machine, const char *arg);
> +    int (*core_type)(MachineState *state, const char *type);
>   
>       BlockInterfaceType block_default_type;
>       int units_per_default_bus;

Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo properties

[wangyanan (Y) via]

在 2023/2/15 10:53, Zhao Liu 写道:
> On Tue, Feb 14, 2023 at 09:46:50AM +0800, wangyanan (Y) wrote:
>> Date: Tue, 14 Feb 2023 09:46:50 +0800
>> From: "wangyanan (Y)" <wangyanan55@huawei.com>
>> Subject: Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo
>>   properties
>>
>> Hi Zhao,
>>
>> 在 2023/2/13 17:50, Zhao Liu 写道:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> Since hybrid cpu topology configuration can benefit not only x86, but
>>> also other architectures/platforms that have supported (in real
>>> machines) or will support hybrid CPU topology, "-hybrid" can be generic.
>>>
>>> So add the generic topology property to configure if support hybrid
>>> cpu topology for architectures/platforms in SmpCompatProps.
>>>
>>> Also rename SmpCompatProps to TopoCompatProps to make this structure
>>> more generic for both smp topology and hybrid topology.
>>>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    include/hw/boards.h | 15 +++++++++++----
>>>    1 file changed, 11 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/include/hw/boards.h b/include/hw/boards.h
>>> index 34ec035b5c9f..17be3485e823 100644
>>> --- a/include/hw/boards.h
>>> +++ b/include/hw/boards.h
>>> @@ -127,19 +127,26 @@ typedef struct {
>>>    } CPUArchIdList;
>>>    /**
>>> - * SMPCompatProps:
>>> - * @prefer_sockets - whether sockets are preferred over cores in smp parsing
>>> + * TopoCompatProps:
>>> + * @hybrid_support - whether hybrid cpu topology are supported by machine.
>> inconsistent with the name in the definition below.
> Thanks! Will fix.
>
>>> + *                   Note that hybrid cpu topology requires to specify the
>>> + *                   topology of each core so that there will no longer be
>>> + *                   a default core topology, thus prefer_sockets won't work
>>> + *                   when hybrid_support is enabled.
>>> + * @prefer_sockets - whether sockets are preferred over cores in smp parsing.
>>> + *                   Not work when hybrid_support is enabled.
>>>     * @dies_supported - whether dies are supported by the machine
>>>     * @clusters_supported - whether clusters are supported by the machine
>>>     * @has_clusters - whether clusters are explicitly specified in the user
>>>     *                 provided SMP configuration
>>>     */
>>>    typedef struct {
>>> +    bool hybrid_supported;
>>>        bool prefer_sockets;
>>>        bool dies_supported;
>>>        bool clusters_supported;
>>>        bool has_clusters;
>>> -} SMPCompatProps;
>>> +} TopoCompatProps;
>> Also here. "Rename SMPCompatProps to TopoCompatProps and
>> move it to cpu-topology.h and adapt the code" should be organized
>> in one or more separate patches, being pre-patches together with
>> the conversion of CpuTopology before.
> Do you think TopoCompatProps/SMPCompatProps should also be moved
> into cpu-topology.h? It seems that SMPCompatProps is a collection
> of properties of MachineClass.
TopoCompatProps holds properties all about CPU topology, I think we
can do this, cpu-topology.h will be included in boards.h any way. But it's
ups to you whether to do this.😉

Thanks,
Yanan
>> And put the "hybrid_supported"
>> extension into another patch. Would this make it easier to review?
> Yes, I agree. Thanks!
>
> Zhao
>
>> Thanks,
>> Yanan
>>>    /**
>>>     * MachineClass:
>>> @@ -281,7 +288,7 @@ struct MachineClass {
>>>        bool nvdimm_supported;
>>>        bool numa_mem_supported;
>>>        bool auto_enable_numa;
>>> -    SMPCompatProps smp_props;
>>> +    TopoCompatProps smp_props;
>>>        const char *default_ram_id;
>>>        HotplugHandler *(*get_hotplug_handler)(MachineState *machine,

Re: [RFC 20/52] s390x: Replace MachineState.smp access with topology helpers

[Thomas Huth]

On 13/02/2023 10.50, Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> When MachineState.topo is introduced, the topology related structures
> become complicated. So we wrapped the access to topology fields of
> MachineState.topo into some helpers, and we are using these helpers
> to replace the use of MachineState.smp.
> 
> In hw/s390x/s390-virtio-ccw.c, s390_init_cpus() needs "threads per core".
> Before s390x supports hybrid, here we use smp-specific interface to get
> "threads per core".
> 
> For other cases, it's straightforward to replace topology access with
> wrapped generic interfaces.
...
> diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> index 3ac7ec9acf4e..d297daed1117 100644
> --- a/target/s390x/kvm/kvm.c
> +++ b/target/s390x/kvm/kvm.c
> @@ -406,9 +406,11 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
>   
>   int kvm_arch_init_vcpu(CPUState *cs)
>   {
> -    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
> +    unsigned int max_cpus;
>       S390CPU *cpu = S390_CPU(cs);
> +
>       kvm_s390_set_cpu_state(cpu, cpu->env.cpu_state);
> +    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
>       cpu->irqstate = g_malloc0(VCPU_IRQ_BUF_SIZE(max_cpus));
>       return 0;
>   }
> @@ -2097,14 +2099,15 @@ int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state)
>   
>   void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu)
>   {
> -    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
> -    struct kvm_s390_irq_state irq_state = {
> -        .buf = (uint64_t) cpu->irqstate,
> -        .len = VCPU_IRQ_BUF_SIZE(max_cpus),
> -    };
> +    unsigned int max_cpus;
> +    struct kvm_s390_irq_state irq_state;
>       CPUState *cs = CPU(cpu);
>       int32_t bytes;
>   
> +    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
> +    irq_state.buf = (uint64_t) cpu->irqstate;
> +    irq_state.len = VCPU_IRQ_BUF_SIZE(max_cpus);

  Hi!

Please don't replace struct initializers like this. There's a reason why 
these structs like irq_state are directly initialized with "= { ... }" at 
the beginning of the function: This automatically clears all fields that are 
not mentioned, e.g. also the "flags" field of struct kvm_s390_irq_state, 
which can be very important for structs that are passed to the kernel via an 
ioctl.
You could use memset(..., 0, ...) instead, but people tend to forget that, 
too, so we settled on using struct initializers at the beginning instead. So 
please stick to that.

  Thanks,
   Thomas


>       if (!kvm_check_extension(kvm_state, KVM_CAP_S390_IRQ_STATE)) {
>           return;
>       }
> diff --git a/target/s390x/tcg/excp_helper.c b/target/s390x/tcg/excp_helper.c
> index bc767f044381..e396a89d5540 100644
> --- a/target/s390x/tcg/excp_helper.c
> +++ b/target/s390x/tcg/excp_helper.c
> @@ -321,7 +321,7 @@ static void do_ext_interrupt(CPUS390XState *env)
>       if ((env->pending_int & INTERRUPT_EMERGENCY_SIGNAL) &&
>           (env->cregs[0] & CR0_EMERGENCY_SIGNAL_SC)) {
>           MachineState *ms = MACHINE(qdev_get_machine());
> -        unsigned int max_cpus = ms->smp.max_cpus;
> +        unsigned int max_cpus = machine_topo_get_max_cpus(ms);
>   
>           lowcore->ext_int_code = cpu_to_be16(EXT_EMERGENCY);
>           cpu_addr = find_first_bit(env->emergency_signals, S390_MAX_CPUS);

Re: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[Zhao Liu]

On Thu, Feb 16, 2023 at 04:38:38PM +0800, wangyanan (Y) wrote:
> Date: Thu, 16 Feb 2023 16:38:38 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 08/52] machine: Add helpers to get cpu topology info from
>  MachineState.topo
> 
> Hi Zhao,
> 
> 在 2023/2/13 17:49, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures
> > become complicated. In the general case (hybrid or smp topology),
> > accessing the topology information needs to determine whether it is
> > currently smp or hybrid topology, and then access the corresponding
> > MachineState.topo.smp or MachineState.topo.hybrid.
> > 
> > The best way to do this is to wrap the access to the topology to
> > avoid having to check each time it is accessed.
> > 
> > The following helpers are provided here:
> > 
> > - General interfaces - no need to worry about whether the underlying
> >    topology is smp or hybrid:
> > 
> > * machine_topo_get_cpus()
> > * machine_topo_get_max_cpus()
> > * machine_topo_is_smp()
> > * machine_topo_get_sockets()
> > * machine_topo_get_dies()
> > * machine_topo_get_clusters()
> > * machine_topo_get_threads();
> > * machine_topo_get_cores();
> > * machine_topo_get_threads_by_idx()
> > * machine_topo_get_cores_by_idx()
> > * machine_topo_get_cores_per_socket()
> > * machine_topo_get_threads_per_socket()
> > 
> > - SMP-specific interfaces - provided for the cases that are clearly
> > known to be smp topology:
> > 
> > * machine_topo_get_smp_cores()
> > * machine_topo_get_smp_threads()
> > 
> > Since for hybrid topology, each core may has different threads, if
> > someone wants "cpus per core", the cpu_index is need to target a
> > specific core (machine_topo_get_threads_by_idx()). But for smp, there is
> > no need to be so troublesome, so for this case, we provide smp-specific
> > interfaces.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/core/machine-topo.c | 142 +++++++++++++++++++++++++++++++++++++++++
> >   include/hw/boards.h    |  35 ++++++++++
> >   2 files changed, 177 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> > index 7223f73f99b0..b20160479629 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -21,6 +21,148 @@
> >   #include "hw/boards.h"
> >   #include "qapi/error.h"
> > +unsigned int machine_topo_get_sockets(const MachineState *ms)
> > +{
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > +                                     ms->topo.hybrid.sockets;
> > +}
> > +
> > +unsigned int machine_topo_get_dies(const MachineState *ms)
> > +{
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
> > +                                     ms->topo.hybrid.dies;
> > +}
> > +
> > +unsigned int machine_topo_get_clusters(const MachineState *ms)
> > +{
> > +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
> > +                                     ms->topo.hybrid.clusters;
> > +}
> > +
> > +unsigned int machine_topo_get_smp_cores(const MachineState *ms)
> > +{
> > +    g_assert(machine_topo_is_smp(ms));
> > +    return ms->topo.smp.cores;
> > +}
> > +
> > +unsigned int machine_topo_get_smp_threads(const MachineState *ms)
> > +{
> > +    g_assert(machine_topo_is_smp(ms));
> > +    return ms->topo.smp.threads;
> > +}
> > +
> > +unsigned int machine_topo_get_threads(const MachineState *ms,
> > +                                      unsigned int cluster_id,
> > +                                      unsigned int core_id)
> > +{
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.threads;
> > +    } else {
> > +        return ms->topo.hybrid.cluster_list[cluster_id]
> > +                   .core_list[core_id].threads;
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +unsigned int machine_topo_get_cores(const MachineState *ms,
> > +                                    unsigned int cluster_id)
> > +{
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores;
> > +    } else {
> > +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
> > +    }
> > +}
> Is it possible to use variadic function so that those two smp specific
> helpers can be avoided? It's a bit wired that we have the generic
> machine_topo_get_threads but also need machine_topo_get_smp_threads
> at the same time.

I am not sure about this, because variadic functions unify function
naming, but eliminate the "smp-specific" information from the name.

Trying to get the cres/threads without considering the cpu index can
only be used in smp scenarios, and I think the caller needs to
understand that he knows it's smp.

> > +
> > +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> > +                                             unsigned int cpu_index)
> > +{
> > +    unsigned cpus_per_die;
> > +    unsigned tmp_idx;
> > +    HybridCluster *cluster;
> > +    HybridCore *core;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.threads;
> > +    }
> > +
> > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > +                            ms->topo.hybrid.dies);
> > +    tmp_idx = cpu_index % cpus_per_die;
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > +
> > +        for (int j = 0; j < cluster->cores; j++) {
> > +            core = &cluster->core_list[j];
> > +
> > +            if (tmp_idx < core->threads) {
> > +                return core->threads;
> > +            } else {
> > +                tmp_idx -= core->threads;
> > +            }
> > +        }
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > +                                           unsigned int cpu_index)
> > +{
> > +    unsigned cpus_per_die;
> > +    unsigned tmp_idx;
> > +    HybridCluster *cluster;
> > +    HybridCore *core;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores;
> > +    }
> > +
> > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > +                            ms->topo.hybrid.dies);
> > +    tmp_idx = cpu_index % cpus_per_die;
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > +
> > +        for (int j = 0; j < cluster->cores; j++) {
> > +            core = &cluster->core_list[j];
> > +
> > +            if (tmp_idx < core->threads) {
> > +                return cluster->cores;
> > +            } else {
> > +                tmp_idx -= core->threads;
> > +            }
> > +        }
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
> > +{
> > +    unsigned int cores_per_die = 0;
> > +
> > +    if (machine_topo_is_smp(ms)) {
> > +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
> > +    }
> > +
> > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
> > +    }
> > +
> > +    return cores_per_die * ms->topo.hybrid.dies;
> > +}
> > +
> > +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
> > +{
> > +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > +                           ms->topo.hybrid.sockets;
> > +    return ms->topo.max_cpus / sockets;
> > +}
> > +
> >   /*
> >    * Report information of a machine's supported CPU topology hierarchy.
> >    * Topology members will be ordered from the largest to the smallest
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 0a61855499e3..34b64b012022 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;
> >   extern GlobalProperty hw_compat_2_1[];
> >   extern const size_t hw_compat_2_1_len;
> > +static inline
> > +unsigned int machine_topo_get_cpus(const MachineState *ms)
> > +{
> > +    return ms->topo.cpus;
> > +}
> > +
> > +static inline
> > +unsigned int machine_topo_get_max_cpus(const MachineState *ms)
> > +{
> > +    return ms->topo.max_cpus;
> > +}
> > +
> > +static inline
> > +bool machine_topo_is_smp(const MachineState *ms)
> > +{
> > +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP;
> > +}
> > +
> > +unsigned int machine_topo_get_sockets(const MachineState *ms);
> > +unsigned int machine_topo_get_dies(const MachineState *ms);
> > +unsigned int machine_topo_get_clusters(const MachineState *ms);
> > +unsigned int machine_topo_get_smp_cores(const MachineState *ms);
> > +unsigned int machine_topo_get_smp_threads(const MachineState *ms);
> > +unsigned int machine_topo_get_threads(const MachineState *ms,
> > +                                      unsigned int cluster_id,
> > +                                      unsigned int core_id);
> > +unsigned int machine_topo_get_cores(const MachineState *ms,
> > +                                    unsigned int cluster_id);
> > +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> > +                                             unsigned int cpu_index);
> > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > +                                           unsigned int cpu_index);
> > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
> > +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
> > +
> >   #endif
> I think it's necessary to document the ablity for each helper.
> For example, at a flance, I cant figure out what
> machine_topo_get_threads_idx
> does. Add some something like:
> /*
>  * Get number of threads within the CPU core where a processor locates,
>  * according to the processor index.
>  *
>  * @param: ...
>  */
> will be friendly to future users.

Yeah, thanks! I will.

> 
> Thanks,
> Yanan

Re: [RFC 12/52] hw/acpi: Replace MachineState.smp access with topology helpers

[Zhao Liu]

On Thu, Feb 16, 2023 at 05:31:11PM +0800, wangyanan (Y) wrote:
> Date: Thu, 16 Feb 2023 17:31:11 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 12/52] hw/acpi: Replace MachineState.smp access with
>  topology helpers
> 
> Hi Zhao,
> 
> 在 2023/2/13 17:49, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > At present, in QEMU only arm needs PPTT table to build cpu topology.
> > 
> > Before QEMU's arm supports hybrid architectures, it's enough to limit
> > the cpu topology of PPTT to smp type through the explicit smp interface
> > (machine_topo_get_smp_threads()).
> > 
> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Cc: Igor Mammedov <imammedo@redhat.com>
> > Cc: Ani Sinha <ani@anisinha.ca>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/acpi/aml-build.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> > index ea331a20d131..693bd8833d10 100644
> > --- a/hw/acpi/aml-build.c
> > +++ b/hw/acpi/aml-build.c
> > @@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
> >               cluster_offset = socket_offset;
> >           }
> > -        if (ms->smp.threads == 1) {
> > +        if (machine_topo_get_smp_threads(ms) == 1) {
> >               build_processor_hierarchy_node(table_data,
> >                   (1 << 1) | /* ACPI Processor ID valid */
> >                   (1 << 3),  /* Node is a Leaf */
> ACPI PPTT table is designed to also support the hybrid CPU topology
> case where nodes on the same CPU topology level can have different
> number of child nodes.
> 
> So to be general, the diff should be:
> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
> index ea331a20d1..dfded95bbc 100644
> --- a/hw/acpi/aml-build.c
> +++ b/hw/acpi/aml-build.c
> @@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker
> *linker, MachineState *ms,
>              cluster_offset = socket_offset;
>          }
> 
> -        if (ms->smp.threads == 1) {
> +        if (machine_topo_get_threads_by_idx(n) == 1) {
>              build_processor_hierarchy_node(table_data,
>                  (1 << 1) | /* ACPI Processor ID valid */
>                  (1 << 3),  /* Node is a Leaf */

Nice! I'll replace that.

> 
> Actually I'm recently working on ARM hmp virtualization which relys on
> PPTT for topology representation, so we will also need PPTT to be general
> for hybrid case anyway.

Good to know that you are considering hybrid support for arm.
BTW, I explained the difference between arm and x86's hybrid in previous
email [1] [2], mainly about whether the cpm model is the same.

I tentatively think that this difference can be solved by arch-specific
coretype(). Do you have any comments on this? Thanks!

[1]: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03884.html
[2]: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03789.html

> 
> Thanks,
> Yanan

Re: [RFC 23/52] arm: Replace MachineState.smp access with topology helpers

[Zhao Liu]

On Thu, Feb 16, 2023 at 06:46:30PM +0800, wangyanan (Y) wrote:
> Date: Thu, 16 Feb 2023 18:46:30 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 23/52] arm: Replace MachineState.smp access with topology
>  helpers
> 
> 在 2023/2/13 17:50, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures
> > become complicated. So we wrapped the access to topology fields of
> > MachineState.topo into some helpers, and we are using these helpers
> > to replace the use of MachineState.smp.
> > 
> > Before arm supports hybrid, here we use smp-specific interface to get
> > "threads per core" and "cores per cluster".
> > 
> > For other cases, it's straightforward to replace topology access with
> > wrapped generic interfaces.
> Sorry. I have not yet understand the necessity of the mixed use
> of generic helpers and smp specific helpers😉. For a machine, the
> topo type is either smp or hybrid. So far the ARM virt machine's
> topo type is always smp, I don't see the difference between
> machine_topo_get_cores and machine_topo_get_smp_cores.

For hybrid, the cpu index is necessary.
But for the common usage of smp, people don't care about the cpu index,
so the cpu index cannot be obtained in many places.

Of course, in this smp case, it is also possible to pass in any cpu index
for machine_topo_get_cores() (such as machine_topo_get_cores(0)), but an
irrelevant cpu index always looks strange...so I introduced the
smp-specific interface.

> 
> When we want to support hybrid for ARM, change the naming
> of variables will be enough.
> 
> Thanks,
> Yanan
> > Cc: Jean-Christophe Dubois <jcd@tribudubois.net>
> > Cc: Andrey Smirnov <andrew.smirnov@gmail.com>
> > Cc: Radoslaw Biernacki <rad@semihalf.com>
> > Cc: Leif Lindholm <quic_llindhol@quicinc.com>
> > Cc: Shannon Zhao <shannon.zhaosl@gmail.com>
> > Cc: Alistair Francis <alistair@alistair23.me>
> > Cc: Edgar E. Iglesias <edgar.iglesias@gmail.com>
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/arm/fsl-imx6.c        |  4 ++--
> >   hw/arm/fsl-imx6ul.c      |  4 ++--
> >   hw/arm/fsl-imx7.c        |  4 ++--
> >   hw/arm/highbank.c        |  2 +-
> >   hw/arm/realview.c        |  2 +-
> >   hw/arm/sbsa-ref.c        |  8 +++----
> >   hw/arm/vexpress.c        |  2 +-
> >   hw/arm/virt-acpi-build.c |  4 ++--
> >   hw/arm/virt.c            | 50 ++++++++++++++++++++++------------------
> >   hw/arm/xlnx-zynqmp.c     |  6 ++---
> >   include/hw/arm/virt.h    |  2 +-
> >   target/arm/cpu.c         |  2 +-
> >   target/arm/cpu_tcg.c     |  2 +-
> >   target/arm/kvm.c         |  2 +-
> >   14 files changed, 50 insertions(+), 44 deletions(-)
> > 
> > diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c
> > index 00dafe3f62de..e94dec5e6c8d 100644
> > --- a/hw/arm/fsl-imx6.c
> > +++ b/hw/arm/fsl-imx6.c
> > @@ -41,7 +41,7 @@ static void fsl_imx6_init(Object *obj)
> >       char name[NAME_SIZE];
> >       int i;
> > -    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX6_NUM_CPUS); i++) {
> > +    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX6_NUM_CPUS); i++) {
> >           snprintf(name, NAME_SIZE, "cpu%d", i);
> >           object_initialize_child(obj, name, &s->cpu[i],
> >                                   ARM_CPU_TYPE_NAME("cortex-a9"));
> > @@ -108,7 +108,7 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp)
> >       FslIMX6State *s = FSL_IMX6(dev);
> >       uint16_t i;
> >       Error *err = NULL;
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >       if (smp_cpus > FSL_IMX6_NUM_CPUS) {
> >           error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
> > diff --git a/hw/arm/fsl-imx6ul.c b/hw/arm/fsl-imx6ul.c
> > index d88d6cc1c5f9..1216b7ff1a92 100644
> > --- a/hw/arm/fsl-imx6ul.c
> > +++ b/hw/arm/fsl-imx6ul.c
> > @@ -160,9 +160,9 @@ static void fsl_imx6ul_realize(DeviceState *dev, Error **errp)
> >       SysBusDevice *sbd;
> >       DeviceState *d;
> > -    if (ms->smp.cpus > 1) {
> > +    if (machine_topo_get_cpus(ms) > 1) {
> >           error_setg(errp, "%s: Only a single CPU is supported (%d requested)",
> > -                   TYPE_FSL_IMX6UL, ms->smp.cpus);
> > +                   TYPE_FSL_IMX6UL, machine_topo_get_cpus(ms));
> >           return;
> >       }
> > diff --git a/hw/arm/fsl-imx7.c b/hw/arm/fsl-imx7.c
> > index afc74807990f..f3e569a6ec29 100644
> > --- a/hw/arm/fsl-imx7.c
> > +++ b/hw/arm/fsl-imx7.c
> > @@ -36,7 +36,7 @@ static void fsl_imx7_init(Object *obj)
> >       char name[NAME_SIZE];
> >       int i;
> > -    for (i = 0; i < MIN(ms->smp.cpus, FSL_IMX7_NUM_CPUS); i++) {
> > +    for (i = 0; i < MIN(machine_topo_get_cpus(ms), FSL_IMX7_NUM_CPUS); i++) {
> >           snprintf(name, NAME_SIZE, "cpu%d", i);
> >           object_initialize_child(obj, name, &s->cpu[i],
> >                                   ARM_CPU_TYPE_NAME("cortex-a7"));
> > @@ -148,7 +148,7 @@ static void fsl_imx7_realize(DeviceState *dev, Error **errp)
> >       int i;
> >       qemu_irq irq;
> >       char name[NAME_SIZE];
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >       if (smp_cpus > FSL_IMX7_NUM_CPUS) {
> >           error_setg(errp, "%s: Only %d CPUs are supported (%d requested)",
> > diff --git a/hw/arm/highbank.c b/hw/arm/highbank.c
> > index f12aacea6b86..22d6987eafe1 100644
> > --- a/hw/arm/highbank.c
> > +++ b/hw/arm/highbank.c
> > @@ -181,7 +181,7 @@ static void calxeda_init(MachineState *machine, enum cxmachines machine_id)
> >       SysBusDevice *busdev;
> >       qemu_irq pic[128];
> >       int n;
> > -    unsigned int smp_cpus = machine->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> >       qemu_irq cpu_irq[4];
> >       qemu_irq cpu_fiq[4];
> >       qemu_irq cpu_virq[4];
> > diff --git a/hw/arm/realview.c b/hw/arm/realview.c
> > index a5aa2f046aec..0a2022a34629 100644
> > --- a/hw/arm/realview.c
> > +++ b/hw/arm/realview.c
> > @@ -87,7 +87,7 @@ static void realview_init(MachineState *machine,
> >       DriveInfo *dinfo;
> >       I2CBus *i2c;
> >       int n;
> > -    unsigned int smp_cpus = machine->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> >       int done_nic = 0;
> >       qemu_irq cpu_irq[4];
> >       int is_mpcore = 0;
> > diff --git a/hw/arm/sbsa-ref.c b/hw/arm/sbsa-ref.c
> > index f778cb6d0979..35f2b83849d5 100644
> > --- a/hw/arm/sbsa-ref.c
> > +++ b/hw/arm/sbsa-ref.c
> > @@ -394,7 +394,7 @@ static void create_secure_ram(SBSAMachineState *sms,
> >   static void create_gic(SBSAMachineState *sms)
> >   {
> > -    unsigned int smp_cpus = MACHINE(sms)->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(MACHINE(sms));
> >       SysBusDevice *gicbusdev;
> >       const char *gictype;
> >       uint32_t redist0_capacity, redist0_count;
> > @@ -674,8 +674,8 @@ static void create_secure_ec(MemoryRegion *mem)
> >   static void sbsa_ref_init(MachineState *machine)
> >   {
> > -    unsigned int smp_cpus = machine->smp.cpus;
> > -    unsigned int max_cpus = machine->smp.max_cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
> >       SBSAMachineState *sms = SBSA_MACHINE(machine);
> >       MachineClass *mc = MACHINE_GET_CLASS(machine);
> >       MemoryRegion *sysmem = get_system_memory();
> > @@ -801,7 +801,7 @@ static void sbsa_ref_init(MachineState *machine)
> >   static const CPUArchIdList *sbsa_ref_possible_cpu_arch_ids(MachineState *ms)
> >   {
> > -    unsigned int max_cpus = ms->smp.max_cpus;
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> >       SBSAMachineState *sms = SBSA_MACHINE(ms);
> >       int n;
> > diff --git a/hw/arm/vexpress.c b/hw/arm/vexpress.c
> > index 34b012b528b0..5e486da27ee8 100644
> > --- a/hw/arm/vexpress.c
> > +++ b/hw/arm/vexpress.c
> > @@ -207,7 +207,7 @@ static void init_cpus(MachineState *ms, const char *cpu_type,
> >       DeviceState *dev;
> >       SysBusDevice *busdev;
> >       int n;
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >       /* Create the actual CPUs */
> >       for (n = 0; n < smp_cpus; n++) {
> > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
> > index 4156111d49f0..985b945e762d 100644
> > --- a/hw/arm/virt-acpi-build.c
> > +++ b/hw/arm/virt-acpi-build.c
> > @@ -67,7 +67,7 @@ static void acpi_dsdt_add_cpus(Aml *scope, VirtMachineState *vms)
> >       MachineState *ms = MACHINE(vms);
> >       uint16_t i;
> > -    for (i = 0; i < ms->smp.cpus; i++) {
> > +    for (i = 0; i < machine_topo_get_cpus(ms); i++) {
> >           Aml *dev = aml_device("C%.03X", i);
> >           aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
> >           aml_append(dev, aml_name_decl("_UID", aml_int(i)));
> > @@ -725,7 +725,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
> >       build_append_int_noprefix(table_data, vms->gic_version, 1);
> >       build_append_int_noprefix(table_data, 0, 3);   /* Reserved */
> > -    for (i = 0; i < MACHINE(vms)->smp.cpus; i++) {
> > +    for (i = 0; i < machine_topo_get_cpus(MACHINE(vms)); i++) {
> >           ARMCPU *armcpu = ARM_CPU(qemu_get_cpu(i));
> >           uint64_t physical_base_address = 0, gich = 0, gicv = 0;
> >           uint32_t vgic_interrupt = vms->virt ? PPI(ARCH_GIC_MAINT_IRQ) : 0;
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index 75f28947de07..ae65ba2c929c 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -346,7 +346,7 @@ static void fdt_add_timer_nodes(const VirtMachineState *vms)
> >       if (vms->gic_version == VIRT_GIC_VERSION_2) {
> >           irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
> >                                GIC_FDT_IRQ_PPI_CPU_WIDTH,
> > -                             (1 << MACHINE(vms)->smp.cpus) - 1);
> > +                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
> >       }
> >       qemu_fdt_add_subnode(ms->fdt, "/timer");
> > @@ -374,7 +374,10 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
> >       int addr_cells = 1;
> >       const MachineState *ms = MACHINE(vms);
> >       const VirtMachineClass *vmc = VIRT_MACHINE_GET_CLASS(vms);
> > -    int smp_cpus = ms->smp.cpus;
> > +    int smp_cpus = machine_topo_get_cpus(ms);
> > +    int smp_clusters = machine_topo_get_clusters(ms);
> > +    int smp_cores = machine_topo_get_smp_cores(ms);
> > +    int smp_threads = machine_topo_get_smp_threads(ms);
> >       /*
> >        * See Linux Documentation/devicetree/bindings/arm/cpus.yaml
> > @@ -461,19 +464,19 @@ static void fdt_add_cpu_nodes(const VirtMachineState *vms)
> >               char *cpu_path = g_strdup_printf("/cpus/cpu@%d", cpu);
> >               char *map_path;
> > -            if (ms->smp.threads > 1) {
> > +            if (smp_threads > 1) {
> >                   map_path = g_strdup_printf(
> >                       "/cpus/cpu-map/socket%d/cluster%d/core%d/thread%d",
> > -                    cpu / (ms->smp.clusters * ms->smp.cores * ms->smp.threads),
> > -                    (cpu / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters,
> > -                    (cpu / ms->smp.threads) % ms->smp.cores,
> > -                    cpu % ms->smp.threads);
> > +                    cpu / (smp_clusters * smp_cores * smp_threads),
> > +                    (cpu / (smp_cores * smp_threads)) % smp_clusters,
> > +                    (cpu / smp_threads) % smp_cores,
> > +                    cpu % smp_threads);
> >               } else {
> >                   map_path = g_strdup_printf(
> >                       "/cpus/cpu-map/socket%d/cluster%d/core%d",
> > -                    cpu / (ms->smp.clusters * ms->smp.cores),
> > -                    (cpu / ms->smp.cores) % ms->smp.clusters,
> > -                    cpu % ms->smp.cores);
> > +                    cpu / (smp_clusters * smp_cores),
> > +                    (cpu / smp_cores) % smp_clusters,
> > +                    cpu % smp_cores);
> >               }
> >               qemu_fdt_add_path(ms->fdt, map_path);
> >               qemu_fdt_setprop_phandle(ms->fdt, map_path, "cpu", cpu_path);
> > @@ -613,7 +616,7 @@ static void fdt_add_pmu_nodes(const VirtMachineState *vms)
> >       if (vms->gic_version == VIRT_GIC_VERSION_2) {
> >           irqflags = deposit32(irqflags, GIC_FDT_IRQ_PPI_CPU_START,
> >                                GIC_FDT_IRQ_PPI_CPU_WIDTH,
> > -                             (1 << MACHINE(vms)->smp.cpus) - 1);
> > +                             (1 << machine_topo_get_cpus(MACHINE(vms))) - 1);
> >       }
> >       qemu_fdt_add_subnode(ms->fdt, "/pmu");
> > @@ -708,7 +711,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
> >       SysBusDevice *gicbusdev;
> >       const char *gictype;
> >       int i;
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >       uint32_t nb_redist_regions = 0;
> >       int revision;
> > @@ -1277,7 +1280,7 @@ static FWCfgState *create_fw_cfg(const VirtMachineState *vms, AddressSpace *as)
> >       char *nodename;
> >       fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16, as);
> > -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
> > +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)machine_topo_get_cpus(ms));
> >       nodename = g_strdup_printf("/fw-cfg@%" PRIx64, base);
> >       qemu_fdt_add_subnode(ms->fdt, nodename);
> > @@ -1908,7 +1911,7 @@ static VirtGICType finalize_gic_version_do(const char *accel_name,
> >   static void finalize_gic_version(VirtMachineState *vms)
> >   {
> >       const char *accel_name = current_accel_name();
> > -    unsigned int max_cpus = MACHINE(vms)->smp.max_cpus;
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
> >       int gics_supported = 0;
> >       /* Determine which GIC versions the current environment supports */
> > @@ -1958,7 +1961,7 @@ static void finalize_gic_version(VirtMachineState *vms)
> >    */
> >   static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem)
> >   {
> > -    int max_cpus = MACHINE(vms)->smp.max_cpus;
> > +    int max_cpus = machine_topo_get_max_cpus(MACHINE(vms));
> >       bool aarch64, pmu, steal_time;
> >       CPUState *cpu;
> > @@ -2032,8 +2035,8 @@ static void machvirt_init(MachineState *machine)
> >       bool firmware_loaded;
> >       bool aarch64 = true;
> >       bool has_ged = !vmc->no_ged;
> > -    unsigned int smp_cpus = machine->smp.cpus;
> > -    unsigned int max_cpus = machine->smp.max_cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(machine);
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(machine);
> >       if (!cpu_type_valid(machine->cpu_type)) {
> >           error_report("mach-virt: CPU type %s not supported", machine->cpu_type);
> > @@ -2664,7 +2667,10 @@ static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int idx)
> >   static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
> >   {
> >       int n;
> > -    unsigned int max_cpus = ms->smp.max_cpus;
> > +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> > +    unsigned int smp_clusters = machine_topo_get_clusters(ms);
> > +    unsigned int smp_cores = machine_topo_get_smp_cores(ms);
> > +    unsigned int smp_threads = machine_topo_get_smp_threads(ms);
> >       VirtMachineState *vms = VIRT_MACHINE(ms);
> >       MachineClass *mc = MACHINE_GET_CLASS(vms);
> > @@ -2684,16 +2690,16 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms)
> >           assert(!mc->smp_props.dies_supported);
> >           ms->possible_cpus->cpus[n].props.has_socket_id = true;
> >           ms->possible_cpus->cpus[n].props.socket_id =
> > -            n / (ms->smp.clusters * ms->smp.cores * ms->smp.threads);
> > +            n / (smp_clusters * smp_cores * smp_threads);
> >           ms->possible_cpus->cpus[n].props.has_cluster_id = true;
> >           ms->possible_cpus->cpus[n].props.cluster_id =
> > -            (n / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters;
> > +            (n / (smp_cores * smp_threads)) % smp_clusters;
> >           ms->possible_cpus->cpus[n].props.has_core_id = true;
> >           ms->possible_cpus->cpus[n].props.core_id =
> > -            (n / ms->smp.threads) % ms->smp.cores;
> > +            (n / smp_threads) % smp_cores;
> >           ms->possible_cpus->cpus[n].props.has_thread_id = true;
> >           ms->possible_cpus->cpus[n].props.thread_id =
> > -            n % ms->smp.threads;
> > +            n % smp_threads;
> >       }
> >       return ms->possible_cpus;
> >   }
> > diff --git a/hw/arm/xlnx-zynqmp.c b/hw/arm/xlnx-zynqmp.c
> > index 335cfc417d70..773de7a51680 100644
> > --- a/hw/arm/xlnx-zynqmp.c
> > +++ b/hw/arm/xlnx-zynqmp.c
> > @@ -213,7 +213,7 @@ static void xlnx_zynqmp_create_rpu(MachineState *ms, XlnxZynqMPState *s,
> >                                      const char *boot_cpu, Error **errp)
> >   {
> >       int i;
> > -    int num_rpus = MIN(ms->smp.cpus - XLNX_ZYNQMP_NUM_APU_CPUS,
> > +    int num_rpus = MIN(machine_topo_get_cpus(ms) - XLNX_ZYNQMP_NUM_APU_CPUS,
> >                          XLNX_ZYNQMP_NUM_RPU_CPUS);
> >       if (num_rpus <= 0) {
> > @@ -376,7 +376,7 @@ static void xlnx_zynqmp_init(Object *obj)
> >       MachineState *ms = MACHINE(qdev_get_machine());
> >       XlnxZynqMPState *s = XLNX_ZYNQMP(obj);
> >       int i;
> > -    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
> > +    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
> >       object_initialize_child(obj, "apu-cluster", &s->apu_cluster,
> >                               TYPE_CPU_CLUSTER);
> > @@ -449,7 +449,7 @@ static void xlnx_zynqmp_realize(DeviceState *dev, Error **errp)
> >       MemoryRegion *system_memory = get_system_memory();
> >       uint8_t i;
> >       uint64_t ram_size;
> > -    int num_apus = MIN(ms->smp.cpus, XLNX_ZYNQMP_NUM_APU_CPUS);
> > +    int num_apus = MIN(machine_topo_get_cpus(ms), XLNX_ZYNQMP_NUM_APU_CPUS);
> >       const char *boot_cpu = s->boot_cpu ? s->boot_cpu : "apu-cpu[0]";
> >       ram_addr_t ddr_low_size, ddr_high_size;
> >       qemu_irq gic_spi[GIC_NUM_SPI_INTR];
> > diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
> > index e1ddbea96bea..e046f530990f 100644
> > --- a/include/hw/arm/virt.h
> > +++ b/include/hw/arm/virt.h
> > @@ -213,7 +213,7 @@ static inline int virt_gicv3_redist_region_count(VirtMachineState *vms)
> >       assert(vms->gic_version != VIRT_GIC_VERSION_2);
> > -    return (MACHINE(vms)->smp.cpus > redist0_capacity &&
> > +    return (machine_topo_get_cpus(MACHINE(vms)) > redist0_capacity &&
> >               vms->highmem_redists) ? 2 : 1;
> >   }
> > diff --git a/target/arm/cpu.c b/target/arm/cpu.c
> > index 5f63316dbf22..a7f1d470eed7 100644
> > --- a/target/arm/cpu.c
> > +++ b/target/arm/cpu.c
> > @@ -2079,7 +2079,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp)
> >   #ifndef CONFIG_USER_ONLY
> >       MachineState *ms = MACHINE(qdev_get_machine());
> > -    unsigned int smp_cpus = ms->smp.cpus;
> > +    unsigned int smp_cpus = machine_topo_get_cpus(ms);
> >       bool has_secure = cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY);
> >       /*
> > diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
> > index ccde5080eb70..a098d797637c 100644
> > --- a/target/arm/cpu_tcg.c
> > +++ b/target/arm/cpu_tcg.c
> > @@ -515,7 +515,7 @@ static uint64_t a15_l2ctlr_read(CPUARMState *env, const ARMCPRegInfo *ri)
> >        * Linux wants the number of processors from here.
> >        * Might as well set the interrupt-controller bit too.
> >        */
> > -    return ((ms->smp.cpus - 1) << 24) | (1 << 23);
> > +    return ((machine_topo_get_cpus(ms) - 1) << 24) | (1 << 23);
> >   }
> >   #endif
> > diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> > index f022c644d2ff..eefded5d203b 100644
> > --- a/target/arm/kvm.c
> > +++ b/target/arm/kvm.c
> > @@ -263,7 +263,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> >       cap_has_mp_state = kvm_check_extension(s, KVM_CAP_MP_STATE);
> > -    if (ms->smp.cpus > 256 &&
> > +    if (machine_topo_get_cpus(ms) > 256 &&
> >           !kvm_check_extension(s, KVM_CAP_ARM_IRQ_LINE_LAYOUT_2)) {
> >           error_report("Using more than 256 vcpus requires a host kernel "
> >                        "with KVM_CAP_ARM_IRQ_LINE_LAYOUT_2");
> 

Re: [RFC 41/52] machine: Introduce core_type() hook

[Zhao Liu]

On Thu, Feb 16, 2023 at 08:15:23PM +0800, wangyanan (Y) wrote:
> Date: Thu, 16 Feb 2023 20:15:23 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 41/52] machine: Introduce core_type() hook
> 
> Hi Zhao,
> 
> 在 2023/2/13 17:50, Zhao Liu 写道:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > Since supported core types are architecture specific, we need this hook
> > to allow archs define its own parsing or validation method.
> > 
> > As the example, add the x86 core_type() which will be used in "-hybrid"
> > parameter parsing.
> > 
> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> >   hw/core/machine-topo.c | 14 ++++++++++++++
> >   hw/core/machine.c      |  1 +
> >   hw/i386/x86.c          | 15 +++++++++++++++
> >   include/hw/boards.h    |  7 +++++++
> >   4 files changed, 37 insertions(+)
> > 
> > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> > index 12c05510c1b5..f9ab08a1252e 100644
> > --- a/hw/core/machine-topo.c
> > +++ b/hw/core/machine-topo.c
> > @@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
> >           return;
> >       }
> >   }
> > +
> > +/*
> > + * machine_parse_hybrid_core_type: the default hook to parse hybrid core
> > + *                                 type corresponding to the coretype
> > + *                                 string option.
> > + */
> > +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> > +{
> > +    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
> > +        return 0;
> > +    }
> > +
> > +    return -1;
> > +}
> Is it possible that coretype can be NULL?
> What would *coretype be if the users don't explicitly specify coretype
> in the command line?

At present, the coretype field cannot be omitted, which requires other code
changes to support omission (if omission is required in the future, there
should be an arch-specific method to supplement the default coretype at the
same time).

> > diff --git a/hw/core/machine.c b/hw/core/machine.c
> > index fad990f49b03..acc32b3be5f6 100644
> > --- a/hw/core/machine.c
> > +++ b/hw/core/machine.c
> > @@ -926,6 +926,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
> >        * On Linux, each node's border has to be 8MB aligned
> >        */
> >       mc->numa_mem_align_shift = 23;
> > +    mc->core_type = machine_parse_hybrid_core_type;
> >       object_class_property_add_str(oc, "kernel",
> >           machine_get_kernel, machine_set_kernel);
> > diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> > index f381fdc43180..f58a90359170 100644
> > --- a/hw/i386/x86.c
> > +++ b/hw/i386/x86.c
> > @@ -1569,6 +1569,20 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
> >       qapi_free_SgxEPCList(list);
> >   }
> > +static int x86_parse_hybrid_core_type(MachineState *ms, const char *coretype)
> > +{
> > +    X86HybridCoreType type;
> > +
> > +    if (strcmp(coretype, "atom") == 0) {
> > +        type = INTEL_ATOM_TYPE;
> > +    } else if (strcmp(coretype, "core") == 0) {
> > +        type = INTEL_CORE_TYPE;
> > +    } else {
> > +        type = INVALID_HYBRID_TYPE;
> > +    }
> What about:
> INTEL_CORE_TYPE_ATOM
> INTEL_CORE_TYPE_CORE
> X86_CORE_TYPE_UNKNOWN ?
> just a suggestion.

It looks better! Thanks.

> 
> Thanks,
> Yanan
> > +    return type;
> > +}
> > +
> >   static void x86_machine_initfn(Object *obj)
> >   {
> >       X86MachineState *x86ms = X86_MACHINE(obj);
> > @@ -1596,6 +1610,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
> >       x86mc->save_tsc_khz = true;
> >       x86mc->fwcfg_dma_enabled = true;
> >       nc->nmi_monitor_handler = x86_nmi;
> > +    mc->core_type = x86_parse_hybrid_core_type;
> >       object_class_property_add(oc, X86_MACHINE_SMM, "OnOffAuto",
> >           x86_machine_get_smm, x86_machine_set_smm,
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 9364c90d5f1a..34ec035b5c9f 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
> >                                  Error **errp);
> >   void machine_parse_smp_config(MachineState *ms,
> >                                 const SMPConfiguration *config, Error **errp);
> > +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
> >   /**
> >    * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
> > @@ -199,6 +200,11 @@ typedef struct {
> >    *    Return the type of KVM corresponding to the kvm-type string option or
> >    *    computed based on other criteria such as the host kernel capabilities.
> >    *    kvm-type may be NULL if it is not needed.
> > + * @core_type:
> > + *    Return the type of hybrid cores corresponding to the coretype string
> > + *    option. The default hook only accept "none" or "" since the most generic
> > + *    core topology should not specify any specific core type. Each arch can
> > + *    define its own core_type() hook to override the default one.
> >    * @numa_mem_supported:
> >    *    true if '--numa node.mem' option is supported and false otherwise
> >    * @hotplug_allowed:
> > @@ -237,6 +243,7 @@ struct MachineClass {
> >       void (*reset)(MachineState *state, ShutdownCause reason);
> >       void (*wakeup)(MachineState *state);
> >       int (*kvm_type)(MachineState *machine, const char *arg);
> > +    int (*core_type)(MachineState *state, const char *type);
> >       BlockInterfaceType block_default_type;
> >       int units_per_default_bus;
> 

Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo properties

[Zhao Liu]

On Thu, Feb 16, 2023 at 08:28:37PM +0800, wangyanan (Y) wrote:
> Date: Thu, 16 Feb 2023 20:28:37 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo
>  properties
> 
> 在 2023/2/15 10:53, Zhao Liu 写道:
> > On Tue, Feb 14, 2023 at 09:46:50AM +0800, wangyanan (Y) wrote:
> > > Date: Tue, 14 Feb 2023 09:46:50 +0800
> > > From: "wangyanan (Y)" <wangyanan55@huawei.com>
> > > Subject: Re: [RFC 42/52] hw/machine: Add hybrid_supported in generic topo
> > >   properties
> > > 
> > > Hi Zhao,
> > > 
> > > 在 2023/2/13 17:50, Zhao Liu 写道:
> > > > From: Zhao Liu <zhao1.liu@intel.com>
> > > > 
> > > > Since hybrid cpu topology configuration can benefit not only x86, but
> > > > also other architectures/platforms that have supported (in real
> > > > machines) or will support hybrid CPU topology, "-hybrid" can be generic.
> > > > 
> > > > So add the generic topology property to configure if support hybrid
> > > > cpu topology for architectures/platforms in SmpCompatProps.
> > > > 
> > > > Also rename SmpCompatProps to TopoCompatProps to make this structure
> > > > more generic for both smp topology and hybrid topology.
> > > > 
> > > > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > > > ---
> > > >    include/hw/boards.h | 15 +++++++++++----
> > > >    1 file changed, 11 insertions(+), 4 deletions(-)
> > > > 
> > > > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > > > index 34ec035b5c9f..17be3485e823 100644
> > > > --- a/include/hw/boards.h
> > > > +++ b/include/hw/boards.h
> > > > @@ -127,19 +127,26 @@ typedef struct {
> > > >    } CPUArchIdList;
> > > >    /**
> > > > - * SMPCompatProps:
> > > > - * @prefer_sockets - whether sockets are preferred over cores in smp parsing
> > > > + * TopoCompatProps:
> > > > + * @hybrid_support - whether hybrid cpu topology are supported by machine.
> > > inconsistent with the name in the definition below.
> > Thanks! Will fix.
> > 
> > > > + *                   Note that hybrid cpu topology requires to specify the
> > > > + *                   topology of each core so that there will no longer be
> > > > + *                   a default core topology, thus prefer_sockets won't work
> > > > + *                   when hybrid_support is enabled.
> > > > + * @prefer_sockets - whether sockets are preferred over cores in smp parsing.
> > > > + *                   Not work when hybrid_support is enabled.
> > > >     * @dies_supported - whether dies are supported by the machine
> > > >     * @clusters_supported - whether clusters are supported by the machine
> > > >     * @has_clusters - whether clusters are explicitly specified in the user
> > > >     *                 provided SMP configuration
> > > >     */
> > > >    typedef struct {
> > > > +    bool hybrid_supported;
> > > >        bool prefer_sockets;
> > > >        bool dies_supported;
> > > >        bool clusters_supported;
> > > >        bool has_clusters;
> > > > -} SMPCompatProps;
> > > > +} TopoCompatProps;
> > > Also here. "Rename SMPCompatProps to TopoCompatProps and
> > > move it to cpu-topology.h and adapt the code" should be organized
> > > in one or more separate patches, being pre-patches together with
> > > the conversion of CpuTopology before.
> > Do you think TopoCompatProps/SMPCompatProps should also be moved
> > into cpu-topology.h? It seems that SMPCompatProps is a collection
> > of properties of MachineClass.
> TopoCompatProps holds properties all about CPU topology, I think we
> can do this, cpu-topology.h will be included in boards.h any way. But it's
> ups to you whether to do this.😉

Yeah, it makes sense to manage all topologically related.
So I will, thanks!

> 
> Thanks,
> Yanan
> > > And put the "hybrid_supported"
> > > extension into another patch. Would this make it easier to review?
> > Yes, I agree. Thanks!
> > 
> > Zhao
> > 
> > > Thanks,
> > > Yanan
> > > >    /**
> > > >     * MachineClass:
> > > > @@ -281,7 +288,7 @@ struct MachineClass {
> > > >        bool nvdimm_supported;
> > > >        bool numa_mem_supported;
> > > >        bool auto_enable_numa;
> > > > -    SMPCompatProps smp_props;
> > > > +    TopoCompatProps smp_props;
> > > >        const char *default_ram_id;
> > > >        HotplugHandler *(*get_hotplug_handler)(MachineState *machine,
> 

Re: [RFC 20/52] s390x: Replace MachineState.smp access with topology helpers

[Zhao Liu]

On Thu, Feb 16, 2023 at 02:38:55PM +0100, Thomas Huth wrote:
> Date: Thu, 16 Feb 2023 14:38:55 +0100
> From: Thomas Huth <thuth@redhat.com>
> Subject: Re: [RFC 20/52] s390x: Replace MachineState.smp access with
>  topology helpers
> 
> On 13/02/2023 10.50, Zhao Liu wrote:
> > From: Zhao Liu <zhao1.liu@intel.com>
> > 
> > When MachineState.topo is introduced, the topology related structures
> > become complicated. So we wrapped the access to topology fields of
> > MachineState.topo into some helpers, and we are using these helpers
> > to replace the use of MachineState.smp.
> > 
> > In hw/s390x/s390-virtio-ccw.c, s390_init_cpus() needs "threads per core".
> > Before s390x supports hybrid, here we use smp-specific interface to get
> > "threads per core".
> > 
> > For other cases, it's straightforward to replace topology access with
> > wrapped generic interfaces.
> ...
> > diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
> > index 3ac7ec9acf4e..d297daed1117 100644
> > --- a/target/s390x/kvm/kvm.c
> > +++ b/target/s390x/kvm/kvm.c
> > @@ -406,9 +406,11 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
> >   int kvm_arch_init_vcpu(CPUState *cs)
> >   {
> > -    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
> > +    unsigned int max_cpus;
> >       S390CPU *cpu = S390_CPU(cs);
> > +
> >       kvm_s390_set_cpu_state(cpu, cpu->env.cpu_state);
> > +    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
> >       cpu->irqstate = g_malloc0(VCPU_IRQ_BUF_SIZE(max_cpus));
> >       return 0;
> >   }
> > @@ -2097,14 +2099,15 @@ int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state)
> >   void kvm_s390_vcpu_interrupt_pre_save(S390CPU *cpu)
> >   {
> > -    unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
> > -    struct kvm_s390_irq_state irq_state = {
> > -        .buf = (uint64_t) cpu->irqstate,
> > -        .len = VCPU_IRQ_BUF_SIZE(max_cpus),
> > -    };
> > +    unsigned int max_cpus;
> > +    struct kvm_s390_irq_state irq_state;
> >       CPUState *cs = CPU(cpu);
> >       int32_t bytes;
> > +    max_cpus = machine_topo_get_max_cpus(MACHINE(qdev_get_machine()));
> > +    irq_state.buf = (uint64_t) cpu->irqstate;
> > +    irq_state.len = VCPU_IRQ_BUF_SIZE(max_cpus);
> 
>  Hi!
> 
> Please don't replace struct initializers like this. There's a reason why
> these structs like irq_state are directly initialized with "= { ... }" at
> the beginning of the function: This automatically clears all fields that are
> not mentioned, e.g. also the "flags" field of struct kvm_s390_irq_state,
> which can be very important for structs that are passed to the kernel via an
> ioctl.
> You could use memset(..., 0, ...) instead, but people tend to forget that,
> too, so we settled on using struct initializers at the beginning instead. So
> please stick to that.

Thanks Thomas! Sorry I didn't notice this, I'll fix it and be careful in the
future.

Zhao

> 
>  Thanks,
>   Thomas
> 
> 
> >       if (!kvm_check_extension(kvm_state, KVM_CAP_S390_IRQ_STATE)) {
> >           return;
> >       }
> > diff --git a/target/s390x/tcg/excp_helper.c b/target/s390x/tcg/excp_helper.c
> > index bc767f044381..e396a89d5540 100644
> > --- a/target/s390x/tcg/excp_helper.c
> > +++ b/target/s390x/tcg/excp_helper.c
> > @@ -321,7 +321,7 @@ static void do_ext_interrupt(CPUS390XState *env)
> >       if ((env->pending_int & INTERRUPT_EMERGENCY_SIGNAL) &&
> >           (env->cregs[0] & CR0_EMERGENCY_SIGNAL_SC)) {
> >           MachineState *ms = MACHINE(qdev_get_machine());
> > -        unsigned int max_cpus = ms->smp.max_cpus;
> > +        unsigned int max_cpus = machine_topo_get_max_cpus(ms);
> >           lowcore->ext_int_code = cpu_to_be16(EXT_EMERGENCY);
> >           cpu_addr = find_first_bit(env->emergency_signals, S390_MAX_CPUS);
> 

Re: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[wangyanan (Y) via]

在 2023/2/17 11:07, Zhao Liu 写道:
> On Thu, Feb 16, 2023 at 04:38:38PM +0800, wangyanan (Y) wrote:
>> Date: Thu, 16 Feb 2023 16:38:38 +0800
>> From: "wangyanan (Y)" <wangyanan55@huawei.com>
>> Subject: Re: [RFC 08/52] machine: Add helpers to get cpu topology info from
>>   MachineState.topo
>>
>> Hi Zhao,
>>
>> 在 2023/2/13 17:49, Zhao Liu 写道:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> When MachineState.topo is introduced, the topology related structures
>>> become complicated. In the general case (hybrid or smp topology),
>>> accessing the topology information needs to determine whether it is
>>> currently smp or hybrid topology, and then access the corresponding
>>> MachineState.topo.smp or MachineState.topo.hybrid.
>>>
>>> The best way to do this is to wrap the access to the topology to
>>> avoid having to check each time it is accessed.
>>>
>>> The following helpers are provided here:
>>>
>>> - General interfaces - no need to worry about whether the underlying
>>>     topology is smp or hybrid:
>>>
>>> * machine_topo_get_cpus()
>>> * machine_topo_get_max_cpus()
>>> * machine_topo_is_smp()
>>> * machine_topo_get_sockets()
>>> * machine_topo_get_dies()
>>> * machine_topo_get_clusters()
>>> * machine_topo_get_threads();
>>> * machine_topo_get_cores();
>>> * machine_topo_get_threads_by_idx()
>>> * machine_topo_get_cores_by_idx()
>>> * machine_topo_get_cores_per_socket()
>>> * machine_topo_get_threads_per_socket()
>>>
>>> - SMP-specific interfaces - provided for the cases that are clearly
>>> known to be smp topology:
>>>
>>> * machine_topo_get_smp_cores()
>>> * machine_topo_get_smp_threads()
>>>
>>> Since for hybrid topology, each core may has different threads, if
>>> someone wants "cpus per core", the cpu_index is need to target a
>>> specific core (machine_topo_get_threads_by_idx()). But for smp, there is
>>> no need to be so troublesome, so for this case, we provide smp-specific
>>> interfaces.
>>>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    hw/core/machine-topo.c | 142 +++++++++++++++++++++++++++++++++++++++++
>>>    include/hw/boards.h    |  35 ++++++++++
>>>    2 files changed, 177 insertions(+)
>>>
>>> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
>>> index 7223f73f99b0..b20160479629 100644
>>> --- a/hw/core/machine-topo.c
>>> +++ b/hw/core/machine-topo.c
>>> @@ -21,6 +21,148 @@
>>>    #include "hw/boards.h"
>>>    #include "qapi/error.h"
>>> +unsigned int machine_topo_get_sockets(const MachineState *ms)
>>> +{
>>> +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
>>> +                                     ms->topo.hybrid.sockets;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_dies(const MachineState *ms)
>>> +{
>>> +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
>>> +                                     ms->topo.hybrid.dies;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_clusters(const MachineState *ms)
>>> +{
>>> +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
>>> +                                     ms->topo.hybrid.clusters;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_smp_cores(const MachineState *ms)
>>> +{
>>> +    g_assert(machine_topo_is_smp(ms));
>>> +    return ms->topo.smp.cores;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_smp_threads(const MachineState *ms)
>>> +{
>>> +    g_assert(machine_topo_is_smp(ms));
>>> +    return ms->topo.smp.threads;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_threads(const MachineState *ms,
>>> +                                      unsigned int cluster_id,
>>> +                                      unsigned int core_id)
>>> +{
>>> +    if (machine_topo_is_smp(ms)) {
>>> +        return ms->topo.smp.threads;
>>> +    } else {
>>> +        return ms->topo.hybrid.cluster_list[cluster_id]
>>> +                   .core_list[core_id].threads;
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_cores(const MachineState *ms,
>>> +                                    unsigned int cluster_id)
>>> +{
>>> +    if (machine_topo_is_smp(ms)) {
>>> +        return ms->topo.smp.cores;
>>> +    } else {
>>> +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
>>> +    }
>>> +}
>> Is it possible to use variadic function so that those two smp specific
>> helpers can be avoided? It's a bit wired that we have the generic
>> machine_topo_get_threads but also need machine_topo_get_smp_threads
>> at the same time.
> I am not sure about this, because variadic functions unify function
> naming, but eliminate the "smp-specific" information from the name.
>
> Trying to get the cres/threads without considering the cpu index can
> only be used in smp scenarios, and I think the caller needs to
> understand that he knows it's smp.
Ok, I get the point.
When it comes to the naming, would it be more concise to remove the
*_get_* in the fun name, such as machine_topo_get_cpus to
machine_topo_cpus, machine_topo_get_clusters to machine_topo_clusters.

And maybe rename machine_topo_get_cores(int cluster_id, int core_id) to
machine_topo_cores_by_ids?

Or machine_topo_get_cores() to machine_topo_cores_by_topo_ids()
and machine_topo_get_cores_by_idx to machine_topo_cores_by_cpu_idx()
>>> +
>>> +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
>>> +                                             unsigned int cpu_index)
>>> +{
>>> +    unsigned cpus_per_die;
>>> +    unsigned tmp_idx;
>>> +    HybridCluster *cluster;
>>> +    HybridCore *core;
>>> +
>>> +    if (machine_topo_is_smp(ms)) {
>>> +        return ms->topo.smp.threads;
>>> +    }
>>> +
>>> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
>>> +                            ms->topo.hybrid.dies);
>>> +    tmp_idx = cpu_index % cpus_per_die;
>>> +
>>> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
>>> +        cluster = &ms->topo.hybrid.cluster_list[i];
>>> +
>>> +        for (int j = 0; j < cluster->cores; j++) {
>>> +            core = &cluster->core_list[j];
>>> +
>>> +            if (tmp_idx < core->threads) {
>>> +                return core->threads;
>>> +            } else {
>>> +                tmp_idx -= core->threads;
>>> +            }
>>> +        }
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
>>> +                                           unsigned int cpu_index)
>>> +{
>>> +    unsigned cpus_per_die;
>>> +    unsigned tmp_idx;
>>> +    HybridCluster *cluster;
>>> +    HybridCore *core;
>>> +
>>> +    if (machine_topo_is_smp(ms)) {
>>> +        return ms->topo.smp.cores;
>>> +    }
>>> +
>>> +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
>>> +                            ms->topo.hybrid.dies);
>>> +    tmp_idx = cpu_index % cpus_per_die;
>>> +
>>> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
>>> +        cluster = &ms->topo.hybrid.cluster_list[i];
>>> +
>>> +        for (int j = 0; j < cluster->cores; j++) {
>>> +            core = &cluster->core_list[j];
>>> +
>>> +            if (tmp_idx < core->threads) {
>>> +                return cluster->cores;
>>> +            } else {
>>> +                tmp_idx -= core->threads;
>>> +            }
>>> +        }
>>> +    }
>>> +
>>> +    return 0;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
>>> +{
>>> +    unsigned int cores_per_die = 0;
>>> +
>>> +    if (machine_topo_is_smp(ms)) {
>>> +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
>>> +    }
>>> +
>>> +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
>>> +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
>>> +    }
>>> +
>>> +    return cores_per_die * ms->topo.hybrid.dies;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
>>> +{
>>> +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
>>> +                           ms->topo.hybrid.sockets;
>>> +    return ms->topo.max_cpus / sockets;
>>> +}
>>> +
>>>    /*
>>>     * Report information of a machine's supported CPU topology hierarchy.
>>>     * Topology members will be ordered from the largest to the smallest
>>> diff --git a/include/hw/boards.h b/include/hw/boards.h
>>> index 0a61855499e3..34b64b012022 100644
>>> --- a/include/hw/boards.h
>>> +++ b/include/hw/boards.h
>>> @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;
>>>    extern GlobalProperty hw_compat_2_1[];
>>>    extern const size_t hw_compat_2_1_len;
>>> +static inline
>>> +unsigned int machine_topo_get_cpus(const MachineState *ms)
>>> +{
>>> +    return ms->topo.cpus;
>>> +}
>>> +
>>> +static inline
>>> +unsigned int machine_topo_get_max_cpus(const MachineState *ms)
>>> +{
>>> +    return ms->topo.max_cpus;
>>> +}
>>> +
>>> +static inline
>>> +bool machine_topo_is_smp(const MachineState *ms)
>>> +{
>>> +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP;
>>> +}
>>> +
>>> +unsigned int machine_topo_get_sockets(const MachineState *ms);
>>> +unsigned int machine_topo_get_dies(const MachineState *ms);
>>> +unsigned int machine_topo_get_clusters(const MachineState *ms);
>>> +unsigned int machine_topo_get_smp_cores(const MachineState *ms);
>>> +unsigned int machine_topo_get_smp_threads(const MachineState *ms);
>>> +unsigned int machine_topo_get_threads(const MachineState *ms,
>>> +                                      unsigned int cluster_id,
>>> +                                      unsigned int core_id);
>>> +unsigned int machine_topo_get_cores(const MachineState *ms,
>>> +                                    unsigned int cluster_id);
>>> +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
>>> +                                             unsigned int cpu_index);
>>> +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
>>> +                                           unsigned int cpu_index);
>>> +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
>>> +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
>>> +
>>>    #endif
>> I think it's necessary to document the ablity for each helper.
>> For example, at a flance, I cant figure out what
>> machine_topo_get_threads_idx
>> does. Add some something like:
>> /*
>>   * Get number of threads within the CPU core where a processor locates,
>>   * according to the processor index.
>>   *
>>   * @param: ...
>>   */
>> will be friendly to future users.
> Yeah, thanks! I will.
>
>> Thanks,
>> Yanan

Re: [RFC 41/52] machine: Introduce core_type() hook

[wangyanan (Y) via]

在 2023/2/17 11:26, Zhao Liu 写道:
> On Thu, Feb 16, 2023 at 08:15:23PM +0800, wangyanan (Y) wrote:
>> Date: Thu, 16 Feb 2023 20:15:23 +0800
>> From: "wangyanan (Y)" <wangyanan55@huawei.com>
>> Subject: Re: [RFC 41/52] machine: Introduce core_type() hook
>>
>> Hi Zhao,
>>
>> 在 2023/2/13 17:50, Zhao Liu 写道:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> Since supported core types are architecture specific, we need this hook
>>> to allow archs define its own parsing or validation method.
>>>
>>> As the example, add the x86 core_type() which will be used in "-hybrid"
>>> parameter parsing.
>>>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    hw/core/machine-topo.c | 14 ++++++++++++++
>>>    hw/core/machine.c      |  1 +
>>>    hw/i386/x86.c          | 15 +++++++++++++++
>>>    include/hw/boards.h    |  7 +++++++
>>>    4 files changed, 37 insertions(+)
>>>
>>> diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
>>> index 12c05510c1b5..f9ab08a1252e 100644
>>> --- a/hw/core/machine-topo.c
>>> +++ b/hw/core/machine-topo.c
>>> @@ -352,3 +352,17 @@ void machine_parse_smp_config(MachineState *ms,
>>>            return;
>>>        }
>>>    }
>>> +
>>> +/*
>>> + * machine_parse_hybrid_core_type: the default hook to parse hybrid core
>>> + *                                 type corresponding to the coretype
>>> + *                                 string option.
>>> + */
>>> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype)
>>> +{
>>> +    if (strcmp(coretype, "") == 0 || strcmp(coretype, "none") == 0) {
>>> +        return 0;
>>> +    }
>>> +
>>> +    return -1;
>>> +}
>> Is it possible that coretype can be NULL?
>> What would *coretype be if the users don't explicitly specify coretype
>> in the command line?
> At present, the coretype field cannot be omitted, which requires other code
> changes to support omission (if omission is required in the future, there
> should be an arch-specific method to supplement the default coretype at the
> same time).
IIUC, we may need to support the handling of omission case at the
beginning. Not all archs have/need the core type concept when they
support hybrid, and if an arch does not have the core type concept,
it's best to forbid it in the CLI and leave the handling to the generic
machine_parse_hybrid_core_type and the arch-specific core_type
hook should be NULL.

Thanks,
Yanan
>
>>> diff --git a/hw/core/machine.c b/hw/core/machine.c
>>> index fad990f49b03..acc32b3be5f6 100644
>>> --- a/hw/core/machine.c
>>> +++ b/hw/core/machine.c
>>> @@ -926,6 +926,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
>>>         * On Linux, each node's border has to be 8MB aligned
>>>         */
>>>        mc->numa_mem_align_shift = 23;
>>> +    mc->core_type = machine_parse_hybrid_core_type;
>>>        object_class_property_add_str(oc, "kernel",
>>>            machine_get_kernel, machine_set_kernel);
>>> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
>>> index f381fdc43180..f58a90359170 100644
>>> --- a/hw/i386/x86.c
>>> +++ b/hw/i386/x86.c
>>> @@ -1569,6 +1569,20 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
>>>        qapi_free_SgxEPCList(list);
>>>    }
>>> +static int x86_parse_hybrid_core_type(MachineState *ms, const char *coretype)
>>> +{
>>> +    X86HybridCoreType type;
>>> +
>>> +    if (strcmp(coretype, "atom") == 0) {
>>> +        type = INTEL_ATOM_TYPE;
>>> +    } else if (strcmp(coretype, "core") == 0) {
>>> +        type = INTEL_CORE_TYPE;
>>> +    } else {
>>> +        type = INVALID_HYBRID_TYPE;
>>> +    }
>> What about:
>> INTEL_CORE_TYPE_ATOM
>> INTEL_CORE_TYPE_CORE
>> X86_CORE_TYPE_UNKNOWN ?
>> just a suggestion.
> It looks better! Thanks.
>
>> Thanks,
>> Yanan
>>> +    return type;
>>> +}
>>> +
>>>    static void x86_machine_initfn(Object *obj)
>>>    {
>>>        X86MachineState *x86ms = X86_MACHINE(obj);
>>> @@ -1596,6 +1610,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
>>>        x86mc->save_tsc_khz = true;
>>>        x86mc->fwcfg_dma_enabled = true;
>>>        nc->nmi_monitor_handler = x86_nmi;
>>> +    mc->core_type = x86_parse_hybrid_core_type;
>>>        object_class_property_add(oc, X86_MACHINE_SMM, "OnOffAuto",
>>>            x86_machine_get_smm, x86_machine_set_smm,
>>> diff --git a/include/hw/boards.h b/include/hw/boards.h
>>> index 9364c90d5f1a..34ec035b5c9f 100644
>>> --- a/include/hw/boards.h
>>> +++ b/include/hw/boards.h
>>> @@ -36,6 +36,7 @@ void machine_set_cpu_numa_node(MachineState *machine,
>>>                                   Error **errp);
>>>    void machine_parse_smp_config(MachineState *ms,
>>>                                  const SMPConfiguration *config, Error **errp);
>>> +int machine_parse_hybrid_core_type(MachineState *ms, const char *coretype);
>>>    /**
>>>     * machine_class_allow_dynamic_sysbus_dev: Add type to list of valid devices
>>> @@ -199,6 +200,11 @@ typedef struct {
>>>     *    Return the type of KVM corresponding to the kvm-type string option or
>>>     *    computed based on other criteria such as the host kernel capabilities.
>>>     *    kvm-type may be NULL if it is not needed.
>>> + * @core_type:
>>> + *    Return the type of hybrid cores corresponding to the coretype string
>>> + *    option. The default hook only accept "none" or "" since the most generic
>>> + *    core topology should not specify any specific core type. Each arch can
>>> + *    define its own core_type() hook to override the default one.
>>>     * @numa_mem_supported:
>>>     *    true if '--numa node.mem' option is supported and false otherwise
>>>     * @hotplug_allowed:
>>> @@ -237,6 +243,7 @@ struct MachineClass {
>>>        void (*reset)(MachineState *state, ShutdownCause reason);
>>>        void (*wakeup)(MachineState *state);
>>>        int (*kvm_type)(MachineState *machine, const char *arg);
>>> +    int (*core_type)(MachineState *state, const char *type);
>>>        BlockInterfaceType block_default_type;
>>>        int units_per_default_bus;

Re: [RFC 12/52] hw/acpi: Replace MachineState.smp access with topology helpers

[wangyanan (Y) via]

在 2023/2/17 11:14, Zhao Liu 写道:
> On Thu, Feb 16, 2023 at 05:31:11PM +0800, wangyanan (Y) wrote:
>> Date: Thu, 16 Feb 2023 17:31:11 +0800
>> From: "wangyanan (Y)" <wangyanan55@huawei.com>
>> Subject: Re: [RFC 12/52] hw/acpi: Replace MachineState.smp access with
>>   topology helpers
>>
>> Hi Zhao,
>>
>> 在 2023/2/13 17:49, Zhao Liu 写道:
>>> From: Zhao Liu <zhao1.liu@intel.com>
>>>
>>> At present, in QEMU only arm needs PPTT table to build cpu topology.
>>>
>>> Before QEMU's arm supports hybrid architectures, it's enough to limit
>>> the cpu topology of PPTT to smp type through the explicit smp interface
>>> (machine_topo_get_smp_threads()).
>>>
>>> Cc: Michael S. Tsirkin <mst@redhat.com>
>>> Cc: Igor Mammedov <imammedo@redhat.com>
>>> Cc: Ani Sinha <ani@anisinha.ca>
>>> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>>> ---
>>>    hw/acpi/aml-build.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>>> index ea331a20d131..693bd8833d10 100644
>>> --- a/hw/acpi/aml-build.c
>>> +++ b/hw/acpi/aml-build.c
>>> @@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms,
>>>                cluster_offset = socket_offset;
>>>            }
>>> -        if (ms->smp.threads == 1) {
>>> +        if (machine_topo_get_smp_threads(ms) == 1) {
>>>                build_processor_hierarchy_node(table_data,
>>>                    (1 << 1) | /* ACPI Processor ID valid */
>>>                    (1 << 3),  /* Node is a Leaf */
>> ACPI PPTT table is designed to also support the hybrid CPU topology
>> case where nodes on the same CPU topology level can have different
>> number of child nodes.
>>
>> So to be general, the diff should be:
>> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c
>> index ea331a20d1..dfded95bbc 100644
>> --- a/hw/acpi/aml-build.c
>> +++ b/hw/acpi/aml-build.c
>> @@ -2044,7 +2044,7 @@ void build_pptt(GArray *table_data, BIOSLinker
>> *linker, MachineState *ms,
>>               cluster_offset = socket_offset;
>>           }
>>
>> -        if (ms->smp.threads == 1) {
>> +        if (machine_topo_get_threads_by_idx(n) == 1) {
>>               build_processor_hierarchy_node(table_data,
>>                   (1 << 1) | /* ACPI Processor ID valid */
>>                   (1 << 3),  /* Node is a Leaf */
> Nice! I'll replace that.
>
>> Actually I'm recently working on ARM hmp virtualization which relys on
>> PPTT for topology representation, so we will also need PPTT to be general
>> for hybrid case anyway.
> Good to know that you are considering hybrid support for arm.
> BTW, I explained the difference between arm and x86's hybrid in previous
> email [1] [2], mainly about whether the cpm model is the same.
>
> I tentatively think that this difference can be solved by arch-specific
> coretype(). Do you have any comments on this? Thanks!
Will look at that. Thanks.
>
> [1]: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03884.html
> [2]: https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg03789.html
>
>> Thanks,
>> Yanan

Re: [RFC 08/52] machine: Add helpers to get cpu topology info from MachineState.topo

[Zhao Liu]

On Fri, Feb 17, 2023 at 03:41:30PM +0800, wangyanan (Y) wrote:
> Date: Fri, 17 Feb 2023 15:41:30 +0800
> From: "wangyanan (Y)" <wangyanan55@huawei.com>
> Subject: Re: [RFC 08/52] machine: Add helpers to get cpu topology info from
>  MachineState.topo
> 
> 在 2023/2/17 11:07, Zhao Liu 写道:
> > On Thu, Feb 16, 2023 at 04:38:38PM +0800, wangyanan (Y) wrote:
> > > Date: Thu, 16 Feb 2023 16:38:38 +0800
> > > From: "wangyanan (Y)" <wangyanan55@huawei.com>
> > > Subject: Re: [RFC 08/52] machine: Add helpers to get cpu topology info from
> > >   MachineState.topo
> > > 
> > > Hi Zhao,
> > > 
> > > 在 2023/2/13 17:49, Zhao Liu 写道:
> > > > From: Zhao Liu <zhao1.liu@intel.com>
> > > > 
> > > > When MachineState.topo is introduced, the topology related structures
> > > > become complicated. In the general case (hybrid or smp topology),
> > > > accessing the topology information needs to determine whether it is
> > > > currently smp or hybrid topology, and then access the corresponding
> > > > MachineState.topo.smp or MachineState.topo.hybrid.
> > > > 
> > > > The best way to do this is to wrap the access to the topology to
> > > > avoid having to check each time it is accessed.
> > > > 
> > > > The following helpers are provided here:
> > > > 
> > > > - General interfaces - no need to worry about whether the underlying
> > > >     topology is smp or hybrid:
> > > > 
> > > > * machine_topo_get_cpus()
> > > > * machine_topo_get_max_cpus()
> > > > * machine_topo_is_smp()
> > > > * machine_topo_get_sockets()
> > > > * machine_topo_get_dies()
> > > > * machine_topo_get_clusters()
> > > > * machine_topo_get_threads();
> > > > * machine_topo_get_cores();
> > > > * machine_topo_get_threads_by_idx()
> > > > * machine_topo_get_cores_by_idx()
> > > > * machine_topo_get_cores_per_socket()
> > > > * machine_topo_get_threads_per_socket()
> > > > 
> > > > - SMP-specific interfaces - provided for the cases that are clearly
> > > > known to be smp topology:
> > > > 
> > > > * machine_topo_get_smp_cores()
> > > > * machine_topo_get_smp_threads()
> > > > 
> > > > Since for hybrid topology, each core may has different threads, if
> > > > someone wants "cpus per core", the cpu_index is need to target a
> > > > specific core (machine_topo_get_threads_by_idx()). But for smp, there is
> > > > no need to be so troublesome, so for this case, we provide smp-specific
> > > > interfaces.
> > > > 
> > > > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> > > > ---
> > > >    hw/core/machine-topo.c | 142 +++++++++++++++++++++++++++++++++++++++++
> > > >    include/hw/boards.h    |  35 ++++++++++
> > > >    2 files changed, 177 insertions(+)
> > > > 
> > > > diff --git a/hw/core/machine-topo.c b/hw/core/machine-topo.c
> > > > index 7223f73f99b0..b20160479629 100644
> > > > --- a/hw/core/machine-topo.c
> > > > +++ b/hw/core/machine-topo.c
> > > > @@ -21,6 +21,148 @@
> > > >    #include "hw/boards.h"
> > > >    #include "qapi/error.h"
> > > > +unsigned int machine_topo_get_sockets(const MachineState *ms)
> > > > +{
> > > > +    return machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > > > +                                     ms->topo.hybrid.sockets;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_dies(const MachineState *ms)
> > > > +{
> > > > +    return machine_topo_is_smp(ms) ? ms->topo.smp.dies :
> > > > +                                     ms->topo.hybrid.dies;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_clusters(const MachineState *ms)
> > > > +{
> > > > +    return machine_topo_is_smp(ms) ? ms->topo.smp.clusters :
> > > > +                                     ms->topo.hybrid.clusters;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_smp_cores(const MachineState *ms)
> > > > +{
> > > > +    g_assert(machine_topo_is_smp(ms));
> > > > +    return ms->topo.smp.cores;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_smp_threads(const MachineState *ms)
> > > > +{
> > > > +    g_assert(machine_topo_is_smp(ms));
> > > > +    return ms->topo.smp.threads;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_threads(const MachineState *ms,
> > > > +                                      unsigned int cluster_id,
> > > > +                                      unsigned int core_id)
> > > > +{
> > > > +    if (machine_topo_is_smp(ms)) {
> > > > +        return ms->topo.smp.threads;
> > > > +    } else {
> > > > +        return ms->topo.hybrid.cluster_list[cluster_id]
> > > > +                   .core_list[core_id].threads;
> > > > +    }
> > > > +
> > > > +    return 0;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_cores(const MachineState *ms,
> > > > +                                    unsigned int cluster_id)
> > > > +{
> > > > +    if (machine_topo_is_smp(ms)) {
> > > > +        return ms->topo.smp.cores;
> > > > +    } else {
> > > > +        return ms->topo.hybrid.cluster_list[cluster_id].cores;
> > > > +    }
> > > > +}
> > > Is it possible to use variadic function so that those two smp specific
> > > helpers can be avoided? It's a bit wired that we have the generic
> > > machine_topo_get_threads but also need machine_topo_get_smp_threads
> > > at the same time.
> > I am not sure about this, because variadic functions unify function
> > naming, but eliminate the "smp-specific" information from the name.
> > 
> > Trying to get the cres/threads without considering the cpu index can
> > only be used in smp scenarios, and I think the caller needs to
> > understand that he knows it's smp.
> Ok, I get the point.
> When it comes to the naming, would it be more concise to remove the
> *_get_* in the fun name, such as machine_topo_get_cpus to
> machine_topo_cpus, machine_topo_get_clusters to machine_topo_clusters.

Good, thanks!

> 
> And maybe rename machine_topo_get_cores(int cluster_id, int core_id) to
> machine_topo_cores_by_ids?
> 
> Or machine_topo_get_cores() to machine_topo_cores_by_topo_ids()
> and machine_topo_get_cores_by_idx to machine_topo_cores_by_cpu_idx()

I like the latter, nice name.

> > > > +
> > > > +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> > > > +                                             unsigned int cpu_index)
> > > > +{
> > > > +    unsigned cpus_per_die;
> > > > +    unsigned tmp_idx;
> > > > +    HybridCluster *cluster;
> > > > +    HybridCore *core;
> > > > +
> > > > +    if (machine_topo_is_smp(ms)) {
> > > > +        return ms->topo.smp.threads;
> > > > +    }
> > > > +
> > > > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > > > +                            ms->topo.hybrid.dies);
> > > > +    tmp_idx = cpu_index % cpus_per_die;
> > > > +
> > > > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > > > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > > > +
> > > > +        for (int j = 0; j < cluster->cores; j++) {
> > > > +            core = &cluster->core_list[j];
> > > > +
> > > > +            if (tmp_idx < core->threads) {
> > > > +                return core->threads;
> > > > +            } else {
> > > > +                tmp_idx -= core->threads;
> > > > +            }
> > > > +        }
> > > > +    }
> > > > +
> > > > +    return 0;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > > > +                                           unsigned int cpu_index)
> > > > +{
> > > > +    unsigned cpus_per_die;
> > > > +    unsigned tmp_idx;
> > > > +    HybridCluster *cluster;
> > > > +    HybridCore *core;
> > > > +
> > > > +    if (machine_topo_is_smp(ms)) {
> > > > +        return ms->topo.smp.cores;
> > > > +    }
> > > > +
> > > > +    cpus_per_die = ms->topo.max_cpus / (ms->topo.hybrid.sockets *
> > > > +                            ms->topo.hybrid.dies);
> > > > +    tmp_idx = cpu_index % cpus_per_die;
> > > > +
> > > > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > > > +        cluster = &ms->topo.hybrid.cluster_list[i];
> > > > +
> > > > +        for (int j = 0; j < cluster->cores; j++) {
> > > > +            core = &cluster->core_list[j];
> > > > +
> > > > +            if (tmp_idx < core->threads) {
> > > > +                return cluster->cores;
> > > > +            } else {
> > > > +                tmp_idx -= core->threads;
> > > > +            }
> > > > +        }
> > > > +    }
> > > > +
> > > > +    return 0;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms)
> > > > +{
> > > > +    unsigned int cores_per_die = 0;
> > > > +
> > > > +    if (machine_topo_is_smp(ms)) {
> > > > +        return ms->topo.smp.cores * ms->topo.smp.clusters * ms->topo.smp.dies;
> > > > +    }
> > > > +
> > > > +    for (int i = 0; i < ms->topo.hybrid.clusters; i++) {
> > > > +        cores_per_die += ms->topo.hybrid.cluster_list[i].cores;
> > > > +    }
> > > > +
> > > > +    return cores_per_die * ms->topo.hybrid.dies;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms)
> > > > +{
> > > > +    unsigned int sockets = machine_topo_is_smp(ms) ? ms->topo.smp.sockets :
> > > > +                           ms->topo.hybrid.sockets;
> > > > +    return ms->topo.max_cpus / sockets;
> > > > +}
> > > > +
> > > >    /*
> > > >     * Report information of a machine's supported CPU topology hierarchy.
> > > >     * Topology members will be ordered from the largest to the smallest
> > > > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > > > index 0a61855499e3..34b64b012022 100644
> > > > --- a/include/hw/boards.h
> > > > +++ b/include/hw/boards.h
> > > > @@ -461,4 +461,39 @@ extern const size_t hw_compat_2_2_len;
> > > >    extern GlobalProperty hw_compat_2_1[];
> > > >    extern const size_t hw_compat_2_1_len;
> > > > +static inline
> > > > +unsigned int machine_topo_get_cpus(const MachineState *ms)
> > > > +{
> > > > +    return ms->topo.cpus;
> > > > +}
> > > > +
> > > > +static inline
> > > > +unsigned int machine_topo_get_max_cpus(const MachineState *ms)
> > > > +{
> > > > +    return ms->topo.max_cpus;
> > > > +}
> > > > +
> > > > +static inline
> > > > +bool machine_topo_is_smp(const MachineState *ms)
> > > > +{
> > > > +    return ms->topo.topo_type == CPU_TOPO_TYPE_SMP;
> > > > +}
> > > > +
> > > > +unsigned int machine_topo_get_sockets(const MachineState *ms);
> > > > +unsigned int machine_topo_get_dies(const MachineState *ms);
> > > > +unsigned int machine_topo_get_clusters(const MachineState *ms);
> > > > +unsigned int machine_topo_get_smp_cores(const MachineState *ms);
> > > > +unsigned int machine_topo_get_smp_threads(const MachineState *ms);
> > > > +unsigned int machine_topo_get_threads(const MachineState *ms,
> > > > +                                      unsigned int cluster_id,
> > > > +                                      unsigned int core_id);
> > > > +unsigned int machine_topo_get_cores(const MachineState *ms,
> > > > +                                    unsigned int cluster_id);
> > > > +unsigned int machine_topo_get_threads_by_idx(const MachineState *ms,
> > > > +                                             unsigned int cpu_index);
> > > > +unsigned int machine_topo_get_cores_by_idx(const MachineState *ms,
> > > > +                                           unsigned int cpu_index);
> > > > +unsigned int machine_topo_get_cores_per_socket(const MachineState *ms);
> > > > +unsigned int machine_topo_get_threads_per_socket(const MachineState *ms);
> > > > +
> > > >    #endif
> > > I think it's necessary to document the ablity for each helper.
> > > For example, at a flance, I cant figure out what
> > > machine_topo_get_threads_idx
> > > does. Add some something like:
> > > /*
> > >   * Get number of threads within the CPU core where a processor locates,
> > >   * according to the processor index.
> > >   *
> > >   * @param: ...
> > >   */
> > > will be friendly to future users.
> > Yeah, thanks! I will.
> > 
> > > Thanks,
> > > Yanan
> 

Re: [RFC 22/52] riscv: Replace MachineState.smp access with topology helpers

[Palmer Dabbelt]

On Tue, 14 Feb 2023 18:57:35 PST (-0800), zhao1.liu@linux.intel.com wrote:
> On Tue, Feb 14, 2023 at 10:17:45AM +0800, Mi, Dapeng1 wrote:
>> Date: Tue, 14 Feb 2023 10:17:45 +0800
>> From: "Mi, Dapeng1" <dapeng1.mi@intel.com>
>> Subject: RE: [RFC 22/52] riscv: Replace MachineState.smp access with
>>  topology helpers
>>
>> > From: Zhao Liu <zhao1.liu@linux.intel.com>
>> > Sent: Monday, February 13, 2023 5:50 PM
>> > To: Eduardo Habkost <eduardo@habkost.net>; Marcel Apfelbaum
>> > <marcel.apfelbaum@gmail.com>; Philippe Mathieu-Daud? <philmd@linaro.org>;
>> > Yanan Wang <wangyanan55@huawei.com>; Michael S . Tsirkin
>> > <mst@redhat.com>; Richard Henderson <richard.henderson@linaro.org>; Paolo
>> > Bonzini <pbonzini@redhat.com>; Eric Blake <eblake@redhat.com>; Markus
>> > Armbruster <armbru@redhat.com>
>> > Cc: qemu-devel@nongnu.org; Wang, Zhenyu Z <zhenyu.z.wang@intel.com>; Mi,
>> > Dapeng1 <dapeng1.mi@intel.com>; Ding, Zhuocheng
>> > <zhuocheng.ding@intel.com>; Robert Hoo <robert.hu@linux.intel.com>;
>> > Christopherson,, Sean <seanjc@google.com>; Like Xu
>> > <like.xu.linux@gmail.com>; Liu, Zhao1 <zhao1.liu@intel.com>; Meng, Bin
>> > <bin.meng@windriver.com>; Palmer Dabbelt <palmer@dabbelt.com>; Alistair
>> > Francis <alistair.francis@wdc.com>; Vijai Kumar K <vijai@behindbytes.com>
>> > Subject: [RFC 22/52] riscv: Replace MachineState.smp access with topology
>> > helpers
>> >
>> > From: Zhao Liu <zhao1.liu@intel.com>
>> >
>> > When MachineState.topo is introduced, the topology related structures
>> > become complicated. So we wrapped the access to topology fields of
>> > MachineState.topo into some helpers, and we are using these helpers
>> > to replace the use of MachineState.smp.
>> >
>> > In the codes of riscv, it's straightforward to replace topology access
>> > with wrapped generic interfaces.
>> >
>> > Cc: Bin Meng <bin.meng@windriver.com>
>> > Cc: Palmer Dabbelt <palmer@dabbelt.com>
>> > Cc: Alistair Francis <alistair.francis@wdc.com>
>> > CC: Vijai Kumar K <vijai@behindbytes.com>
>> > Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
>> > ---
>> >  hw/riscv/microchip_pfsoc.c | 11 ++++++-----
>> >  hw/riscv/numa.c            | 21 +++++++++++----------
>> >  hw/riscv/opentitan.c       |  8 ++++----
>> >  hw/riscv/shakti_c.c        |  2 +-
>> >  hw/riscv/sifive_e.c        | 10 ++++++----
>> >  hw/riscv/sifive_u.c        | 28 ++++++++++++++--------------
>> >  hw/riscv/virt.c            | 24 +++++++++++++-----------
>> >  7 files changed, 55 insertions(+), 49 deletions(-)
>> >
>> > diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c
>> > index 2b91e49561f1..30295cce17e7 100644
>> > --- a/hw/riscv/microchip_pfsoc.c
>> > +++ b/hw/riscv/microchip_pfsoc.c
>> > @@ -164,7 +164,8 @@ static void microchip_pfsoc_soc_instance_init(Object
>> > *obj)
>> >
>> >      object_initialize_child(OBJECT(&s->u_cluster), "u-cpus", &s->u_cpus,
>> >                              TYPE_RISCV_HART_ARRAY);
>> > -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
>> > +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts",
>> > +                         machine_topo_get_cpus(ms) - 1);
>> >      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
>> >      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type",
>> >                           TYPE_RISCV_CPU_SIFIVE_U54);
>> > @@ -249,10 +250,10 @@ static void microchip_pfsoc_soc_realize(DeviceState
>> > *dev, Error **errp)
>> >
>> >      /* CLINT */
>> >      riscv_aclint_swi_create(memmap[MICROCHIP_PFSOC_CLINT].base,
>> > -        0, ms->smp.cpus, false);
>> > +        0, machine_topo_get_cpus(ms), false);
>> >      riscv_aclint_mtimer_create(
>> >          memmap[MICROCHIP_PFSOC_CLINT].base + RISCV_ACLINT_SWI_SIZE,
>> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
>> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, machine_topo_get_cpus(ms),
>> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>> >          CLINT_TIMEBASE_FREQ, false);
>> >
>> > @@ -276,11 +277,11 @@ static void microchip_pfsoc_soc_realize(DeviceState
>> > *dev, Error **errp)
>> >                                  l2lim_mem);
>> >
>> >      /* create PLIC hart topology configuration string */
>> > -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
>> > +    plic_hart_config =
>> > riscv_plic_hart_config_string(machine_topo_get_cpus(ms));
>> >
>> >      /* PLIC */
>> >      s->plic = sifive_plic_create(memmap[MICROCHIP_PFSOC_PLIC].base,
>> > -        plic_hart_config, ms->smp.cpus, 0,
>> > +        plic_hart_config, machine_topo_get_cpus(ms), 0,
>> >          MICROCHIP_PFSOC_PLIC_NUM_SOURCES,
>> >          MICROCHIP_PFSOC_PLIC_NUM_PRIORITIES,
>> >          MICROCHIP_PFSOC_PLIC_PRIORITY_BASE,
>> > diff --git a/hw/riscv/numa.c b/hw/riscv/numa.c
>> > index 472010256183..1fabdc42e767 100644
>> > --- a/hw/riscv/numa.c
>> > +++ b/hw/riscv/numa.c
>> > @@ -37,13 +37,14 @@ int riscv_socket_count(const MachineState *ms)
>> >
>> >  int riscv_socket_first_hartid(const MachineState *ms, int socket_id)
>> >  {
>> > -    int i, first_hartid = ms->smp.cpus;
>> > +    int i, first_hartid, cpus = machine_topo_get_cpus(ms);
>> >
>> > +    first_hartid = cpus;
>> >      if (!numa_enabled(ms)) {
>> >          return (!socket_id) ? 0 : -1;
>> >      }
>> >
>> > -    for (i = 0; i < ms->smp.cpus; i++) {
>> > +    for (i = 0; i < cpus; i++) {
>> >          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
>> >              continue;
>> >          }
>> > @@ -52,18 +53,18 @@ int riscv_socket_first_hartid(const MachineState *ms,
>> > int socket_id)
>> >          }
>> >      }
>> >
>> > -    return (first_hartid < ms->smp.cpus) ? first_hartid : -1;
>> > +    return (first_hartid < cpus) ? first_hartid : -1;
>> >  }
>> >
>> >  int riscv_socket_last_hartid(const MachineState *ms, int socket_id)
>> >  {
>> > -    int i, last_hartid = -1;
>> > +    int i, last_hartid = -1, cpus = machine_topo_get_cpus(ms);
>> >
>> >      if (!numa_enabled(ms)) {
>> > -        return (!socket_id) ? ms->smp.cpus - 1 : -1;
>> > +        return (!socket_id) ? cpus - 1 : -1;
>> >      }
>> >
>> > -    for (i = 0; i < ms->smp.cpus; i++) {
>> > +    for (i = 0; i < cpus; i++) {
>> >          if (ms->possible_cpus->cpus[i].props.node_id != socket_id) {
>> >              continue;
>> >          }
>> > @@ -72,7 +73,7 @@ int riscv_socket_last_hartid(const MachineState *ms, int
>> > socket_id)
>> >          }
>> >      }
>> >
>> > -    return (last_hartid < ms->smp.cpus) ? last_hartid : -1;
>> > +    return (last_hartid < cpus) ? last_hartid : -1;
>> >  }
>> >
>> >  int riscv_socket_hart_count(const MachineState *ms, int socket_id)
>> > @@ -80,7 +81,7 @@ int riscv_socket_hart_count(const MachineState *ms, int
>> > socket_id)
>> >      int first_hartid, last_hartid;
>> >
>> >      if (!numa_enabled(ms)) {
>> > -        return (!socket_id) ? ms->smp.cpus : -1;
>> > +        return (!socket_id) ? machine_topo_get_cpus(ms) : -1;
>> >      }
>> >
>> >      first_hartid = riscv_socket_first_hartid(ms, socket_id);
>> > @@ -208,7 +209,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
>> > MachineState *ms, int idx)
>> >      int64_t nidx = 0;
>> >
>> >      if (ms->numa_state->num_nodes) {
>> > -        nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes);
>> > +        nidx = idx / (machine_topo_get_cpus(ms) / ms->numa_state->num_nodes);
>> >          if (ms->numa_state->num_nodes <= nidx) {
>> >              nidx = ms->numa_state->num_nodes - 1;
>> >          }
>> > @@ -220,7 +221,7 @@ int64_t riscv_numa_get_default_cpu_node_id(const
>> > MachineState *ms, int idx)
>> >  const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms)
>> >  {
>> >      int n;
>> > -    unsigned int max_cpus = ms->smp.max_cpus;
>> > +    unsigned int max_cpus = machine_topo_get_max_cpus(ms);
>> >
>> >      if (ms->possible_cpus) {
>> >          assert(ms->possible_cpus->len == max_cpus);
>> > diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c
>> > index 353f030d80c1..fb59c43dcc08 100644
>> > --- a/hw/riscv/opentitan.c
>> > +++ b/hw/riscv/opentitan.c
>> > @@ -143,11 +143,11 @@ static void lowrisc_ibex_soc_realize(DeviceState
>> > *dev_soc, Error **errp)
>> >      MachineState *ms = MACHINE(qdev_get_machine());
>> >      LowRISCIbexSoCState *s = RISCV_IBEX_SOC(dev_soc);
>> >      MemoryRegion *sys_mem = get_system_memory();
>> > -    int i;
>> > +    int i, cpus = machine_topo_get_cpus(ms);
>> >
>> >      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
>> >                              &error_abort);
>> > -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
>> > +    object_property_set_int(OBJECT(&s->cpus), "num-harts", cpus,
>> >                              &error_abort);
>> >      object_property_set_int(OBJECT(&s->cpus), "resetvec", s->resetvec,
>> >                              &error_abort);
>> > @@ -186,10 +186,10 @@ static void lowrisc_ibex_soc_realize(DeviceState
>> > *dev_soc, Error **errp)
>> >      }
>> >      sysbus_mmio_map(SYS_BUS_DEVICE(&s->plic), 0,
>> > memmap[IBEX_DEV_PLIC].base);
>> >
>> > -    for (i = 0; i < ms->smp.cpus; i++) {
>> > +    for (i = 0; i < cpus; i++) {
>> >          CPUState *cpu = qemu_get_cpu(i);
>> >
>> > -        qdev_connect_gpio_out(DEVICE(&s->plic), ms->smp.cpus + i,
>> > +        qdev_connect_gpio_out(DEVICE(&s->plic), cpus + i,
>> >                                qdev_get_gpio_in(DEVICE(cpu), IRQ_M_EXT));
>> >      }
>> >
>> > diff --git a/hw/riscv/shakti_c.c b/hw/riscv/shakti_c.c
>> > index e43cc9445cb1..34499510e2f0 100644
>> > --- a/hw/riscv/shakti_c.c
>> > +++ b/hw/riscv/shakti_c.c
>> > @@ -110,7 +110,7 @@ static void shakti_c_soc_state_realize(DeviceState *dev,
>> > Error **errp)
>> >      sysbus_realize(SYS_BUS_DEVICE(&sss->cpus), &error_abort);
>> >
>> >      sss->plic = sifive_plic_create(shakti_c_memmap[SHAKTI_C_PLIC].base,
>> > -        (char *)SHAKTI_C_PLIC_HART_CONFIG, ms->smp.cpus, 0,
>> > +        (char *)SHAKTI_C_PLIC_HART_CONFIG, machine_topo_get_cpus(ms), 0,
>> >          SHAKTI_C_PLIC_NUM_SOURCES,
>> >          SHAKTI_C_PLIC_NUM_PRIORITIES,
>> >          SHAKTI_C_PLIC_PRIORITY_BASE,
>> > diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c
>> > index 3e3f4b0088ca..ca3a7a0877cd 100644
>> > --- a/hw/riscv/sifive_e.c
>> > +++ b/hw/riscv/sifive_e.c
>> > @@ -178,7 +178,8 @@ static void sifive_e_soc_init(Object *obj)
>> >      SiFiveESoCState *s = RISCV_E_SOC(obj);
>> >
>> >      object_initialize_child(obj, "cpus", &s->cpus, TYPE_RISCV_HART_ARRAY);
>> > -    object_property_set_int(OBJECT(&s->cpus), "num-harts", ms->smp.cpus,
>> > +    object_property_set_int(OBJECT(&s->cpus), "num-harts",
>> > +                            machine_topo_get_cpus(ms),
>> >                              &error_abort);
>>
>> With this change, we don't need to start a new line for '&error_abort);', just move
>> it to previous line.
>
> Yes, this is no more than 80 characters.

With that fixed

Reviewed-by: Palmer Dabbelt <palmer@rivosinc.com>
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>

Thanks!

>> >      object_property_set_int(OBJECT(&s->cpus), "resetvec", 0x1004,
>> > &error_abort);
>> >      object_initialize_child(obj, "riscv.sifive.e.gpio0", &s->gpio,
>> > @@ -191,6 +192,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
>> > **errp)
>> >      const MemMapEntry *memmap = sifive_e_memmap;
>> >      SiFiveESoCState *s = RISCV_E_SOC(dev);
>> >      MemoryRegion *sys_mem = get_system_memory();
>> > +    unsigned int cpus = machine_topo_get_cpus(ms);
>> >
>> >      object_property_set_str(OBJECT(&s->cpus), "cpu-type", ms->cpu_type,
>> >                              &error_abort);
>> > @@ -204,7 +206,7 @@ static void sifive_e_soc_realize(DeviceState *dev, Error
>> > **errp)
>> >
>> >      /* MMIO */
>> >      s->plic = sifive_plic_create(memmap[SIFIVE_E_DEV_PLIC].base,
>> > -        (char *)SIFIVE_E_PLIC_HART_CONFIG, ms->smp.cpus, 0,
>> > +        (char *)SIFIVE_E_PLIC_HART_CONFIG, cpus, 0,
>> >          SIFIVE_E_PLIC_NUM_SOURCES,
>> >          SIFIVE_E_PLIC_NUM_PRIORITIES,
>> >          SIFIVE_E_PLIC_PRIORITY_BASE,
>> > @@ -215,10 +217,10 @@ static void sifive_e_soc_realize(DeviceState *dev,
>> > Error **errp)
>> >          SIFIVE_E_PLIC_CONTEXT_STRIDE,
>> >          memmap[SIFIVE_E_DEV_PLIC].size);
>> >      riscv_aclint_swi_create(memmap[SIFIVE_E_DEV_CLINT].base,
>> > -        0, ms->smp.cpus, false);
>> > +        0, cpus, false);
>> >      riscv_aclint_mtimer_create(memmap[SIFIVE_E_DEV_CLINT].base +
>> >              RISCV_ACLINT_SWI_SIZE,
>> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
>> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
>> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>> >          RISCV_ACLINT_DEFAULT_TIMEBASE_FREQ, false);
>> >      create_unimplemented_device("riscv.sifive.e.aon",
>> > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c
>> > index d3ab7a9cda5d..c7f969510209 100644
>> > --- a/hw/riscv/sifive_u.c
>> > +++ b/hw/riscv/sifive_u.c
>> > @@ -99,7 +99,7 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >      MachineState *ms = MACHINE(s);
>> >      uint64_t mem_size = ms->ram_size;
>> >      void *fdt;
>> > -    int cpu, fdt_size;
>> > +    int cpu, fdt_size, num_cpus = machine_topo_get_cpus(ms);
>> >      uint32_t *cells;
>> >      char *nodename;
>> >      uint32_t plic_phandle, prci_phandle, gpio_phandle, phandle = 1;
>> > @@ -175,7 +175,7 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0);
>> >      qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1);
>> >
>> > -    for (cpu = ms->smp.cpus - 1; cpu >= 0; cpu--) {
>> > +    for (cpu = num_cpus - 1; cpu >= 0; cpu--) {
>> >          int cpu_phandle = phandle++;
>> >          nodename = g_strdup_printf("/cpus/cpu@%d", cpu);
>> >          char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
>> > @@ -207,8 +207,8 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >          g_free(nodename);
>> >      }
>> >
>> > -    cells =  g_new0(uint32_t, ms->smp.cpus * 4);
>> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
>> > +    cells =  g_new0(uint32_t, num_cpus * 4);
>> > +    for (cpu = 0; cpu < num_cpus; cpu++) {
>> >          nodename =
>> >              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
>> >          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
>> > @@ -227,7 +227,7 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >          0x0, memmap[SIFIVE_U_DEV_CLINT].base,
>> >          0x0, memmap[SIFIVE_U_DEV_CLINT].size);
>> >      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
>> > -        cells, ms->smp.cpus * sizeof(uint32_t) * 4);
>> > +        cells, num_cpus * sizeof(uint32_t) * 4);
>> >      g_free(cells);
>> >      g_free(nodename);
>> >
>> > @@ -258,8 +258,8 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >      g_free(nodename);
>> >
>> >      plic_phandle = phandle++;
>> > -    cells =  g_new0(uint32_t, ms->smp.cpus * 4 - 2);
>> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
>> > +    cells =  g_new0(uint32_t, machine_topo_get_cpus(ms) * 4 - 2);
>> > +    for (cpu = 0; cpu < machine_topo_get_cpus(ms); cpu++) {
>> >          nodename =
>> >              g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu);
>> >          uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename);
>> > @@ -283,7 +283,7 @@ static void create_fdt(SiFiveUState *s, const
>> > MemMapEntry *memmap,
>> >          (char **)&plic_compat, ARRAY_SIZE(plic_compat));
>> >      qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0);
>> >      qemu_fdt_setprop(fdt, nodename, "interrupts-extended",
>> > -        cells, (ms->smp.cpus * 4 - 2) * sizeof(uint32_t));
>> > +        cells, (machine_topo_get_cpus(ms) * 4 - 2) * sizeof(uint32_t));
>> >      qemu_fdt_setprop_cells(fdt, nodename, "reg",
>> >          0x0, memmap[SIFIVE_U_DEV_PLIC].base,
>> >          0x0, memmap[SIFIVE_U_DEV_PLIC].size);
>> > @@ -797,10 +797,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
>> > Error **errp)
>> >      MemoryRegion *mask_rom = g_new(MemoryRegion, 1);
>> >      MemoryRegion *l2lim_mem = g_new(MemoryRegion, 1);
>> >      char *plic_hart_config;
>> > -    int i, j;
>> > +    int i, j, cpus = machine_topo_get_cpus(ms);
>> >      NICInfo *nd = &nd_table[0];
>> >
>> > -    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", ms->smp.cpus - 1);
>> > +    qdev_prop_set_uint32(DEVICE(&s->u_cpus), "num-harts", cpus - 1);
>> >      qdev_prop_set_uint32(DEVICE(&s->u_cpus), "hartid-base", 1);
>> >      qdev_prop_set_string(DEVICE(&s->u_cpus), "cpu-type", s->cpu_type);
>> >      qdev_prop_set_uint64(DEVICE(&s->u_cpus), "resetvec", 0x1004);
>> > @@ -837,11 +837,11 @@ static void sifive_u_soc_realize(DeviceState *dev,
>> > Error **errp)
>> >                                  l2lim_mem);
>> >
>> >      /* create PLIC hart topology configuration string */
>> > -    plic_hart_config = riscv_plic_hart_config_string(ms->smp.cpus);
>> > +    plic_hart_config = riscv_plic_hart_config_string(cpus);
>> >
>> >      /* MMIO */
>> >      s->plic = sifive_plic_create(memmap[SIFIVE_U_DEV_PLIC].base,
>> > -        plic_hart_config, ms->smp.cpus, 0,
>> > +        plic_hart_config, cpus, 0,
>> >          SIFIVE_U_PLIC_NUM_SOURCES,
>> >          SIFIVE_U_PLIC_NUM_PRIORITIES,
>> >          SIFIVE_U_PLIC_PRIORITY_BASE,
>> > @@ -857,10 +857,10 @@ static void sifive_u_soc_realize(DeviceState *dev,
>> > Error **errp)
>> >      sifive_uart_create(system_memory, memmap[SIFIVE_U_DEV_UART1].base,
>> >          serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ));
>> >      riscv_aclint_swi_create(memmap[SIFIVE_U_DEV_CLINT].base, 0,
>> > -        ms->smp.cpus, false);
>> > +        cpus, false);
>> >      riscv_aclint_mtimer_create(memmap[SIFIVE_U_DEV_CLINT].base +
>> >              RISCV_ACLINT_SWI_SIZE,
>> > -        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, ms->smp.cpus,
>> > +        RISCV_ACLINT_DEFAULT_MTIMER_SIZE, 0, cpus,
>> >          RISCV_ACLINT_DEFAULT_MTIMECMP, RISCV_ACLINT_DEFAULT_MTIME,
>> >          CLINT_TIMEBASE_FREQ, false);
>> >
>> > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
>> > index b81081c70b1c..4b0a3dab5ec1 100644
>> > --- a/hw/riscv/virt.c
>> > +++ b/hw/riscv/virt.c
>> > @@ -506,16 +506,17 @@ static void create_fdt_imsic(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      char *imsic_name;
>> >      MachineState *ms = MACHINE(s);
>> >      int socket_count = riscv_socket_count(ms);
>> > -    uint32_t imsic_max_hart_per_socket, imsic_guest_bits;
>> > +    uint32_t imsic_max_hart_per_socket, imsic_guest_bits, cpus;
>> >      uint32_t *imsic_cells, *imsic_regs, imsic_addr, imsic_size;
>> >
>> > +    cpus = machine_topo_get_cpus(ms);
>> >      *msi_m_phandle = (*phandle)++;
>> >      *msi_s_phandle = (*phandle)++;
>> > -    imsic_cells = g_new0(uint32_t, ms->smp.cpus * 2);
>> > +    imsic_cells = g_new0(uint32_t, cpus * 2);
>> >      imsic_regs = g_new0(uint32_t, socket_count * 4);
>> >
>> >      /* M-level IMSIC node */
>> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
>> > +    for (cpu = 0; cpu < cpus; cpu++) {
>> >          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
>> >          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_M_EXT);
>> >      }
>> > @@ -544,7 +545,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
>> >          NULL, 0);
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
>> > -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
>> > +        imsic_cells, cpus * sizeof(uint32_t) * 2);
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
>> >          socket_count * sizeof(uint32_t) * 4);
>> >      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
>> > @@ -562,7 +563,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      g_free(imsic_name);
>> >
>> >      /* S-level IMSIC node */
>> > -    for (cpu = 0; cpu < ms->smp.cpus; cpu++) {
>> > +    for (cpu = 0; cpu < cpus; cpu++) {
>> >          imsic_cells[cpu * 2 + 0] = cpu_to_be32(intc_phandles[cpu]);
>> >          imsic_cells[cpu * 2 + 1] = cpu_to_be32(IRQ_S_EXT);
>> >      }
>> > @@ -593,7 +594,7 @@ static void create_fdt_imsic(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "msi-controller",
>> >          NULL, 0);
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "interrupts-extended",
>> > -        imsic_cells, ms->smp.cpus * sizeof(uint32_t) * 2);
>> > +        imsic_cells, cpus * sizeof(uint32_t) * 2);
>> >      qemu_fdt_setprop(ms->fdt, imsic_name, "reg", imsic_regs,
>> >          socket_count * sizeof(uint32_t) * 4);
>> >      qemu_fdt_setprop_cell(ms->fdt, imsic_name, "riscv,num-ids",
>> > @@ -732,7 +733,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      char *clust_name;
>> >      int socket, phandle_pos;
>> >      MachineState *ms = MACHINE(s);
>> > -    uint32_t msi_m_phandle = 0, msi_s_phandle = 0;
>> > +    uint32_t msi_m_phandle = 0, msi_s_phandle = 0, cpus;
>> >      uint32_t *intc_phandles, xplic_phandles[MAX_NODES];
>> >      int socket_count = riscv_socket_count(ms);
>> >
>> > @@ -743,9 +744,10 @@ static void create_fdt_sockets(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >      qemu_fdt_setprop_cell(ms->fdt, "/cpus", "#address-cells", 0x1);
>> >      qemu_fdt_add_subnode(ms->fdt, "/cpus/cpu-map");
>> >
>> > -    intc_phandles = g_new0(uint32_t, ms->smp.cpus);
>> > +    cpus = machine_topo_get_cpus(ms);
>> > +    intc_phandles = g_new0(uint32_t, cpus);
>> >
>> > -    phandle_pos = ms->smp.cpus;
>> > +    phandle_pos = cpus;
>> >      for (socket = (socket_count - 1); socket >= 0; socket--) {
>> >          phandle_pos -= s->soc[socket].num_harts;
>> >
>> > @@ -776,7 +778,7 @@ static void create_fdt_sockets(RISCVVirtState *s, const
>> > MemMapEntry *memmap,
>> >          *msi_pcie_phandle = msi_s_phandle;
>> >      }
>> >
>> > -    phandle_pos = ms->smp.cpus;
>> > +    phandle_pos = cpus;
>> >      for (socket = (socket_count - 1); socket >= 0; socket--) {
>> >          phandle_pos -= s->soc[socket].num_harts;
>> >
>> > @@ -1113,7 +1115,7 @@ static FWCfgState *create_fw_cfg(const
>> > MachineState *ms)
>> >
>> >      fw_cfg = fw_cfg_init_mem_wide(base + 8, base, 8, base + 16,
>> >                                    &address_space_memory);
>> > -    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS, (uint16_t)ms->smp.cpus);
>> > +    fw_cfg_add_i16(fw_cfg, FW_CFG_NB_CPUS,
>> > (uint16_t)machine_topo_get_cpus(ms));
>> >
>> >      return fw_cfg;
>> >  }
>> > --
>> > 2.34.1
>>

Re: [RFC 00/52] Introduce hybrid CPU topology

[Zhao Liu]

Hi Daniel and folks,

Let me pick up this thread again to discuss some updates from my work
on QOM CPU topology. I would like to know if I'm on the right track. :-)

On Mon, Feb 13, 2023 at 01:38:28PM +0000, Daniel P. Berrangé wrote:
>

[snip]

> 
> IIUC the functionality offered by -hybrid should be a superset
> of the -smp functionality. IOW, -smp ought to be possible to
> re-implement -smp as an alias for -hybrid, such that internally
> code only ever has to deal with the modern approach. Having to
> keep support for both -smp and -hybrid throughout the code is
> undesirable IMHO. Keeping the compat at the CLI parsing level
> limits the burden.
> 
> 
> As a more general thought, rather than introducing a new top level
> command line argument -hybrid, I'm thinking we should possibly just
> define this all using QOM and thus re-use the existing -object
> argument. 
> 
> I'm also finding the above example command lines quite difficult
> to understand, as there is alot of implicit linkage and expansion
> between the different levels. With devices we're much more
> explicitly with the parent/child relationships, and have to
> express everything with no automatic expansion, linking it all
> together via the id=/bus= properties.  This is quite a bit more
> verbose, but it is also very effective at letting us express
> arbitrarily complex relationships.
> 
> I think it would be worth exploring that approach for the CPU
> topology expression too.
> 
> If we followed the more explicit device approach to modelling
> then instead of:
> 
>  -cpu core,...
>  -cpu atom,...
>  -hybrid socket,sockets=1
>  -hybrid die,dies=1
>  -hybrid cluster,clusters=4
>  -hybrid core,cores=1,coretype="core",threads=2,clusterid=0-2
>  -hybrid core,cores=4,coretype="atom",threads=1
> 
> we would end up with something like
> 
>   -object cpu-socket,id=sock0
>   -object cpu-die,id=die0,parent=sock0
>   -object cpu-cluster,id=cluster0,parent=die0
>   -object cpu-cluster,id=cluster1,parent=die0
>   -object cpu-cluster,id=cluster2,parent=die0
>   -object cpu-cluster,id=cluster3,parent=die0
>   -object x86-cpu-model-atom,id=cpu0,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu1,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu2,parent=cluster0
>   -object x86-cpu-model-atom,id=cpu3,parent=cluster0
>   -object x86-cpu-model-core,id=cpu4,parent=cluster0,threads=2
>   -object x86-cpu-model-atom,id=cpu5,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu6,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu7,parent=cluster1
>   -object x86-cpu-model-atom,id=cpu8,parent=cluster1
>   -object x86-cpu-model-core,id=cpu9,parent=cluster1,threads=2
>   -object x86-cpu-model-atom,id=cpu10,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu11,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu12,parent=cluster2
>   -object x86-cpu-model-atom,id=cpu13,parent=cluster2
>   -object x86-cpu-model-core,id=cpu14,parent=cluster2,threads=2
>   -object x86-cpu-model-atom,id=cpu15,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu16,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu17,parent=cluster3
>   -object x86-cpu-model-atom,id=cpu18,parent=cluster3
>   -object x86-cpu-model-core,id=cpu19,parent=cluster3,threads=2
> 
> The really obvious downside is that it is much more verbose.

I find the "core" and "cluster" already have the abstraction as the
device, and Andreas also tried to abstract "socket" as the device.

So I find in this QOM way, all CPU topology levels should be
abstracted to devices and created via "-device". And I also need to
extend "cluster" device to support VM case as a general CPU topology
abstraction, not only just used in TCG emulation.

I've done a preliminary POC so far, where I can create the CPU topology
via "-device", and also added the ability to create the "child"
relationship in "-device" (since neither the "link" relationship nor
the bus looks like a good representation of the CPU hierarchy).

There's an example:

-device cpu-socket,id=sock0 \
-device cpu-die,id=die0,parent=sock0 \
-device cpu-die,id=die1,parent=sock0 \
-device cpu-cluster,id=cluster0,parent=die0 \
-device cpu-cluster,id=cluster1,parent=die0 \
-device cpu-cluster,id=cluster2,parent=die1 \
-device x86-intel-core,id=core0,parent=cluster0,threads=3 \
-device x86-intel-atom,id=core1,parent=cluster0,threads=2 \
-device x86-core,id=core2,parent=cluster1,threads=1 \
-device x86-intel-core,id=core3,parent=cluster2,threads=5 \

with the above format, I could build the the more accurate canonical
path for CPUs like this:

(QEMU) query-hotpluggable-cpus
{
    "arguments": {},
    "execute": "query-hotpluggable-cpus"
}
{
    "return": [
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 1,
                "socket-id": 0,
                "thread-id": 4
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die1/cluster2/core3/host-x86_64-cpu[10]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 1,
                "socket-id": 0,
                "thread-id": 3
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die1/cluster2/core3/host-x86_64-cpu[9]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 1,
                "socket-id": 0,
                "thread-id": 2
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die1/cluster2/core3/host-x86_64-cpu[8]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 1,
                "socket-id": 0,
                "thread-id": 1
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die1/cluster2/core3/host-x86_64-cpu[7]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 1,
                "socket-id": 0,
                "thread-id": 0
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die1/cluster2/core3/host-x86_64-cpu[6]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 1,
                "core-id": 0,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 0
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster1/core2/host-x86_64-cpu[5]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 1,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 1
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster0/core1/host-x86_64-cpu[4]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 1,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 0
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster0/core1/host-x86_64-cpu[3]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 2
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster0/core0/host-x86_64-cpu[2]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 1
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster0/core0/host-x86_64-cpu[1]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        },
        {
            "props": {
                "cluster-id": 0,
                "core-id": 0,
                "die-id": 0,
                "socket-id": 0,
                "thread-id": 0
            },
            "qom-path": "/machine/peripheral/cpu-slot/sock0/die0/cluster0/core0/host-x86_64-cpu[0]",
            "type": "host-x86_64-cpu",
            "vcpus-count": 1
        }
    ]
}

Of course, I'm still a bit far from the full QOM topology support, but
would like to hear comments at the POC stage as well! :-)

Thanks and BR,
Zhao

> 
> This example only has 20 CPUs. For a VM with say 1000 CPUs
> this will be very big, but that doesn't neccesarily make it
> wrong.
> 
> On the flipside
> 
>  * It is really clear exactly how many CPUs I've added
> 
>  * The relationship between the topology levels is clear
> 
>  * Every CPU has a unique ID given that can be used in
>    later QMP commands
> 
>  * Whether or not 'threads' are permitted is now a property
>    of the specific CPU model implementation, not the global
>    config. IOW we can express that some CPU models allowing
>    for threads, and some don't.
> 
>  * The -cpu arg is also obsoleted, replaced by the
>    -object x86-cpu-model-core. This might facilitate the
>    modelling of machines with CPUs from different architectures.
> 
> 
> We could potentially compress the leaf node level by expressing
> how many instances of an object we want. it we want. ie, define
> a more convenient shorthand syntax to creating many instances of
> an object. so eg
> 
>   -object-set $TYPE,$PROPS,idbase=foo,count=4
> 
> would be functionally identical to
> 
>   -object $TYPE,$PROPS,id=foo.0
>   -object $TYPE,$PROPS,id=foo.1
>   -object $TYPE,$PROPS,id=foo.2
>   -object $TYPE,$PROPS,id=foo.3
> 
> QEMU just expands it and creates all the objects internally.
> 
> So the huge example I have above for 20 cpus would become much
> shorter: e.g.
> 
>   -object cpu-socket,id=sock0
>   -object cpu-die,id=die0,parent=sock0
>   -object cpu-cluster,id=cluster0,parent=die0
>   -object cpu-cluster,id=cluster1,parent=die0
>   -object cpu-cluster,id=cluster2,parent=die0
>   -object cpu-cluster,id=cluster3,parent=die0
>   -object-set x86-cpu-core-atom,idbase=cpu0,parent=cluster0,count=4
>   -object-set x86-cpu-core-core,id=cpu1,parent=cluster0,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu2,parent=cluster1,count=4
>   -object-set x86-cpu-core-core,id=cpu3,parent=cluster1,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu4,parent=cluster2,count=4
>   -object-set x86-cpu-core-core,id=cpu5,parent=cluster2,threads=2,count=1
>   -object-set x86-cpu-core-atom,idbase=cpu6,parent=cluster3,count=4
>   -object-set x86-cpu-core-core,id=cpu7,parent=cluster3,threads=2,count=1
> 
> IOW, the size of the CLI config only depends on the number of elements
> in the hierarchy, and is independant of the number of leaf CPU cores.
> 
> Obviously in describing all of the above, I've ignored any complexity
> of dealing with our existing code implementation and pain of getting
> it converted to the new model.
> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
>