* [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model
@ 2020-02-13 18:16 Babu Moger
2020-02-13 18:16 ` [PATCH v4 01/16] hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs Babu Moger
` (15 more replies)
0 siblings, 16 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
This series fixes APIC ID encoding problem reported on AMD EPYC cpu model.
https://bugzilla.redhat.com/show_bug.cgi?id=1728166
Currently, the APIC ID is decoded based on the sequence
sockets->dies->cores->threads. This works for most standard AMD and other
vendors' configurations, but this decoding sequence does not follow that of
AMD's APIC ID enumeration strictly. In some cases this can cause CPU topology
inconsistency. When booting a guest VM, the kernel tries to validate the
topology, and finds it inconsistent with the enumeration of EPYC cpu models.
To fix the problem we need to build the topology as per the Processor
Programming Reference (PPR) for AMD Family 17h Model 01h, Revision B1
Processors. It is available at https://www.amd.com/system/files/TechDocs/55570-B1_PUB.zip
Here is the text from the PPR.
Operating systems are expected to use Core::X86::Cpuid::SizeId[ApicIdSize], the
number of least significant bits in the Initial APIC ID that indicate core ID
within a processor, in constructing per-core CPUID masks.
Core::X86::Cpuid::SizeId[ApicIdSize] determines the maximum number of cores
(MNC) that the processor could theoretically support, not the actual number of
cores that are actually implemented or enabled on the processor, as indicated
by Core::X86::Cpuid::SizeId[NC].
Each Core::X86::Apic::ApicId[ApicId] register is preset as follows:
• ApicId[6] = Socket ID.
• ApicId[5:4] = Node ID.
• ApicId[3] = Logical CCX L3 complex ID
• ApicId[2:0]= (SMT) ? {LogicalCoreID[1:0],ThreadId} : {1'b0,LogicalCoreID[1:0]}
v4:
Generated the patches on top of git://github.com/ehabkost/qemu.git (x86-next).
Changes from v3.
1. Moved the arch_id calculation inside the function x86_cpus_init. With this change,
we dont need to change common numa code(suggested by Igor).
2. Introduced the model specific handlers inside X86CPUDefinitions.
These handlers are loaded into X86MachineState during the init(suggested by Eduardo).
Removed the string comparison for EPYC model.
3. Removed llc_id from x86CPU. Added node_id inside X86CPUTopoIDs to take care of
EPYC apicid decoding.
4. Removed init_apicid_fn hanlder from MachineClass. Kept all the code changes
inside the x86.
5. Added new handler function apicid_pkg_offset for pkg_offset calculation.
6. And some other minor changes.
v3:
https://lore.kernel.org/qemu-devel/157541968844.46157.17994918142533791313.stgit@naples-babu.amd.com/
1. Consolidated the topology information in structure X86CPUTopoInfo.
2. Changed the ccx_id to llc_id as commented by upstream.
3. Generalized the apic id decoding. It is mostly similar to current apic id
except that it adds new field llc_id when numa configured. Removes all the
hardcoded values.
4. Removed the earlier parse_numa split. And moved the numa node initialization
inside the numa_complete_configuration. This is bit cleaner as commented by
Eduardo.
5. Added new function init_apicid_fn inside machine_class structure. This
will be used to update the apic id handler specific to cpu model.
6. Updated the cpuid unit tests.
7. TODO : Need to figure out how to dynamically update the handlers using cpu models.
I might some guidance on that.
v2:
https://lore.kernel.org/qemu-devel/156779689013.21957.1631551572950676212.stgit@localhost.localdomain/
1. Introduced the new property epyc to enable new epyc mode.
2. Separated the epyc mode and non epyc mode function.
3. Introduced function pointers in PCMachineState to handle the
differences.
4. Mildly tested different combinations to make things are working as expected.
5. TODO : Setting the epyc feature bit needs to be worked out. This feature is
supported only on AMD EPYC models. I may need some guidance on that.
v1:
https://lore.kernel.org/qemu-devel/20190731232032.51786-1-babu.moger@amd.com/
---
Babu Moger (16):
hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs
hw/i386: Introduce X86CPUTopoInfo to contain topology info
hw/i386: Consolidate topology functions
hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
machine: Add SMP Sockets in CpuTopology
hw/i386: Update structures for nodes_per_pkg
hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids
hw/386: Add EPYC mode topology decoding functions
target/i386: Cleanup and use the EPYC mode topology functions
hw/i386: Introduce apicid functions inside X86MachineState
target/i386: Load apicid model specific handlers from X86CPUDefinition
hw/i386: Use the apicid handlers from X86MachineState
target/i386: Add EPYC model specific handlers
hw/i386: Move arch_id decode inside x86_cpus_init
i386: Fix pkg_id offset for EPYC cpu models
tests: Update the Unit tests
hw/core/machine.c | 1
hw/i386/pc.c | 51 +++++-----
hw/i386/x86.c | 73 ++++++++++----
include/hw/boards.h | 2
include/hw/i386/topology.h | 219 +++++++++++++++++++++++++++++-------------
include/hw/i386/x86.h | 28 +++++
target/i386/cpu.c | 228 +++++++++++++++++---------------------------
target/i386/cpu.h | 3 +
tests/test-x86-cpuid.c | 115 +++++++++++++---------
vl.c | 1
10 files changed, 421 insertions(+), 300 deletions(-)
--
Signature
^ permalink raw reply [flat|nested] 44+ messages in thread
* [PATCH v4 01/16] hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
@ 2020-02-13 18:16 ` Babu Moger
2020-02-13 18:16 ` [PATCH v4 02/16] hw/i386: Introduce X86CPUTopoInfo to contain topology info Babu Moger
` (14 subsequent siblings)
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Rename few data structures related to X86 topology. X86CPUTopoIDs will
have individual arch ids. Next patch introduces X86CPUTopoInfo which will
have all topology information(like cores, threads etc..).
Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
hw/i386/pc.c | 42 +++++++++++++++++++++---------------------
hw/i386/x86.c | 18 +++++++++---------
include/hw/i386/topology.h | 40 ++++++++++++++++++++--------------------
3 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 58867f987d..3a580f6a60 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1733,7 +1733,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
int idx;
CPUState *cs;
CPUArchId *cpu_slot;
- X86CPUTopoInfo topo;
+ X86CPUTopoIDs topo_ids;
X86CPU *cpu = X86_CPU(dev);
CPUX86State *env = &cpu->env;
MachineState *ms = MACHINE(hotplug_dev);
@@ -1799,12 +1799,12 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
return;
}
- topo.pkg_id = cpu->socket_id;
- topo.die_id = cpu->die_id;
- topo.core_id = cpu->core_id;
- topo.smt_id = cpu->thread_id;
+ topo_ids.pkg_id = cpu->socket_id;
+ topo_ids.die_id = cpu->die_id;
+ topo_ids.core_id = cpu->core_id;
+ topo_ids.smt_id = cpu->thread_id;
cpu->apic_id = apicid_from_topo_ids(x86ms->smp_dies, smp_cores,
- smp_threads, &topo);
+ smp_threads, &topo_ids);
}
cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
@@ -1812,11 +1812,11 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
MachineState *ms = MACHINE(pcms);
x86_topo_ids_from_apicid(cpu->apic_id, x86ms->smp_dies,
- smp_cores, smp_threads, &topo);
+ smp_cores, smp_threads, &topo_ids);
error_setg(errp,
"Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with"
" APIC ID %" PRIu32 ", valid index range 0:%d",
- topo.pkg_id, topo.die_id, topo.core_id, topo.smt_id,
+ topo_ids.pkg_id, topo_ids.die_id, topo_ids.core_id, topo_ids.smt_id,
cpu->apic_id, ms->possible_cpus->len - 1);
return;
}
@@ -1834,34 +1834,34 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
* once -smp refactoring is complete and there will be CPU private
* CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
x86_topo_ids_from_apicid(cpu->apic_id, x86ms->smp_dies,
- smp_cores, smp_threads, &topo);
- if (cpu->socket_id != -1 && cpu->socket_id != topo.pkg_id) {
+ smp_cores, smp_threads, &topo_ids);
+ if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
- " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo.pkg_id);
+ " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo_ids.pkg_id);
return;
}
- cpu->socket_id = topo.pkg_id;
+ cpu->socket_id = topo_ids.pkg_id;
- if (cpu->die_id != -1 && cpu->die_id != topo.die_id) {
+ if (cpu->die_id != -1 && cpu->die_id != topo_ids.die_id) {
error_setg(errp, "property die-id: %u doesn't match set apic-id:"
- " 0x%x (die-id: %u)", cpu->die_id, cpu->apic_id, topo.die_id);
+ " 0x%x (die-id: %u)", cpu->die_id, cpu->apic_id, topo_ids.die_id);
return;
}
- cpu->die_id = topo.die_id;
+ cpu->die_id = topo_ids.die_id;
- if (cpu->core_id != -1 && cpu->core_id != topo.core_id) {
+ if (cpu->core_id != -1 && cpu->core_id != topo_ids.core_id) {
error_setg(errp, "property core-id: %u doesn't match set apic-id:"
- " 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id, topo.core_id);
+ " 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id, topo_ids.core_id);
return;
}
- cpu->core_id = topo.core_id;
+ cpu->core_id = topo_ids.core_id;
- if (cpu->thread_id != -1 && cpu->thread_id != topo.smt_id) {
+ if (cpu->thread_id != -1 && cpu->thread_id != topo_ids.smt_id) {
error_setg(errp, "property thread-id: %u doesn't match set apic-id:"
- " 0x%x (thread-id: %u)", cpu->thread_id, cpu->apic_id, topo.smt_id);
+ " 0x%x (thread-id: %u)", cpu->thread_id, cpu->apic_id, topo_ids.smt_id);
return;
}
- cpu->thread_id = topo.smt_id;
+ cpu->thread_id = topo_ids.smt_id;
if (hyperv_feat_enabled(cpu, HYPERV_FEAT_VPINDEX) &&
!kvm_hv_vpindex_settable()) {
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 394edc2f72..4a112c8e30 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -138,14 +138,14 @@ x86_cpu_index_to_props(MachineState *ms, unsigned cpu_index)
int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
{
- X86CPUTopoInfo topo;
+ X86CPUTopoIDs topo_ids;
X86MachineState *x86ms = X86_MACHINE(ms);
assert(idx < ms->possible_cpus->len);
x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
x86ms->smp_dies, ms->smp.cores,
- ms->smp.threads, &topo);
- return topo.pkg_id % ms->numa_state->num_nodes;
+ ms->smp.threads, &topo_ids);
+ return topo_ids.pkg_id % ms->numa_state->num_nodes;
}
const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
@@ -167,7 +167,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
for (i = 0; i < ms->possible_cpus->len; i++) {
- X86CPUTopoInfo topo;
+ X86CPUTopoIDs topo_ids;
ms->possible_cpus->cpus[i].type = ms->cpu_type;
ms->possible_cpus->cpus[i].vcpus_count = 1;
@@ -175,17 +175,17 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
x86_cpu_apic_id_from_index(x86ms, i);
x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
x86ms->smp_dies, ms->smp.cores,
- ms->smp.threads, &topo);
+ ms->smp.threads, &topo_ids);
ms->possible_cpus->cpus[i].props.has_socket_id = true;
- ms->possible_cpus->cpus[i].props.socket_id = topo.pkg_id;
+ ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
if (x86ms->smp_dies > 1) {
ms->possible_cpus->cpus[i].props.has_die_id = true;
- ms->possible_cpus->cpus[i].props.die_id = topo.die_id;
+ ms->possible_cpus->cpus[i].props.die_id = topo_ids.die_id;
}
ms->possible_cpus->cpus[i].props.has_core_id = true;
- ms->possible_cpus->cpus[i].props.core_id = topo.core_id;
+ ms->possible_cpus->cpus[i].props.core_id = topo_ids.core_id;
ms->possible_cpus->cpus[i].props.has_thread_id = true;
- ms->possible_cpus->cpus[i].props.thread_id = topo.smt_id;
+ ms->possible_cpus->cpus[i].props.thread_id = topo_ids.smt_id;
}
return ms->possible_cpus;
}
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 4ff5b2da6c..6c184f3115 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -45,12 +45,12 @@
*/
typedef uint32_t apic_id_t;
-typedef struct X86CPUTopoInfo {
+typedef struct X86CPUTopoIDs {
unsigned pkg_id;
unsigned die_id;
unsigned core_id;
unsigned smt_id;
-} X86CPUTopoInfo;
+} X86CPUTopoIDs;
/* Return the bit width needed for 'count' IDs
*/
@@ -122,12 +122,12 @@ static inline unsigned apicid_pkg_offset(unsigned nr_dies,
static inline apic_id_t apicid_from_topo_ids(unsigned nr_dies,
unsigned nr_cores,
unsigned nr_threads,
- const X86CPUTopoInfo *topo)
+ const X86CPUTopoIDs *topo_ids)
{
- return (topo->pkg_id << apicid_pkg_offset(nr_dies, nr_cores, nr_threads)) |
- (topo->die_id << apicid_die_offset(nr_dies, nr_cores, nr_threads)) |
- (topo->core_id << apicid_core_offset(nr_dies, nr_cores, nr_threads)) |
- topo->smt_id;
+ return (topo_ids->pkg_id << apicid_pkg_offset(nr_dies, nr_cores, nr_threads)) |
+ (topo_ids->die_id << apicid_die_offset(nr_dies, nr_cores, nr_threads)) |
+ (topo_ids->core_id << apicid_core_offset(nr_dies, nr_cores, nr_threads)) |
+ topo_ids->smt_id;
}
/* Calculate thread/core/package IDs for a specific topology,
@@ -137,12 +137,12 @@ static inline void x86_topo_ids_from_idx(unsigned nr_dies,
unsigned nr_cores,
unsigned nr_threads,
unsigned cpu_index,
- X86CPUTopoInfo *topo)
+ X86CPUTopoIDs *topo_ids)
{
- topo->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
- topo->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
- topo->core_id = cpu_index / nr_threads % nr_cores;
- topo->smt_id = cpu_index % nr_threads;
+ topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
+ topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
+ topo_ids->core_id = cpu_index / nr_threads % nr_cores;
+ topo_ids->smt_id = cpu_index % nr_threads;
}
/* Calculate thread/core/package IDs for a specific topology,
@@ -152,17 +152,17 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
unsigned nr_dies,
unsigned nr_cores,
unsigned nr_threads,
- X86CPUTopoInfo *topo)
+ X86CPUTopoIDs *topo_ids)
{
- topo->smt_id = apicid &
+ topo_ids->smt_id = apicid &
~(0xFFFFFFFFUL << apicid_smt_width(nr_dies, nr_cores, nr_threads));
- topo->core_id =
+ topo_ids->core_id =
(apicid >> apicid_core_offset(nr_dies, nr_cores, nr_threads)) &
~(0xFFFFFFFFUL << apicid_core_width(nr_dies, nr_cores, nr_threads));
- topo->die_id =
+ topo_ids->die_id =
(apicid >> apicid_die_offset(nr_dies, nr_cores, nr_threads)) &
~(0xFFFFFFFFUL << apicid_die_width(nr_dies, nr_cores, nr_threads));
- topo->pkg_id = apicid >> apicid_pkg_offset(nr_dies, nr_cores, nr_threads);
+ topo_ids->pkg_id = apicid >> apicid_pkg_offset(nr_dies, nr_cores, nr_threads);
}
/* Make APIC ID for the CPU 'cpu_index'
@@ -174,9 +174,9 @@ static inline apic_id_t x86_apicid_from_cpu_idx(unsigned nr_dies,
unsigned nr_threads,
unsigned cpu_index)
{
- X86CPUTopoInfo topo;
- x86_topo_ids_from_idx(nr_dies, nr_cores, nr_threads, cpu_index, &topo);
- return apicid_from_topo_ids(nr_dies, nr_cores, nr_threads, &topo);
+ X86CPUTopoIDs topo_ids;
+ x86_topo_ids_from_idx(nr_dies, nr_cores, nr_threads, cpu_index, &topo_ids);
+ return apicid_from_topo_ids(nr_dies, nr_cores, nr_threads, &topo_ids);
}
#endif /* HW_I386_TOPOLOGY_H */
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 02/16] hw/i386: Introduce X86CPUTopoInfo to contain topology info
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
2020-02-13 18:16 ` [PATCH v4 01/16] hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs Babu Moger
@ 2020-02-13 18:16 ` Babu Moger
2020-02-13 18:16 ` [PATCH v4 03/16] hw/i386: Consolidate topology functions Babu Moger
` (13 subsequent siblings)
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
This is an effort to re-arrange few data structure for better readability.
Add X86CPUTopoInfo which will have all the topology informations required
to build the cpu topology. There is no functional changes.
Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
hw/i386/pc.c | 14 ++++++++------
hw/i386/x86.c | 28 +++++++++++++++++++++-------
include/hw/i386/topology.h | 38 ++++++++++++++++++++++++--------------
3 files changed, 53 insertions(+), 27 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 3a580f6a60..2adf7f6afa 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1741,6 +1741,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
X86MachineState *x86ms = X86_MACHINE(pcms);
unsigned int smp_cores = ms->smp.cores;
unsigned int smp_threads = ms->smp.threads;
+ X86CPUTopoInfo topo_info;
if(!object_dynamic_cast(OBJECT(cpu), ms->cpu_type)) {
error_setg(errp, "Invalid CPU type, expected cpu type: '%s'",
@@ -1748,6 +1749,10 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
return;
}
+ topo_info.dies_per_pkg = x86ms->smp_dies;
+ topo_info.cores_per_die = smp_cores;
+ topo_info.threads_per_core = smp_threads;
+
env->nr_dies = x86ms->smp_dies;
/*
@@ -1803,16 +1808,14 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
topo_ids.die_id = cpu->die_id;
topo_ids.core_id = cpu->core_id;
topo_ids.smt_id = cpu->thread_id;
- cpu->apic_id = apicid_from_topo_ids(x86ms->smp_dies, smp_cores,
- smp_threads, &topo_ids);
+ cpu->apic_id = apicid_from_topo_ids(&topo_info, &topo_ids);
}
cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
if (!cpu_slot) {
MachineState *ms = MACHINE(pcms);
- x86_topo_ids_from_apicid(cpu->apic_id, x86ms->smp_dies,
- smp_cores, smp_threads, &topo_ids);
+ x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
error_setg(errp,
"Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with"
" APIC ID %" PRIu32 ", valid index range 0:%d",
@@ -1833,8 +1836,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
/* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
* once -smp refactoring is complete and there will be CPU private
* CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
- x86_topo_ids_from_apicid(cpu->apic_id, x86ms->smp_dies,
- smp_cores, smp_threads, &topo_ids);
+ x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
" 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo_ids.pkg_id);
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 4a112c8e30..f18cab8e5c 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -65,11 +65,15 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
{
MachineState *ms = MACHINE(x86ms);
X86MachineClass *x86mc = X86_MACHINE_GET_CLASS(x86ms);
+ X86CPUTopoInfo topo_info;
uint32_t correct_id;
static bool warned;
- correct_id = x86_apicid_from_cpu_idx(x86ms->smp_dies, ms->smp.cores,
- ms->smp.threads, cpu_index);
+ topo_info.dies_per_pkg = x86ms->smp_dies;
+ topo_info.cores_per_die = ms->smp.cores;
+ topo_info.threads_per_core = ms->smp.threads;
+
+ correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
if (x86mc->compat_apic_id_mode) {
if (cpu_index != correct_id && !warned && !qtest_enabled()) {
error_report("APIC IDs set in compatibility mode, "
@@ -140,19 +144,25 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
{
X86CPUTopoIDs topo_ids;
X86MachineState *x86ms = X86_MACHINE(ms);
+ X86CPUTopoInfo topo_info;
+
+ topo_info.dies_per_pkg = x86ms->smp_dies;
+ topo_info.cores_per_die = ms->smp.cores;
+ topo_info.threads_per_core = ms->smp.threads;
+
assert(idx < ms->possible_cpus->len);
x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
- x86ms->smp_dies, ms->smp.cores,
- ms->smp.threads, &topo_ids);
+ &topo_info, &topo_ids);
return topo_ids.pkg_id % ms->numa_state->num_nodes;
}
const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
{
X86MachineState *x86ms = X86_MACHINE(ms);
- int i;
unsigned int max_cpus = ms->smp.max_cpus;
+ X86CPUTopoInfo topo_info;
+ int i;
if (ms->possible_cpus) {
/*
@@ -166,6 +176,11 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
+
+ topo_info.dies_per_pkg = x86ms->smp_dies;
+ topo_info.cores_per_die = ms->smp.cores;
+ topo_info.threads_per_core = ms->smp.threads;
+
for (i = 0; i < ms->possible_cpus->len; i++) {
X86CPUTopoIDs topo_ids;
@@ -174,8 +189,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
ms->possible_cpus->cpus[i].arch_id =
x86_cpu_apic_id_from_index(x86ms, i);
x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
- x86ms->smp_dies, ms->smp.cores,
- ms->smp.threads, &topo_ids);
+ &topo_info, &topo_ids);
ms->possible_cpus->cpus[i].props.has_socket_id = true;
ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
if (x86ms->smp_dies > 1) {
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 6c184f3115..cf1935d548 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -52,6 +52,12 @@ typedef struct X86CPUTopoIDs {
unsigned smt_id;
} X86CPUTopoIDs;
+typedef struct X86CPUTopoInfo {
+ unsigned dies_per_pkg;
+ unsigned cores_per_die;
+ unsigned threads_per_core;
+} X86CPUTopoInfo;
+
/* Return the bit width needed for 'count' IDs
*/
static unsigned apicid_bitwidth_for_count(unsigned count)
@@ -119,11 +125,13 @@ static inline unsigned apicid_pkg_offset(unsigned nr_dies,
*
* The caller must make sure core_id < nr_cores and smt_id < nr_threads.
*/
-static inline apic_id_t apicid_from_topo_ids(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads,
+static inline apic_id_t apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
const X86CPUTopoIDs *topo_ids)
{
+ unsigned nr_dies = topo_info->dies_per_pkg;
+ unsigned nr_cores = topo_info->cores_per_die;
+ unsigned nr_threads = topo_info->threads_per_core;
+
return (topo_ids->pkg_id << apicid_pkg_offset(nr_dies, nr_cores, nr_threads)) |
(topo_ids->die_id << apicid_die_offset(nr_dies, nr_cores, nr_threads)) |
(topo_ids->core_id << apicid_core_offset(nr_dies, nr_cores, nr_threads)) |
@@ -133,12 +141,14 @@ static inline apic_id_t apicid_from_topo_ids(unsigned nr_dies,
/* Calculate thread/core/package IDs for a specific topology,
* based on (contiguous) CPU index
*/
-static inline void x86_topo_ids_from_idx(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads,
+static inline void x86_topo_ids_from_idx(X86CPUTopoInfo *topo_info,
unsigned cpu_index,
X86CPUTopoIDs *topo_ids)
{
+ unsigned nr_dies = topo_info->dies_per_pkg;
+ unsigned nr_cores = topo_info->cores_per_die;
+ unsigned nr_threads = topo_info->threads_per_core;
+
topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
topo_ids->core_id = cpu_index / nr_threads % nr_cores;
@@ -149,11 +159,13 @@ static inline void x86_topo_ids_from_idx(unsigned nr_dies,
* based on APIC ID
*/
static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
- unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads,
+ X86CPUTopoInfo *topo_info,
X86CPUTopoIDs *topo_ids)
{
+ unsigned nr_dies = topo_info->dies_per_pkg;
+ unsigned nr_cores = topo_info->cores_per_die;
+ unsigned nr_threads = topo_info->threads_per_core;
+
topo_ids->smt_id = apicid &
~(0xFFFFFFFFUL << apicid_smt_width(nr_dies, nr_cores, nr_threads));
topo_ids->core_id =
@@ -169,14 +181,12 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
*
* 'cpu_index' is a sequential, contiguous ID for the CPU.
*/
-static inline apic_id_t x86_apicid_from_cpu_idx(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads,
+static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info,
unsigned cpu_index)
{
X86CPUTopoIDs topo_ids;
- x86_topo_ids_from_idx(nr_dies, nr_cores, nr_threads, cpu_index, &topo_ids);
- return apicid_from_topo_ids(nr_dies, nr_cores, nr_threads, &topo_ids);
+ x86_topo_ids_from_idx(topo_info, cpu_index, &topo_ids);
+ return apicid_from_topo_ids(topo_info, &topo_ids);
}
#endif /* HW_I386_TOPOLOGY_H */
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 03/16] hw/i386: Consolidate topology functions
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
2020-02-13 18:16 ` [PATCH v4 01/16] hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs Babu Moger
2020-02-13 18:16 ` [PATCH v4 02/16] hw/i386: Introduce X86CPUTopoInfo to contain topology info Babu Moger
@ 2020-02-13 18:16 ` Babu Moger
2020-02-13 18:16 ` [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo Babu Moger
` (12 subsequent siblings)
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Now that we have all the parameters in X86CPUTopoInfo, we can just
pass the structure to calculate the offsets and width.
Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
---
include/hw/i386/topology.h | 64 ++++++++++++++------------------------------
target/i386/cpu.c | 23 ++++++++--------
2 files changed, 32 insertions(+), 55 deletions(-)
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index cf1935d548..ba52d49079 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -69,56 +69,42 @@ static unsigned apicid_bitwidth_for_count(unsigned count)
/* Bit width of the SMT_ID (thread ID) field on the APIC ID
*/
-static inline unsigned apicid_smt_width(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_smt_width(X86CPUTopoInfo *topo_info)
{
- return apicid_bitwidth_for_count(nr_threads);
+ return apicid_bitwidth_for_count(topo_info->threads_per_core);
}
/* Bit width of the Core_ID field
*/
-static inline unsigned apicid_core_width(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_core_width(X86CPUTopoInfo *topo_info)
{
- return apicid_bitwidth_for_count(nr_cores);
+ return apicid_bitwidth_for_count(topo_info->cores_per_die);
}
/* Bit width of the Die_ID field */
-static inline unsigned apicid_die_width(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_die_width(X86CPUTopoInfo *topo_info)
{
- return apicid_bitwidth_for_count(nr_dies);
+ return apicid_bitwidth_for_count(topo_info->dies_per_pkg);
}
/* Bit offset of the Core_ID field
*/
-static inline unsigned apicid_core_offset(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)
{
- return apicid_smt_width(nr_dies, nr_cores, nr_threads);
+ return apicid_smt_width(topo_info);
}
/* Bit offset of the Die_ID field */
-static inline unsigned apicid_die_offset(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_die_offset(X86CPUTopoInfo *topo_info)
{
- return apicid_core_offset(nr_dies, nr_cores, nr_threads) +
- apicid_core_width(nr_dies, nr_cores, nr_threads);
+ return apicid_core_offset(topo_info) + apicid_core_width(topo_info);
}
/* Bit offset of the Pkg_ID (socket ID) field
*/
-static inline unsigned apicid_pkg_offset(unsigned nr_dies,
- unsigned nr_cores,
- unsigned nr_threads)
+static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
{
- return apicid_die_offset(nr_dies, nr_cores, nr_threads) +
- apicid_die_width(nr_dies, nr_cores, nr_threads);
+ return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
}
/* Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
@@ -128,13 +114,9 @@ static inline unsigned apicid_pkg_offset(unsigned nr_dies,
static inline apic_id_t apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
const X86CPUTopoIDs *topo_ids)
{
- unsigned nr_dies = topo_info->dies_per_pkg;
- unsigned nr_cores = topo_info->cores_per_die;
- unsigned nr_threads = topo_info->threads_per_core;
-
- return (topo_ids->pkg_id << apicid_pkg_offset(nr_dies, nr_cores, nr_threads)) |
- (topo_ids->die_id << apicid_die_offset(nr_dies, nr_cores, nr_threads)) |
- (topo_ids->core_id << apicid_core_offset(nr_dies, nr_cores, nr_threads)) |
+ return (topo_ids->pkg_id << apicid_pkg_offset(topo_info)) |
+ (topo_ids->die_id << apicid_die_offset(topo_info)) |
+ (topo_ids->core_id << apicid_core_offset(topo_info)) |
topo_ids->smt_id;
}
@@ -162,19 +144,15 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid,
X86CPUTopoInfo *topo_info,
X86CPUTopoIDs *topo_ids)
{
- unsigned nr_dies = topo_info->dies_per_pkg;
- unsigned nr_cores = topo_info->cores_per_die;
- unsigned nr_threads = topo_info->threads_per_core;
-
topo_ids->smt_id = apicid &
- ~(0xFFFFFFFFUL << apicid_smt_width(nr_dies, nr_cores, nr_threads));
+ ~(0xFFFFFFFFUL << apicid_smt_width(topo_info));
topo_ids->core_id =
- (apicid >> apicid_core_offset(nr_dies, nr_cores, nr_threads)) &
- ~(0xFFFFFFFFUL << apicid_core_width(nr_dies, nr_cores, nr_threads));
+ (apicid >> apicid_core_offset(topo_info)) &
+ ~(0xFFFFFFFFUL << apicid_core_width(topo_info));
topo_ids->die_id =
- (apicid >> apicid_die_offset(nr_dies, nr_cores, nr_threads)) &
- ~(0xFFFFFFFFUL << apicid_die_width(nr_dies, nr_cores, nr_threads));
- topo_ids->pkg_id = apicid >> apicid_pkg_offset(nr_dies, nr_cores, nr_threads);
+ (apicid >> apicid_die_offset(topo_info)) &
+ ~(0xFFFFFFFFUL << apicid_die_width(topo_info));
+ topo_ids->pkg_id = apicid >> apicid_pkg_offset(topo_info);
}
/* Make APIC ID for the CPU 'cpu_index'
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index ba5e9d5d6b..7e630f47ac 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5316,6 +5316,11 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
uint32_t die_offset;
uint32_t limit;
uint32_t signature[3];
+ X86CPUTopoInfo topo_info;
+
+ topo_info.dies_per_pkg = env->nr_dies;
+ topo_info.cores_per_die = cs->nr_cores;
+ topo_info.threads_per_core = cs->nr_threads;
/* Calculate & apply limits for different index ranges */
if (index >= 0xC0000000) {
@@ -5402,8 +5407,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
eax, ebx, ecx, edx);
break;
case 3: /* L3 cache info */
- die_offset = apicid_die_offset(env->nr_dies,
- cs->nr_cores, cs->nr_threads);
+ die_offset = apicid_die_offset(&topo_info);
if (cpu->enable_l3_cache) {
encode_cache_cpuid4(env->cache_info_cpuid4.l3_cache,
(1 << die_offset), cs->nr_cores,
@@ -5494,14 +5498,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
switch (count) {
case 0:
- *eax = apicid_core_offset(env->nr_dies,
- cs->nr_cores, cs->nr_threads);
+ *eax = apicid_core_offset(&topo_info);
*ebx = cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
break;
case 1:
- *eax = apicid_pkg_offset(env->nr_dies,
- cs->nr_cores, cs->nr_threads);
+ *eax = apicid_pkg_offset(&topo_info);
*ebx = cs->nr_cores * cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
break;
@@ -5525,20 +5527,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
*edx = cpu->apic_id;
switch (count) {
case 0:
- *eax = apicid_core_offset(env->nr_dies, cs->nr_cores,
- cs->nr_threads);
+ *eax = apicid_core_offset(&topo_info);
*ebx = cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
break;
case 1:
- *eax = apicid_die_offset(env->nr_dies, cs->nr_cores,
- cs->nr_threads);
+ *eax = apicid_die_offset(&topo_info);
*ebx = cs->nr_cores * cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
break;
case 2:
- *eax = apicid_pkg_offset(env->nr_dies, cs->nr_cores,
- cs->nr_threads);
+ *eax = apicid_pkg_offset(&topo_info);
*ebx = env->nr_dies * cs->nr_cores * cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
break;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (2 preceding siblings ...)
2020-02-13 18:16 ` [PATCH v4 03/16] hw/i386: Consolidate topology functions Babu Moger
@ 2020-02-13 18:16 ` Babu Moger
2020-02-21 17:05 ` Igor Mammedov
2020-02-13 18:16 ` [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology Babu Moger
` (11 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Initialize all the parameters in one function init_topo_info.
Move the data structure X86CPUTopoIDs and X86CPUTopoInfo into
x86.h.
Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
hw/i386/pc.c | 4 +---
hw/i386/x86.c | 14 +++-----------
include/hw/i386/topology.h | 26 ++++++++++----------------
include/hw/i386/x86.h | 17 +++++++++++++++++
4 files changed, 31 insertions(+), 30 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2adf7f6afa..9803413dd9 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1749,9 +1749,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
return;
}
- topo_info.dies_per_pkg = x86ms->smp_dies;
- topo_info.cores_per_die = smp_cores;
- topo_info.threads_per_core = smp_threads;
+ init_topo_info(&topo_info, x86ms);
env->nr_dies = x86ms->smp_dies;
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index f18cab8e5c..083effb2f5 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -63,15 +63,12 @@ static size_t pvh_start_addr;
uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
unsigned int cpu_index)
{
- MachineState *ms = MACHINE(x86ms);
X86MachineClass *x86mc = X86_MACHINE_GET_CLASS(x86ms);
X86CPUTopoInfo topo_info;
uint32_t correct_id;
static bool warned;
- topo_info.dies_per_pkg = x86ms->smp_dies;
- topo_info.cores_per_die = ms->smp.cores;
- topo_info.threads_per_core = ms->smp.threads;
+ init_topo_info(&topo_info, x86ms);
correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
if (x86mc->compat_apic_id_mode) {
@@ -146,10 +143,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
X86MachineState *x86ms = X86_MACHINE(ms);
X86CPUTopoInfo topo_info;
- topo_info.dies_per_pkg = x86ms->smp_dies;
- topo_info.cores_per_die = ms->smp.cores;
- topo_info.threads_per_core = ms->smp.threads;
-
+ init_topo_info(&topo_info, x86ms);
assert(idx < ms->possible_cpus->len);
x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
@@ -177,9 +171,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
- topo_info.dies_per_pkg = x86ms->smp_dies;
- topo_info.cores_per_die = ms->smp.cores;
- topo_info.threads_per_core = ms->smp.threads;
+ init_topo_info(&topo_info, x86ms);
for (i = 0; i < ms->possible_cpus->len; i++) {
X86CPUTopoIDs topo_ids;
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index ba52d49079..ef0ab0b6a3 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -40,23 +40,17 @@
#include "qemu/bitops.h"
+#include "hw/i386/x86.h"
-/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
- */
-typedef uint32_t apic_id_t;
-
-typedef struct X86CPUTopoIDs {
- unsigned pkg_id;
- unsigned die_id;
- unsigned core_id;
- unsigned smt_id;
-} X86CPUTopoIDs;
-
-typedef struct X86CPUTopoInfo {
- unsigned dies_per_pkg;
- unsigned cores_per_die;
- unsigned threads_per_core;
-} X86CPUTopoInfo;
+static inline void init_topo_info(X86CPUTopoInfo *topo_info,
+ const X86MachineState *x86ms)
+{
+ MachineState *ms = MACHINE(x86ms);
+
+ topo_info->dies_per_pkg = x86ms->smp_dies;
+ topo_info->cores_per_die = ms->smp.cores;
+ topo_info->threads_per_core = ms->smp.threads;
+}
/* Return the bit width needed for 'count' IDs
*/
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 4b84917885..ad62b01cf2 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -36,6 +36,23 @@ typedef struct {
bool compat_apic_id_mode;
} X86MachineClass;
+/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
+ */
+typedef uint32_t apic_id_t;
+
+typedef struct X86CPUTopoIDs {
+ unsigned pkg_id;
+ unsigned die_id;
+ unsigned core_id;
+ unsigned smt_id;
+} X86CPUTopoIDs;
+
+typedef struct X86CPUTopoInfo {
+ unsigned dies_per_pkg;
+ unsigned cores_per_die;
+ unsigned threads_per_core;
+} X86CPUTopoInfo;
+
typedef struct {
/*< private >*/
MachineState parent;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (3 preceding siblings ...)
2020-02-13 18:16 ` [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo Babu Moger
@ 2020-02-13 18:16 ` Babu Moger
2020-02-24 8:37 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg Babu Moger
` (10 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:16 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Store the smp sockets in CpuTopology. The socket information required to
build the apic id in EPYC mode. Right now socket information is not passed
to down when decoding the apic id. Add the socket information here.
Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
hw/core/machine.c | 1 +
hw/i386/pc.c | 1 +
include/hw/boards.h | 2 ++
vl.c | 1 +
4 files changed, 5 insertions(+)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 023548b4f3..ea03c25565 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -789,6 +789,7 @@ static void smp_parse(MachineState *ms, QemuOpts *opts)
ms->smp.cpus = cpus;
ms->smp.cores = cores;
ms->smp.threads = threads;
+ ms->smp.sockets = sockets;
}
if (ms->smp.cpus > 1) {
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 9803413dd9..f13721ac43 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -978,6 +978,7 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
ms->smp.cpus = cpus;
ms->smp.cores = cores;
ms->smp.threads = threads;
+ ms->smp.sockets = sockets;
x86ms->smp_dies = dies;
}
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 24cbeecbae..a50a2d779e 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -256,12 +256,14 @@ typedef struct DeviceMemoryState {
* @cpus: the number of present logical processors on the machine
* @cores: the number of cores in one package
* @threads: the number of threads in one core
+ * @sockets: the number of sockets on the machine
* @max_cpus: the maximum number of logical processors on the machine
*/
typedef struct CpuTopology {
unsigned int cpus;
unsigned int cores;
unsigned int threads;
+ unsigned int sockets;
unsigned int max_cpus;
} CpuTopology;
diff --git a/vl.c b/vl.c
index 94508300c3..3f6d396a95 100644
--- a/vl.c
+++ b/vl.c
@@ -3819,6 +3819,7 @@ int main(int argc, char **argv, char **envp)
current_machine->smp.max_cpus = machine_class->default_cpus;
current_machine->smp.cores = 1;
current_machine->smp.threads = 1;
+ current_machine->smp.sockets = 1;
machine_class->smp_parse(current_machine,
qemu_opts_find(qemu_find_opts("smp-opts"), NULL));
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (4 preceding siblings ...)
2020-02-13 18:16 ` [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-24 8:34 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids Babu Moger
` (9 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Update structures X86CPUTopoIDs and CPUX86State to hold the nodes_per_pkg.
This is required to build EPYC mode topology.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
hw/i386/pc.c | 1 +
hw/i386/x86.c | 2 ++
include/hw/i386/topology.h | 2 ++
include/hw/i386/x86.h | 1 +
target/i386/cpu.c | 1 +
target/i386/cpu.h | 1 +
6 files changed, 8 insertions(+)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index f13721ac43..02fdb3d506 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1753,6 +1753,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
init_topo_info(&topo_info, x86ms);
env->nr_dies = x86ms->smp_dies;
+ env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
/*
* If APIC ID is not set,
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 083effb2f5..3d944f68e6 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -89,11 +89,13 @@ void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
Object *cpu = NULL;
Error *local_err = NULL;
CPUX86State *env = NULL;
+ MachineState *ms = MACHINE(x86ms);
cpu = object_new(MACHINE(x86ms)->cpu_type);
env = &X86_CPU(cpu)->env;
env->nr_dies = x86ms->smp_dies;
+ env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
object_property_set_uint(cpu, apic_id, "apic-id", &local_err);
object_property_set_bool(cpu, true, "realized", &local_err);
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index ef0ab0b6a3..522c77e6a9 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -41,12 +41,14 @@
#include "qemu/bitops.h"
#include "hw/i386/x86.h"
+#include "sysemu/numa.h"
static inline void init_topo_info(X86CPUTopoInfo *topo_info,
const X86MachineState *x86ms)
{
MachineState *ms = MACHINE(x86ms);
+ topo_info->nodes_per_pkg = ms->numa_state->num_nodes / ms->smp.sockets;
topo_info->dies_per_pkg = x86ms->smp_dies;
topo_info->cores_per_die = ms->smp.cores;
topo_info->threads_per_core = ms->smp.threads;
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index ad62b01cf2..d76fd0bbb1 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -48,6 +48,7 @@ typedef struct X86CPUTopoIDs {
} X86CPUTopoIDs;
typedef struct X86CPUTopoInfo {
+ unsigned nodes_per_pkg;
unsigned dies_per_pkg;
unsigned cores_per_die;
unsigned threads_per_core;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 7e630f47ac..5d6edfd09b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6761,6 +6761,7 @@ static void x86_cpu_initfn(Object *obj)
FeatureWord w;
env->nr_dies = 1;
+ env->nr_nodes = 1;
cpu_set_cpustate_pointers(cpu);
object_property_add(obj, "family", "int",
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index af282936a7..627a8cb9be 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1603,6 +1603,7 @@ typedef struct CPUX86State {
TPRAccess tpr_access_type;
unsigned nr_dies;
+ unsigned nr_nodes;
} CPUX86State;
struct kvm_msrs;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (5 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-20 12:43 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions Babu Moger
` (8 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
For consistancy rename apicid_from_topo_ids to x86_apicid_from_topo_ids.
No functional change.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
hw/i386/pc.c | 2 +-
include/hw/i386/topology.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 02fdb3d506..be72a49716 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1808,7 +1808,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
topo_ids.die_id = cpu->die_id;
topo_ids.core_id = cpu->core_id;
topo_ids.smt_id = cpu->thread_id;
- cpu->apic_id = apicid_from_topo_ids(&topo_info, &topo_ids);
+ cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
}
cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 522c77e6a9..3158157430 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -107,8 +107,8 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
*
* The caller must make sure core_id < nr_cores and smt_id < nr_threads.
*/
-static inline apic_id_t apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
- const X86CPUTopoIDs *topo_ids)
+static inline apic_id_t x86_apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
+ const X86CPUTopoIDs *topo_ids)
{
return (topo_ids->pkg_id << apicid_pkg_offset(topo_info)) |
(topo_ids->die_id << apicid_die_offset(topo_info)) |
@@ -160,7 +160,7 @@ static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info,
{
X86CPUTopoIDs topo_ids;
x86_topo_ids_from_idx(topo_info, cpu_index, &topo_ids);
- return apicid_from_topo_ids(topo_info, &topo_ids);
+ return x86_apicid_from_topo_ids(topo_info, &topo_ids);
}
#endif /* HW_I386_TOPOLOGY_H */
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (6 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-24 8:50 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions Babu Moger
` (7 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
These functions add support for building EPYC mode topology given the smp
details like numa nodes, cores, threads and sockets.
The new apic id decoding is mostly similar to current apic id decoding
except that it adds a new field llc_id when numa configured. Removes all
the hardcoded values. Subsequent patches will use these functions to build
the topology.
Following functions are added.
apicid_llc_width_epyc
apicid_llc_offset_epyc
apicid_pkg_offset_epyc
apicid_from_topo_ids_epyc
x86_topo_ids_from_idx_epyc
x86_topo_ids_from_apicid_epyc
x86_apicid_from_cpu_idx_epyc
The topology details are available in Processor Programming Reference (PPR)
for AMD Family 17h Model 01h, Revision B1 Processors.
https://www.amd.com/system/files/TechDocs/55570-B1_PUB.zip
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
include/hw/i386/topology.h | 93 ++++++++++++++++++++++++++++++++++++++++++++
include/hw/i386/x86.h | 1
2 files changed, 94 insertions(+)
diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
index 3158157430..d9319dc2ac 100644
--- a/include/hw/i386/topology.h
+++ b/include/hw/i386/topology.h
@@ -83,6 +83,11 @@ static inline unsigned apicid_die_width(X86CPUTopoInfo *topo_info)
return apicid_bitwidth_for_count(topo_info->dies_per_pkg);
}
+/* Bit width of the node_id field per socket */
+static inline unsigned apicid_node_width_epyc(X86CPUTopoInfo *topo_info)
+{
+ return apicid_bitwidth_for_count(MAX(topo_info->nodes_per_pkg, 1));
+}
/* Bit offset of the Core_ID field
*/
static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)
@@ -103,6 +108,94 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
}
+#define LLC_OFFSET 3 /* Minimum LLC offset if numa configured */
+
+/* Bit offset of the node_id field */
+static inline unsigned apicid_node_offset_epyc(X86CPUTopoInfo *topo_info)
+{
+ unsigned offset = apicid_die_offset(topo_info) +
+ apicid_die_width(topo_info);
+
+ if (topo_info->nodes_per_pkg) {
+ return MAX(LLC_OFFSET, offset);
+ } else {
+ return offset;
+ }
+}
+
+/* Bit offset of the Pkg_ID (socket ID) field */
+static inline unsigned apicid_pkg_offset_epyc(X86CPUTopoInfo *topo_info)
+{
+ return apicid_node_offset_epyc(topo_info) + apicid_node_width_epyc(topo_info);
+}
+
+/*
+ * Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
+ *
+ * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
+ */
+static inline apic_id_t x86_apicid_from_topo_ids_epyc(X86CPUTopoInfo *topo_info,
+ const X86CPUTopoIDs *topo_ids)
+{
+ return (topo_ids->pkg_id << apicid_pkg_offset_epyc(topo_info)) |
+ (topo_ids->node_id << apicid_node_offset_epyc(topo_info)) |
+ (topo_ids->die_id << apicid_die_offset(topo_info)) |
+ (topo_ids->core_id << apicid_core_offset(topo_info)) |
+ topo_ids->smt_id;
+}
+
+static inline void x86_topo_ids_from_idx_epyc(X86CPUTopoInfo *topo_info,
+ unsigned cpu_index,
+ X86CPUTopoIDs *topo_ids)
+{
+ unsigned nr_nodes = MAX(topo_info->nodes_per_pkg, 1);
+ unsigned nr_dies = topo_info->dies_per_pkg;
+ unsigned nr_cores = topo_info->cores_per_die;
+ unsigned nr_threads = topo_info->threads_per_core;
+ unsigned cores_per_node = DIV_ROUND_UP((nr_dies * nr_cores * nr_threads),
+ nr_nodes);
+
+ topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
+ topo_ids->node_id = (cpu_index / cores_per_node) % nr_nodes;
+ topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
+ topo_ids->core_id = cpu_index / nr_threads % nr_cores;
+ topo_ids->smt_id = cpu_index % nr_threads;
+}
+
+/*
+ * Calculate thread/core/package IDs for a specific topology,
+ * based on APIC ID
+ */
+static inline void x86_topo_ids_from_apicid_epyc(apic_id_t apicid,
+ X86CPUTopoInfo *topo_info,
+ X86CPUTopoIDs *topo_ids)
+{
+ topo_ids->smt_id = apicid &
+ ~(0xFFFFFFFFUL << apicid_smt_width(topo_info));
+ topo_ids->core_id =
+ (apicid >> apicid_core_offset(topo_info)) &
+ ~(0xFFFFFFFFUL << apicid_core_width(topo_info));
+ topo_ids->die_id =
+ (apicid >> apicid_die_offset(topo_info)) &
+ ~(0xFFFFFFFFUL << apicid_die_width(topo_info));
+ topo_ids->node_id =
+ (apicid >> apicid_node_offset_epyc(topo_info)) &
+ ~(0xFFFFFFFFUL << apicid_node_width_epyc(topo_info));
+ topo_ids->pkg_id = apicid >> apicid_pkg_offset_epyc(topo_info);
+}
+
+/*
+ * Make APIC ID for the CPU 'cpu_index'
+ *
+ * 'cpu_index' is a sequential, contiguous ID for the CPU.
+ */
+static inline apic_id_t x86_apicid_from_cpu_idx_epyc(X86CPUTopoInfo *topo_info,
+ unsigned cpu_index)
+{
+ X86CPUTopoIDs topo_ids;
+ x86_topo_ids_from_idx_epyc(topo_info, cpu_index, &topo_ids);
+ return x86_apicid_from_topo_ids_epyc(topo_info, &topo_ids);
+}
/* Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
*
* The caller must make sure core_id < nr_cores and smt_id < nr_threads.
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index d76fd0bbb1..38c2d27910 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -42,6 +42,7 @@ typedef uint32_t apic_id_t;
typedef struct X86CPUTopoIDs {
unsigned pkg_id;
+ unsigned node_id;
unsigned die_id;
unsigned core_id;
unsigned smt_id;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (7 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-24 8:52 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState Babu Moger
` (6 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Use the new functions from topology.h and delete the unused code. Given the
sockets, nodes, cores and threads, the new functions generate apic id for EPYC
mode. Removes all the hardcoded values.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
target/i386/cpu.c | 162 +++++++++++------------------------------------------
1 file changed, 35 insertions(+), 127 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 5d6edfd09b..19675eb696 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
}
}
-/*
- * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
- * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
- * Define the constants to build the cpu topology. Right now, TOPOEXT
- * feature is enabled only on EPYC. So, these constants are based on
- * EPYC supported configurations. We may need to handle the cases if
- * these values change in future.
- */
-/* Maximum core complexes in a node */
-#define MAX_CCX 2
-/* Maximum cores in a core complex */
-#define MAX_CORES_IN_CCX 4
-/* Maximum cores in a node */
-#define MAX_CORES_IN_NODE 8
-/* Maximum nodes in a socket */
-#define MAX_NODES_PER_SOCKET 4
-
-/*
- * Figure out the number of nodes required to build this config.
- * Max cores in a node is 8
- */
-static int nodes_in_socket(int nr_cores)
-{
- int nodes;
-
- nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
-
- /* Hardware does not support config with 3 nodes, return 4 in that case */
- return (nodes == 3) ? 4 : nodes;
-}
-
-/*
- * Decide the number of cores in a core complex with the given nr_cores using
- * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
- * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
- * L3 cache is shared across all cores in a core complex. So, this will also
- * tell us how many cores are sharing the L3 cache.
- */
-static int cores_in_core_complex(int nr_cores)
-{
- int nodes;
-
- /* Check if we can fit all the cores in one core complex */
- if (nr_cores <= MAX_CORES_IN_CCX) {
- return nr_cores;
- }
- /* Get the number of nodes required to build this config */
- nodes = nodes_in_socket(nr_cores);
-
- /*
- * Divide the cores accros all the core complexes
- * Return rounded up value
- */
- return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
-}
-
/* Encode cache info for CPUID[8000001D] */
-static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
- uint32_t *eax, uint32_t *ebx,
- uint32_t *ecx, uint32_t *edx)
+static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
+ X86CPUTopoInfo *topo_info,
+ uint32_t *eax, uint32_t *ebx,
+ uint32_t *ecx, uint32_t *edx)
{
uint32_t l3_cores;
+ unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
+
assert(cache->size == cache->line_size * cache->associativity *
cache->partitions * cache->sets);
@@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
/* L3 is shared among multiple cores */
if (cache->level == 3) {
- l3_cores = cores_in_core_complex(cs->nr_cores);
- *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
+ l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
+ topo_info->cores_per_die *
+ topo_info->threads_per_core),
+ nodes);
+ *eax |= (l3_cores - 1) << 14;
} else {
- *eax |= ((cs->nr_threads - 1) << 14);
+ *eax |= ((topo_info->threads_per_core - 1) << 14);
}
assert(cache->line_size > 0);
@@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
(cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
}
-/* Data structure to hold the configuration info for a given core index */
-struct core_topology {
- /* core complex id of the current core index */
- int ccx_id;
- /*
- * Adjusted core index for this core in the topology
- * This can be 0,1,2,3 with max 4 cores in a core complex
- */
- int core_id;
- /* Node id for this core index */
- int node_id;
- /* Number of nodes in this config */
- int num_nodes;
-};
-
-/*
- * Build the configuration closely match the EPYC hardware. Using the EPYC
- * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
- * right now. This could change in future.
- * nr_cores : Total number of cores in the config
- * core_id : Core index of the current CPU
- * topo : Data structure to hold all the config info for this core index
- */
-static void build_core_topology(int nr_cores, int core_id,
- struct core_topology *topo)
-{
- int nodes, cores_in_ccx;
-
- /* First get the number of nodes required */
- nodes = nodes_in_socket(nr_cores);
-
- cores_in_ccx = cores_in_core_complex(nr_cores);
-
- topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
- topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
- topo->core_id = core_id % cores_in_ccx;
- topo->num_nodes = nodes;
-}
-
/* Encode cache info for CPUID[8000001E] */
-static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
+static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
uint32_t *eax, uint32_t *ebx,
uint32_t *ecx, uint32_t *edx)
{
- struct core_topology topo = {0};
- unsigned long nodes;
+ X86CPUTopoIDs topo_ids = {0};
+ unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
int shift;
- build_core_topology(cs->nr_cores, cpu->core_id, &topo);
+ x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
+
*eax = cpu->apic_id;
/*
* CPUID_Fn8000001E_EBX
@@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
* 3 Core complex id
* 1:0 Core id
*/
- if (cs->nr_threads - 1) {
- *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
- (topo.ccx_id << 2) | topo.core_id;
- } else {
- *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
- }
+ *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
+ (topo_ids.core_id);
/*
* CPUID_Fn8000001E_ECX
* 31:11 Reserved
@@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
* 2 Socket id
* 1:0 Node id
*/
- if (topo.num_nodes <= 4) {
- *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
- topo.node_id;
+
+ if (nodes <= 4) {
+ *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
} else {
/*
* Node id fix up. Actual hardware supports up to 4 nodes. But with
@@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
* number of nodes. find_last_bit returns last set bit(0 based). Left
* shift(+1) the socket id to represent all the nodes.
*/
- nodes = topo.num_nodes - 1;
+ nodes -= 1;
shift = find_last_bit(&nodes, 8);
- *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
- topo.node_id;
+ *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
+ topo_ids.node_id;
}
*edx = 0;
}
@@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
uint32_t signature[3];
X86CPUTopoInfo topo_info;
+ topo_info.nodes_per_pkg = env->nr_nodes;
topo_info.dies_per_pkg = env->nr_dies;
topo_info.cores_per_die = cs->nr_cores;
topo_info.threads_per_core = cs->nr_threads;
@@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
}
switch (count) {
case 0: /* L1 dcache info */
- encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
- eax, ebx, ecx, edx);
+ encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
+ &topo_info, eax, ebx, ecx, edx);
break;
case 1: /* L1 icache info */
- encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
- eax, ebx, ecx, edx);
+ encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
+ &topo_info, eax, ebx, ecx, edx);
break;
case 2: /* L2 cache info */
- encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
- eax, ebx, ecx, edx);
+ encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
+ &topo_info, eax, ebx, ecx, edx);
break;
case 3: /* L3 cache info */
- encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
- eax, ebx, ecx, edx);
+ encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
+ &topo_info, eax, ebx, ecx, edx);
break;
default: /* end of info */
*eax = *ebx = *ecx = *edx = 0;
@@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
break;
case 0x8000001E:
assert(cpu->core_id <= 255);
- encode_topo_cpuid8000001e(cs, cpu,
- eax, ebx, ecx, edx);
+ encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
break;
case 0xC0000000:
*eax = env->cpuid_xlevel2;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (8 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-24 17:01 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 11/16] target/i386: Load apicid model specific handlers from X86CPUDefinition Babu Moger
` (5 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Introduce model specific apicid functions inside X86MachineState.
These functions will be loaded from X86CPUDefinition.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
include/hw/i386/x86.h | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 38c2d27910..75c2462770 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -79,6 +79,15 @@ typedef struct {
uint16_t boot_cpus;
unsigned smp_dies;
+ /* Apic id specific handlers */
+ uint32_t (*apicid_from_cpu_idx)(X86CPUTopoInfo *topo_info,
+ unsigned cpu_index);
+ void (*topo_ids_from_apicid)(apic_id_t apicid, X86CPUTopoInfo *topo_info,
+ X86CPUTopoIDs *topo_ids);
+ apic_id_t (*apicid_from_topo_ids)(X86CPUTopoInfo *topo_info,
+ const X86CPUTopoIDs *topo_ids);
+ uint32_t (*apicid_pkg_offset)(X86CPUTopoInfo *topo_info);
+
/*
* Address space used by IOAPIC device. All IOAPIC interrupts
* will be translated to MSI messages in the address space.
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 11/16] target/i386: Load apicid model specific handlers from X86CPUDefinition
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (9 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-13 18:17 ` [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState Babu Moger
` (4 subsequent siblings)
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Load the model specific handlers if available or else default handlers
will be loaded. Add the model specific handlers if apicid decoding
differs from the standard sequential numbering.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
target/i386/cpu.c | 38 ++++++++++++++++++++++++++++++++++++++
target/i386/cpu.h | 1 +
2 files changed, 39 insertions(+)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 19675eb696..389b68d765 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1614,6 +1614,16 @@ typedef struct X86CPUDefinition {
FeatureWordArray features;
const char *model_id;
CPUCaches *cache_info;
+
+ /* Apic id specific handlers */
+ uint32_t (*apicid_from_cpu_idx)(X86CPUTopoInfo *topo_info,
+ unsigned cpu_index);
+ void (*topo_ids_from_apicid)(apic_id_t apicid, X86CPUTopoInfo *topo_info,
+ X86CPUTopoIDs *topo_ids);
+ apic_id_t (*apicid_from_topo_ids)(X86CPUTopoInfo *topo_info,
+ const X86CPUTopoIDs *topo_ids);
+ uint32_t (*apicid_pkg_offset)(X86CPUTopoInfo *topo_info);
+
/*
* Definitions for alternative versions of CPU model.
* List is terminated by item with version == 0.
@@ -1654,6 +1664,34 @@ static const X86CPUVersionDefinition *x86_cpu_def_get_versions(X86CPUDefinition
return def->versions ?: default_version_list;
}
+void cpu_x86_init_apicid_fns(MachineState *machine)
+{
+ X86CPUClass *xcc = X86_CPU_CLASS(object_class_by_name(machine->cpu_type));
+ X86CPUModel *model = xcc->model;
+ X86CPUDefinition *def = model->cpudef;
+ X86MachineState *x86ms = X86_MACHINE(machine);
+
+ if (def) {
+ if (def->apicid_from_cpu_idx) {
+ x86ms->apicid_from_cpu_idx = def->apicid_from_cpu_idx;
+ }
+ if (def->topo_ids_from_apicid) {
+ x86ms->topo_ids_from_apicid = def->topo_ids_from_apicid;
+ }
+ if (def->apicid_from_topo_ids) {
+ x86ms->apicid_from_topo_ids = def->apicid_from_topo_ids;
+ }
+ if (def->apicid_pkg_offset) {
+ x86ms->apicid_pkg_offset = def->apicid_pkg_offset;
+ }
+ } else {
+ x86ms->apicid_from_cpu_idx = x86_apicid_from_cpu_idx;
+ x86ms->topo_ids_from_apicid = x86_topo_ids_from_apicid;
+ x86ms->apicid_from_topo_ids = x86_apicid_from_topo_ids;
+ x86ms->apicid_pkg_offset = apicid_pkg_offset;
+ }
+}
+
static CPUCaches epyc_cache_info = {
.l1d_cache = &(CPUCacheInfo) {
.type = DATA_CACHE,
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 627a8cb9be..64a1cca690 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1889,6 +1889,7 @@ void cpu_clear_apic_feature(CPUX86State *env);
void host_cpuid(uint32_t function, uint32_t count,
uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
void host_vendor_fms(char *vendor, int *family, int *model, int *stepping);
+void cpu_x86_init_apicid_fns(MachineState *machine);
/* helper.c */
bool x86_cpu_tlb_fill(CPUState *cs, vaddr address, int size,
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (10 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 11/16] target/i386: Load apicid model specific handlers from X86CPUDefinition Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-24 17:19 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 13/16] target/i386: Add EPYC model specific handlers Babu Moger
` (3 subsequent siblings)
15 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Check and Load the apicid handlers from X86CPUDefinition if available.
Update the calling convention for the apicid handlers.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
hw/i386/pc.c | 6 +++---
hw/i386/x86.c | 27 +++++++++++++++++++++++----
2 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index be72a49716..93063af6a8 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1808,14 +1808,14 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
topo_ids.die_id = cpu->die_id;
topo_ids.core_id = cpu->core_id;
topo_ids.smt_id = cpu->thread_id;
- cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
+ cpu->apic_id = x86ms->apicid_from_topo_ids(&topo_info, &topo_ids);
}
cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
if (!cpu_slot) {
MachineState *ms = MACHINE(pcms);
- x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+ x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
error_setg(errp,
"Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with"
" APIC ID %" PRIu32 ", valid index range 0:%d",
@@ -1836,7 +1836,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
/* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
* once -smp refactoring is complete and there will be CPU private
* CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
- x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
+ x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
" 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo_ids.pkg_id);
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 3d944f68e6..b825861b85 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -52,6 +52,22 @@
/* Physical Address of PVH entry point read from kernel ELF NOTE */
static size_t pvh_start_addr;
+/*
+ * Check for apicid handlers in X86MachineState. Load them if
+ * not loaded already. These handlers are loaded from X86CPUDefinition.
+ */
+static void x86_check_apicid_handlers(MachineState *ms)
+{
+ X86MachineState *x86ms = X86_MACHINE(ms);
+
+ if (!x86ms->apicid_from_cpu_idx ||
+ !x86ms->topo_ids_from_apicid ||
+ !x86ms->apicid_from_topo_ids ||
+ !x86ms->apicid_pkg_offset) {
+ cpu_x86_init_apicid_fns(ms);
+ }
+}
+
/*
* Calculates initial APIC ID for a specific CPU index
*
@@ -70,7 +86,7 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
init_topo_info(&topo_info, x86ms);
- correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
+ correct_id = x86ms->apicid_from_cpu_idx(&topo_info, cpu_index);
if (x86mc->compat_apic_id_mode) {
if (cpu_index != correct_id && !warned && !qtest_enabled()) {
error_report("APIC IDs set in compatibility mode, "
@@ -148,8 +164,8 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
init_topo_info(&topo_info, x86ms);
assert(idx < ms->possible_cpus->len);
- x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
- &topo_info, &topo_ids);
+ x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
+ &topo_info, &topo_ids);
return topo_ids.pkg_id % ms->numa_state->num_nodes;
}
@@ -169,6 +185,9 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
return ms->possible_cpus;
}
+ /* Initialize apicid handlers */
+ x86_check_apicid_handlers(ms);
+
ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
sizeof(CPUArchId) * max_cpus);
ms->possible_cpus->len = max_cpus;
@@ -182,7 +201,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
ms->possible_cpus->cpus[i].vcpus_count = 1;
ms->possible_cpus->cpus[i].arch_id =
x86_cpu_apic_id_from_index(x86ms, i);
- x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
+ x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
&topo_info, &topo_ids);
ms->possible_cpus->cpus[i].props.has_socket_id = true;
ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 13/16] target/i386: Add EPYC model specific handlers
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (11 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-13 18:17 ` [PATCH v4 14/16] hw/i386: Move arch_id decode inside x86_cpus_init Babu Moger
` (2 subsequent siblings)
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Add the new EPYC model specific handlers to fix the apicid decoding.
The APIC ID is decoded based on the sequence sockets->dies->cores->threads.
This works fine for most standard AMD and other vendors' configurations,
but this decoding sequence does not follow that of AMD's APIC ID enumeration
strictly. In some cases this can cause CPU topology inconsistency.
When booting a guest VM, the kernel tries to validate the topology, and finds
it inconsistent with the enumeration of EPYC cpu models. The more details are
in the bug https://bugzilla.redhat.com/show_bug.cgi?id=1728166.
To fix the problem we need to build the topology as per the Processor
Programming Reference (PPR) for AMD Family 17h Model 01h, Revision B1
Processors.
It is available at https://www.amd.com/system/files/TechDocs/55570-B1_PUB.zip
Here is the text from the PPR.
Operating systems are expected to use Core::X86::Cpuid::SizeId[ApicIdSize], the
number of least significant bits in the Initial APIC ID that indicate core ID
within a processor, in constructing per-core CPUID masks.
Core::X86::Cpuid::SizeId[ApicIdSize] determines the maximum number of cores
(MNC) that the processor could theoretically support, not the actual number of
cores that are actually implemented or enabled on the processor, as indicated
by Core::X86::Cpuid::SizeId[NC].
Each Core::X86::Apic::ApicId[ApicId] register is preset as follows:
• ApicId[6] = Socket ID.
• ApicId[5:4] = Node ID.
• ApicId[3] = Logical CCX L3 complex ID
• ApicId[2:0]= (SMT) ? {LogicalCoreID[1:0],ThreadId} : {1'b0,LogicalCoreID[1:0]}
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
target/i386/cpu.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 389b68d765..082865d72b 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3835,6 +3835,10 @@ static X86CPUDefinition builtin_x86_defs[] = {
.xlevel = 0x8000001E,
.model_id = "AMD EPYC Processor",
.cache_info = &epyc_cache_info,
+ .apicid_from_cpu_idx = x86_apicid_from_cpu_idx_epyc,
+ .topo_ids_from_apicid = x86_topo_ids_from_apicid_epyc,
+ .apicid_from_topo_ids = x86_apicid_from_topo_ids_epyc,
+ .apicid_pkg_offset = apicid_pkg_offset_epyc,
.versions = (X86CPUVersionDefinition[]) {
{ .version = 1 },
{
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 14/16] hw/i386: Move arch_id decode inside x86_cpus_init
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (12 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 13/16] target/i386: Add EPYC model specific handlers Babu Moger
@ 2020-02-13 18:17 ` Babu Moger
2020-02-13 18:18 ` [PATCH v4 15/16] i386: Fix pkg_id offset for EPYC cpu models Babu Moger
2020-02-13 18:18 ` [PATCH v4 16/16] tests: Update the Unit tests Babu Moger
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:17 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Apicid calculation depends on knowing the total number of numa nodes
for EPYC cpu models. Right now, we are calculating the arch_id while
parsing the numa(parse_numa). At this time, it is not known how many
total numa nodes are configured in the system.
Move the arch_id inside x86_cpus_init. At this time smp parameter is already
completed and numa node information is available.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
hw/i386/x86.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index b825861b85..472dc3eb77 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -127,6 +127,9 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
MachineState *ms = MACHINE(x86ms);
MachineClass *mc = MACHINE_GET_CLASS(x86ms);
+ /* Initialize apicid handlers first */
+ x86_check_apicid_handlers(ms);
+
x86_cpu_set_default_version(default_cpu_version);
/*
@@ -140,6 +143,11 @@ void x86_cpus_init(X86MachineState *x86ms, int default_cpu_version)
x86ms->apic_id_limit = x86_cpu_apic_id_from_index(x86ms,
ms->smp.max_cpus - 1) + 1;
possible_cpus = mc->possible_cpu_arch_ids(ms);
+
+ for (i = 0; i < ms->smp.cpus; i++) {
+ ms->possible_cpus->cpus[i].arch_id = x86_cpu_apic_id_from_index(x86ms, i);
+ }
+
for (i = 0; i < ms->smp.cpus; i++) {
x86_cpu_new(x86ms, possible_cpus->cpus[i].arch_id, &error_fatal);
}
@@ -164,8 +172,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
init_topo_info(&topo_info, x86ms);
assert(idx < ms->possible_cpus->len);
- x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
- &topo_info, &topo_ids);
+ x86_topo_ids_from_idx(&topo_info, idx, &topo_ids);
return topo_ids.pkg_id % ms->numa_state->num_nodes;
}
@@ -199,10 +206,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
ms->possible_cpus->cpus[i].type = ms->cpu_type;
ms->possible_cpus->cpus[i].vcpus_count = 1;
- ms->possible_cpus->cpus[i].arch_id =
- x86_cpu_apic_id_from_index(x86ms, i);
- x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
- &topo_info, &topo_ids);
+ x86_topo_ids_from_idx(&topo_info, i, &topo_ids);
ms->possible_cpus->cpus[i].props.has_socket_id = true;
ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
if (x86ms->smp_dies > 1) {
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 15/16] i386: Fix pkg_id offset for EPYC cpu models
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (13 preceding siblings ...)
2020-02-13 18:17 ` [PATCH v4 14/16] hw/i386: Move arch_id decode inside x86_cpus_init Babu Moger
@ 2020-02-13 18:18 ` Babu Moger
2020-02-13 18:18 ` [PATCH v4 16/16] tests: Update the Unit tests Babu Moger
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:18 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
If the system is numa configured the pkg_offset needs
to be adjusted for EPYC cpu models. Fix it calling the
model specific handler.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
hw/i386/pc.c | 1 +
hw/i386/x86.c | 4 ++++
target/i386/cpu.c | 4 ++--
target/i386/cpu.h | 1 +
4 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 93063af6a8..e56780b1f0 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1754,6 +1754,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
env->nr_dies = x86ms->smp_dies;
env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
+ env->pkg_offset = x86ms->apicid_pkg_offset(&topo_info);
/*
* If APIC ID is not set,
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 472dc3eb77..99f2a67d69 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -106,12 +106,16 @@ void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
Error *local_err = NULL;
CPUX86State *env = NULL;
MachineState *ms = MACHINE(x86ms);
+ X86CPUTopoInfo topo_info;
+
+ init_topo_info(&topo_info, x86ms);
cpu = object_new(MACHINE(x86ms)->cpu_type);
env = &X86_CPU(cpu)->env;
env->nr_dies = x86ms->smp_dies;
env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
+ env->pkg_offset = x86ms->apicid_pkg_offset(&topo_info);
object_property_set_uint(cpu, apic_id, "apic-id", &local_err);
object_property_set_bool(cpu, true, "realized", &local_err);
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 082865d72b..b793da7c6e 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5454,7 +5454,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
*ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
break;
case 1:
- *eax = apicid_pkg_offset(&topo_info);
+ *eax = env->pkg_offset;
*ebx = cs->nr_cores * cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
break;
@@ -5488,7 +5488,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
*ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
break;
case 2:
- *eax = apicid_pkg_offset(&topo_info);
+ *eax = env->pkg_offset;
*ebx = env->nr_dies * cs->nr_cores * cs->nr_threads;
*ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
break;
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 64a1cca690..641e1cacdf 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1604,6 +1604,7 @@ typedef struct CPUX86State {
unsigned nr_dies;
unsigned nr_nodes;
+ unsigned pkg_offset;
} CPUX86State;
struct kvm_msrs;
^ permalink raw reply related [flat|nested] 44+ messages in thread
* [PATCH v4 16/16] tests: Update the Unit tests
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
` (14 preceding siblings ...)
2020-02-13 18:18 ` [PATCH v4 15/16] i386: Fix pkg_id offset for EPYC cpu models Babu Moger
@ 2020-02-13 18:18 ` Babu Moger
15 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-13 18:18 UTC (permalink / raw)
To: ehabkost, marcel.apfelbaum, pbonzini, rth, mst, imammedo; +Cc: qemu-devel
Since the topology routines have changed, update
the unit tests to use the new APIs.
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
tests/test-x86-cpuid.c | 115 ++++++++++++++++++++++++++++--------------------
1 file changed, 68 insertions(+), 47 deletions(-)
diff --git a/tests/test-x86-cpuid.c b/tests/test-x86-cpuid.c
index 1942287f33..00553c1d77 100644
--- a/tests/test-x86-cpuid.c
+++ b/tests/test-x86-cpuid.c
@@ -28,79 +28,100 @@
static void test_topo_bits(void)
{
+ X86CPUTopoInfo topo_info = {0};
+
/* simple tests for 1 thread per core, 1 core per die, 1 die per package */
- g_assert_cmpuint(apicid_smt_width(1, 1, 1), ==, 0);
- g_assert_cmpuint(apicid_core_width(1, 1, 1), ==, 0);
- g_assert_cmpuint(apicid_die_width(1, 1, 1), ==, 0);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 1};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 0);
+ g_assert_cmpuint(apicid_core_width(&topo_info), ==, 0);
+ g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 1, 1, 0), ==, 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 1, 1, 1), ==, 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 1, 1, 2), ==, 2);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 1, 1, 3), ==, 3);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 3), ==, 3);
/* Test field width calculation for multiple values
*/
- g_assert_cmpuint(apicid_smt_width(1, 1, 2), ==, 1);
- g_assert_cmpuint(apicid_smt_width(1, 1, 3), ==, 2);
- g_assert_cmpuint(apicid_smt_width(1, 1, 4), ==, 2);
-
- g_assert_cmpuint(apicid_smt_width(1, 1, 14), ==, 4);
- g_assert_cmpuint(apicid_smt_width(1, 1, 15), ==, 4);
- g_assert_cmpuint(apicid_smt_width(1, 1, 16), ==, 4);
- g_assert_cmpuint(apicid_smt_width(1, 1, 17), ==, 5);
-
-
- g_assert_cmpuint(apicid_core_width(1, 30, 2), ==, 5);
- g_assert_cmpuint(apicid_core_width(1, 31, 2), ==, 5);
- g_assert_cmpuint(apicid_core_width(1, 32, 2), ==, 5);
- g_assert_cmpuint(apicid_core_width(1, 33, 2), ==, 6);
-
- g_assert_cmpuint(apicid_die_width(1, 30, 2), ==, 0);
- g_assert_cmpuint(apicid_die_width(2, 30, 2), ==, 1);
- g_assert_cmpuint(apicid_die_width(3, 30, 2), ==, 2);
- g_assert_cmpuint(apicid_die_width(4, 30, 2), ==, 2);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 2};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 1);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 3};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 4};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
+
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 14};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 15};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 16};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 4);
+ topo_info = (X86CPUTopoInfo) {0, 1, 1, 17};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 5);
+
+
+ topo_info = (X86CPUTopoInfo) {0, 1, 30, 2};
+ g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
+ topo_info = (X86CPUTopoInfo) {0, 1, 31, 2};
+ g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
+ topo_info = (X86CPUTopoInfo) {0, 1, 32, 2};
+ g_assert_cmpuint(apicid_core_width(&topo_info), ==, 5);
+ topo_info = (X86CPUTopoInfo) {0, 1, 33, 2};
+ g_assert_cmpuint(apicid_core_width(&topo_info), ==, 6);
+
+ topo_info = (X86CPUTopoInfo) {0, 1, 30, 2};
+ g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0);
+ topo_info = (X86CPUTopoInfo) {0, 2, 30, 2};
+ g_assert_cmpuint(apicid_die_width(&topo_info), ==, 1);
+ topo_info = (X86CPUTopoInfo) {0, 3, 30, 2};
+ g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
+ topo_info = (X86CPUTopoInfo) {0, 4, 30, 2};
+ g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);
/* build a weird topology and see if IDs are calculated correctly
*/
/* This will use 2 bits for thread ID and 3 bits for core ID
*/
- g_assert_cmpuint(apicid_smt_width(1, 6, 3), ==, 2);
- g_assert_cmpuint(apicid_core_offset(1, 6, 3), ==, 2);
- g_assert_cmpuint(apicid_die_offset(1, 6, 3), ==, 5);
- g_assert_cmpuint(apicid_pkg_offset(1, 6, 3), ==, 5);
-
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 0), ==, 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 1), ==, 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 2), ==, 2);
-
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 1 * 3 + 0), ==,
+ topo_info = (X86CPUTopoInfo) {0, 1, 6, 3};
+ g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2);
+ g_assert_cmpuint(apicid_core_offset(&topo_info), ==, 2);
+ g_assert_cmpuint(apicid_die_offset(&topo_info), ==, 5);
+ g_assert_cmpuint(apicid_pkg_offset(&topo_info), ==, 5);
+
+ topo_info = (X86CPUTopoInfo) {0, 1, 6, 3};
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 0), ==, 0);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1), ==, 1);
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2), ==, 2);
+
+ topo_info = (X86CPUTopoInfo) {0, 1, 6, 3};
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 0), ==,
(1 << 2) | 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 1 * 3 + 1), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 1), ==,
(1 << 2) | 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 1 * 3 + 2), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 1 * 3 + 2), ==,
(1 << 2) | 2);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 2 * 3 + 0), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 0), ==,
(2 << 2) | 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 2 * 3 + 1), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 1), ==,
(2 << 2) | 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 2 * 3 + 2), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 2 * 3 + 2), ==,
(2 << 2) | 2);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 5 * 3 + 0), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 0), ==,
(5 << 2) | 0);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 5 * 3 + 1), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 1), ==,
(5 << 2) | 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3, 5 * 3 + 2), ==,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info, 5 * 3 + 2), ==,
(5 << 2) | 2);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
1 * 6 * 3 + 0 * 3 + 0), ==, (1 << 5));
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
1 * 6 * 3 + 1 * 3 + 1), ==, (1 << 5) | (1 << 2) | 1);
- g_assert_cmpuint(x86_apicid_from_cpu_idx(1, 6, 3,
+ g_assert_cmpuint(x86_apicid_from_cpu_idx(&topo_info,
3 * 6 * 3 + 5 * 3 + 2), ==, (3 << 5) | (5 << 2) | 2);
}
^ permalink raw reply related [flat|nested] 44+ messages in thread
* Re: [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids
2020-02-13 18:17 ` [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids Babu Moger
@ 2020-02-20 12:43 ` Igor Mammedov
0 siblings, 0 replies; 44+ messages in thread
From: Igor Mammedov @ 2020-02-20 12:43 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:12 -0600
Babu Moger <babu.moger@amd.com> wrote:
> For consistancy rename apicid_from_topo_ids to x86_apicid_from_topo_ids.
> No functional change.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> hw/i386/pc.c | 2 +-
> include/hw/i386/topology.h | 6 +++---
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 02fdb3d506..be72a49716 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1808,7 +1808,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> topo_ids.die_id = cpu->die_id;
> topo_ids.core_id = cpu->core_id;
> topo_ids.smt_id = cpu->thread_id;
> - cpu->apic_id = apicid_from_topo_ids(&topo_info, &topo_ids);
> + cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
> }
>
> cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
> index 522c77e6a9..3158157430 100644
> --- a/include/hw/i386/topology.h
> +++ b/include/hw/i386/topology.h
> @@ -107,8 +107,8 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
> *
> * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
> */
> -static inline apic_id_t apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
> - const X86CPUTopoIDs *topo_ids)
> +static inline apic_id_t x86_apicid_from_topo_ids(X86CPUTopoInfo *topo_info,
> + const X86CPUTopoIDs *topo_ids)
> {
> return (topo_ids->pkg_id << apicid_pkg_offset(topo_info)) |
> (topo_ids->die_id << apicid_die_offset(topo_info)) |
> @@ -160,7 +160,7 @@ static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info,
> {
> X86CPUTopoIDs topo_ids;
> x86_topo_ids_from_idx(topo_info, cpu_index, &topo_ids);
> - return apicid_from_topo_ids(topo_info, &topo_ids);
> + return x86_apicid_from_topo_ids(topo_info, &topo_ids);
> }
>
> #endif /* HW_I386_TOPOLOGY_H */
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
2020-02-13 18:16 ` [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo Babu Moger
@ 2020-02-21 17:05 ` Igor Mammedov
2020-02-21 17:51 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-21 17:05 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:16:51 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Initialize all the parameters in one function init_topo_info.
is it possible to squash it in 2/16
>
> Move the data structure X86CPUTopoIDs and X86CPUTopoInfo into
> x86.h.
A reason why it's moved should be here.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> hw/i386/pc.c | 4 +---
> hw/i386/x86.c | 14 +++-----------
> include/hw/i386/topology.h | 26 ++++++++++----------------
> include/hw/i386/x86.h | 17 +++++++++++++++++
> 4 files changed, 31 insertions(+), 30 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 2adf7f6afa..9803413dd9 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1749,9 +1749,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> return;
> }
>
> - topo_info.dies_per_pkg = x86ms->smp_dies;
> - topo_info.cores_per_die = smp_cores;
> - topo_info.threads_per_core = smp_threads;
> + init_topo_info(&topo_info, x86ms);
>
> env->nr_dies = x86ms->smp_dies;
>
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index f18cab8e5c..083effb2f5 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -63,15 +63,12 @@ static size_t pvh_start_addr;
> uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
> unsigned int cpu_index)
> {
> - MachineState *ms = MACHINE(x86ms);
> X86MachineClass *x86mc = X86_MACHINE_GET_CLASS(x86ms);
> X86CPUTopoInfo topo_info;
> uint32_t correct_id;
> static bool warned;
>
> - topo_info.dies_per_pkg = x86ms->smp_dies;
> - topo_info.cores_per_die = ms->smp.cores;
> - topo_info.threads_per_core = ms->smp.threads;
> + init_topo_info(&topo_info, x86ms);
>
> correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
> if (x86mc->compat_apic_id_mode) {
> @@ -146,10 +143,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
> X86MachineState *x86ms = X86_MACHINE(ms);
> X86CPUTopoInfo topo_info;
>
> - topo_info.dies_per_pkg = x86ms->smp_dies;
> - topo_info.cores_per_die = ms->smp.cores;
> - topo_info.threads_per_core = ms->smp.threads;
> -
> + init_topo_info(&topo_info, x86ms);
>
> assert(idx < ms->possible_cpus->len);
> x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
> @@ -177,9 +171,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
> sizeof(CPUArchId) * max_cpus);
> ms->possible_cpus->len = max_cpus;
>
> - topo_info.dies_per_pkg = x86ms->smp_dies;
> - topo_info.cores_per_die = ms->smp.cores;
> - topo_info.threads_per_core = ms->smp.threads;
> + init_topo_info(&topo_info, x86ms);
>
> for (i = 0; i < ms->possible_cpus->len; i++) {
> X86CPUTopoIDs topo_ids;
> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
> index ba52d49079..ef0ab0b6a3 100644
> --- a/include/hw/i386/topology.h
> +++ b/include/hw/i386/topology.h
> @@ -40,23 +40,17 @@
>
>
> #include "qemu/bitops.h"
> +#include "hw/i386/x86.h"
>
> -/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
> - */
> -typedef uint32_t apic_id_t;
> -
> -typedef struct X86CPUTopoIDs {
> - unsigned pkg_id;
> - unsigned die_id;
> - unsigned core_id;
> - unsigned smt_id;
> -} X86CPUTopoIDs;
> -
> -typedef struct X86CPUTopoInfo {
> - unsigned dies_per_pkg;
> - unsigned cores_per_die;
> - unsigned threads_per_core;
> -} X86CPUTopoInfo;
> +static inline void init_topo_info(X86CPUTopoInfo *topo_info,
> + const X86MachineState *x86ms)
> +{
> + MachineState *ms = MACHINE(x86ms);
> +
> + topo_info->dies_per_pkg = x86ms->smp_dies;
> + topo_info->cores_per_die = ms->smp.cores;
> + topo_info->threads_per_core = ms->smp.threads;
> +}
>
> /* Return the bit width needed for 'count' IDs
> */
> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> index 4b84917885..ad62b01cf2 100644
> --- a/include/hw/i386/x86.h
> +++ b/include/hw/i386/x86.h
> @@ -36,6 +36,23 @@ typedef struct {
> bool compat_apic_id_mode;
> } X86MachineClass;
>
> +/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
> + */
> +typedef uint32_t apic_id_t;
> +
> +typedef struct X86CPUTopoIDs {
> + unsigned pkg_id;
> + unsigned die_id;
> + unsigned core_id;
> + unsigned smt_id;
> +} X86CPUTopoIDs;
> +
> +typedef struct X86CPUTopoInfo {
> + unsigned dies_per_pkg;
> + unsigned cores_per_die;
> + unsigned threads_per_core;
> +} X86CPUTopoInfo;
> +
> typedef struct {
> /*< private >*/
> MachineState parent;
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
2020-02-21 17:05 ` Igor Mammedov
@ 2020-02-21 17:51 ` Babu Moger
2020-02-24 8:18 ` Igor Mammedov
0 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-21 17:51 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/21/20 11:05 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:16:51 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> Initialize all the parameters in one function init_topo_info.
>
> is it possible to squash it in 2/16
>
Sure. We can do that.
>
>>
>> Move the data structure X86CPUTopoIDs and X86CPUTopoInfo into
>> x86.h.
> A reason why it's moved should be here.
Apicid functions will be part of X86MachineState data structure(patches
introduced later).These functions will use X86CPUTopoIDs and
X86CPUTopoInfo definition. Will add these details. Thanks
>
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
>> ---
>> hw/i386/pc.c | 4 +---
>> hw/i386/x86.c | 14 +++-----------
>> include/hw/i386/topology.h | 26 ++++++++++----------------
>> include/hw/i386/x86.h | 17 +++++++++++++++++
>> 4 files changed, 31 insertions(+), 30 deletions(-)
>>
>> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> index 2adf7f6afa..9803413dd9 100644
>> --- a/hw/i386/pc.c
>> +++ b/hw/i386/pc.c
>> @@ -1749,9 +1749,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
>> return;
>> }
>>
>> - topo_info.dies_per_pkg = x86ms->smp_dies;
>> - topo_info.cores_per_die = smp_cores;
>> - topo_info.threads_per_core = smp_threads;
>> + init_topo_info(&topo_info, x86ms);
>>
>> env->nr_dies = x86ms->smp_dies;
>>
>> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
>> index f18cab8e5c..083effb2f5 100644
>> --- a/hw/i386/x86.c
>> +++ b/hw/i386/x86.c
>> @@ -63,15 +63,12 @@ static size_t pvh_start_addr;
>> uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
>> unsigned int cpu_index)
>> {
>> - MachineState *ms = MACHINE(x86ms);
>> X86MachineClass *x86mc = X86_MACHINE_GET_CLASS(x86ms);
>> X86CPUTopoInfo topo_info;
>> uint32_t correct_id;
>> static bool warned;
>>
>> - topo_info.dies_per_pkg = x86ms->smp_dies;
>> - topo_info.cores_per_die = ms->smp.cores;
>> - topo_info.threads_per_core = ms->smp.threads;
>> + init_topo_info(&topo_info, x86ms);
>>
>> correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
>> if (x86mc->compat_apic_id_mode) {
>> @@ -146,10 +143,7 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
>> X86MachineState *x86ms = X86_MACHINE(ms);
>> X86CPUTopoInfo topo_info;
>>
>> - topo_info.dies_per_pkg = x86ms->smp_dies;
>> - topo_info.cores_per_die = ms->smp.cores;
>> - topo_info.threads_per_core = ms->smp.threads;
>> -
>> + init_topo_info(&topo_info, x86ms);
>>
>> assert(idx < ms->possible_cpus->len);
>> x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
>> @@ -177,9 +171,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
>> sizeof(CPUArchId) * max_cpus);
>> ms->possible_cpus->len = max_cpus;
>>
>> - topo_info.dies_per_pkg = x86ms->smp_dies;
>> - topo_info.cores_per_die = ms->smp.cores;
>> - topo_info.threads_per_core = ms->smp.threads;
>> + init_topo_info(&topo_info, x86ms);
>>
>> for (i = 0; i < ms->possible_cpus->len; i++) {
>> X86CPUTopoIDs topo_ids;
>> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
>> index ba52d49079..ef0ab0b6a3 100644
>> --- a/include/hw/i386/topology.h
>> +++ b/include/hw/i386/topology.h
>> @@ -40,23 +40,17 @@
>>
>>
>> #include "qemu/bitops.h"
>> +#include "hw/i386/x86.h"
>>
>> -/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
>> - */
>> -typedef uint32_t apic_id_t;
>> -
>> -typedef struct X86CPUTopoIDs {
>> - unsigned pkg_id;
>> - unsigned die_id;
>> - unsigned core_id;
>> - unsigned smt_id;
>> -} X86CPUTopoIDs;
>> -
>> -typedef struct X86CPUTopoInfo {
>> - unsigned dies_per_pkg;
>> - unsigned cores_per_die;
>> - unsigned threads_per_core;
>> -} X86CPUTopoInfo;
>> +static inline void init_topo_info(X86CPUTopoInfo *topo_info,
>> + const X86MachineState *x86ms)
>> +{
>> + MachineState *ms = MACHINE(x86ms);
>> +
>> + topo_info->dies_per_pkg = x86ms->smp_dies;
>> + topo_info->cores_per_die = ms->smp.cores;
>> + topo_info->threads_per_core = ms->smp.threads;
>> +}
>>
>> /* Return the bit width needed for 'count' IDs
>> */
>> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
>> index 4b84917885..ad62b01cf2 100644
>> --- a/include/hw/i386/x86.h
>> +++ b/include/hw/i386/x86.h
>> @@ -36,6 +36,23 @@ typedef struct {
>> bool compat_apic_id_mode;
>> } X86MachineClass;
>>
>> +/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
>> + */
>> +typedef uint32_t apic_id_t;
>> +
>> +typedef struct X86CPUTopoIDs {
>> + unsigned pkg_id;
>> + unsigned die_id;
>> + unsigned core_id;
>> + unsigned smt_id;
>> +} X86CPUTopoIDs;
>> +
>> +typedef struct X86CPUTopoInfo {
>> + unsigned dies_per_pkg;
>> + unsigned cores_per_die;
>> + unsigned threads_per_core;
>> +} X86CPUTopoInfo;
>> +
>> typedef struct {
>> /*< private >*/
>> MachineState parent;
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
2020-02-21 17:51 ` Babu Moger
@ 2020-02-24 8:18 ` Igor Mammedov
2020-02-24 16:54 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 8:18 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Fri, 21 Feb 2020 11:51:15 -0600
Babu Moger <babu.moger@amd.com> wrote:
> On 2/21/20 11:05 AM, Igor Mammedov wrote:
> > On Thu, 13 Feb 2020 12:16:51 -0600
> > Babu Moger <babu.moger@amd.com> wrote:
> >
> >> Initialize all the parameters in one function init_topo_info.
> >
> > is it possible to squash it in 2/16
> >
> Sure. We can do that.
> >
> >>
> >> Move the data structure X86CPUTopoIDs and X86CPUTopoInfo into
> >> x86.h.
> > A reason why it's moved should be here.
>
> Apicid functions will be part of X86MachineState data structure(patches
> introduced later).These functions will use X86CPUTopoIDs and
> X86CPUTopoInfo definition. Will add these details. Thanks
why not just include topology.h into the X86MachineState header,
and keep topo structures/functions where they are now?
(I dislike a little scattering consolidated pieces across multiple files,
but what worries me more is that it makes target/i386/cpu.c via
topology.h -> x86.h chain pull in a lot of unrelated dependencies)
So I'd keep X86CPUTopoIDs and X86CPUTopoInfo in topology.h
[...]
> >> +static inline void init_topo_info(X86CPUTopoInfo *topo_info,
> >> + const X86MachineState *x86ms)
> >> +{
> >> + MachineState *ms = MACHINE(x86ms);
> >> +
> >> + topo_info->dies_per_pkg = x86ms->smp_dies;
> >> + topo_info->cores_per_die = ms->smp.cores;
> >> + topo_info->threads_per_core = ms->smp.threads;
> >> +}
this is pure machine specific helper, and aren't used anywhere else
beside machine code.
Suggest to put it in pc.c or x86.c to keep topology.h machine independent.
> >>
> >> /* Return the bit width needed for 'count' IDs
> >> */
> >> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> >> index 4b84917885..ad62b01cf2 100644
> >> --- a/include/hw/i386/x86.h
> >> +++ b/include/hw/i386/x86.h
> >> @@ -36,6 +36,23 @@ typedef struct {
> >> bool compat_apic_id_mode;
> >> } X86MachineClass;
> >>
> >> +/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
> >> + */
> >> +typedef uint32_t apic_id_t;
> >> +
> >> +typedef struct X86CPUTopoIDs {
> >> + unsigned pkg_id;
> >> + unsigned die_id;
> >> + unsigned core_id;
> >> + unsigned smt_id;
> >> +} X86CPUTopoIDs;
> >> +
> >> +typedef struct X86CPUTopoInfo {
> >> + unsigned dies_per_pkg;
> >> + unsigned cores_per_die;
> >> + unsigned threads_per_core;
> >> +} X86CPUTopoInfo;
> >> +
> >> typedef struct {
> >> /*< private >*/
> >> MachineState parent;
> >>
> >
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg
2020-02-13 18:17 ` [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg Babu Moger
@ 2020-02-24 8:34 ` Igor Mammedov
2020-02-24 17:12 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 8:34 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:04 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Update structures X86CPUTopoIDs and CPUX86State to hold the nodes_per_pkg.
> This is required to build EPYC mode topology.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> hw/i386/pc.c | 1 +
> hw/i386/x86.c | 2 ++
> include/hw/i386/topology.h | 2 ++
> include/hw/i386/x86.h | 1 +
> target/i386/cpu.c | 1 +
> target/i386/cpu.h | 1 +
> 6 files changed, 8 insertions(+)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index f13721ac43..02fdb3d506 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1753,6 +1753,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> init_topo_info(&topo_info, x86ms);
>
> env->nr_dies = x86ms->smp_dies;
> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
it would be better if calculation would result in valid result
so you won't have to later scatter MAX(env->nr_nodes, 1) everywhere.
also I'd use earlier intialized:
env->nr_nodes = topo_info->nodes_per_pkg
to avoid repeating calculation
> /*
> * If APIC ID is not set,
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index 083effb2f5..3d944f68e6 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -89,11 +89,13 @@ void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
> Object *cpu = NULL;
> Error *local_err = NULL;
> CPUX86State *env = NULL;
> + MachineState *ms = MACHINE(x86ms);
>
> cpu = object_new(MACHINE(x86ms)->cpu_type);
>
> env = &X86_CPU(cpu)->env;
> env->nr_dies = x86ms->smp_dies;
> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
Is this really necessary? (I think pc_cpu_pre_plug should take care of setting it)
> object_property_set_uint(cpu, apic_id, "apic-id", &local_err);
> object_property_set_bool(cpu, true, "realized", &local_err);
> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
> index ef0ab0b6a3..522c77e6a9 100644
> --- a/include/hw/i386/topology.h
> +++ b/include/hw/i386/topology.h
> @@ -41,12 +41,14 @@
>
> #include "qemu/bitops.h"
> #include "hw/i386/x86.h"
> +#include "sysemu/numa.h"
>
> static inline void init_topo_info(X86CPUTopoInfo *topo_info,
> const X86MachineState *x86ms)
> {
> MachineState *ms = MACHINE(x86ms);
>
> + topo_info->nodes_per_pkg = ms->numa_state->num_nodes / ms->smp.sockets;
> topo_info->dies_per_pkg = x86ms->smp_dies;
> topo_info->cores_per_die = ms->smp.cores;
> topo_info->threads_per_core = ms->smp.threads;
> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> index ad62b01cf2..d76fd0bbb1 100644
> --- a/include/hw/i386/x86.h
> +++ b/include/hw/i386/x86.h
> @@ -48,6 +48,7 @@ typedef struct X86CPUTopoIDs {
> } X86CPUTopoIDs;
>
> typedef struct X86CPUTopoInfo {
> + unsigned nodes_per_pkg;
> unsigned dies_per_pkg;
> unsigned cores_per_die;
> unsigned threads_per_core;
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 7e630f47ac..5d6edfd09b 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6761,6 +6761,7 @@ static void x86_cpu_initfn(Object *obj)
> FeatureWord w;
>
> env->nr_dies = 1;
> + env->nr_nodes = 1;
> cpu_set_cpustate_pointers(cpu);
>
> object_property_add(obj, "family", "int",
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index af282936a7..627a8cb9be 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -1603,6 +1603,7 @@ typedef struct CPUX86State {
> TPRAccess tpr_access_type;
>
> unsigned nr_dies;
> + unsigned nr_nodes;
> } CPUX86State;
>
> struct kvm_msrs;
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology
2020-02-13 18:16 ` [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology Babu Moger
@ 2020-02-24 8:37 ` Igor Mammedov
0 siblings, 0 replies; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 8:37 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:16:58 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Store the smp sockets in CpuTopology. The socket information required to
> build the apic id in EPYC mode. Right now socket information is not passed
> to down when decoding the apic id. Add the socket information here.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> hw/core/machine.c | 1 +
> hw/i386/pc.c | 1 +
> include/hw/boards.h | 2 ++
> vl.c | 1 +
> 4 files changed, 5 insertions(+)
>
> diff --git a/hw/core/machine.c b/hw/core/machine.c
> index 023548b4f3..ea03c25565 100644
> --- a/hw/core/machine.c
> +++ b/hw/core/machine.c
> @@ -789,6 +789,7 @@ static void smp_parse(MachineState *ms, QemuOpts *opts)
> ms->smp.cpus = cpus;
> ms->smp.cores = cores;
> ms->smp.threads = threads;
> + ms->smp.sockets = sockets;
> }
>
> if (ms->smp.cpus > 1) {
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 9803413dd9..f13721ac43 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -978,6 +978,7 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
> ms->smp.cpus = cpus;
> ms->smp.cores = cores;
> ms->smp.threads = threads;
> + ms->smp.sockets = sockets;
> x86ms->smp_dies = dies;
> }
>
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 24cbeecbae..a50a2d779e 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -256,12 +256,14 @@ typedef struct DeviceMemoryState {
> * @cpus: the number of present logical processors on the machine
> * @cores: the number of cores in one package
> * @threads: the number of threads in one core
> + * @sockets: the number of sockets on the machine
> * @max_cpus: the maximum number of logical processors on the machine
> */
> typedef struct CpuTopology {
> unsigned int cpus;
> unsigned int cores;
> unsigned int threads;
> + unsigned int sockets;
> unsigned int max_cpus;
> } CpuTopology;
>
> diff --git a/vl.c b/vl.c
> index 94508300c3..3f6d396a95 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -3819,6 +3819,7 @@ int main(int argc, char **argv, char **envp)
> current_machine->smp.max_cpus = machine_class->default_cpus;
> current_machine->smp.cores = 1;
> current_machine->smp.threads = 1;
> + current_machine->smp.sockets = 1;
>
> machine_class->smp_parse(current_machine,
> qemu_opts_find(qemu_find_opts("smp-opts"), NULL));
>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions
2020-02-13 18:17 ` [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions Babu Moger
@ 2020-02-24 8:50 ` Igor Mammedov
2020-02-24 17:24 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 8:50 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:18 -0600
Babu Moger <babu.moger@amd.com> wrote:
> These functions add support for building EPYC mode topology given the smp
> details like numa nodes, cores, threads and sockets.
>
> The new apic id decoding is mostly similar to current apic id decoding
> except that it adds a new field llc_id when numa configured. Removes all
llc_id/nodes_per_pkg
> the hardcoded values. Subsequent patches will use these functions to build
> the topology.
>
> Following functions are added.
> apicid_llc_width_epyc
> apicid_llc_offset_epyc
> apicid_pkg_offset_epyc
> apicid_from_topo_ids_epyc
> x86_topo_ids_from_idx_epyc
> x86_topo_ids_from_apicid_epyc
> x86_apicid_from_cpu_idx_epyc
>
> The topology details are available in Processor Programming Reference (PPR)
> for AMD Family 17h Model 01h, Revision B1 Processors.
> https://www.amd.com/system/files/TechDocs/55570-B1_PUB.zip
also checkpatch doesn't like this patch for too long strings.
with that fixed
Acked-by: Igor Mammedov <imammedo@redhat.com>
PS:
similar checkpatch warnings should be fixed in preceding patches
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> include/hw/i386/topology.h | 93 ++++++++++++++++++++++++++++++++++++++++++++
> include/hw/i386/x86.h | 1
> 2 files changed, 94 insertions(+)
>
> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
> index 3158157430..d9319dc2ac 100644
> --- a/include/hw/i386/topology.h
> +++ b/include/hw/i386/topology.h
> @@ -83,6 +83,11 @@ static inline unsigned apicid_die_width(X86CPUTopoInfo *topo_info)
> return apicid_bitwidth_for_count(topo_info->dies_per_pkg);
> }
>
> +/* Bit width of the node_id field per socket */
> +static inline unsigned apicid_node_width_epyc(X86CPUTopoInfo *topo_info)
> +{
> + return apicid_bitwidth_for_count(MAX(topo_info->nodes_per_pkg, 1));
> +}
> /* Bit offset of the Core_ID field
> */
> static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)
> @@ -103,6 +108,94 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
> return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
> }
>
> +#define LLC_OFFSET 3 /* Minimum LLC offset if numa configured */
> +
> +/* Bit offset of the node_id field */
> +static inline unsigned apicid_node_offset_epyc(X86CPUTopoInfo *topo_info)
> +{
> + unsigned offset = apicid_die_offset(topo_info) +
> + apicid_die_width(topo_info);
> +
> + if (topo_info->nodes_per_pkg) {
> + return MAX(LLC_OFFSET, offset);
> + } else {
> + return offset;
> + }
> +}
> +
> +/* Bit offset of the Pkg_ID (socket ID) field */
> +static inline unsigned apicid_pkg_offset_epyc(X86CPUTopoInfo *topo_info)
> +{
> + return apicid_node_offset_epyc(topo_info) + apicid_node_width_epyc(topo_info);
> +}
> +
> +/*
> + * Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
> + *
> + * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
> + */
> +static inline apic_id_t x86_apicid_from_topo_ids_epyc(X86CPUTopoInfo *topo_info,
> + const X86CPUTopoIDs *topo_ids)
> +{
> + return (topo_ids->pkg_id << apicid_pkg_offset_epyc(topo_info)) |
> + (topo_ids->node_id << apicid_node_offset_epyc(topo_info)) |
> + (topo_ids->die_id << apicid_die_offset(topo_info)) |
> + (topo_ids->core_id << apicid_core_offset(topo_info)) |
> + topo_ids->smt_id;
> +}
> +
> +static inline void x86_topo_ids_from_idx_epyc(X86CPUTopoInfo *topo_info,
> + unsigned cpu_index,
> + X86CPUTopoIDs *topo_ids)
> +{
> + unsigned nr_nodes = MAX(topo_info->nodes_per_pkg, 1);
> + unsigned nr_dies = topo_info->dies_per_pkg;
> + unsigned nr_cores = topo_info->cores_per_die;
> + unsigned nr_threads = topo_info->threads_per_core;
> + unsigned cores_per_node = DIV_ROUND_UP((nr_dies * nr_cores * nr_threads),
> + nr_nodes);
> +
> + topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
> + topo_ids->node_id = (cpu_index / cores_per_node) % nr_nodes;
> + topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
> + topo_ids->core_id = cpu_index / nr_threads % nr_cores;
> + topo_ids->smt_id = cpu_index % nr_threads;
> +}
> +
> +/*
> + * Calculate thread/core/package IDs for a specific topology,
> + * based on APIC ID
> + */
> +static inline void x86_topo_ids_from_apicid_epyc(apic_id_t apicid,
> + X86CPUTopoInfo *topo_info,
> + X86CPUTopoIDs *topo_ids)
> +{
> + topo_ids->smt_id = apicid &
> + ~(0xFFFFFFFFUL << apicid_smt_width(topo_info));
> + topo_ids->core_id =
> + (apicid >> apicid_core_offset(topo_info)) &
> + ~(0xFFFFFFFFUL << apicid_core_width(topo_info));
> + topo_ids->die_id =
> + (apicid >> apicid_die_offset(topo_info)) &
> + ~(0xFFFFFFFFUL << apicid_die_width(topo_info));
> + topo_ids->node_id =
> + (apicid >> apicid_node_offset_epyc(topo_info)) &
> + ~(0xFFFFFFFFUL << apicid_node_width_epyc(topo_info));
> + topo_ids->pkg_id = apicid >> apicid_pkg_offset_epyc(topo_info);
> +}
> +
> +/*
> + * Make APIC ID for the CPU 'cpu_index'
> + *
> + * 'cpu_index' is a sequential, contiguous ID for the CPU.
> + */
> +static inline apic_id_t x86_apicid_from_cpu_idx_epyc(X86CPUTopoInfo *topo_info,
> + unsigned cpu_index)
> +{
> + X86CPUTopoIDs topo_ids;
> + x86_topo_ids_from_idx_epyc(topo_info, cpu_index, &topo_ids);
> + return x86_apicid_from_topo_ids_epyc(topo_info, &topo_ids);
> +}
> /* Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
> *
> * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> index d76fd0bbb1..38c2d27910 100644
> --- a/include/hw/i386/x86.h
> +++ b/include/hw/i386/x86.h
> @@ -42,6 +42,7 @@ typedef uint32_t apic_id_t;
>
> typedef struct X86CPUTopoIDs {
> unsigned pkg_id;
> + unsigned node_id;
> unsigned die_id;
> unsigned core_id;
> unsigned smt_id;
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-13 18:17 ` [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions Babu Moger
@ 2020-02-24 8:52 ` Igor Mammedov
2020-02-24 17:29 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 8:52 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:25 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Use the new functions from topology.h and delete the unused code. Given the
> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
> mode. Removes all the hardcoded values.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
modulo MAX() macro, looks fine to me
> ---
> target/i386/cpu.c | 162 +++++++++++------------------------------------------
> 1 file changed, 35 insertions(+), 127 deletions(-)
>
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 5d6edfd09b..19675eb696 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
> }
> }
>
> -/*
> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
> - * Define the constants to build the cpu topology. Right now, TOPOEXT
> - * feature is enabled only on EPYC. So, these constants are based on
> - * EPYC supported configurations. We may need to handle the cases if
> - * these values change in future.
> - */
> -/* Maximum core complexes in a node */
> -#define MAX_CCX 2
> -/* Maximum cores in a core complex */
> -#define MAX_CORES_IN_CCX 4
> -/* Maximum cores in a node */
> -#define MAX_CORES_IN_NODE 8
> -/* Maximum nodes in a socket */
> -#define MAX_NODES_PER_SOCKET 4
> -
> -/*
> - * Figure out the number of nodes required to build this config.
> - * Max cores in a node is 8
> - */
> -static int nodes_in_socket(int nr_cores)
> -{
> - int nodes;
> -
> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
> -
> - /* Hardware does not support config with 3 nodes, return 4 in that case */
> - return (nodes == 3) ? 4 : nodes;
> -}
> -
> -/*
> - * Decide the number of cores in a core complex with the given nr_cores using
> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
> - * L3 cache is shared across all cores in a core complex. So, this will also
> - * tell us how many cores are sharing the L3 cache.
> - */
> -static int cores_in_core_complex(int nr_cores)
> -{
> - int nodes;
> -
> - /* Check if we can fit all the cores in one core complex */
> - if (nr_cores <= MAX_CORES_IN_CCX) {
> - return nr_cores;
> - }
> - /* Get the number of nodes required to build this config */
> - nodes = nodes_in_socket(nr_cores);
> -
> - /*
> - * Divide the cores accros all the core complexes
> - * Return rounded up value
> - */
> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
> -}
> -
> /* Encode cache info for CPUID[8000001D] */
> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> - uint32_t *eax, uint32_t *ebx,
> - uint32_t *ecx, uint32_t *edx)
> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
> + X86CPUTopoInfo *topo_info,
> + uint32_t *eax, uint32_t *ebx,
> + uint32_t *ecx, uint32_t *edx)
> {
> uint32_t l3_cores;
> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
> +
> assert(cache->size == cache->line_size * cache->associativity *
> cache->partitions * cache->sets);
>
> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>
> /* L3 is shared among multiple cores */
> if (cache->level == 3) {
> - l3_cores = cores_in_core_complex(cs->nr_cores);
> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
> + topo_info->cores_per_die *
> + topo_info->threads_per_core),
> + nodes);
> + *eax |= (l3_cores - 1) << 14;
> } else {
> - *eax |= ((cs->nr_threads - 1) << 14);
> + *eax |= ((topo_info->threads_per_core - 1) << 14);
> }
>
> assert(cache->line_size > 0);
> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
> }
>
> -/* Data structure to hold the configuration info for a given core index */
> -struct core_topology {
> - /* core complex id of the current core index */
> - int ccx_id;
> - /*
> - * Adjusted core index for this core in the topology
> - * This can be 0,1,2,3 with max 4 cores in a core complex
> - */
> - int core_id;
> - /* Node id for this core index */
> - int node_id;
> - /* Number of nodes in this config */
> - int num_nodes;
> -};
> -
> -/*
> - * Build the configuration closely match the EPYC hardware. Using the EPYC
> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
> - * right now. This could change in future.
> - * nr_cores : Total number of cores in the config
> - * core_id : Core index of the current CPU
> - * topo : Data structure to hold all the config info for this core index
> - */
> -static void build_core_topology(int nr_cores, int core_id,
> - struct core_topology *topo)
> -{
> - int nodes, cores_in_ccx;
> -
> - /* First get the number of nodes required */
> - nodes = nodes_in_socket(nr_cores);
> -
> - cores_in_ccx = cores_in_core_complex(nr_cores);
> -
> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
> - topo->core_id = core_id % cores_in_ccx;
> - topo->num_nodes = nodes;
> -}
> -
> /* Encode cache info for CPUID[8000001E] */
> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
> uint32_t *eax, uint32_t *ebx,
> uint32_t *ecx, uint32_t *edx)
> {
> - struct core_topology topo = {0};
> - unsigned long nodes;
> + X86CPUTopoIDs topo_ids = {0};
> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
> int shift;
>
> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
> +
> *eax = cpu->apic_id;
> /*
> * CPUID_Fn8000001E_EBX
> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> * 3 Core complex id
> * 1:0 Core id
> */
> - if (cs->nr_threads - 1) {
> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
> - (topo.ccx_id << 2) | topo.core_id;
> - } else {
> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
> - }
> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
> + (topo_ids.core_id);
> /*
> * CPUID_Fn8000001E_ECX
> * 31:11 Reserved
> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> * 2 Socket id
> * 1:0 Node id
> */
> - if (topo.num_nodes <= 4) {
> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
> - topo.node_id;
> +
> + if (nodes <= 4) {
> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
> } else {
> /*
> * Node id fix up. Actual hardware supports up to 4 nodes. But with
> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> * number of nodes. find_last_bit returns last set bit(0 based). Left
> * shift(+1) the socket id to represent all the nodes.
> */
> - nodes = topo.num_nodes - 1;
> + nodes -= 1;
> shift = find_last_bit(&nodes, 8);
> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
> - topo.node_id;
> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
> + topo_ids.node_id;
> }
> *edx = 0;
> }
> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> uint32_t signature[3];
> X86CPUTopoInfo topo_info;
>
> + topo_info.nodes_per_pkg = env->nr_nodes;
> topo_info.dies_per_pkg = env->nr_dies;
> topo_info.cores_per_die = cs->nr_cores;
> topo_info.threads_per_core = cs->nr_threads;
> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> }
> switch (count) {
> case 0: /* L1 dcache info */
> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
> - eax, ebx, ecx, edx);
> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
> + &topo_info, eax, ebx, ecx, edx);
> break;
> case 1: /* L1 icache info */
> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
> - eax, ebx, ecx, edx);
> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
> + &topo_info, eax, ebx, ecx, edx);
> break;
> case 2: /* L2 cache info */
> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
> - eax, ebx, ecx, edx);
> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
> + &topo_info, eax, ebx, ecx, edx);
> break;
> case 3: /* L3 cache info */
> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
> - eax, ebx, ecx, edx);
> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
> + &topo_info, eax, ebx, ecx, edx);
> break;
> default: /* end of info */
> *eax = *ebx = *ecx = *edx = 0;
> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> break;
> case 0x8000001E:
> assert(cpu->core_id <= 255);
> - encode_topo_cpuid8000001e(cs, cpu,
> - eax, ebx, ecx, edx);
> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
> break;
> case 0xC0000000:
> *eax = env->cpuid_xlevel2;
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo
2020-02-24 8:18 ` Igor Mammedov
@ 2020-02-24 16:54 ` Babu Moger
0 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-24 16:54 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 2:18 AM, Igor Mammedov wrote:
> On Fri, 21 Feb 2020 11:51:15 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> On 2/21/20 11:05 AM, Igor Mammedov wrote:
>>> On Thu, 13 Feb 2020 12:16:51 -0600
>>> Babu Moger <babu.moger@amd.com> wrote:
>>>
>>>> Initialize all the parameters in one function init_topo_info.
>>>
>>> is it possible to squash it in 2/16
>>>
>> Sure. We can do that.
>>>
>>>>
>>>> Move the data structure X86CPUTopoIDs and X86CPUTopoInfo into
>>>> x86.h.
>>> A reason why it's moved should be here.
>>
>> Apicid functions will be part of X86MachineState data structure(patches
>> introduced later).These functions will use X86CPUTopoIDs and
>> X86CPUTopoInfo definition. Will add these details. Thanks
>
> why not just include topology.h into the X86MachineState header,
> and keep topo structures/functions where they are now?
> (I dislike a little scattering consolidated pieces across multiple files,
> but what worries me more is that it makes target/i386/cpu.c via
> topology.h -> x86.h chain pull in a lot of unrelated dependencies)
>
> So I'd keep X86CPUTopoIDs and X86CPUTopoInfo in topology.h
Ok. Sure. we can do that.
>
> [...]
>>>> +static inline void init_topo_info(X86CPUTopoInfo *topo_info,
>>>> + const X86MachineState *x86ms)
>>>> +{
>>>> + MachineState *ms = MACHINE(x86ms);
>>>> +
>>>> + topo_info->dies_per_pkg = x86ms->smp_dies;
>>>> + topo_info->cores_per_die = ms->smp.cores;
>>>> + topo_info->threads_per_core = ms->smp.threads;
>>>> +}
>
> this is pure machine specific helper, and aren't used anywhere else
> beside machine code.
> Suggest to put it in pc.c or x86.c to keep topology.h machine independent.
Ok. Will do.
>
>>>>
>>>> /* Return the bit width needed for 'count' IDs
>>>> */
>>>> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
>>>> index 4b84917885..ad62b01cf2 100644
>>>> --- a/include/hw/i386/x86.h
>>>> +++ b/include/hw/i386/x86.h
>>>> @@ -36,6 +36,23 @@ typedef struct {
>>>> bool compat_apic_id_mode;
>>>> } X86MachineClass;
>>>>
>>>> +/* APIC IDs can be 32-bit, but beware: APIC IDs > 255 require x2APIC support
>>>> + */
>>>> +typedef uint32_t apic_id_t;
>>>> +
>>>> +typedef struct X86CPUTopoIDs {
>>>> + unsigned pkg_id;
>>>> + unsigned die_id;
>>>> + unsigned core_id;
>>>> + unsigned smt_id;
>>>> +} X86CPUTopoIDs;
>>>> +
>>>> +typedef struct X86CPUTopoInfo {
>>>> + unsigned dies_per_pkg;
>>>> + unsigned cores_per_die;
>>>> + unsigned threads_per_core;
>>>> +} X86CPUTopoInfo;
>>>> +
>>>> typedef struct {
>>>> /*< private >*/
>>>> MachineState parent;
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState
2020-02-13 18:17 ` [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState Babu Moger
@ 2020-02-24 17:01 ` Igor Mammedov
2020-02-24 17:30 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 17:01 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:32 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Introduce model specific apicid functions inside X86MachineState.
> These functions will be loaded from X86CPUDefinition.
>
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> include/hw/i386/x86.h | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> index 38c2d27910..75c2462770 100644
> --- a/include/hw/i386/x86.h
> +++ b/include/hw/i386/x86.h
> @@ -79,6 +79,15 @@ typedef struct {
> uint16_t boot_cpus;
> unsigned smp_dies;
>
> + /* Apic id specific handlers */
> + uint32_t (*apicid_from_cpu_idx)(X86CPUTopoInfo *topo_info,
> + unsigned cpu_index);
> + void (*topo_ids_from_apicid)(apic_id_t apicid, X86CPUTopoInfo *topo_info,
> + X86CPUTopoIDs *topo_ids);
> + apic_id_t (*apicid_from_topo_ids)(X86CPUTopoInfo *topo_info,
> + const X86CPUTopoIDs *topo_ids);
> + uint32_t (*apicid_pkg_offset)(X86CPUTopoInfo *topo_info);
> +
> /*
> * Address space used by IOAPIC device. All IOAPIC interrupts
> * will be translated to MSI messages in the address space.
>
>
Suggest to move defaults into this patch as well. i.e.
move parts that replace direct calls with hooks from 12/16
and set default hooks (i.e. non EPYC ones) in x86's machine_class_init().
This way It's immediately obvious what you are changing and then
followup EPYC related patch won't be cluttered by non related defaults
conversion.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg
2020-02-24 8:34 ` Igor Mammedov
@ 2020-02-24 17:12 ` Babu Moger
2020-02-25 7:42 ` Igor Mammedov
0 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-24 17:12 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 2:34 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:17:04 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> Update structures X86CPUTopoIDs and CPUX86State to hold the nodes_per_pkg.
>> This is required to build EPYC mode topology.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> hw/i386/pc.c | 1 +
>> hw/i386/x86.c | 2 ++
>> include/hw/i386/topology.h | 2 ++
>> include/hw/i386/x86.h | 1 +
>> target/i386/cpu.c | 1 +
>> target/i386/cpu.h | 1 +
>> 6 files changed, 8 insertions(+)
>>
>> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> index f13721ac43..02fdb3d506 100644
>> --- a/hw/i386/pc.c
>> +++ b/hw/i386/pc.c
>> @@ -1753,6 +1753,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
>> init_topo_info(&topo_info, x86ms);
>>
>> env->nr_dies = x86ms->smp_dies;
>> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
>
> it would be better if calculation would result in valid result
> so you won't have to later scatter MAX(env->nr_nodes, 1) everywhere.
Ok. Sure.
>
> also I'd use earlier intialized:
> env->nr_nodes = topo_info->nodes_per_pkg
> to avoid repeating calculation
yes. Will do it.
>
>> /*
>> * If APIC ID is not set,
>> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
>> index 083effb2f5..3d944f68e6 100644
>> --- a/hw/i386/x86.c
>> +++ b/hw/i386/x86.c
>> @@ -89,11 +89,13 @@ void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
>> Object *cpu = NULL;
>> Error *local_err = NULL;
>> CPUX86State *env = NULL;
>> + MachineState *ms = MACHINE(x86ms);
>>
>> cpu = object_new(MACHINE(x86ms)->cpu_type);
>>
>> env = &X86_CPU(cpu)->env;
>> env->nr_dies = x86ms->smp_dies;
>> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
>
> Is this really necessary? (I think pc_cpu_pre_plug should take care of setting it)
This does not seem necessary. I can add as a separate patch to remove env
initialization from x86_cpu_new.
>
>> object_property_set_uint(cpu, apic_id, "apic-id", &local_err);
>> object_property_set_bool(cpu, true, "realized", &local_err);
>> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
>> index ef0ab0b6a3..522c77e6a9 100644
>> --- a/include/hw/i386/topology.h
>> +++ b/include/hw/i386/topology.h
>> @@ -41,12 +41,14 @@
>>
>> #include "qemu/bitops.h"
>> #include "hw/i386/x86.h"
>> +#include "sysemu/numa.h"
>>
>> static inline void init_topo_info(X86CPUTopoInfo *topo_info,
>> const X86MachineState *x86ms)
>> {
>> MachineState *ms = MACHINE(x86ms);
>>
>> + topo_info->nodes_per_pkg = ms->numa_state->num_nodes / ms->smp.sockets;
>> topo_info->dies_per_pkg = x86ms->smp_dies;
>> topo_info->cores_per_die = ms->smp.cores;
>> topo_info->threads_per_core = ms->smp.threads;
>> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
>> index ad62b01cf2..d76fd0bbb1 100644
>> --- a/include/hw/i386/x86.h
>> +++ b/include/hw/i386/x86.h
>> @@ -48,6 +48,7 @@ typedef struct X86CPUTopoIDs {
>> } X86CPUTopoIDs;
>>
>> typedef struct X86CPUTopoInfo {
>> + unsigned nodes_per_pkg;
>> unsigned dies_per_pkg;
>> unsigned cores_per_die;
>> unsigned threads_per_core;
>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>> index 7e630f47ac..5d6edfd09b 100644
>> --- a/target/i386/cpu.c
>> +++ b/target/i386/cpu.c
>> @@ -6761,6 +6761,7 @@ static void x86_cpu_initfn(Object *obj)
>> FeatureWord w;
>>
>> env->nr_dies = 1;
>> + env->nr_nodes = 1;
>> cpu_set_cpustate_pointers(cpu);
>>
>> object_property_add(obj, "family", "int",
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index af282936a7..627a8cb9be 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -1603,6 +1603,7 @@ typedef struct CPUX86State {
>> TPRAccess tpr_access_type;
>>
>> unsigned nr_dies;
>> + unsigned nr_nodes;
>> } CPUX86State;
>>
>> struct kvm_msrs;
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-13 18:17 ` [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState Babu Moger
@ 2020-02-24 17:19 ` Igor Mammedov
2020-02-24 17:58 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-24 17:19 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Thu, 13 Feb 2020 12:17:46 -0600
Babu Moger <babu.moger@amd.com> wrote:
> Check and Load the apicid handlers from X86CPUDefinition if available.
> Update the calling convention for the apicid handlers.
Previous and this patch look too complicated for the task at the hand.
In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
reference to Machine into i386/cpu.c (even though it's just a helper function)
and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
businesses to make up APIC-IDs).
I'd rather do opposite and get rid of the last explicit dependency to
ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
so for this series I'd just try to avoid adding more Machine dependencies.
All 11/16 does is basically using hooks as a switch "I'm EPYC" to
set epyc specific encoding topo routines.
It could be accomplished by a simple Boolean flag like
X86CPUDefinition::use_epyc_apic_id_encoding
and then cpu_x86_init_apicid_fns() could be replaced with trivial
helper like:
x86_use_epyc_apic_id_encoding(char *cpu_type)
{
X86CPUClass *xcc = ... cpu_type ...
return xcc->model->cpudef->use_epyc_apic_id_encoding
}
then machine could override default[1] hooks using this helper
as the trigger
x86_cpus_init()
{
// no need in dedicated function as it's the only instance it's going to be called ever
if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
x86ms->apicid_from_cpu_idx = ...epyc...
x86ms->topo_ids_from_apicid = ...epyc...
x86ms->apicid_from_topo_ids = ...epyc...
x86ms->apicid_pkg_offset = ...epyc...
}
}
That would be less invasive and won't create non necessary dependencies.
---
1) defaults should be set in x86_machine_class_init()
Eduardo, what's your take on this?
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> hw/i386/pc.c | 6 +++---
> hw/i386/x86.c | 27 +++++++++++++++++++++++----
> 2 files changed, 26 insertions(+), 7 deletions(-)
>
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index be72a49716..93063af6a8 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1808,14 +1808,14 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> topo_ids.die_id = cpu->die_id;
> topo_ids.core_id = cpu->core_id;
> topo_ids.smt_id = cpu->thread_id;
> - cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
> + cpu->apic_id = x86ms->apicid_from_topo_ids(&topo_info, &topo_ids);
> }
>
> cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
> if (!cpu_slot) {
> MachineState *ms = MACHINE(pcms);
>
> - x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
> + x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
> error_setg(errp,
> "Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with"
> " APIC ID %" PRIu32 ", valid index range 0:%d",
> @@ -1836,7 +1836,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> /* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
> * once -smp refactoring is complete and there will be CPU private
> * CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
> - x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
> + x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
> if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
> error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
> " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo_ids.pkg_id);
> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> index 3d944f68e6..b825861b85 100644
> --- a/hw/i386/x86.c
> +++ b/hw/i386/x86.c
> @@ -52,6 +52,22 @@
> /* Physical Address of PVH entry point read from kernel ELF NOTE */
> static size_t pvh_start_addr;
>
> +/*
> + * Check for apicid handlers in X86MachineState. Load them if
> + * not loaded already. These handlers are loaded from X86CPUDefinition.
> + */
> +static void x86_check_apicid_handlers(MachineState *ms)
> +{
> + X86MachineState *x86ms = X86_MACHINE(ms);
> +
> + if (!x86ms->apicid_from_cpu_idx ||
> + !x86ms->topo_ids_from_apicid ||
> + !x86ms->apicid_from_topo_ids ||
> + !x86ms->apicid_pkg_offset) {
> + cpu_x86_init_apicid_fns(ms);
> + }
> +}
> +
> /*
> * Calculates initial APIC ID for a specific CPU index
> *
> @@ -70,7 +86,7 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
>
> init_topo_info(&topo_info, x86ms);
>
> - correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
> + correct_id = x86ms->apicid_from_cpu_idx(&topo_info, cpu_index);
> if (x86mc->compat_apic_id_mode) {
> if (cpu_index != correct_id && !warned && !qtest_enabled()) {
> error_report("APIC IDs set in compatibility mode, "
> @@ -148,8 +164,8 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
> init_topo_info(&topo_info, x86ms);
>
> assert(idx < ms->possible_cpus->len);
> - x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
> - &topo_info, &topo_ids);
> + x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
> + &topo_info, &topo_ids);
> return topo_ids.pkg_id % ms->numa_state->num_nodes;
> }
>
> @@ -169,6 +185,9 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
> return ms->possible_cpus;
> }
>
> + /* Initialize apicid handlers */
> + x86_check_apicid_handlers(ms);
> +
> ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
> sizeof(CPUArchId) * max_cpus);
> ms->possible_cpus->len = max_cpus;
> @@ -182,7 +201,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
> ms->possible_cpus->cpus[i].vcpus_count = 1;
> ms->possible_cpus->cpus[i].arch_id =
> x86_cpu_apic_id_from_index(x86ms, i);
> - x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
> + x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
> &topo_info, &topo_ids);
> ms->possible_cpus->cpus[i].props.has_socket_id = true;
> ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions
2020-02-24 8:50 ` Igor Mammedov
@ 2020-02-24 17:24 ` Babu Moger
0 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-24 17:24 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 2:50 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:17:18 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> These functions add support for building EPYC mode topology given the smp
>> details like numa nodes, cores, threads and sockets.
>>
>> The new apic id decoding is mostly similar to current apic id decoding
>> except that it adds a new field llc_id when numa configured. Removes all
>
> llc_id/nodes_per_pkg
Good catch. Thanks.
>
>> the hardcoded values. Subsequent patches will use these functions to build
>> the topology.
>>
>> Following functions are added.
>> apicid_llc_width_epyc
>> apicid_llc_offset_epyc
>> apicid_pkg_offset_epyc
>> apicid_from_topo_ids_epyc
>> x86_topo_ids_from_idx_epyc
>> x86_topo_ids_from_apicid_epyc
>> x86_apicid_from_cpu_idx_epyc
>>
>> The topology details are available in Processor Programming Reference (PPR)
>> for AMD Family 17h Model 01h, Revision B1 Processors.
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F55570-B1_PUB.zip&data=02%7C01%7Cbabu.moger%40amd.com%7C07f6ec6272fa4a0bb44608d7b9069ef6%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637181310409402221&sdata=nPUTNqiXVSihj8mHt%2FO6sSGCL4uTtPsfvIiEJ0WosaM%3D&reserved=0
>
> also checkpatch doesn't like this patch for too long strings.
Yes. I saw that. Some are bit tricky. I may need to break the definitions
like this below.
static inline apic_id_t
x86_apicid_from_topo_ids_epyc(X86CPUTopoInfo *topo_info,
const X86CPUTopoIDs *topo_ids)
> with that fixed
> Acked-by: Igor Mammedov <imammedo@redhat.com>
>
>
> PS:
> similar checkpatch warnings should be fixed in preceding patches
Sure. Will take care.
>
>
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> include/hw/i386/topology.h | 93 ++++++++++++++++++++++++++++++++++++++++++++
>> include/hw/i386/x86.h | 1
>> 2 files changed, 94 insertions(+)
>>
>> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
>> index 3158157430..d9319dc2ac 100644
>> --- a/include/hw/i386/topology.h
>> +++ b/include/hw/i386/topology.h
>> @@ -83,6 +83,11 @@ static inline unsigned apicid_die_width(X86CPUTopoInfo *topo_info)
>> return apicid_bitwidth_for_count(topo_info->dies_per_pkg);
>> }
>>
>> +/* Bit width of the node_id field per socket */
>> +static inline unsigned apicid_node_width_epyc(X86CPUTopoInfo *topo_info)
>> +{
>> + return apicid_bitwidth_for_count(MAX(topo_info->nodes_per_pkg, 1));
>> +}
>> /* Bit offset of the Core_ID field
>> */
>> static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)
>> @@ -103,6 +108,94 @@ static inline unsigned apicid_pkg_offset(X86CPUTopoInfo *topo_info)
>> return apicid_die_offset(topo_info) + apicid_die_width(topo_info);
>> }
>>
>> +#define LLC_OFFSET 3 /* Minimum LLC offset if numa configured */
>> +
>> +/* Bit offset of the node_id field */
>> +static inline unsigned apicid_node_offset_epyc(X86CPUTopoInfo *topo_info)
>> +{
>> + unsigned offset = apicid_die_offset(topo_info) +
>> + apicid_die_width(topo_info);
>> +
>> + if (topo_info->nodes_per_pkg) {
>> + return MAX(LLC_OFFSET, offset);
>> + } else {
>> + return offset;
>> + }
>> +}
>> +
>> +/* Bit offset of the Pkg_ID (socket ID) field */
>> +static inline unsigned apicid_pkg_offset_epyc(X86CPUTopoInfo *topo_info)
>> +{
>> + return apicid_node_offset_epyc(topo_info) + apicid_node_width_epyc(topo_info);
>> +}
>> +
>> +/*
>> + * Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
>> + *
>> + * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
>> + */
>> +static inline apic_id_t x86_apicid_from_topo_ids_epyc(X86CPUTopoInfo *topo_info,
>> + const X86CPUTopoIDs *topo_ids)
>> +{
>> + return (topo_ids->pkg_id << apicid_pkg_offset_epyc(topo_info)) |
>> + (topo_ids->node_id << apicid_node_offset_epyc(topo_info)) |
>> + (topo_ids->die_id << apicid_die_offset(topo_info)) |
>> + (topo_ids->core_id << apicid_core_offset(topo_info)) |
>> + topo_ids->smt_id;
>> +}
>> +
>> +static inline void x86_topo_ids_from_idx_epyc(X86CPUTopoInfo *topo_info,
>> + unsigned cpu_index,
>> + X86CPUTopoIDs *topo_ids)
>> +{
>> + unsigned nr_nodes = MAX(topo_info->nodes_per_pkg, 1);
>> + unsigned nr_dies = topo_info->dies_per_pkg;
>> + unsigned nr_cores = topo_info->cores_per_die;
>> + unsigned nr_threads = topo_info->threads_per_core;
>> + unsigned cores_per_node = DIV_ROUND_UP((nr_dies * nr_cores * nr_threads),
>> + nr_nodes);
>> +
>> + topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
>> + topo_ids->node_id = (cpu_index / cores_per_node) % nr_nodes;
>> + topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies;
>> + topo_ids->core_id = cpu_index / nr_threads % nr_cores;
>> + topo_ids->smt_id = cpu_index % nr_threads;
>> +}
>> +
>> +/*
>> + * Calculate thread/core/package IDs for a specific topology,
>> + * based on APIC ID
>> + */
>> +static inline void x86_topo_ids_from_apicid_epyc(apic_id_t apicid,
>> + X86CPUTopoInfo *topo_info,
>> + X86CPUTopoIDs *topo_ids)
>> +{
>> + topo_ids->smt_id = apicid &
>> + ~(0xFFFFFFFFUL << apicid_smt_width(topo_info));
>> + topo_ids->core_id =
>> + (apicid >> apicid_core_offset(topo_info)) &
>> + ~(0xFFFFFFFFUL << apicid_core_width(topo_info));
>> + topo_ids->die_id =
>> + (apicid >> apicid_die_offset(topo_info)) &
>> + ~(0xFFFFFFFFUL << apicid_die_width(topo_info));
>> + topo_ids->node_id =
>> + (apicid >> apicid_node_offset_epyc(topo_info)) &
>> + ~(0xFFFFFFFFUL << apicid_node_width_epyc(topo_info));
>> + topo_ids->pkg_id = apicid >> apicid_pkg_offset_epyc(topo_info);
>> +}
>> +
>> +/*
>> + * Make APIC ID for the CPU 'cpu_index'
>> + *
>> + * 'cpu_index' is a sequential, contiguous ID for the CPU.
>> + */
>> +static inline apic_id_t x86_apicid_from_cpu_idx_epyc(X86CPUTopoInfo *topo_info,
>> + unsigned cpu_index)
>> +{
>> + X86CPUTopoIDs topo_ids;
>> + x86_topo_ids_from_idx_epyc(topo_info, cpu_index, &topo_ids);
>> + return x86_apicid_from_topo_ids_epyc(topo_info, &topo_ids);
>> +}
>> /* Make APIC ID for the CPU based on Pkg_ID, Core_ID, SMT_ID
>> *
>> * The caller must make sure core_id < nr_cores and smt_id < nr_threads.
>> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
>> index d76fd0bbb1..38c2d27910 100644
>> --- a/include/hw/i386/x86.h
>> +++ b/include/hw/i386/x86.h
>> @@ -42,6 +42,7 @@ typedef uint32_t apic_id_t;
>>
>> typedef struct X86CPUTopoIDs {
>> unsigned pkg_id;
>> + unsigned node_id;
>> unsigned die_id;
>> unsigned core_id;
>> unsigned smt_id;
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-24 8:52 ` Igor Mammedov
@ 2020-02-24 17:29 ` Babu Moger
2020-02-25 7:49 ` Igor Mammedov
0 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-24 17:29 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 2:52 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:17:25 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> Use the new functions from topology.h and delete the unused code. Given the
>> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
>> mode. Removes all the hardcoded values.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>
> modulo MAX() macro, looks fine to me
Igor, Sorry. What do you mean here?
>
>> ---
>> target/i386/cpu.c | 162 +++++++++++------------------------------------------
>> 1 file changed, 35 insertions(+), 127 deletions(-)
>>
>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>> index 5d6edfd09b..19675eb696 100644
>> --- a/target/i386/cpu.c
>> +++ b/target/i386/cpu.c
>> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
>> }
>> }
>>
>> -/*
>> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
>> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
>> - * Define the constants to build the cpu topology. Right now, TOPOEXT
>> - * feature is enabled only on EPYC. So, these constants are based on
>> - * EPYC supported configurations. We may need to handle the cases if
>> - * these values change in future.
>> - */
>> -/* Maximum core complexes in a node */
>> -#define MAX_CCX 2
>> -/* Maximum cores in a core complex */
>> -#define MAX_CORES_IN_CCX 4
>> -/* Maximum cores in a node */
>> -#define MAX_CORES_IN_NODE 8
>> -/* Maximum nodes in a socket */
>> -#define MAX_NODES_PER_SOCKET 4
>> -
>> -/*
>> - * Figure out the number of nodes required to build this config.
>> - * Max cores in a node is 8
>> - */
>> -static int nodes_in_socket(int nr_cores)
>> -{
>> - int nodes;
>> -
>> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
>> -
>> - /* Hardware does not support config with 3 nodes, return 4 in that case */
>> - return (nodes == 3) ? 4 : nodes;
>> -}
>> -
>> -/*
>> - * Decide the number of cores in a core complex with the given nr_cores using
>> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
>> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
>> - * L3 cache is shared across all cores in a core complex. So, this will also
>> - * tell us how many cores are sharing the L3 cache.
>> - */
>> -static int cores_in_core_complex(int nr_cores)
>> -{
>> - int nodes;
>> -
>> - /* Check if we can fit all the cores in one core complex */
>> - if (nr_cores <= MAX_CORES_IN_CCX) {
>> - return nr_cores;
>> - }
>> - /* Get the number of nodes required to build this config */
>> - nodes = nodes_in_socket(nr_cores);
>> -
>> - /*
>> - * Divide the cores accros all the core complexes
>> - * Return rounded up value
>> - */
>> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
>> -}
>> -
>> /* Encode cache info for CPUID[8000001D] */
>> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>> - uint32_t *eax, uint32_t *ebx,
>> - uint32_t *ecx, uint32_t *edx)
>> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
>> + X86CPUTopoInfo *topo_info,
>> + uint32_t *eax, uint32_t *ebx,
>> + uint32_t *ecx, uint32_t *edx)
>> {
>> uint32_t l3_cores;
>> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
>> +
>> assert(cache->size == cache->line_size * cache->associativity *
>> cache->partitions * cache->sets);
>>
>> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>
>> /* L3 is shared among multiple cores */
>> if (cache->level == 3) {
>> - l3_cores = cores_in_core_complex(cs->nr_cores);
>> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
>> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
>> + topo_info->cores_per_die *
>> + topo_info->threads_per_core),
>> + nodes);
>> + *eax |= (l3_cores - 1) << 14;
>> } else {
>> - *eax |= ((cs->nr_threads - 1) << 14);
>> + *eax |= ((topo_info->threads_per_core - 1) << 14);
>> }
>>
>> assert(cache->line_size > 0);
>> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
>> }
>>
>> -/* Data structure to hold the configuration info for a given core index */
>> -struct core_topology {
>> - /* core complex id of the current core index */
>> - int ccx_id;
>> - /*
>> - * Adjusted core index for this core in the topology
>> - * This can be 0,1,2,3 with max 4 cores in a core complex
>> - */
>> - int core_id;
>> - /* Node id for this core index */
>> - int node_id;
>> - /* Number of nodes in this config */
>> - int num_nodes;
>> -};
>> -
>> -/*
>> - * Build the configuration closely match the EPYC hardware. Using the EPYC
>> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
>> - * right now. This could change in future.
>> - * nr_cores : Total number of cores in the config
>> - * core_id : Core index of the current CPU
>> - * topo : Data structure to hold all the config info for this core index
>> - */
>> -static void build_core_topology(int nr_cores, int core_id,
>> - struct core_topology *topo)
>> -{
>> - int nodes, cores_in_ccx;
>> -
>> - /* First get the number of nodes required */
>> - nodes = nodes_in_socket(nr_cores);
>> -
>> - cores_in_ccx = cores_in_core_complex(nr_cores);
>> -
>> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
>> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
>> - topo->core_id = core_id % cores_in_ccx;
>> - topo->num_nodes = nodes;
>> -}
>> -
>> /* Encode cache info for CPUID[8000001E] */
>> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
>> uint32_t *eax, uint32_t *ebx,
>> uint32_t *ecx, uint32_t *edx)
>> {
>> - struct core_topology topo = {0};
>> - unsigned long nodes;
>> + X86CPUTopoIDs topo_ids = {0};
>> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
>> int shift;
>>
>> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
>> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
>> +
>> *eax = cpu->apic_id;
>> /*
>> * CPUID_Fn8000001E_EBX
>> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>> * 3 Core complex id
>> * 1:0 Core id
>> */
>> - if (cs->nr_threads - 1) {
>> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
>> - (topo.ccx_id << 2) | topo.core_id;
>> - } else {
>> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
>> - }
>> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
>> + (topo_ids.core_id);
>> /*
>> * CPUID_Fn8000001E_ECX
>> * 31:11 Reserved
>> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>> * 2 Socket id
>> * 1:0 Node id
>> */
>> - if (topo.num_nodes <= 4) {
>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
>> - topo.node_id;
>> +
>> + if (nodes <= 4) {
>> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
>> } else {
>> /*
>> * Node id fix up. Actual hardware supports up to 4 nodes. But with
>> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>> * number of nodes. find_last_bit returns last set bit(0 based). Left
>> * shift(+1) the socket id to represent all the nodes.
>> */
>> - nodes = topo.num_nodes - 1;
>> + nodes -= 1;
>> shift = find_last_bit(&nodes, 8);
>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
>> - topo.node_id;
>> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
>> + topo_ids.node_id;
>> }
>> *edx = 0;
>> }
>> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>> uint32_t signature[3];
>> X86CPUTopoInfo topo_info;
>>
>> + topo_info.nodes_per_pkg = env->nr_nodes;
>> topo_info.dies_per_pkg = env->nr_dies;
>> topo_info.cores_per_die = cs->nr_cores;
>> topo_info.threads_per_core = cs->nr_threads;
>> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>> }
>> switch (count) {
>> case 0: /* L1 dcache info */
>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
>> - eax, ebx, ecx, edx);
>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
>> + &topo_info, eax, ebx, ecx, edx);
>> break;
>> case 1: /* L1 icache info */
>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
>> - eax, ebx, ecx, edx);
>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
>> + &topo_info, eax, ebx, ecx, edx);
>> break;
>> case 2: /* L2 cache info */
>> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
>> - eax, ebx, ecx, edx);
>> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
>> + &topo_info, eax, ebx, ecx, edx);
>> break;
>> case 3: /* L3 cache info */
>> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
>> - eax, ebx, ecx, edx);
>> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
>> + &topo_info, eax, ebx, ecx, edx);
>> break;
>> default: /* end of info */
>> *eax = *ebx = *ecx = *edx = 0;
>> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>> break;
>> case 0x8000001E:
>> assert(cpu->core_id <= 255);
>> - encode_topo_cpuid8000001e(cs, cpu,
>> - eax, ebx, ecx, edx);
>> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
>> break;
>> case 0xC0000000:
>> *eax = env->cpuid_xlevel2;
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState
2020-02-24 17:01 ` Igor Mammedov
@ 2020-02-24 17:30 ` Babu Moger
0 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-24 17:30 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 11:01 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:17:32 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> Introduce model specific apicid functions inside X86MachineState.
>> These functions will be loaded from X86CPUDefinition.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> include/hw/i386/x86.h | 9 +++++++++
>> 1 file changed, 9 insertions(+)
>>
>> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
>> index 38c2d27910..75c2462770 100644
>> --- a/include/hw/i386/x86.h
>> +++ b/include/hw/i386/x86.h
>> @@ -79,6 +79,15 @@ typedef struct {
>> uint16_t boot_cpus;
>> unsigned smp_dies;
>>
>> + /* Apic id specific handlers */
>> + uint32_t (*apicid_from_cpu_idx)(X86CPUTopoInfo *topo_info,
>> + unsigned cpu_index);
>> + void (*topo_ids_from_apicid)(apic_id_t apicid, X86CPUTopoInfo *topo_info,
>> + X86CPUTopoIDs *topo_ids);
>> + apic_id_t (*apicid_from_topo_ids)(X86CPUTopoInfo *topo_info,
>> + const X86CPUTopoIDs *topo_ids);
>> + uint32_t (*apicid_pkg_offset)(X86CPUTopoInfo *topo_info);
>> +
>> /*
>> * Address space used by IOAPIC device. All IOAPIC interrupts
>> * will be translated to MSI messages in the address space.
>>
>>
>
> Suggest to move defaults into this patch as well. i.e.
> move parts that replace direct calls with hooks from 12/16
> and set default hooks (i.e. non EPYC ones) in x86's machine_class_init().
>
> This way It's immediately obvious what you are changing and then
> followup EPYC related patch won't be cluttered by non related defaults
> conversion.
>
Ok. Sure.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-24 17:19 ` Igor Mammedov
@ 2020-02-24 17:58 ` Babu Moger
2020-02-24 22:31 ` Eduardo Habkost
0 siblings, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-24 17:58 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/24/20 11:19 AM, Igor Mammedov wrote:
> On Thu, 13 Feb 2020 12:17:46 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> Check and Load the apicid handlers from X86CPUDefinition if available.
>> Update the calling convention for the apicid handlers.
>
> Previous and this patch look too complicated for the task at the hand.
> In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
> reference to Machine into i386/cpu.c (even though it's just a helper function)
> and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
> businesses to make up APIC-IDs).
>
> I'd rather do opposite and get rid of the last explicit dependency to
> ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
> so for this series I'd just try to avoid adding more Machine dependencies.
>
> All 11/16 does is basically using hooks as a switch "I'm EPYC" to
> set epyc specific encoding topo routines.
>
> It could be accomplished by a simple Boolean flag like
> X86CPUDefinition::use_epyc_apic_id_encoding
>
> and then cpu_x86_init_apicid_fns() could be replaced with trivial
> helper like:
>
> x86_use_epyc_apic_id_encoding(char *cpu_type)
> {
> X86CPUClass *xcc = ... cpu_type ...
> return xcc->model->cpudef->use_epyc_apic_id_encoding
> }
>
> then machine could override default[1] hooks using this helper
> as the trigger
> x86_cpus_init()
> {
> // no need in dedicated function as it's the only instance it's going to be called ever
> if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
> x86ms->apicid_from_cpu_idx = ...epyc...
> x86ms->topo_ids_from_apicid = ...epyc...
> x86ms->apicid_from_topo_ids = ...epyc...
> x86ms->apicid_pkg_offset = ...epyc...
> }
> }
>
> That would be less invasive and won't create non necessary dependencies.
Yes. We can achieve the task here with your approach mentioned above. But,
we still will have a scaling issue. In future if a "new cpu model" comes
up its own decoding, then we need to add another bolean flag use_new
_cpu_apic_id_encoding. And then do that same check again. In that sense,
the current approach is bit generic. Lets also hear from Eduardo.
>
> ---
> 1) defaults should be set in x86_machine_class_init()
>
> Eduardo, what's your take on this?
>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> hw/i386/pc.c | 6 +++---
>> hw/i386/x86.c | 27 +++++++++++++++++++++++----
>> 2 files changed, 26 insertions(+), 7 deletions(-)
>>
>> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> index be72a49716..93063af6a8 100644
>> --- a/hw/i386/pc.c
>> +++ b/hw/i386/pc.c
>> @@ -1808,14 +1808,14 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
>> topo_ids.die_id = cpu->die_id;
>> topo_ids.core_id = cpu->core_id;
>> topo_ids.smt_id = cpu->thread_id;
>> - cpu->apic_id = x86_apicid_from_topo_ids(&topo_info, &topo_ids);
>> + cpu->apic_id = x86ms->apicid_from_topo_ids(&topo_info, &topo_ids);
>> }
>>
>> cpu_slot = pc_find_cpu_slot(MACHINE(pcms), cpu->apic_id, &idx);
>> if (!cpu_slot) {
>> MachineState *ms = MACHINE(pcms);
>>
>> - x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
>> + x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
>> error_setg(errp,
>> "Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with"
>> " APIC ID %" PRIu32 ", valid index range 0:%d",
>> @@ -1836,7 +1836,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
>> /* TODO: move socket_id/core_id/thread_id checks into x86_cpu_realizefn()
>> * once -smp refactoring is complete and there will be CPU private
>> * CPUState::nr_cores and CPUState::nr_threads fields instead of globals */
>> - x86_topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
>> + x86ms->topo_ids_from_apicid(cpu->apic_id, &topo_info, &topo_ids);
>> if (cpu->socket_id != -1 && cpu->socket_id != topo_ids.pkg_id) {
>> error_setg(errp, "property socket-id: %u doesn't match set apic-id:"
>> " 0x%x (socket-id: %u)", cpu->socket_id, cpu->apic_id, topo_ids.pkg_id);
>> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
>> index 3d944f68e6..b825861b85 100644
>> --- a/hw/i386/x86.c
>> +++ b/hw/i386/x86.c
>> @@ -52,6 +52,22 @@
>> /* Physical Address of PVH entry point read from kernel ELF NOTE */
>> static size_t pvh_start_addr;
>>
>> +/*
>> + * Check for apicid handlers in X86MachineState. Load them if
>> + * not loaded already. These handlers are loaded from X86CPUDefinition.
>> + */
>> +static void x86_check_apicid_handlers(MachineState *ms)
>> +{
>> + X86MachineState *x86ms = X86_MACHINE(ms);
>> +
>> + if (!x86ms->apicid_from_cpu_idx ||
>> + !x86ms->topo_ids_from_apicid ||
>> + !x86ms->apicid_from_topo_ids ||
>> + !x86ms->apicid_pkg_offset) {
>> + cpu_x86_init_apicid_fns(ms);
>> + }
>> +}
>> +
>> /*
>> * Calculates initial APIC ID for a specific CPU index
>> *
>> @@ -70,7 +86,7 @@ uint32_t x86_cpu_apic_id_from_index(X86MachineState *x86ms,
>>
>> init_topo_info(&topo_info, x86ms);
>>
>> - correct_id = x86_apicid_from_cpu_idx(&topo_info, cpu_index);
>> + correct_id = x86ms->apicid_from_cpu_idx(&topo_info, cpu_index);
>> if (x86mc->compat_apic_id_mode) {
>> if (cpu_index != correct_id && !warned && !qtest_enabled()) {
>> error_report("APIC IDs set in compatibility mode, "
>> @@ -148,8 +164,8 @@ int64_t x86_get_default_cpu_node_id(const MachineState *ms, int idx)
>> init_topo_info(&topo_info, x86ms);
>>
>> assert(idx < ms->possible_cpus->len);
>> - x86_topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
>> - &topo_info, &topo_ids);
>> + x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[idx].arch_id,
>> + &topo_info, &topo_ids);
>> return topo_ids.pkg_id % ms->numa_state->num_nodes;
>> }
>>
>> @@ -169,6 +185,9 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
>> return ms->possible_cpus;
>> }
>>
>> + /* Initialize apicid handlers */
>> + x86_check_apicid_handlers(ms);
>> +
>> ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
>> sizeof(CPUArchId) * max_cpus);
>> ms->possible_cpus->len = max_cpus;
>> @@ -182,7 +201,7 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms)
>> ms->possible_cpus->cpus[i].vcpus_count = 1;
>> ms->possible_cpus->cpus[i].arch_id =
>> x86_cpu_apic_id_from_index(x86ms, i);
>> - x86_topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
>> + x86ms->topo_ids_from_apicid(ms->possible_cpus->cpus[i].arch_id,
>> &topo_info, &topo_ids);
>> ms->possible_cpus->cpus[i].props.has_socket_id = true;
>> ms->possible_cpus->cpus[i].props.socket_id = topo_ids.pkg_id;
>>
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-24 17:58 ` Babu Moger
@ 2020-02-24 22:31 ` Eduardo Habkost
2020-02-24 23:13 ` Babu Moger
2020-02-25 8:16 ` Igor Mammedov
0 siblings, 2 replies; 44+ messages in thread
From: Eduardo Habkost @ 2020-02-24 22:31 UTC (permalink / raw)
To: Babu Moger; +Cc: mst, qemu-devel, pbonzini, Igor Mammedov, rth
On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
>
>
> On 2/24/20 11:19 AM, Igor Mammedov wrote:
> > On Thu, 13 Feb 2020 12:17:46 -0600
> > Babu Moger <babu.moger@amd.com> wrote:
> >
> >> Check and Load the apicid handlers from X86CPUDefinition if available.
> >> Update the calling convention for the apicid handlers.
> >
> > Previous and this patch look too complicated for the task at the hand.
> > In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
> > reference to Machine into i386/cpu.c (even though it's just a helper function)
> > and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
> > businesses to make up APIC-IDs).
> >
> > I'd rather do opposite and get rid of the last explicit dependency to
> > ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
> > so for this series I'd just try to avoid adding more Machine dependencies.
> >
> > All 11/16 does is basically using hooks as a switch "I'm EPYC" to
> > set epyc specific encoding topo routines.
> >
> > It could be accomplished by a simple Boolean flag like
> > X86CPUDefinition::use_epyc_apic_id_encoding
> >
> > and then cpu_x86_init_apicid_fns() could be replaced with trivial
> > helper like:
> >
> > x86_use_epyc_apic_id_encoding(char *cpu_type)
> > {
> > X86CPUClass *xcc = ... cpu_type ...
> > return xcc->model->cpudef->use_epyc_apic_id_encoding
> > }
> >
> > then machine could override default[1] hooks using this helper
> > as the trigger
> > x86_cpus_init()
> > {
> > // no need in dedicated function as it's the only instance it's going to be called ever
> > if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
> > x86ms->apicid_from_cpu_idx = ...epyc...
> > x86ms->topo_ids_from_apicid = ...epyc...
> > x86ms->apicid_from_topo_ids = ...epyc...
> > x86ms->apicid_pkg_offset = ...epyc...
> > }
> > }
> >
> > That would be less invasive and won't create non necessary dependencies.
>
> Yes. We can achieve the task here with your approach mentioned above. But,
> we still will have a scaling issue. In future if a "new cpu model" comes
> up its own decoding, then we need to add another bolean flag use_new
> _cpu_apic_id_encoding. And then do that same check again. In that sense,
> the current approach is bit generic. Lets also hear from Eduardo.
To be honest, I really hope the number of APIC ID initialization
variations won't grow in the future.
In either case, X86MachineState really doesn't seem to be the
right place to save the function pointers. Whether we choose a
boolean flag or a collection of function pointers, model-specific
information belong to x86CPUClass and/or X86CPUDefinition, not
MachineState.
See the reply I sent at:
https://lore.kernel.org/qemu-devel/20200128200438.GJ18770@habkost.net/
] If you need a CPU model to provide special behavior,
] you have two options:
]
] * Add a method pointer to X86CPUClass and/or X86CPUDefinition
] * Add a QOM property to enable/disable special behavior, and
] include the property in the CPU model definition.
]
] The second option might be preferable long term, but might
] require more work because the property would become visible in
] query-cpu-model-expansion and in the command line. The first
] option may be acceptable to avoid extra user-visible complexity
] in the first version.
]
]
]
] > + pcms->apicid_from_cpu_idx = x86_apicid_from_cpu_idx_epyc;
] > + pcms->topo_ids_from_apicid = x86_topo_ids_from_apicid_epyc;
] > + pcms->apicid_from_topo_ids = x86_apicid_from_topo_ids_epyc;
]
] Why do you need to override the function pointers in
] PCMachineState instead of just looking up the relevant info at
] X86CPUClass?
--
Eduardo
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-24 22:31 ` Eduardo Habkost
@ 2020-02-24 23:13 ` Babu Moger
2020-02-25 15:32 ` Eduardo Habkost
2020-02-25 8:16 ` Igor Mammedov
1 sibling, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-02-24 23:13 UTC (permalink / raw)
To: Eduardo Habkost; +Cc: mst, qemu-devel, pbonzini, Igor Mammedov, rth
On 2/24/20 4:31 PM, Eduardo Habkost wrote:
> On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
>>
>>
>> On 2/24/20 11:19 AM, Igor Mammedov wrote:
>>> On Thu, 13 Feb 2020 12:17:46 -0600
>>> Babu Moger <babu.moger@amd.com> wrote:
>>>
>>>> Check and Load the apicid handlers from X86CPUDefinition if available.
>>>> Update the calling convention for the apicid handlers.
>>>
>>> Previous and this patch look too complicated for the task at the hand.
>>> In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
>>> reference to Machine into i386/cpu.c (even though it's just a helper function)
>>> and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
>>> businesses to make up APIC-IDs).
>>>
>>> I'd rather do opposite and get rid of the last explicit dependency to
>>> ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
>>> so for this series I'd just try to avoid adding more Machine dependencies.
>>>
>>> All 11/16 does is basically using hooks as a switch "I'm EPYC" to
>>> set epyc specific encoding topo routines.
>>>
>>> It could be accomplished by a simple Boolean flag like
>>> X86CPUDefinition::use_epyc_apic_id_encoding
>>>
>>> and then cpu_x86_init_apicid_fns() could be replaced with trivial
>>> helper like:
>>>
>>> x86_use_epyc_apic_id_encoding(char *cpu_type)
>>> {
>>> X86CPUClass *xcc = ... cpu_type ...
>>> return xcc->model->cpudef->use_epyc_apic_id_encoding
>>> }
>>>
>>> then machine could override default[1] hooks using this helper
>>> as the trigger
>>> x86_cpus_init()
>>> {
>>> // no need in dedicated function as it's the only instance it's going to be called ever
>>> if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
>>> x86ms->apicid_from_cpu_idx = ...epyc...
>>> x86ms->topo_ids_from_apicid = ...epyc...
>>> x86ms->apicid_from_topo_ids = ...epyc...
>>> x86ms->apicid_pkg_offset = ...epyc...
>>> }
>>> }
>>>
>>> That would be less invasive and won't create non necessary dependencies.
>>
>> Yes. We can achieve the task here with your approach mentioned above. But,
>> we still will have a scaling issue. In future if a "new cpu model" comes
>> up its own decoding, then we need to add another bolean flag use_new
>> _cpu_apic_id_encoding. And then do that same check again. In that sense,
>> the current approach is bit generic. Lets also hear from Eduardo.
>
> To be honest, I really hope the number of APIC ID initialization
> variations won't grow in the future.
>
> In either case, X86MachineState really doesn't seem to be the
> right place to save the function pointers. Whether we choose a
> boolean flag or a collection of function pointers, model-specific
> information belong to x86CPUClass and/or X86CPUDefinition, not
> MachineState.
My bad. I completely missed that part. Yes. You mentioned that earlier.
I can move the functions pointers to X86CPUClass and initialize the
pointers from X86CPUDefinition. Thanks
>
> See the reply I sent at:
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fqemu-devel%2F20200128200438.GJ18770%40habkost.net%2F&data=02%7C01%7Cbabu.moger%40amd.com%7Cda1d1c9f34af4475596108d7b9795fef%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637181803279890359&sdata=Z%2B%2BA2%2FMIVQZfGtUe1aBLzttCQnCpZKEwshOhoVAg1%2BU%3D&reserved=0
>
> ] If you need a CPU model to provide special behavior,
> ] you have two options:
> ]
> ] * Add a method pointer to X86CPUClass and/or X86CPUDefinition
> ] * Add a QOM property to enable/disable special behavior, and
> ] include the property in the CPU model definition.
> ]
> ] The second option might be preferable long term, but might
> ] require more work because the property would become visible in
> ] query-cpu-model-expansion and in the command line. The first
> ] option may be acceptable to avoid extra user-visible complexity
> ] in the first version.
> ]
> ]
> ]
> ] > + pcms->apicid_from_cpu_idx = x86_apicid_from_cpu_idx_epyc;
> ] > + pcms->topo_ids_from_apicid = x86_topo_ids_from_apicid_epyc;
> ] > + pcms->apicid_from_topo_ids = x86_apicid_from_topo_ids_epyc;
> ]
> ] Why do you need to override the function pointers in
> ] PCMachineState instead of just looking up the relevant info at
> ] X86CPUClass?
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg
2020-02-24 17:12 ` Babu Moger
@ 2020-02-25 7:42 ` Igor Mammedov
0 siblings, 0 replies; 44+ messages in thread
From: Igor Mammedov @ 2020-02-25 7:42 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Mon, 24 Feb 2020 11:12:41 -0600
Babu Moger <babu.moger@amd.com> wrote:
> On 2/24/20 2:34 AM, Igor Mammedov wrote:
> > On Thu, 13 Feb 2020 12:17:04 -0600
> > Babu Moger <babu.moger@amd.com> wrote:
> >
> >> Update structures X86CPUTopoIDs and CPUX86State to hold the nodes_per_pkg.
> >> This is required to build EPYC mode topology.
> >>
> >> Signed-off-by: Babu Moger <babu.moger@amd.com>
> >> ---
> >> hw/i386/pc.c | 1 +
> >> hw/i386/x86.c | 2 ++
> >> include/hw/i386/topology.h | 2 ++
> >> include/hw/i386/x86.h | 1 +
> >> target/i386/cpu.c | 1 +
> >> target/i386/cpu.h | 1 +
> >> 6 files changed, 8 insertions(+)
> >>
> >> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> >> index f13721ac43..02fdb3d506 100644
> >> --- a/hw/i386/pc.c
> >> +++ b/hw/i386/pc.c
> >> @@ -1753,6 +1753,7 @@ static void pc_cpu_pre_plug(HotplugHandler *hotplug_dev,
> >> init_topo_info(&topo_info, x86ms);
> >>
> >> env->nr_dies = x86ms->smp_dies;
> >> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
> >
> > it would be better if calculation would result in valid result
> > so you won't have to later scatter MAX(env->nr_nodes, 1) everywhere.
>
> Ok. Sure.
> >
> > also I'd use earlier intialized:
> > env->nr_nodes = topo_info->nodes_per_pkg
> > to avoid repeating calculation
>
> yes. Will do it.
>
> >
> >> /*
> >> * If APIC ID is not set,
> >> diff --git a/hw/i386/x86.c b/hw/i386/x86.c
> >> index 083effb2f5..3d944f68e6 100644
> >> --- a/hw/i386/x86.c
> >> +++ b/hw/i386/x86.c
> >> @@ -89,11 +89,13 @@ void x86_cpu_new(X86MachineState *x86ms, int64_t apic_id, Error **errp)
> >> Object *cpu = NULL;
> >> Error *local_err = NULL;
> >> CPUX86State *env = NULL;
> >> + MachineState *ms = MACHINE(x86ms);
> >>
> >> cpu = object_new(MACHINE(x86ms)->cpu_type);
> >>
> >> env = &X86_CPU(cpu)->env;
> >> env->nr_dies = x86ms->smp_dies;
> >> + env->nr_nodes = ms->numa_state->num_nodes / ms->smp.sockets;
> >
> > Is this really necessary? (I think pc_cpu_pre_plug should take care of setting it)
>
> This does not seem necessary. I can add as a separate patch to remove env
> initialization from x86_cpu_new.
it doesn't have to be part of this series, but it's up to you
how to send it
>
> >
> >> object_property_set_uint(cpu, apic_id, "apic-id", &local_err);
> >> object_property_set_bool(cpu, true, "realized", &local_err);
> >> diff --git a/include/hw/i386/topology.h b/include/hw/i386/topology.h
> >> index ef0ab0b6a3..522c77e6a9 100644
> >> --- a/include/hw/i386/topology.h
> >> +++ b/include/hw/i386/topology.h
> >> @@ -41,12 +41,14 @@
> >>
> >> #include "qemu/bitops.h"
> >> #include "hw/i386/x86.h"
> >> +#include "sysemu/numa.h"
> >>
> >> static inline void init_topo_info(X86CPUTopoInfo *topo_info,
> >> const X86MachineState *x86ms)
> >> {
> >> MachineState *ms = MACHINE(x86ms);
> >>
> >> + topo_info->nodes_per_pkg = ms->numa_state->num_nodes / ms->smp.sockets;
> >> topo_info->dies_per_pkg = x86ms->smp_dies;
> >> topo_info->cores_per_die = ms->smp.cores;
> >> topo_info->threads_per_core = ms->smp.threads;
> >> diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
> >> index ad62b01cf2..d76fd0bbb1 100644
> >> --- a/include/hw/i386/x86.h
> >> +++ b/include/hw/i386/x86.h
> >> @@ -48,6 +48,7 @@ typedef struct X86CPUTopoIDs {
> >> } X86CPUTopoIDs;
> >>
> >> typedef struct X86CPUTopoInfo {
> >> + unsigned nodes_per_pkg;
> >> unsigned dies_per_pkg;
> >> unsigned cores_per_die;
> >> unsigned threads_per_core;
> >> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> >> index 7e630f47ac..5d6edfd09b 100644
> >> --- a/target/i386/cpu.c
> >> +++ b/target/i386/cpu.c
> >> @@ -6761,6 +6761,7 @@ static void x86_cpu_initfn(Object *obj)
> >> FeatureWord w;
> >>
> >> env->nr_dies = 1;
> >> + env->nr_nodes = 1;
> >> cpu_set_cpustate_pointers(cpu);
> >>
> >> object_property_add(obj, "family", "int",
> >> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> >> index af282936a7..627a8cb9be 100644
> >> --- a/target/i386/cpu.h
> >> +++ b/target/i386/cpu.h
> >> @@ -1603,6 +1603,7 @@ typedef struct CPUX86State {
> >> TPRAccess tpr_access_type;
> >>
> >> unsigned nr_dies;
> >> + unsigned nr_nodes;
> >> } CPUX86State;
> >>
> >> struct kvm_msrs;
> >>
> >
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-24 17:29 ` Babu Moger
@ 2020-02-25 7:49 ` Igor Mammedov
2020-02-25 15:10 ` Babu Moger
2020-03-02 17:09 ` Babu Moger
0 siblings, 2 replies; 44+ messages in thread
From: Igor Mammedov @ 2020-02-25 7:49 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Mon, 24 Feb 2020 11:29:37 -0600
Babu Moger <babu.moger@amd.com> wrote:
> On 2/24/20 2:52 AM, Igor Mammedov wrote:
> > On Thu, 13 Feb 2020 12:17:25 -0600
> > Babu Moger <babu.moger@amd.com> wrote:
> >
> >> Use the new functions from topology.h and delete the unused code. Given the
> >> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
> >> mode. Removes all the hardcoded values.
> >>
> >> Signed-off-by: Babu Moger <babu.moger@amd.com>
> >
> > modulo MAX() macro, looks fine to me
>
> Igor, Sorry. What do you mean here?
I meant s/MAX(topo_info->nodes_per_pkg, 1)/topo_info->nodes_per_pkg/
after it's made sure that topo_info->nodes_per_pkg is always valid.
(I believe I've commented on that somewhere. Series isn't split nicely,
so I've ended up applying it all and then reviewing so comments might
look out of the place sometimes, hopefully next revision will be easier
to review)
> >
> >> ---
> >> target/i386/cpu.c | 162 +++++++++++------------------------------------------
> >> 1 file changed, 35 insertions(+), 127 deletions(-)
> >>
> >> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> >> index 5d6edfd09b..19675eb696 100644
> >> --- a/target/i386/cpu.c
> >> +++ b/target/i386/cpu.c
> >> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
> >> }
> >> }
> >>
> >> -/*
> >> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
> >> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
> >> - * Define the constants to build the cpu topology. Right now, TOPOEXT
> >> - * feature is enabled only on EPYC. So, these constants are based on
> >> - * EPYC supported configurations. We may need to handle the cases if
> >> - * these values change in future.
> >> - */
> >> -/* Maximum core complexes in a node */
> >> -#define MAX_CCX 2
> >> -/* Maximum cores in a core complex */
> >> -#define MAX_CORES_IN_CCX 4
> >> -/* Maximum cores in a node */
> >> -#define MAX_CORES_IN_NODE 8
> >> -/* Maximum nodes in a socket */
> >> -#define MAX_NODES_PER_SOCKET 4
> >> -
> >> -/*
> >> - * Figure out the number of nodes required to build this config.
> >> - * Max cores in a node is 8
> >> - */
> >> -static int nodes_in_socket(int nr_cores)
> >> -{
> >> - int nodes;
> >> -
> >> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
> >> -
> >> - /* Hardware does not support config with 3 nodes, return 4 in that case */
> >> - return (nodes == 3) ? 4 : nodes;
> >> -}
> >> -
> >> -/*
> >> - * Decide the number of cores in a core complex with the given nr_cores using
> >> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
> >> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
> >> - * L3 cache is shared across all cores in a core complex. So, this will also
> >> - * tell us how many cores are sharing the L3 cache.
> >> - */
> >> -static int cores_in_core_complex(int nr_cores)
> >> -{
> >> - int nodes;
> >> -
> >> - /* Check if we can fit all the cores in one core complex */
> >> - if (nr_cores <= MAX_CORES_IN_CCX) {
> >> - return nr_cores;
> >> - }
> >> - /* Get the number of nodes required to build this config */
> >> - nodes = nodes_in_socket(nr_cores);
> >> -
> >> - /*
> >> - * Divide the cores accros all the core complexes
> >> - * Return rounded up value
> >> - */
> >> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
> >> -}
> >> -
> >> /* Encode cache info for CPUID[8000001D] */
> >> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >> - uint32_t *eax, uint32_t *ebx,
> >> - uint32_t *ecx, uint32_t *edx)
> >> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
> >> + X86CPUTopoInfo *topo_info,
> >> + uint32_t *eax, uint32_t *ebx,
> >> + uint32_t *ecx, uint32_t *edx)
> >> {
> >> uint32_t l3_cores;
> >> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
> >> +
> >> assert(cache->size == cache->line_size * cache->associativity *
> >> cache->partitions * cache->sets);
> >>
> >> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >>
> >> /* L3 is shared among multiple cores */
> >> if (cache->level == 3) {
> >> - l3_cores = cores_in_core_complex(cs->nr_cores);
> >> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
> >> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
> >> + topo_info->cores_per_die *
> >> + topo_info->threads_per_core),
> >> + nodes);
> >> + *eax |= (l3_cores - 1) << 14;
> >> } else {
> >> - *eax |= ((cs->nr_threads - 1) << 14);
> >> + *eax |= ((topo_info->threads_per_core - 1) << 14);
> >> }
> >>
> >> assert(cache->line_size > 0);
> >> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
> >> }
> >>
> >> -/* Data structure to hold the configuration info for a given core index */
> >> -struct core_topology {
> >> - /* core complex id of the current core index */
> >> - int ccx_id;
> >> - /*
> >> - * Adjusted core index for this core in the topology
> >> - * This can be 0,1,2,3 with max 4 cores in a core complex
> >> - */
> >> - int core_id;
> >> - /* Node id for this core index */
> >> - int node_id;
> >> - /* Number of nodes in this config */
> >> - int num_nodes;
> >> -};
> >> -
> >> -/*
> >> - * Build the configuration closely match the EPYC hardware. Using the EPYC
> >> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
> >> - * right now. This could change in future.
> >> - * nr_cores : Total number of cores in the config
> >> - * core_id : Core index of the current CPU
> >> - * topo : Data structure to hold all the config info for this core index
> >> - */
> >> -static void build_core_topology(int nr_cores, int core_id,
> >> - struct core_topology *topo)
> >> -{
> >> - int nodes, cores_in_ccx;
> >> -
> >> - /* First get the number of nodes required */
> >> - nodes = nodes_in_socket(nr_cores);
> >> -
> >> - cores_in_ccx = cores_in_core_complex(nr_cores);
> >> -
> >> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
> >> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
> >> - topo->core_id = core_id % cores_in_ccx;
> >> - topo->num_nodes = nodes;
> >> -}
> >> -
> >> /* Encode cache info for CPUID[8000001E] */
> >> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
> >> uint32_t *eax, uint32_t *ebx,
> >> uint32_t *ecx, uint32_t *edx)
> >> {
> >> - struct core_topology topo = {0};
> >> - unsigned long nodes;
> >> + X86CPUTopoIDs topo_ids = {0};
> >> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
> >> int shift;
> >>
> >> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
> >> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
> >> +
> >> *eax = cpu->apic_id;
> >> /*
> >> * CPUID_Fn8000001E_EBX
> >> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >> * 3 Core complex id
> >> * 1:0 Core id
> >> */
> >> - if (cs->nr_threads - 1) {
> >> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
> >> - (topo.ccx_id << 2) | topo.core_id;
> >> - } else {
> >> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
> >> - }
> >> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
> >> + (topo_ids.core_id);
> >> /*
> >> * CPUID_Fn8000001E_ECX
> >> * 31:11 Reserved
> >> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >> * 2 Socket id
> >> * 1:0 Node id
> >> */
> >> - if (topo.num_nodes <= 4) {
> >> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
> >> - topo.node_id;
> >> +
> >> + if (nodes <= 4) {
> >> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
> >> } else {
> >> /*
> >> * Node id fix up. Actual hardware supports up to 4 nodes. But with
> >> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >> * number of nodes. find_last_bit returns last set bit(0 based). Left
> >> * shift(+1) the socket id to represent all the nodes.
> >> */
> >> - nodes = topo.num_nodes - 1;
> >> + nodes -= 1;
> >> shift = find_last_bit(&nodes, 8);
> >> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
> >> - topo.node_id;
> >> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
> >> + topo_ids.node_id;
> >> }
> >> *edx = 0;
> >> }
> >> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >> uint32_t signature[3];
> >> X86CPUTopoInfo topo_info;
> >>
> >> + topo_info.nodes_per_pkg = env->nr_nodes;
> >> topo_info.dies_per_pkg = env->nr_dies;
> >> topo_info.cores_per_die = cs->nr_cores;
> >> topo_info.threads_per_core = cs->nr_threads;
> >> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >> }
> >> switch (count) {
> >> case 0: /* L1 dcache info */
> >> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
> >> - eax, ebx, ecx, edx);
> >> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
> >> + &topo_info, eax, ebx, ecx, edx);
> >> break;
> >> case 1: /* L1 icache info */
> >> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
> >> - eax, ebx, ecx, edx);
> >> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
> >> + &topo_info, eax, ebx, ecx, edx);
> >> break;
> >> case 2: /* L2 cache info */
> >> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
> >> - eax, ebx, ecx, edx);
> >> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
> >> + &topo_info, eax, ebx, ecx, edx);
> >> break;
> >> case 3: /* L3 cache info */
> >> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
> >> - eax, ebx, ecx, edx);
> >> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
> >> + &topo_info, eax, ebx, ecx, edx);
> >> break;
> >> default: /* end of info */
> >> *eax = *ebx = *ecx = *edx = 0;
> >> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >> break;
> >> case 0x8000001E:
> >> assert(cpu->core_id <= 255);
> >> - encode_topo_cpuid8000001e(cs, cpu,
> >> - eax, ebx, ecx, edx);
> >> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
> >> break;
> >> case 0xC0000000:
> >> *eax = env->cpuid_xlevel2;
> >>
> >
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-24 22:31 ` Eduardo Habkost
2020-02-24 23:13 ` Babu Moger
@ 2020-02-25 8:16 ` Igor Mammedov
2020-02-25 15:26 ` Eduardo Habkost
1 sibling, 1 reply; 44+ messages in thread
From: Igor Mammedov @ 2020-02-25 8:16 UTC (permalink / raw)
To: Eduardo Habkost; +Cc: Babu Moger, rth, pbonzini, qemu-devel, mst
On Mon, 24 Feb 2020 17:31:49 -0500
Eduardo Habkost <ehabkost@redhat.com> wrote:
> On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
> >
> >
> > On 2/24/20 11:19 AM, Igor Mammedov wrote:
> > > On Thu, 13 Feb 2020 12:17:46 -0600
> > > Babu Moger <babu.moger@amd.com> wrote:
> > >
> > >> Check and Load the apicid handlers from X86CPUDefinition if available.
> > >> Update the calling convention for the apicid handlers.
> > >
> > > Previous and this patch look too complicated for the task at the hand.
> > > In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
> > > reference to Machine into i386/cpu.c (even though it's just a helper function)
> > > and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
> > > businesses to make up APIC-IDs).
> > >
> > > I'd rather do opposite and get rid of the last explicit dependency to
> > > ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
> > > so for this series I'd just try to avoid adding more Machine dependencies.
> > >
> > > All 11/16 does is basically using hooks as a switch "I'm EPYC" to
> > > set epyc specific encoding topo routines.
> > >
> > > It could be accomplished by a simple Boolean flag like
> > > X86CPUDefinition::use_epyc_apic_id_encoding
> > >
> > > and then cpu_x86_init_apicid_fns() could be replaced with trivial
> > > helper like:
> > >
> > > x86_use_epyc_apic_id_encoding(char *cpu_type)
> > > {
> > > X86CPUClass *xcc = ... cpu_type ...
> > > return xcc->model->cpudef->use_epyc_apic_id_encoding
> > > }
> > >
> > > then machine could override default[1] hooks using this helper
> > > as the trigger
> > > x86_cpus_init()
> > > {
> > > // no need in dedicated function as it's the only instance it's going to be called ever
> > > if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
> > > x86ms->apicid_from_cpu_idx = ...epyc...
> > > x86ms->topo_ids_from_apicid = ...epyc...
> > > x86ms->apicid_from_topo_ids = ...epyc...
> > > x86ms->apicid_pkg_offset = ...epyc...
> > > }
> > > }
> > >
> > > That would be less invasive and won't create non necessary dependencies.
> >
> > Yes. We can achieve the task here with your approach mentioned above. But,
> > we still will have a scaling issue. In future if a "new cpu model" comes
> > up its own decoding, then we need to add another bolean flag use_new
> > _cpu_apic_id_encoding. And then do that same check again. In that sense,
> > the current approach is bit generic. Lets also hear from Eduardo.
Without another encoding on horizon, it looks like over-engineering.
So lets think of a more generic approach later when need arises.
> To be honest, I really hope the number of APIC ID initialization
> variations won't grow in the future.
> In either case, X86MachineState really doesn't seem to be the
> right place to save the function pointers. Whether we choose a
> boolean flag or a collection of function pointers, model-specific
> information belong to x86CPUClass and/or X86CPUDefinition, not
> MachineState.
That's where I disagree, generating APIC ID and it's assignment
shouldn't be part of CPU model. On real machines it's done by
board/firmware, there board is purpose build for specific CPU.
The same should be in QEMU case, where universal QEMU board
adapts its APIC initialization to used CPU and not other way
around (i.e. it's not CPU's job to generate IDs).
So hooks in machine look like a good approach to me.
I's fine to add small helper to CPU code to help board decide
what APIC encoding to use, but I strongly disagree in putting
more logic/data than that into CPU model.
What CPU does inside (I mean CPUID handling) it's separate
business and in that case CPU usually knows what it's encoding
and can use epyc/non_epyc functions directly if necessary.
> See the reply I sent at:
> https://lore.kernel.org/qemu-devel/20200128200438.GJ18770@habkost.net/
>
> ] If you need a CPU model to provide special behavior,
> ] you have two options:
> ]
> ] * Add a method pointer to X86CPUClass and/or X86CPUDefinition
> ] * Add a QOM property to enable/disable special behavior, and
> ] include the property in the CPU model definition.
> ]
> ] The second option might be preferable long term, but might
> ] require more work because the property would become visible in
> ] query-cpu-model-expansion and in the command line. The first
> ] option may be acceptable to avoid extra user-visible complexity
> ] in the first version.
> ]
> ]
> ]
> ] > + pcms->apicid_from_cpu_idx = x86_apicid_from_cpu_idx_epyc;
> ] > + pcms->topo_ids_from_apicid = x86_topo_ids_from_apicid_epyc;
> ] > + pcms->apicid_from_topo_ids = x86_apicid_from_topo_ids_epyc;
> ]
> ] Why do you need to override the function pointers in
> ] PCMachineState instead of just looking up the relevant info at
> ] X86CPUClass?
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-25 7:49 ` Igor Mammedov
@ 2020-02-25 15:10 ` Babu Moger
2020-03-02 17:09 ` Babu Moger
1 sibling, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-25 15:10 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/25/20 1:49 AM, Igor Mammedov wrote:
> On Mon, 24 Feb 2020 11:29:37 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> On 2/24/20 2:52 AM, Igor Mammedov wrote:
>>> On Thu, 13 Feb 2020 12:17:25 -0600
>>> Babu Moger <babu.moger@amd.com> wrote:
>>>
>>>> Use the new functions from topology.h and delete the unused code. Given the
>>>> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
>>>> mode. Removes all the hardcoded values.
>>>>
>>>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>>>
>>> modulo MAX() macro, looks fine to me
>>
>> Igor, Sorry. What do you mean here?
>
> I meant s/MAX(topo_info->nodes_per_pkg, 1)/topo_info->nodes_per_pkg/
>
> after it's made sure that topo_info->nodes_per_pkg is always valid.
>
>
> (I believe I've commented on that somewhere. Series isn't split nicely,
> so I've ended up applying it all and then reviewing so comments might
> look out of the place sometimes, hopefully next revision will be easier
> to review)
Ok. I got it. Thanks
>
>>>
>>>> ---
>>>> target/i386/cpu.c | 162 +++++++++++------------------------------------------
>>>> 1 file changed, 35 insertions(+), 127 deletions(-)
>>>>
>>>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>>>> index 5d6edfd09b..19675eb696 100644
>>>> --- a/target/i386/cpu.c
>>>> +++ b/target/i386/cpu.c
>>>> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
>>>> }
>>>> }
>>>>
>>>> -/*
>>>> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
>>>> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
>>>> - * Define the constants to build the cpu topology. Right now, TOPOEXT
>>>> - * feature is enabled only on EPYC. So, these constants are based on
>>>> - * EPYC supported configurations. We may need to handle the cases if
>>>> - * these values change in future.
>>>> - */
>>>> -/* Maximum core complexes in a node */
>>>> -#define MAX_CCX 2
>>>> -/* Maximum cores in a core complex */
>>>> -#define MAX_CORES_IN_CCX 4
>>>> -/* Maximum cores in a node */
>>>> -#define MAX_CORES_IN_NODE 8
>>>> -/* Maximum nodes in a socket */
>>>> -#define MAX_NODES_PER_SOCKET 4
>>>> -
>>>> -/*
>>>> - * Figure out the number of nodes required to build this config.
>>>> - * Max cores in a node is 8
>>>> - */
>>>> -static int nodes_in_socket(int nr_cores)
>>>> -{
>>>> - int nodes;
>>>> -
>>>> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
>>>> -
>>>> - /* Hardware does not support config with 3 nodes, return 4 in that case */
>>>> - return (nodes == 3) ? 4 : nodes;
>>>> -}
>>>> -
>>>> -/*
>>>> - * Decide the number of cores in a core complex with the given nr_cores using
>>>> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
>>>> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
>>>> - * L3 cache is shared across all cores in a core complex. So, this will also
>>>> - * tell us how many cores are sharing the L3 cache.
>>>> - */
>>>> -static int cores_in_core_complex(int nr_cores)
>>>> -{
>>>> - int nodes;
>>>> -
>>>> - /* Check if we can fit all the cores in one core complex */
>>>> - if (nr_cores <= MAX_CORES_IN_CCX) {
>>>> - return nr_cores;
>>>> - }
>>>> - /* Get the number of nodes required to build this config */
>>>> - nodes = nodes_in_socket(nr_cores);
>>>> -
>>>> - /*
>>>> - * Divide the cores accros all the core complexes
>>>> - * Return rounded up value
>>>> - */
>>>> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
>>>> -}
>>>> -
>>>> /* Encode cache info for CPUID[8000001D] */
>>>> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>> - uint32_t *eax, uint32_t *ebx,
>>>> - uint32_t *ecx, uint32_t *edx)
>>>> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
>>>> + X86CPUTopoInfo *topo_info,
>>>> + uint32_t *eax, uint32_t *ebx,
>>>> + uint32_t *ecx, uint32_t *edx)
>>>> {
>>>> uint32_t l3_cores;
>>>> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
>>>> +
>>>> assert(cache->size == cache->line_size * cache->associativity *
>>>> cache->partitions * cache->sets);
>>>>
>>>> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>>
>>>> /* L3 is shared among multiple cores */
>>>> if (cache->level == 3) {
>>>> - l3_cores = cores_in_core_complex(cs->nr_cores);
>>>> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
>>>> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
>>>> + topo_info->cores_per_die *
>>>> + topo_info->threads_per_core),
>>>> + nodes);
>>>> + *eax |= (l3_cores - 1) << 14;
>>>> } else {
>>>> - *eax |= ((cs->nr_threads - 1) << 14);
>>>> + *eax |= ((topo_info->threads_per_core - 1) << 14);
>>>> }
>>>>
>>>> assert(cache->line_size > 0);
>>>> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
>>>> }
>>>>
>>>> -/* Data structure to hold the configuration info for a given core index */
>>>> -struct core_topology {
>>>> - /* core complex id of the current core index */
>>>> - int ccx_id;
>>>> - /*
>>>> - * Adjusted core index for this core in the topology
>>>> - * This can be 0,1,2,3 with max 4 cores in a core complex
>>>> - */
>>>> - int core_id;
>>>> - /* Node id for this core index */
>>>> - int node_id;
>>>> - /* Number of nodes in this config */
>>>> - int num_nodes;
>>>> -};
>>>> -
>>>> -/*
>>>> - * Build the configuration closely match the EPYC hardware. Using the EPYC
>>>> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
>>>> - * right now. This could change in future.
>>>> - * nr_cores : Total number of cores in the config
>>>> - * core_id : Core index of the current CPU
>>>> - * topo : Data structure to hold all the config info for this core index
>>>> - */
>>>> -static void build_core_topology(int nr_cores, int core_id,
>>>> - struct core_topology *topo)
>>>> -{
>>>> - int nodes, cores_in_ccx;
>>>> -
>>>> - /* First get the number of nodes required */
>>>> - nodes = nodes_in_socket(nr_cores);
>>>> -
>>>> - cores_in_ccx = cores_in_core_complex(nr_cores);
>>>> -
>>>> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
>>>> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
>>>> - topo->core_id = core_id % cores_in_ccx;
>>>> - topo->num_nodes = nodes;
>>>> -}
>>>> -
>>>> /* Encode cache info for CPUID[8000001E] */
>>>> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
>>>> uint32_t *eax, uint32_t *ebx,
>>>> uint32_t *ecx, uint32_t *edx)
>>>> {
>>>> - struct core_topology topo = {0};
>>>> - unsigned long nodes;
>>>> + X86CPUTopoIDs topo_ids = {0};
>>>> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
>>>> int shift;
>>>>
>>>> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
>>>> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
>>>> +
>>>> *eax = cpu->apic_id;
>>>> /*
>>>> * CPUID_Fn8000001E_EBX
>>>> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * 3 Core complex id
>>>> * 1:0 Core id
>>>> */
>>>> - if (cs->nr_threads - 1) {
>>>> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
>>>> - (topo.ccx_id << 2) | topo.core_id;
>>>> - } else {
>>>> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
>>>> - }
>>>> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
>>>> + (topo_ids.core_id);
>>>> /*
>>>> * CPUID_Fn8000001E_ECX
>>>> * 31:11 Reserved
>>>> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * 2 Socket id
>>>> * 1:0 Node id
>>>> */
>>>> - if (topo.num_nodes <= 4) {
>>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
>>>> - topo.node_id;
>>>> +
>>>> + if (nodes <= 4) {
>>>> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
>>>> } else {
>>>> /*
>>>> * Node id fix up. Actual hardware supports up to 4 nodes. But with
>>>> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * number of nodes. find_last_bit returns last set bit(0 based). Left
>>>> * shift(+1) the socket id to represent all the nodes.
>>>> */
>>>> - nodes = topo.num_nodes - 1;
>>>> + nodes -= 1;
>>>> shift = find_last_bit(&nodes, 8);
>>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
>>>> - topo.node_id;
>>>> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
>>>> + topo_ids.node_id;
>>>> }
>>>> *edx = 0;
>>>> }
>>>> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> uint32_t signature[3];
>>>> X86CPUTopoInfo topo_info;
>>>>
>>>> + topo_info.nodes_per_pkg = env->nr_nodes;
>>>> topo_info.dies_per_pkg = env->nr_dies;
>>>> topo_info.cores_per_die = cs->nr_cores;
>>>> topo_info.threads_per_core = cs->nr_threads;
>>>> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> }
>>>> switch (count) {
>>>> case 0: /* L1 dcache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 1: /* L1 icache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 2: /* L2 cache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 3: /* L3 cache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> default: /* end of info */
>>>> *eax = *ebx = *ecx = *edx = 0;
>>>> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> break;
>>>> case 0x8000001E:
>>>> assert(cpu->core_id <= 255);
>>>> - encode_topo_cpuid8000001e(cs, cpu,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
>>>> break;
>>>> case 0xC0000000:
>>>> *eax = env->cpuid_xlevel2;
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-25 8:16 ` Igor Mammedov
@ 2020-02-25 15:26 ` Eduardo Habkost
0 siblings, 0 replies; 44+ messages in thread
From: Eduardo Habkost @ 2020-02-25 15:26 UTC (permalink / raw)
To: Igor Mammedov; +Cc: Babu Moger, rth, pbonzini, qemu-devel, mst
On Tue, Feb 25, 2020 at 09:16:32AM +0100, Igor Mammedov wrote:
> On Mon, 24 Feb 2020 17:31:49 -0500
> Eduardo Habkost <ehabkost@redhat.com> wrote:
>
> > On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
> > >
> > >
> > > On 2/24/20 11:19 AM, Igor Mammedov wrote:
> > > > On Thu, 13 Feb 2020 12:17:46 -0600
> > > > Babu Moger <babu.moger@amd.com> wrote:
> > > >
> > > >> Check and Load the apicid handlers from X86CPUDefinition if available.
> > > >> Update the calling convention for the apicid handlers.
> > > >
> > > > Previous and this patch look too complicated for the task at the hand.
> > > > In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
> > > > reference to Machine into i386/cpu.c (even though it's just a helper function)
> > > > and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
> > > > businesses to make up APIC-IDs).
> > > >
> > > > I'd rather do opposite and get rid of the last explicit dependency to
> > > > ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
> > > > so for this series I'd just try to avoid adding more Machine dependencies.
> > > >
> > > > All 11/16 does is basically using hooks as a switch "I'm EPYC" to
> > > > set epyc specific encoding topo routines.
> > > >
> > > > It could be accomplished by a simple Boolean flag like
> > > > X86CPUDefinition::use_epyc_apic_id_encoding
> > > >
> > > > and then cpu_x86_init_apicid_fns() could be replaced with trivial
> > > > helper like:
> > > >
> > > > x86_use_epyc_apic_id_encoding(char *cpu_type)
> > > > {
> > > > X86CPUClass *xcc = ... cpu_type ...
> > > > return xcc->model->cpudef->use_epyc_apic_id_encoding
> > > > }
> > > >
> > > > then machine could override default[1] hooks using this helper
> > > > as the trigger
> > > > x86_cpus_init()
> > > > {
> > > > // no need in dedicated function as it's the only instance it's going to be called ever
> > > > if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
> > > > x86ms->apicid_from_cpu_idx = ...epyc...
> > > > x86ms->topo_ids_from_apicid = ...epyc...
> > > > x86ms->apicid_from_topo_ids = ...epyc...
> > > > x86ms->apicid_pkg_offset = ...epyc...
[1]
> > > > }
> > > > }
> > > >
> > > > That would be less invasive and won't create non necessary dependencies.
> > >
> > > Yes. We can achieve the task here with your approach mentioned above. But,
> > > we still will have a scaling issue. In future if a "new cpu model" comes
> > > up its own decoding, then we need to add another bolean flag use_new
> > > _cpu_apic_id_encoding. And then do that same check again. In that sense,
> > > the current approach is bit generic. Lets also hear from Eduardo.
>
> Without another encoding on horizon, it looks like over-engineering.
> So lets think of a more generic approach later when need arises.
>
>
> > To be honest, I really hope the number of APIC ID initialization
> > variations won't grow in the future.
> > In either case, X86MachineState really doesn't seem to be the
> > right place to save the function pointers. Whether we choose a
> > boolean flag or a collection of function pointers, model-specific
> > information belong to x86CPUClass and/or X86CPUDefinition, not
> > MachineState.
>
> That's where I disagree, generating APIC ID and it's assignment
> shouldn't be part of CPU model. On real machines it's done by
> board/firmware, there board is purpose build for specific CPU.
>
> The same should be in QEMU case, where universal QEMU board
> adapts its APIC initialization to used CPU and not other way
> around (i.e. it's not CPU's job to generate IDs).
> So hooks in machine look like a good approach to me.
>
> I's fine to add small helper to CPU code to help board decide
> what APIC encoding to use, but I strongly disagree in putting
> more logic/data than that into CPU model.
>
> What CPU does inside (I mean CPUID handling) it's separate
> business and in that case CPU usually knows what it's encoding
> and can use epyc/non_epyc functions directly if necessary.
I see your point. I was assuming that the function pointers
would stay in cpu.c/X86CPUDefinition, then it wouldn't make sense
to copy the pointers to X86MachineClass.
If the functions are going to live in pc.c, then adding the
function pointers to X86MachineState might make sense and your
suggestion above[1] sounds reasonable.
--
Eduardo
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-24 23:13 ` Babu Moger
@ 2020-02-25 15:32 ` Eduardo Habkost
2020-02-25 15:41 ` Babu Moger
0 siblings, 1 reply; 44+ messages in thread
From: Eduardo Habkost @ 2020-02-25 15:32 UTC (permalink / raw)
To: Babu Moger; +Cc: mst, qemu-devel, pbonzini, Igor Mammedov, rth
On Mon, Feb 24, 2020 at 05:13:18PM -0600, Babu Moger wrote:
>
>
> On 2/24/20 4:31 PM, Eduardo Habkost wrote:
> > On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
> >>
> >>
> >> On 2/24/20 11:19 AM, Igor Mammedov wrote:
> >>> On Thu, 13 Feb 2020 12:17:46 -0600
> >>> Babu Moger <babu.moger@amd.com> wrote:
> >>>
> >>>> Check and Load the apicid handlers from X86CPUDefinition if available.
> >>>> Update the calling convention for the apicid handlers.
> >>>
> >>> Previous and this patch look too complicated for the task at the hand.
> >>> In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
> >>> reference to Machine into i386/cpu.c (even though it's just a helper function)
> >>> and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
> >>> businesses to make up APIC-IDs).
> >>>
> >>> I'd rather do opposite and get rid of the last explicit dependency to
> >>> ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
> >>> so for this series I'd just try to avoid adding more Machine dependencies.
> >>>
> >>> All 11/16 does is basically using hooks as a switch "I'm EPYC" to
> >>> set epyc specific encoding topo routines.
> >>>
> >>> It could be accomplished by a simple Boolean flag like
> >>> X86CPUDefinition::use_epyc_apic_id_encoding
> >>>
> >>> and then cpu_x86_init_apicid_fns() could be replaced with trivial
> >>> helper like:
> >>>
> >>> x86_use_epyc_apic_id_encoding(char *cpu_type)
> >>> {
> >>> X86CPUClass *xcc = ... cpu_type ...
> >>> return xcc->model->cpudef->use_epyc_apic_id_encoding
> >>> }
> >>>
> >>> then machine could override default[1] hooks using this helper
> >>> as the trigger
> >>> x86_cpus_init()
> >>> {
> >>> // no need in dedicated function as it's the only instance it's going to be called ever
> >>> if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
> >>> x86ms->apicid_from_cpu_idx = ...epyc...
> >>> x86ms->topo_ids_from_apicid = ...epyc...
> >>> x86ms->apicid_from_topo_ids = ...epyc...
> >>> x86ms->apicid_pkg_offset = ...epyc...
> >>> }
> >>> }
> >>>
> >>> That would be less invasive and won't create non necessary dependencies.
> >>
> >> Yes. We can achieve the task here with your approach mentioned above. But,
> >> we still will have a scaling issue. In future if a "new cpu model" comes
> >> up its own decoding, then we need to add another bolean flag use_new
> >> _cpu_apic_id_encoding. And then do that same check again. In that sense,
> >> the current approach is bit generic. Lets also hear from Eduardo.
> >
> > To be honest, I really hope the number of APIC ID initialization
> > variations won't grow in the future.
> >
> > In either case, X86MachineState really doesn't seem to be the
> > right place to save the function pointers. Whether we choose a
> > boolean flag or a collection of function pointers, model-specific
> > information belong to x86CPUClass and/or X86CPUDefinition, not
> > MachineState.
>
> My bad. I completely missed that part. Yes. You mentioned that earlier.
> I can move the functions pointers to X86CPUClass and initialize the
> pointers from X86CPUDefinition. Thanks
See my reply to Igor before doing that.
Summary is: if the function implementations are provided by the
CPU, the pointers belong to X86CPUClass. If the APIC ID
calculation logic lives in pc.c, the pointers probably can stay
in X86MachineState.
--
Eduardo
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState
2020-02-25 15:32 ` Eduardo Habkost
@ 2020-02-25 15:41 ` Babu Moger
0 siblings, 0 replies; 44+ messages in thread
From: Babu Moger @ 2020-02-25 15:41 UTC (permalink / raw)
To: Eduardo Habkost; +Cc: mst, qemu-devel, pbonzini, Igor Mammedov, rth
On 2/25/20 9:32 AM, Eduardo Habkost wrote:
> On Mon, Feb 24, 2020 at 05:13:18PM -0600, Babu Moger wrote:
>>
>>
>> On 2/24/20 4:31 PM, Eduardo Habkost wrote:
>>> On Mon, Feb 24, 2020 at 11:58:09AM -0600, Babu Moger wrote:
>>>>
>>>>
>>>> On 2/24/20 11:19 AM, Igor Mammedov wrote:
>>>>> On Thu, 13 Feb 2020 12:17:46 -0600
>>>>> Babu Moger <babu.moger@amd.com> wrote:
>>>>>
>>>>>> Check and Load the apicid handlers from X86CPUDefinition if available.
>>>>>> Update the calling convention for the apicid handlers.
>>>>>
>>>>> Previous and this patch look too complicated for the task at the hand.
>>>>> In particular, cpu_x86_init_apicid_fns() from previous patch adds 1 more
>>>>> reference to Machine into i386/cpu.c (even though it's just a helper function)
>>>>> and I think un-necessary hooks to X86CPUDefinition (it's not really CPU's
>>>>> businesses to make up APIC-IDs).
>>>>>
>>>>> I'd rather do opposite and get rid of the last explicit dependency to
>>>>> ms->smp.cpus from cpu.c. But well, it's out of scope of this series,
>>>>> so for this series I'd just try to avoid adding more Machine dependencies.
>>>>>
>>>>> All 11/16 does is basically using hooks as a switch "I'm EPYC" to
>>>>> set epyc specific encoding topo routines.
>>>>>
>>>>> It could be accomplished by a simple Boolean flag like
>>>>> X86CPUDefinition::use_epyc_apic_id_encoding
>>>>>
>>>>> and then cpu_x86_init_apicid_fns() could be replaced with trivial
>>>>> helper like:
>>>>>
>>>>> x86_use_epyc_apic_id_encoding(char *cpu_type)
>>>>> {
>>>>> X86CPUClass *xcc = ... cpu_type ...
>>>>> return xcc->model->cpudef->use_epyc_apic_id_encoding
>>>>> }
>>>>>
>>>>> then machine could override default[1] hooks using this helper
>>>>> as the trigger
>>>>> x86_cpus_init()
>>>>> {
>>>>> // no need in dedicated function as it's the only instance it's going to be called ever
>>>>> if (x86_use_epyc_apic_id_encoding(ms->cpu_type)) {
>>>>> x86ms->apicid_from_cpu_idx = ...epyc...
>>>>> x86ms->topo_ids_from_apicid = ...epyc...
>>>>> x86ms->apicid_from_topo_ids = ...epyc...
>>>>> x86ms->apicid_pkg_offset = ...epyc...
>>>>> }
>>>>> }
>>>>>
>>>>> That would be less invasive and won't create non necessary dependencies.
>>>>
>>>> Yes. We can achieve the task here with your approach mentioned above. But,
>>>> we still will have a scaling issue. In future if a "new cpu model" comes
>>>> up its own decoding, then we need to add another bolean flag use_new
>>>> _cpu_apic_id_encoding. And then do that same check again. In that sense,
>>>> the current approach is bit generic. Lets also hear from Eduardo.
>>>
>>> To be honest, I really hope the number of APIC ID initialization
>>> variations won't grow in the future.
>>>
>>> In either case, X86MachineState really doesn't seem to be the
>>> right place to save the function pointers. Whether we choose a
>>> boolean flag or a collection of function pointers, model-specific
>>> information belong to x86CPUClass and/or X86CPUDefinition, not
>>> MachineState.
>>
>> My bad. I completely missed that part. Yes. You mentioned that earlier.
>> I can move the functions pointers to X86CPUClass and initialize the
>> pointers from X86CPUDefinition. Thanks
>
> See my reply to Igor before doing that.
>
> Summary is: if the function implementations are provided by the
> CPU, the pointers belong to X86CPUClass. If the APIC ID
> calculation logic lives in pc.c, the pointers probably can stay
> in X86MachineState.
>
Ok. Sure. I will leave those pointers in X86MachineState.
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-02-25 7:49 ` Igor Mammedov
2020-02-25 15:10 ` Babu Moger
@ 2020-03-02 17:09 ` Babu Moger
2020-03-03 8:47 ` Igor Mammedov
1 sibling, 1 reply; 44+ messages in thread
From: Babu Moger @ 2020-03-02 17:09 UTC (permalink / raw)
To: Igor Mammedov; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On 2/25/20 1:49 AM, Igor Mammedov wrote:
> On Mon, 24 Feb 2020 11:29:37 -0600
> Babu Moger <babu.moger@amd.com> wrote:
>
>> On 2/24/20 2:52 AM, Igor Mammedov wrote:
>>> On Thu, 13 Feb 2020 12:17:25 -0600
>>> Babu Moger <babu.moger@amd.com> wrote:
>>>
>>>> Use the new functions from topology.h and delete the unused code. Given the
>>>> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
>>>> mode. Removes all the hardcoded values.
>>>>
>>>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>>>
>>> modulo MAX() macro, looks fine to me
>>
>> Igor, Sorry. What do you mean here?
>
> I meant s/MAX(topo_info->nodes_per_pkg, 1)/topo_info->nodes_per_pkg/
>
> after it's made sure that topo_info->nodes_per_pkg is always valid.
>
Noticed that we cannot change it in all the places and assign to valid
value 1. We need this information to know weather the system is numa
configured in topology.h. This is similar to ms->numa_state->num_nodes.
This value is > 0 if system is numa configured else it is 0. I need this
information while generating the apicid. I have added comments about it in
topology.h. Hope this is not a problem.
>
> (I believe I've commented on that somewhere. Series isn't split nicely,
> so I've ended up applying it all and then reviewing so comments might
> look out of the place sometimes, hopefully next revision will be easier
> to review)
>
>>>
>>>> ---
>>>> target/i386/cpu.c | 162 +++++++++++------------------------------------------
>>>> 1 file changed, 35 insertions(+), 127 deletions(-)
>>>>
>>>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>>>> index 5d6edfd09b..19675eb696 100644
>>>> --- a/target/i386/cpu.c
>>>> +++ b/target/i386/cpu.c
>>>> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
>>>> }
>>>> }
>>>>
>>>> -/*
>>>> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
>>>> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
>>>> - * Define the constants to build the cpu topology. Right now, TOPOEXT
>>>> - * feature is enabled only on EPYC. So, these constants are based on
>>>> - * EPYC supported configurations. We may need to handle the cases if
>>>> - * these values change in future.
>>>> - */
>>>> -/* Maximum core complexes in a node */
>>>> -#define MAX_CCX 2
>>>> -/* Maximum cores in a core complex */
>>>> -#define MAX_CORES_IN_CCX 4
>>>> -/* Maximum cores in a node */
>>>> -#define MAX_CORES_IN_NODE 8
>>>> -/* Maximum nodes in a socket */
>>>> -#define MAX_NODES_PER_SOCKET 4
>>>> -
>>>> -/*
>>>> - * Figure out the number of nodes required to build this config.
>>>> - * Max cores in a node is 8
>>>> - */
>>>> -static int nodes_in_socket(int nr_cores)
>>>> -{
>>>> - int nodes;
>>>> -
>>>> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
>>>> -
>>>> - /* Hardware does not support config with 3 nodes, return 4 in that case */
>>>> - return (nodes == 3) ? 4 : nodes;
>>>> -}
>>>> -
>>>> -/*
>>>> - * Decide the number of cores in a core complex with the given nr_cores using
>>>> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
>>>> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
>>>> - * L3 cache is shared across all cores in a core complex. So, this will also
>>>> - * tell us how many cores are sharing the L3 cache.
>>>> - */
>>>> -static int cores_in_core_complex(int nr_cores)
>>>> -{
>>>> - int nodes;
>>>> -
>>>> - /* Check if we can fit all the cores in one core complex */
>>>> - if (nr_cores <= MAX_CORES_IN_CCX) {
>>>> - return nr_cores;
>>>> - }
>>>> - /* Get the number of nodes required to build this config */
>>>> - nodes = nodes_in_socket(nr_cores);
>>>> -
>>>> - /*
>>>> - * Divide the cores accros all the core complexes
>>>> - * Return rounded up value
>>>> - */
>>>> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
>>>> -}
>>>> -
>>>> /* Encode cache info for CPUID[8000001D] */
>>>> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>> - uint32_t *eax, uint32_t *ebx,
>>>> - uint32_t *ecx, uint32_t *edx)
>>>> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
>>>> + X86CPUTopoInfo *topo_info,
>>>> + uint32_t *eax, uint32_t *ebx,
>>>> + uint32_t *ecx, uint32_t *edx)
>>>> {
>>>> uint32_t l3_cores;
>>>> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
>>>> +
>>>> assert(cache->size == cache->line_size * cache->associativity *
>>>> cache->partitions * cache->sets);
>>>>
>>>> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>>
>>>> /* L3 is shared among multiple cores */
>>>> if (cache->level == 3) {
>>>> - l3_cores = cores_in_core_complex(cs->nr_cores);
>>>> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
>>>> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
>>>> + topo_info->cores_per_die *
>>>> + topo_info->threads_per_core),
>>>> + nodes);
>>>> + *eax |= (l3_cores - 1) << 14;
>>>> } else {
>>>> - *eax |= ((cs->nr_threads - 1) << 14);
>>>> + *eax |= ((topo_info->threads_per_core - 1) << 14);
>>>> }
>>>>
>>>> assert(cache->line_size > 0);
>>>> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
>>>> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
>>>> }
>>>>
>>>> -/* Data structure to hold the configuration info for a given core index */
>>>> -struct core_topology {
>>>> - /* core complex id of the current core index */
>>>> - int ccx_id;
>>>> - /*
>>>> - * Adjusted core index for this core in the topology
>>>> - * This can be 0,1,2,3 with max 4 cores in a core complex
>>>> - */
>>>> - int core_id;
>>>> - /* Node id for this core index */
>>>> - int node_id;
>>>> - /* Number of nodes in this config */
>>>> - int num_nodes;
>>>> -};
>>>> -
>>>> -/*
>>>> - * Build the configuration closely match the EPYC hardware. Using the EPYC
>>>> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
>>>> - * right now. This could change in future.
>>>> - * nr_cores : Total number of cores in the config
>>>> - * core_id : Core index of the current CPU
>>>> - * topo : Data structure to hold all the config info for this core index
>>>> - */
>>>> -static void build_core_topology(int nr_cores, int core_id,
>>>> - struct core_topology *topo)
>>>> -{
>>>> - int nodes, cores_in_ccx;
>>>> -
>>>> - /* First get the number of nodes required */
>>>> - nodes = nodes_in_socket(nr_cores);
>>>> -
>>>> - cores_in_ccx = cores_in_core_complex(nr_cores);
>>>> -
>>>> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
>>>> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
>>>> - topo->core_id = core_id % cores_in_ccx;
>>>> - topo->num_nodes = nodes;
>>>> -}
>>>> -
>>>> /* Encode cache info for CPUID[8000001E] */
>>>> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
>>>> uint32_t *eax, uint32_t *ebx,
>>>> uint32_t *ecx, uint32_t *edx)
>>>> {
>>>> - struct core_topology topo = {0};
>>>> - unsigned long nodes;
>>>> + X86CPUTopoIDs topo_ids = {0};
>>>> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
>>>> int shift;
>>>>
>>>> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
>>>> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
>>>> +
>>>> *eax = cpu->apic_id;
>>>> /*
>>>> * CPUID_Fn8000001E_EBX
>>>> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * 3 Core complex id
>>>> * 1:0 Core id
>>>> */
>>>> - if (cs->nr_threads - 1) {
>>>> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
>>>> - (topo.ccx_id << 2) | topo.core_id;
>>>> - } else {
>>>> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
>>>> - }
>>>> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
>>>> + (topo_ids.core_id);
>>>> /*
>>>> * CPUID_Fn8000001E_ECX
>>>> * 31:11 Reserved
>>>> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * 2 Socket id
>>>> * 1:0 Node id
>>>> */
>>>> - if (topo.num_nodes <= 4) {
>>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
>>>> - topo.node_id;
>>>> +
>>>> + if (nodes <= 4) {
>>>> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
>>>> } else {
>>>> /*
>>>> * Node id fix up. Actual hardware supports up to 4 nodes. But with
>>>> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
>>>> * number of nodes. find_last_bit returns last set bit(0 based). Left
>>>> * shift(+1) the socket id to represent all the nodes.
>>>> */
>>>> - nodes = topo.num_nodes - 1;
>>>> + nodes -= 1;
>>>> shift = find_last_bit(&nodes, 8);
>>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
>>>> - topo.node_id;
>>>> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
>>>> + topo_ids.node_id;
>>>> }
>>>> *edx = 0;
>>>> }
>>>> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> uint32_t signature[3];
>>>> X86CPUTopoInfo topo_info;
>>>>
>>>> + topo_info.nodes_per_pkg = env->nr_nodes;
>>>> topo_info.dies_per_pkg = env->nr_dies;
>>>> topo_info.cores_per_die = cs->nr_cores;
>>>> topo_info.threads_per_core = cs->nr_threads;
>>>> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> }
>>>> switch (count) {
>>>> case 0: /* L1 dcache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 1: /* L1 icache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 2: /* L2 cache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> case 3: /* L3 cache info */
>>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
>>>> + &topo_info, eax, ebx, ecx, edx);
>>>> break;
>>>> default: /* end of info */
>>>> *eax = *ebx = *ecx = *edx = 0;
>>>> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
>>>> break;
>>>> case 0x8000001E:
>>>> assert(cpu->core_id <= 255);
>>>> - encode_topo_cpuid8000001e(cs, cpu,
>>>> - eax, ebx, ecx, edx);
>>>> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
>>>> break;
>>>> case 0xC0000000:
>>>> *eax = env->cpuid_xlevel2;
>>>>
>>>
>>
>
^ permalink raw reply [flat|nested] 44+ messages in thread
* Re: [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions
2020-03-02 17:09 ` Babu Moger
@ 2020-03-03 8:47 ` Igor Mammedov
0 siblings, 0 replies; 44+ messages in thread
From: Igor Mammedov @ 2020-03-03 8:47 UTC (permalink / raw)
To: Babu Moger; +Cc: ehabkost, mst, qemu-devel, pbonzini, rth
On Mon, 2 Mar 2020 11:09:14 -0600
Babu Moger <babu.moger@amd.com> wrote:
> On 2/25/20 1:49 AM, Igor Mammedov wrote:
> > On Mon, 24 Feb 2020 11:29:37 -0600
> > Babu Moger <babu.moger@amd.com> wrote:
> >
> >> On 2/24/20 2:52 AM, Igor Mammedov wrote:
> >>> On Thu, 13 Feb 2020 12:17:25 -0600
> >>> Babu Moger <babu.moger@amd.com> wrote:
> >>>
> >>>> Use the new functions from topology.h and delete the unused code. Given the
> >>>> sockets, nodes, cores and threads, the new functions generate apic id for EPYC
> >>>> mode. Removes all the hardcoded values.
> >>>>
> >>>> Signed-off-by: Babu Moger <babu.moger@amd.com>
> >>>
> >>> modulo MAX() macro, looks fine to me
> >>
> >> Igor, Sorry. What do you mean here?
> >
> > I meant s/MAX(topo_info->nodes_per_pkg, 1)/topo_info->nodes_per_pkg/
> >
> > after it's made sure that topo_info->nodes_per_pkg is always valid.
> >
> Noticed that we cannot change it in all the places and assign to valid
> value 1. We need this information to know weather the system is numa
> configured in topology.h. This is similar to ms->numa_state->num_nodes.
> This value is > 0 if system is numa configured else it is 0. I need this
> information while generating the apicid. I have added comments about it in
> topology.h. Hope this is not a problem.
It should be fine as far as that's the only and best way to handle it
and it's clear from patches why it's done this way.
> >
> > (I believe I've commented on that somewhere. Series isn't split nicely,
> > so I've ended up applying it all and then reviewing so comments might
> > look out of the place sometimes, hopefully next revision will be easier
> > to review)
> >
> >>>
> >>>> ---
> >>>> target/i386/cpu.c | 162 +++++++++++------------------------------------------
> >>>> 1 file changed, 35 insertions(+), 127 deletions(-)
> >>>>
> >>>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> >>>> index 5d6edfd09b..19675eb696 100644
> >>>> --- a/target/i386/cpu.c
> >>>> +++ b/target/i386/cpu.c
> >>>> @@ -338,68 +338,15 @@ static void encode_cache_cpuid80000006(CPUCacheInfo *l2,
> >>>> }
> >>>> }
> >>>>
> >>>> -/*
> >>>> - * Definitions used for building CPUID Leaf 0x8000001D and 0x8000001E
> >>>> - * Please refer to the AMD64 Architecture Programmer’s Manual Volume 3.
> >>>> - * Define the constants to build the cpu topology. Right now, TOPOEXT
> >>>> - * feature is enabled only on EPYC. So, these constants are based on
> >>>> - * EPYC supported configurations. We may need to handle the cases if
> >>>> - * these values change in future.
> >>>> - */
> >>>> -/* Maximum core complexes in a node */
> >>>> -#define MAX_CCX 2
> >>>> -/* Maximum cores in a core complex */
> >>>> -#define MAX_CORES_IN_CCX 4
> >>>> -/* Maximum cores in a node */
> >>>> -#define MAX_CORES_IN_NODE 8
> >>>> -/* Maximum nodes in a socket */
> >>>> -#define MAX_NODES_PER_SOCKET 4
> >>>> -
> >>>> -/*
> >>>> - * Figure out the number of nodes required to build this config.
> >>>> - * Max cores in a node is 8
> >>>> - */
> >>>> -static int nodes_in_socket(int nr_cores)
> >>>> -{
> >>>> - int nodes;
> >>>> -
> >>>> - nodes = DIV_ROUND_UP(nr_cores, MAX_CORES_IN_NODE);
> >>>> -
> >>>> - /* Hardware does not support config with 3 nodes, return 4 in that case */
> >>>> - return (nodes == 3) ? 4 : nodes;
> >>>> -}
> >>>> -
> >>>> -/*
> >>>> - * Decide the number of cores in a core complex with the given nr_cores using
> >>>> - * following set constants MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE and
> >>>> - * MAX_NODES_PER_SOCKET. Maintain symmetry as much as possible
> >>>> - * L3 cache is shared across all cores in a core complex. So, this will also
> >>>> - * tell us how many cores are sharing the L3 cache.
> >>>> - */
> >>>> -static int cores_in_core_complex(int nr_cores)
> >>>> -{
> >>>> - int nodes;
> >>>> -
> >>>> - /* Check if we can fit all the cores in one core complex */
> >>>> - if (nr_cores <= MAX_CORES_IN_CCX) {
> >>>> - return nr_cores;
> >>>> - }
> >>>> - /* Get the number of nodes required to build this config */
> >>>> - nodes = nodes_in_socket(nr_cores);
> >>>> -
> >>>> - /*
> >>>> - * Divide the cores accros all the core complexes
> >>>> - * Return rounded up value
> >>>> - */
> >>>> - return DIV_ROUND_UP(nr_cores, nodes * MAX_CCX);
> >>>> -}
> >>>> -
> >>>> /* Encode cache info for CPUID[8000001D] */
> >>>> -static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >>>> - uint32_t *eax, uint32_t *ebx,
> >>>> - uint32_t *ecx, uint32_t *edx)
> >>>> +static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,
> >>>> + X86CPUTopoInfo *topo_info,
> >>>> + uint32_t *eax, uint32_t *ebx,
> >>>> + uint32_t *ecx, uint32_t *edx)
> >>>> {
> >>>> uint32_t l3_cores;
> >>>> + unsigned nodes = MAX(topo_info->nodes_per_pkg, 1);
> >>>> +
> >>>> assert(cache->size == cache->line_size * cache->associativity *
> >>>> cache->partitions * cache->sets);
> >>>>
> >>>> @@ -408,10 +355,13 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >>>>
> >>>> /* L3 is shared among multiple cores */
> >>>> if (cache->level == 3) {
> >>>> - l3_cores = cores_in_core_complex(cs->nr_cores);
> >>>> - *eax |= ((l3_cores * cs->nr_threads) - 1) << 14;
> >>>> + l3_cores = DIV_ROUND_UP((topo_info->dies_per_pkg *
> >>>> + topo_info->cores_per_die *
> >>>> + topo_info->threads_per_core),
> >>>> + nodes);
> >>>> + *eax |= (l3_cores - 1) << 14;
> >>>> } else {
> >>>> - *eax |= ((cs->nr_threads - 1) << 14);
> >>>> + *eax |= ((topo_info->threads_per_core - 1) << 14);
> >>>> }
> >>>>
> >>>> assert(cache->line_size > 0);
> >>>> @@ -431,55 +381,17 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, CPUState *cs,
> >>>> (cache->complex_indexing ? CACHE_COMPLEX_IDX : 0);
> >>>> }
> >>>>
> >>>> -/* Data structure to hold the configuration info for a given core index */
> >>>> -struct core_topology {
> >>>> - /* core complex id of the current core index */
> >>>> - int ccx_id;
> >>>> - /*
> >>>> - * Adjusted core index for this core in the topology
> >>>> - * This can be 0,1,2,3 with max 4 cores in a core complex
> >>>> - */
> >>>> - int core_id;
> >>>> - /* Node id for this core index */
> >>>> - int node_id;
> >>>> - /* Number of nodes in this config */
> >>>> - int num_nodes;
> >>>> -};
> >>>> -
> >>>> -/*
> >>>> - * Build the configuration closely match the EPYC hardware. Using the EPYC
> >>>> - * hardware configuration values (MAX_CCX, MAX_CORES_IN_CCX, MAX_CORES_IN_NODE)
> >>>> - * right now. This could change in future.
> >>>> - * nr_cores : Total number of cores in the config
> >>>> - * core_id : Core index of the current CPU
> >>>> - * topo : Data structure to hold all the config info for this core index
> >>>> - */
> >>>> -static void build_core_topology(int nr_cores, int core_id,
> >>>> - struct core_topology *topo)
> >>>> -{
> >>>> - int nodes, cores_in_ccx;
> >>>> -
> >>>> - /* First get the number of nodes required */
> >>>> - nodes = nodes_in_socket(nr_cores);
> >>>> -
> >>>> - cores_in_ccx = cores_in_core_complex(nr_cores);
> >>>> -
> >>>> - topo->node_id = core_id / (cores_in_ccx * MAX_CCX);
> >>>> - topo->ccx_id = (core_id % (cores_in_ccx * MAX_CCX)) / cores_in_ccx;
> >>>> - topo->core_id = core_id % cores_in_ccx;
> >>>> - topo->num_nodes = nodes;
> >>>> -}
> >>>> -
> >>>> /* Encode cache info for CPUID[8000001E] */
> >>>> -static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >>>> +static void encode_topo_cpuid8000001e(X86CPUTopoInfo *topo_info, X86CPU *cpu,
> >>>> uint32_t *eax, uint32_t *ebx,
> >>>> uint32_t *ecx, uint32_t *edx)
> >>>> {
> >>>> - struct core_topology topo = {0};
> >>>> - unsigned long nodes;
> >>>> + X86CPUTopoIDs topo_ids = {0};
> >>>> + unsigned long nodes = MAX(topo_info->nodes_per_pkg, 1);
> >>>> int shift;
> >>>>
> >>>> - build_core_topology(cs->nr_cores, cpu->core_id, &topo);
> >>>> + x86_topo_ids_from_apicid_epyc(cpu->apic_id, topo_info, &topo_ids);
> >>>> +
> >>>> *eax = cpu->apic_id;
> >>>> /*
> >>>> * CPUID_Fn8000001E_EBX
> >>>> @@ -496,12 +408,8 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >>>> * 3 Core complex id
> >>>> * 1:0 Core id
> >>>> */
> >>>> - if (cs->nr_threads - 1) {
> >>>> - *ebx = ((cs->nr_threads - 1) << 8) | (topo.node_id << 3) |
> >>>> - (topo.ccx_id << 2) | topo.core_id;
> >>>> - } else {
> >>>> - *ebx = (topo.node_id << 4) | (topo.ccx_id << 3) | topo.core_id;
> >>>> - }
> >>>> + *ebx = ((topo_info->threads_per_core - 1) << 8) | (topo_ids.node_id << 3) |
> >>>> + (topo_ids.core_id);
> >>>> /*
> >>>> * CPUID_Fn8000001E_ECX
> >>>> * 31:11 Reserved
> >>>> @@ -510,9 +418,9 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >>>> * 2 Socket id
> >>>> * 1:0 Node id
> >>>> */
> >>>> - if (topo.num_nodes <= 4) {
> >>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << 2) |
> >>>> - topo.node_id;
> >>>> +
> >>>> + if (nodes <= 4) {
> >>>> + *ecx = ((nodes - 1) << 8) | (topo_ids.pkg_id << 2) | topo_ids.node_id;
> >>>> } else {
> >>>> /*
> >>>> * Node id fix up. Actual hardware supports up to 4 nodes. But with
> >>>> @@ -527,10 +435,10 @@ static void encode_topo_cpuid8000001e(CPUState *cs, X86CPU *cpu,
> >>>> * number of nodes. find_last_bit returns last set bit(0 based). Left
> >>>> * shift(+1) the socket id to represent all the nodes.
> >>>> */
> >>>> - nodes = topo.num_nodes - 1;
> >>>> + nodes -= 1;
> >>>> shift = find_last_bit(&nodes, 8);
> >>>> - *ecx = ((topo.num_nodes - 1) << 8) | (cpu->socket_id << (shift + 1)) |
> >>>> - topo.node_id;
> >>>> + *ecx = (nodes << 8) | (topo_ids.pkg_id << (shift + 1)) |
> >>>> + topo_ids.node_id;
> >>>> }
> >>>> *edx = 0;
> >>>> }
> >>>> @@ -5318,6 +5226,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >>>> uint32_t signature[3];
> >>>> X86CPUTopoInfo topo_info;
> >>>>
> >>>> + topo_info.nodes_per_pkg = env->nr_nodes;
> >>>> topo_info.dies_per_pkg = env->nr_dies;
> >>>> topo_info.cores_per_die = cs->nr_cores;
> >>>> topo_info.threads_per_core = cs->nr_threads;
> >>>> @@ -5737,20 +5646,20 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >>>> }
> >>>> switch (count) {
> >>>> case 0: /* L1 dcache info */
> >>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache, cs,
> >>>> - eax, ebx, ecx, edx);
> >>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1d_cache,
> >>>> + &topo_info, eax, ebx, ecx, edx);
> >>>> break;
> >>>> case 1: /* L1 icache info */
> >>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache, cs,
> >>>> - eax, ebx, ecx, edx);
> >>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l1i_cache,
> >>>> + &topo_info, eax, ebx, ecx, edx);
> >>>> break;
> >>>> case 2: /* L2 cache info */
> >>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache, cs,
> >>>> - eax, ebx, ecx, edx);
> >>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l2_cache,
> >>>> + &topo_info, eax, ebx, ecx, edx);
> >>>> break;
> >>>> case 3: /* L3 cache info */
> >>>> - encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache, cs,
> >>>> - eax, ebx, ecx, edx);
> >>>> + encode_cache_cpuid8000001d(env->cache_info_amd.l3_cache,
> >>>> + &topo_info, eax, ebx, ecx, edx);
> >>>> break;
> >>>> default: /* end of info */
> >>>> *eax = *ebx = *ecx = *edx = 0;
> >>>> @@ -5759,8 +5668,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
> >>>> break;
> >>>> case 0x8000001E:
> >>>> assert(cpu->core_id <= 255);
> >>>> - encode_topo_cpuid8000001e(cs, cpu,
> >>>> - eax, ebx, ecx, edx);
> >>>> + encode_topo_cpuid8000001e(&topo_info, cpu, eax, ebx, ecx, edx);
> >>>> break;
> >>>> case 0xC0000000:
> >>>> *eax = env->cpuid_xlevel2;
> >>>>
> >>>
> >>
> >
>
^ permalink raw reply [flat|nested] 44+ messages in thread
end of thread, other threads:[~2020-03-03 8:48 UTC | newest]
Thread overview: 44+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-13 18:16 [PATCH v4 00/16] APIC ID fixes for AMD EPYC CPU model Babu Moger
2020-02-13 18:16 ` [PATCH v4 01/16] hw/i386: Rename X86CPUTopoInfo structure to X86CPUTopoIDs Babu Moger
2020-02-13 18:16 ` [PATCH v4 02/16] hw/i386: Introduce X86CPUTopoInfo to contain topology info Babu Moger
2020-02-13 18:16 ` [PATCH v4 03/16] hw/i386: Consolidate topology functions Babu Moger
2020-02-13 18:16 ` [PATCH v4 04/16] hw/i386: Introduce init_topo_info to initialize X86CPUTopoInfo Babu Moger
2020-02-21 17:05 ` Igor Mammedov
2020-02-21 17:51 ` Babu Moger
2020-02-24 8:18 ` Igor Mammedov
2020-02-24 16:54 ` Babu Moger
2020-02-13 18:16 ` [PATCH v4 05/16] machine: Add SMP Sockets in CpuTopology Babu Moger
2020-02-24 8:37 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 06/16] hw/i386: Update structures for nodes_per_pkg Babu Moger
2020-02-24 8:34 ` Igor Mammedov
2020-02-24 17:12 ` Babu Moger
2020-02-25 7:42 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 07/16] hw/i386: Rename apicid_from_topo_ids to x86_apicid_from_topo_ids Babu Moger
2020-02-20 12:43 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 08/16] hw/386: Add EPYC mode topology decoding functions Babu Moger
2020-02-24 8:50 ` Igor Mammedov
2020-02-24 17:24 ` Babu Moger
2020-02-13 18:17 ` [PATCH v4 09/16] target/i386: Cleanup and use the EPYC mode topology functions Babu Moger
2020-02-24 8:52 ` Igor Mammedov
2020-02-24 17:29 ` Babu Moger
2020-02-25 7:49 ` Igor Mammedov
2020-02-25 15:10 ` Babu Moger
2020-03-02 17:09 ` Babu Moger
2020-03-03 8:47 ` Igor Mammedov
2020-02-13 18:17 ` [PATCH v4 10/16] hw/i386: Introduce apicid functions inside X86MachineState Babu Moger
2020-02-24 17:01 ` Igor Mammedov
2020-02-24 17:30 ` Babu Moger
2020-02-13 18:17 ` [PATCH v4 11/16] target/i386: Load apicid model specific handlers from X86CPUDefinition Babu Moger
2020-02-13 18:17 ` [PATCH v4 12/16] hw/i386: Use the apicid handlers from X86MachineState Babu Moger
2020-02-24 17:19 ` Igor Mammedov
2020-02-24 17:58 ` Babu Moger
2020-02-24 22:31 ` Eduardo Habkost
2020-02-24 23:13 ` Babu Moger
2020-02-25 15:32 ` Eduardo Habkost
2020-02-25 15:41 ` Babu Moger
2020-02-25 8:16 ` Igor Mammedov
2020-02-25 15:26 ` Eduardo Habkost
2020-02-13 18:17 ` [PATCH v4 13/16] target/i386: Add EPYC model specific handlers Babu Moger
2020-02-13 18:17 ` [PATCH v4 14/16] hw/i386: Move arch_id decode inside x86_cpus_init Babu Moger
2020-02-13 18:18 ` [PATCH v4 15/16] i386: Fix pkg_id offset for EPYC cpu models Babu Moger
2020-02-13 18:18 ` [PATCH v4 16/16] tests: Update the Unit tests Babu Moger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).