* [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation
@ 2020-05-28 16:20 Vishal Verma
2020-05-28 16:20 ` [PATCH v3 1/3] diffs-allowed: add the SRAT AML to diffs-allowed Vishal Verma
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Vishal Verma @ 2020-05-28 16:20 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Xiao Guangrong, Michael S. Tsirkin,
jingqi.liu, Dave Hansen, Eduardo Habkost, Vishal Verma,
Paolo Bonzini, Igor Mammedov, Dan Williams, Richard Henderson
Changes since v2:
- Change a repetitive OBJECT(dev) to a stored 'Object' (Igor)
- No need to return 'numamem' back to build_srat (Igor)
Changes since v1:
- Use error_abort for getters (Igor)
- Free the device list (Igor)
- Refactor the NVDIMM related portion into hw/acpi/nvdimm.c (Igor)
- Rebase onto latest master
- Add Jingqi's Reviewed-by
On the command line, one can specify a NUMA node for NVDIMM devices. If
we set up the topology to give NVDIMMs their own nodes, i.e. not
containing any CPUs or regular memory, qemu doesn't populate SRAT memory
affinity structures for these nodes. However the NFIT does reference
those proximity domains.
As a result, Linux, while parsing the SRAT, fails to initialize node
related structures for these nodes, and they never end up in the
nodes_possible map. When these are onlined at a later point (via
hotplug), this causes problems.
I've followed the instructions in bios-tables-test.c to update the
expected SRAT binary, and the tests (make check) pass. Patches 1 and 3
are the relevant ones for the binary update.
Patch 2 is the main patch which changes SRAT generation.
Vishal Verma (3):
diffs-allowed: add the SRAT AML to diffs-allowed
hw/acpi-build: account for NVDIMM numa nodes in SRAT
tests/acpi: update expected SRAT files
hw/acpi/nvdimm.c | 23 +++++++++++++++++++++++
hw/i386/acpi-build.c | 5 +++++
include/hw/mem/nvdimm.h | 1 +
tests/data/acpi/pc/SRAT.dimmpxm | Bin 392 -> 392 bytes
tests/data/acpi/q35/SRAT.dimmpxm | Bin 392 -> 392 bytes
5 files changed, 29 insertions(+)
--
2.26.2
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v3 1/3] diffs-allowed: add the SRAT AML to diffs-allowed
2020-05-28 16:20 [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation Vishal Verma
@ 2020-05-28 16:20 ` Vishal Verma
2020-05-28 16:20 ` [PATCH v3 2/3] hw/acpi-build: account for NVDIMM numa nodes in SRAT Vishal Verma
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2020-05-28 16:20 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Xiao Guangrong, Michael S. Tsirkin,
jingqi.liu, Dave Hansen, Eduardo Habkost, Vishal Verma,
Paolo Bonzini, Igor Mammedov, Dan Williams, Richard Henderson
In anticipation of a change to the SRAT generation in qemu, add the AML
file to diffs-allowed.
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
tests/qtest/bios-tables-test-allowed-diff.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index dfb8523c8b..83d3ea5032 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1 +1,2 @@
/* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/pc/SRAT.dimmpxm",
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 2/3] hw/acpi-build: account for NVDIMM numa nodes in SRAT
2020-05-28 16:20 [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation Vishal Verma
2020-05-28 16:20 ` [PATCH v3 1/3] diffs-allowed: add the SRAT AML to diffs-allowed Vishal Verma
@ 2020-05-28 16:20 ` Vishal Verma
2020-05-28 16:20 ` [PATCH v3 3/3] tests/acpi: update expected SRAT files Vishal Verma
2020-05-28 23:03 ` [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation no-reply
3 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2020-05-28 16:20 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Xiao Guangrong, Michael S. Tsirkin,
jingqi.liu, Dave Hansen, Eduardo Habkost, Vishal Verma,
Paolo Bonzini, Igor Mammedov, Dan Williams, Richard Henderson
NVDIMMs can belong to their own proximity domains, as described by the
NFIT. In such cases, the SRAT needs to have Memory Affinity structures
in the SRAT for these NVDIMMs, otherwise Linux doesn't populate node
data structures properly during NUMA initialization. See the following
for an example failure case.
https://lore.kernel.org/linux-nvdimm/20200416225438.15208-1-vishal.l.verma@intel.com/
Fix this by adding device address range and node information from
NVDIMMs to the SRAT in build_srat().
The relevant command line options to exercise this are below. Nodes 0-1
contain CPUs and regular memory, and nodes 2-3 are the NVDIMM address
space.
-numa node,nodeid=0,mem=2048M,
-numa node,nodeid=1,mem=2048M,
-numa node,nodeid=2,mem=0,
-object memory-backend-file,id=nvmem0,share,mem-path=nvdimm-0,size=16384M,align=128M
-device nvdimm,memdev=nvmem0,id=nv0,label-size=2M,node=2
-numa node,nodeid=3,mem=0,
-object memory-backend-file,id=nvmem1,share,mem-path=nvdimm-1,size=16384M,align=128M
-device nvdimm,memdev=nvmem1,id=nv1,label-size=2M,node=3
Cc: Jingqi Liu <jingqi.liu@intel.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Jingqi Liu <jingqi.liu@intel.com>
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
hw/acpi/nvdimm.c | 23 +++++++++++++++++++++++
hw/i386/acpi-build.c | 5 +++++
include/hw/mem/nvdimm.h | 1 +
3 files changed, 29 insertions(+)
diff --git a/hw/acpi/nvdimm.c b/hw/acpi/nvdimm.c
index 9316d12b70..8f7cc16add 100644
--- a/hw/acpi/nvdimm.c
+++ b/hw/acpi/nvdimm.c
@@ -28,6 +28,7 @@
#include "qemu/osdep.h"
#include "qemu/uuid.h"
+#include "qapi/error.h"
#include "hw/acpi/acpi.h"
#include "hw/acpi/aml-build.h"
#include "hw/acpi/bios-linker-loader.h"
@@ -1334,6 +1335,28 @@ static void nvdimm_build_ssdt(GArray *table_offsets, GArray *table_data,
free_aml_allocator();
}
+void nvdimm_build_srat(GArray *table_data)
+{
+ GSList *device_list = nvdimm_get_device_list();
+
+ for (; device_list; device_list = device_list->next) {
+ AcpiSratMemoryAffinity *numamem = NULL;
+ DeviceState *dev = device_list->data;
+ Object *obj = OBJECT(dev);
+ uint64_t addr, size;
+ int node;
+
+ node = object_property_get_int(obj, PC_DIMM_NODE_PROP, &error_abort);
+ addr = object_property_get_uint(obj, PC_DIMM_ADDR_PROP, &error_abort);
+ size = object_property_get_uint(obj, PC_DIMM_SIZE_PROP, &error_abort);
+
+ numamem = acpi_data_push(table_data, sizeof *numamem);
+ build_srat_memory(numamem, addr, size, node,
+ MEM_AFFINITY_ENABLED | MEM_AFFINITY_NON_VOLATILE);
+ }
+ g_slist_free(device_list);
+}
+
void nvdimm_build_acpi(GArray *table_offsets, GArray *table_data,
BIOSLinker *linker, NVDIMMState *state,
uint32_t ram_slots)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 2e15f6848e..d996525e2c 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2428,6 +2428,11 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
MEM_AFFINITY_ENABLED);
}
}
+
+ if (machine->nvdimms_state->is_enabled) {
+ nvdimm_build_srat(table_data);
+ }
+
slots = (table_data->len - numa_start) / sizeof *numamem;
for (; slots < pcms->numa_nodes + 2; slots++) {
numamem = acpi_data_push(table_data, sizeof *numamem);
diff --git a/include/hw/mem/nvdimm.h b/include/hw/mem/nvdimm.h
index a3c08955e8..b67a1aedf6 100644
--- a/include/hw/mem/nvdimm.h
+++ b/include/hw/mem/nvdimm.h
@@ -155,6 +155,7 @@ typedef struct NVDIMMState NVDIMMState;
void nvdimm_init_acpi_state(NVDIMMState *state, MemoryRegion *io,
struct AcpiGenericAddress dsm_io,
FWCfgState *fw_cfg, Object *owner);
+void nvdimm_build_srat(GArray *table_data);
void nvdimm_build_acpi(GArray *table_offsets, GArray *table_data,
BIOSLinker *linker, NVDIMMState *state,
uint32_t ram_slots);
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v3 3/3] tests/acpi: update expected SRAT files
2020-05-28 16:20 [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation Vishal Verma
2020-05-28 16:20 ` [PATCH v3 1/3] diffs-allowed: add the SRAT AML to diffs-allowed Vishal Verma
2020-05-28 16:20 ` [PATCH v3 2/3] hw/acpi-build: account for NVDIMM numa nodes in SRAT Vishal Verma
@ 2020-05-28 16:20 ` Vishal Verma
2020-05-28 23:03 ` [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation no-reply
3 siblings, 0 replies; 6+ messages in thread
From: Vishal Verma @ 2020-05-28 16:20 UTC (permalink / raw)
To: qemu-devel
Cc: Laurent Vivier, Thomas Huth, Xiao Guangrong, Michael S. Tsirkin,
jingqi.liu, Dave Hansen, Eduardo Habkost, Vishal Verma,
Paolo Bonzini, Igor Mammedov, Dan Williams, Richard Henderson
Update the expected SRAT files for the change to account for NVDIMM numa
nodes in the SRAT.
AML Diff:
--- /tmp/asl-V49YJ0.dsl 2020-04-27 18:50:52.680043327 -0600
+++ /tmp/asl-48AZJ0.dsl 2020-04-27 18:50:52.679043344 -0600
@@ -3,7 +3,7 @@
* AML/ASL+ Disassembler version 20190509 (64-bit version)
* Copyright (c) 2000 - 2019 Intel Corporation
*
- * Disassembly of tests/data/acpi/pc/SRAT.dimmpxm, Mon Apr 27 18:50:52 2020
+ * Disassembly of /tmp/aml-U3BZJ0, Mon Apr 27 18:50:52 2020
*
* ACPI Data Table [SRAT]
*
@@ -13,7 +13,7 @@
[000h 0000 4] Signature : "SRAT" [System Resource Affinity Table]
[004h 0004 4] Table Length : 00000188
[008h 0008 1] Revision : 01
-[009h 0009 1] Checksum : 80
+[009h 0009 1] Checksum : 68
[00Ah 0010 6] Oem ID : "BOCHS "
[010h 0016 8] Oem Table ID : "BXPCSRAT"
[018h 0024 4] Oem Revision : 00000001
@@ -140,15 +140,15 @@
[138h 0312 1] Subtable Type : 01 [Memory Affinity]
[139h 0313 1] Length : 28
-[13Ah 0314 4] Proximity Domain : 00000000
+[13Ah 0314 4] Proximity Domain : 00000002
[13Eh 0318 2] Reserved1 : 0000
-[140h 0320 8] Base Address : 0000000000000000
-[148h 0328 8] Address Length : 0000000000000000
+[140h 0320 8] Base Address : 0000000108000000
+[148h 0328 8] Address Length : 0000000008000000
[150h 0336 4] Reserved2 : 00000000
-[154h 0340 4] Flags (decoded below) : 00000000
- Enabled : 0
+[154h 0340 4] Flags (decoded below) : 00000005
+ Enabled : 1
Hot Pluggable : 0
- Non-Volatile : 0
+ Non-Volatile : 1
[158h 0344 8] Reserved3 : 0000000000000000
[160h 0352 1] Subtable Type : 01 [Memory Affinity]
@@ -167,7 +167,7 @@
Raw Table Data: Length 392 (0x188)
- 0000: 53 52 41 54 88 01 00 00 01 80 42 4F 43 48 53 20 // SRAT......BOCHS
+ 0000: 53 52 41 54 88 01 00 00 01 68 42 4F 43 48 53 20 // SRAT.....hBOCHS
0010: 42 58 50 43 53 52 41 54 01 00 00 00 42 58 50 43 // BXPCSRAT....BXPC
0020: 01 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0030: 00 10 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
@@ -186,9 +186,9 @@
0100: 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 // ................
0110: 01 28 03 00 00 00 00 00 00 00 00 06 00 00 00 00 // .(..............
0120: 00 00 00 02 00 00 00 00 00 00 00 00 01 00 00 00 // ................
- 0130: 00 00 00 00 00 00 00 00 01 28 00 00 00 00 00 00 // .........(......
- 0140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
- 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
+ 0130: 00 00 00 00 00 00 00 00 01 28 02 00 00 00 00 00 // .........(......
+ 0140: 00 00 00 08 01 00 00 00 00 00 00 08 00 00 00 00 // ................
+ 0150: 00 00 00 00 05 00 00 00 00 00 00 00 00 00 00 00 // ................
0160: 01 28 03 00 00 00 00 00 00 00 00 00 01 00 00 00 // .(..............
0170: 00 00 00 F8 00 00 00 00 00 00 00 00 03 00 00 00 // ................
0180: 00 00 00 00 00 00 00 00 // ........
Signed-off-by: Vishal Verma <vishal.l.verma@intel.com>
---
tests/data/acpi/pc/SRAT.dimmpxm | Bin 392 -> 392 bytes
tests/data/acpi/q35/SRAT.dimmpxm | Bin 392 -> 392 bytes
tests/qtest/bios-tables-test-allowed-diff.h | 1 -
3 files changed, 1 deletion(-)
diff --git a/tests/data/acpi/pc/SRAT.dimmpxm b/tests/data/acpi/pc/SRAT.dimmpxm
index f5c0267ea24bb404b6b4e687390140378fbdc3f1..5a13c61b9041c6045c29643bf93a111fb1c0c76a 100644
GIT binary patch
delta 51
scmeBR?qKE$4ss0XU}Rum%-G0fz$nec00kUCF%aN@Pz(&LlS3Je0lmQmhyVZp
delta 51
icmeBR?qKE$4ss0XU}RumY}m+Uz$ndt8%z#mGzI{_tp$hx
diff --git a/tests/data/acpi/q35/SRAT.dimmpxm b/tests/data/acpi/q35/SRAT.dimmpxm
index f5c0267ea24bb404b6b4e687390140378fbdc3f1..5a13c61b9041c6045c29643bf93a111fb1c0c76a 100644
GIT binary patch
delta 51
scmeBR?qKE$4ss0XU}Rum%-G0fz$nec00kUCF%aN@Pz(&LlS3Je0lmQmhyVZp
delta 51
icmeBR?qKE$4ss0XU}RumY}m+Uz$ndt8%z#mGzI{_tp$hx
diff --git a/tests/qtest/bios-tables-test-allowed-diff.h b/tests/qtest/bios-tables-test-allowed-diff.h
index 83d3ea5032..dfb8523c8b 100644
--- a/tests/qtest/bios-tables-test-allowed-diff.h
+++ b/tests/qtest/bios-tables-test-allowed-diff.h
@@ -1,2 +1 @@
/* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/pc/SRAT.dimmpxm",
--
2.26.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation
2020-05-28 16:20 [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation Vishal Verma
` (2 preceding siblings ...)
2020-05-28 16:20 ` [PATCH v3 3/3] tests/acpi: update expected SRAT files Vishal Verma
@ 2020-05-28 23:03 ` no-reply
2020-06-02 5:37 ` Michael S. Tsirkin
3 siblings, 1 reply; 6+ messages in thread
From: no-reply @ 2020-05-28 23:03 UTC (permalink / raw)
To: vishal.l.verma
Cc: lvivier, thuth, xiaoguangrong.eric, mst, jingqi.liu, dave.hansen,
qemu-devel, imammedo, vishal.l.verma, pbonzini, dan.j.williams,
rth, ehabkost
Patchew URL: https://patchew.org/QEMU/20200528162011.16258-1-vishal.l.verma@intel.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Message-id: 20200528162011.16258-1-vishal.l.verma@intel.com
Subject: [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation
Type: series
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Switched to a new branch 'test'
7e9fa62 tests/acpi: update expected SRAT files
e8c3427 hw/acpi-build: account for NVDIMM numa nodes in SRAT
7598dc9 diffs-allowed: add the SRAT AML to diffs-allowed
=== OUTPUT BEGIN ===
1/3 Checking commit 7598dc9bc984 (diffs-allowed: add the SRAT AML to diffs-allowed)
2/3 Checking commit e8c342740610 (hw/acpi-build: account for NVDIMM numa nodes in SRAT)
3/3 Checking commit 7e9fa62e9d26 (tests/acpi: update expected SRAT files)
ERROR: Do not add expected files together with tests, follow instructions in tests/qtest/bios-tables-test.c: both tests/data/acpi/q35/SRAT.dimmpxm and tests/qtest/bios-tables-test-allowed-diff.h found
ERROR: Do not add expected files together with tests, follow instructions in tests/qtest/bios-tables-test.c: both tests/data/acpi/q35/SRAT.dimmpxm and tests/qtest/bios-tables-test-allowed-diff.h found
total: 2 errors, 0 warnings, 1 lines checked
Patch 3/3 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20200528162011.16258-1-vishal.l.verma@intel.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation
2020-05-28 23:03 ` [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation no-reply
@ 2020-06-02 5:37 ` Michael S. Tsirkin
0 siblings, 0 replies; 6+ messages in thread
From: Michael S. Tsirkin @ 2020-06-02 5:37 UTC (permalink / raw)
To: qemu-devel
Cc: lvivier, thuth, xiaoguangrong.eric, imammedo, vishal.l.verma,
dave.hansen, jingqi.liu, pbonzini, dan.j.williams, rth, ehabkost
On Thu, May 28, 2020 at 04:03:51PM -0700, no-reply@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/20200528162011.16258-1-vishal.l.verma@intel.com/
>
>
>
> Hi,
>
> This series seems to have some coding style problems. See output below for
> more information:
>
> Message-id: 20200528162011.16258-1-vishal.l.verma@intel.com
> Subject: [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation
> Type: series
>
> === TEST SCRIPT BEGIN ===
> #!/bin/bash
> git rev-parse base > /dev/null || exit 0
> git config --local diff.renamelimit 0
> git config --local diff.renames True
> git config --local diff.algorithm histogram
> ./scripts/checkpatch.pl --mailback base..
> === TEST SCRIPT END ===
>
> Switched to a new branch 'test'
> 7e9fa62 tests/acpi: update expected SRAT files
> e8c3427 hw/acpi-build: account for NVDIMM numa nodes in SRAT
> 7598dc9 diffs-allowed: add the SRAT AML to diffs-allowed
>
> === OUTPUT BEGIN ===
> 1/3 Checking commit 7598dc9bc984 (diffs-allowed: add the SRAT AML to diffs-allowed)
> 2/3 Checking commit e8c342740610 (hw/acpi-build: account for NVDIMM numa nodes in SRAT)
> 3/3 Checking commit 7e9fa62e9d26 (tests/acpi: update expected SRAT files)
> ERROR: Do not add expected files together with tests, follow instructions in tests/qtest/bios-tables-test.c: both tests/data/acpi/q35/SRAT.dimmpxm and tests/qtest/bios-tables-test-allowed-diff.h found
>
> ERROR: Do not add expected files together with tests, follow instructions in tests/qtest/bios-tables-test.c: both tests/data/acpi/q35/SRAT.dimmpxm and tests/qtest/bios-tables-test-allowed-diff.h found
>
> total: 2 errors, 0 warnings, 1 lines checked
>
> Patch 3/3 has style problems, please review. If any of these errors
> are false positives report them to the maintainer, see
> CHECKPATCH in MAINTAINERS.
This is a bug I introduced in checkpatch, pls ignore.
> === OUTPUT END ===
>
> Test command exited with code: 1
>
>
> The full log is available at
> http://patchew.org/logs/20200528162011.16258-1-vishal.l.verma@intel.com/testing.checkpatch/?type=message.
> ---
> Email generated automatically by Patchew [https://patchew.org/].
> Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-06-02 5:38 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-28 16:20 [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation Vishal Verma
2020-05-28 16:20 ` [PATCH v3 1/3] diffs-allowed: add the SRAT AML to diffs-allowed Vishal Verma
2020-05-28 16:20 ` [PATCH v3 2/3] hw/acpi-build: account for NVDIMM numa nodes in SRAT Vishal Verma
2020-05-28 16:20 ` [PATCH v3 3/3] tests/acpi: update expected SRAT files Vishal Verma
2020-05-28 23:03 ` [PATCH v3 0/3] account for NVDIMM nodes during SRAT generation no-reply
2020-06-02 5:37 ` Michael S. Tsirkin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.