* [PATCH v2 0/2] Specify empty NUMA node @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin There are multiple empty nodes, where no memory resides in, are allowed in virtual machine (VM) backed by QEMU. I don't find anywhere to document how the corresponding device nodes are populated. This series adds one section in Documentation/devicetree/bindings/numa.txt to document how the empty NUMA nodes are handled: no device nodes are existing for these empty NUMA nodes, but the NUMA IDs and distance map are still valid in "numa-distance-map-v1" compatible device node. PATCH[1] adds one section in Documentation/devicetree/bindings/numa.txt to specify how device nodes are populated for these empty NUMA nodes. PATCH[2] fetches NUMA IDs and distance maps from "numa-distance-map-v1" compatible device node. Changelog ========= * No device nodes for empty NUMA nodes (Rob) * Add patch to fetch NUMA IDs and distance map from the "numa-distance-map-v1" compatible device node (Rob) Gavin Shan (2): Documentation, dt, numa: Add note to empty NUMA node of, numa: Fetch empty NUMA node ID from distance map Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- drivers/of/of_numa.c | 2 + 2 files changed, 47 insertions(+), 1 deletion(-) -- 2.23.0 ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2 0/2] Specify empty NUMA node @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin There are multiple empty nodes, where no memory resides in, are allowed in virtual machine (VM) backed by QEMU. I don't find anywhere to document how the corresponding device nodes are populated. This series adds one section in Documentation/devicetree/bindings/numa.txt to document how the empty NUMA nodes are handled: no device nodes are existing for these empty NUMA nodes, but the NUMA IDs and distance map are still valid in "numa-distance-map-v1" compatible device node. PATCH[1] adds one section in Documentation/devicetree/bindings/numa.txt to specify how device nodes are populated for these empty NUMA nodes. PATCH[2] fetches NUMA IDs and distance maps from "numa-distance-map-v1" compatible device node. Changelog ========= * No device nodes for empty NUMA nodes (Rob) * Add patch to fetch NUMA IDs and distance map from the "numa-distance-map-v1" compatible device node (Rob) Gavin Shan (2): Documentation, dt, numa: Add note to empty NUMA node of, numa: Fetch empty NUMA node ID from distance map Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- drivers/of/of_numa.c | 2 + 2 files changed, 47 insertions(+), 1 deletion(-) -- 2.23.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2 0/2] Specify empty NUMA node @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: robh, linux-efi, maz, rdunlap, linux-kernel, shan.gavin, catalin.marinas, will, kvmarm, linux-arm-kernel There are multiple empty nodes, where no memory resides in, are allowed in virtual machine (VM) backed by QEMU. I don't find anywhere to document how the corresponding device nodes are populated. This series adds one section in Documentation/devicetree/bindings/numa.txt to document how the empty NUMA nodes are handled: no device nodes are existing for these empty NUMA nodes, but the NUMA IDs and distance map are still valid in "numa-distance-map-v1" compatible device node. PATCH[1] adds one section in Documentation/devicetree/bindings/numa.txt to specify how device nodes are populated for these empty NUMA nodes. PATCH[2] fetches NUMA IDs and distance maps from "numa-distance-map-v1" compatible device node. Changelog ========= * No device nodes for empty NUMA nodes (Rob) * Add patch to fetch NUMA IDs and distance map from the "numa-distance-map-v1" compatible device node (Rob) Gavin Shan (2): Documentation, dt, numa: Add note to empty NUMA node of, numa: Fetch empty NUMA node ID from distance map Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- drivers/of/of_numa.c | 2 + 2 files changed, 47 insertions(+), 1 deletion(-) -- 2.23.0 _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node 2021-09-27 6:41 ` Gavin Shan (?) @ 2021-09-27 6:41 ` Gavin Shan -1 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin The empty memory nodes, where no memory resides in, are allowed. The NUMA node IDs are still valid and parsed, but memory may be added to them through hotplug afterwards. Currently, QEMU fails to boot when multiple empty memory nodes are specified. It's caused by device-tree population failure and duplicated memory node names. The device-tree specification doesn't provide how empty NUMA nodes are handled. Besides, I finds difficulty to get where this case is documented. So lets add a section for empty memory nodes to cover it in NUMA binding document. Signed-off-by: Gavin Shan <gshan@redhat.com> --- Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt index 21b35053ca5a..42f282c2f3cc 100644 --- a/Documentation/devicetree/bindings/numa.txt +++ b/Documentation/devicetree/bindings/numa.txt @@ -103,7 +103,51 @@ Example: }; ============================================================================== -4 - Example dts +4 - Empty memory nodes +============================================================================== + +Empty memory nodes, which no memory resides in, are allowed. There are no +device nodes for these empty memory nodes. However, the NUMA node IDs and +distance maps are still valid and memory may be added into them through +hotplug afterwards. + +Example: + + memory@0 { + device_type = "memory"; + reg = <0x0 0x0 0x0 0x80000000>; + numa-node-id = <0>; + }; + + memory@80000000 { + device_type = "memory"; + reg = <0x0 0x80000000 0x0 0x80000000>; + numa-node-id = <1>; + }; + + /* Empty memory node 2 and 3 */ + distance-map { + compatible = "numa-distance-map-v1"; + distance-matrix = <0 0 10>, + <0 1 20>, + <0 2 40>, + <0 3 20>, + <1 0 20>, + <1 1 10>, + <1 2 20>, + <1 3 40>, + <2 0 40>, + <2 1 20>, + <2 2 10>, + <2 3 20>, + <3 0 20>, + <3 1 40>, + <3 2 20>, + <3 3 10>; + }; + +============================================================================== +5 - Example dts ============================================================================== Dual socket system consists of 2 boards connected through ccn bus and -- 2.23.0 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin The empty memory nodes, where no memory resides in, are allowed. The NUMA node IDs are still valid and parsed, but memory may be added to them through hotplug afterwards. Currently, QEMU fails to boot when multiple empty memory nodes are specified. It's caused by device-tree population failure and duplicated memory node names. The device-tree specification doesn't provide how empty NUMA nodes are handled. Besides, I finds difficulty to get where this case is documented. So lets add a section for empty memory nodes to cover it in NUMA binding document. Signed-off-by: Gavin Shan <gshan@redhat.com> --- Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt index 21b35053ca5a..42f282c2f3cc 100644 --- a/Documentation/devicetree/bindings/numa.txt +++ b/Documentation/devicetree/bindings/numa.txt @@ -103,7 +103,51 @@ Example: }; ============================================================================== -4 - Example dts +4 - Empty memory nodes +============================================================================== + +Empty memory nodes, which no memory resides in, are allowed. There are no +device nodes for these empty memory nodes. However, the NUMA node IDs and +distance maps are still valid and memory may be added into them through +hotplug afterwards. + +Example: + + memory@0 { + device_type = "memory"; + reg = <0x0 0x0 0x0 0x80000000>; + numa-node-id = <0>; + }; + + memory@80000000 { + device_type = "memory"; + reg = <0x0 0x80000000 0x0 0x80000000>; + numa-node-id = <1>; + }; + + /* Empty memory node 2 and 3 */ + distance-map { + compatible = "numa-distance-map-v1"; + distance-matrix = <0 0 10>, + <0 1 20>, + <0 2 40>, + <0 3 20>, + <1 0 20>, + <1 1 10>, + <1 2 20>, + <1 3 40>, + <2 0 40>, + <2 1 20>, + <2 2 10>, + <2 3 20>, + <3 0 20>, + <3 1 40>, + <3 2 20>, + <3 3 10>; + }; + +============================================================================== +5 - Example dts ============================================================================== Dual socket system consists of 2 boards connected through ccn bus and -- 2.23.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: robh, linux-efi, maz, rdunlap, linux-kernel, shan.gavin, catalin.marinas, will, kvmarm, linux-arm-kernel The empty memory nodes, where no memory resides in, are allowed. The NUMA node IDs are still valid and parsed, but memory may be added to them through hotplug afterwards. Currently, QEMU fails to boot when multiple empty memory nodes are specified. It's caused by device-tree population failure and duplicated memory node names. The device-tree specification doesn't provide how empty NUMA nodes are handled. Besides, I finds difficulty to get where this case is documented. So lets add a section for empty memory nodes to cover it in NUMA binding document. Signed-off-by: Gavin Shan <gshan@redhat.com> --- Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/Documentation/devicetree/bindings/numa.txt b/Documentation/devicetree/bindings/numa.txt index 21b35053ca5a..42f282c2f3cc 100644 --- a/Documentation/devicetree/bindings/numa.txt +++ b/Documentation/devicetree/bindings/numa.txt @@ -103,7 +103,51 @@ Example: }; ============================================================================== -4 - Example dts +4 - Empty memory nodes +============================================================================== + +Empty memory nodes, which no memory resides in, are allowed. There are no +device nodes for these empty memory nodes. However, the NUMA node IDs and +distance maps are still valid and memory may be added into them through +hotplug afterwards. + +Example: + + memory@0 { + device_type = "memory"; + reg = <0x0 0x0 0x0 0x80000000>; + numa-node-id = <0>; + }; + + memory@80000000 { + device_type = "memory"; + reg = <0x0 0x80000000 0x0 0x80000000>; + numa-node-id = <1>; + }; + + /* Empty memory node 2 and 3 */ + distance-map { + compatible = "numa-distance-map-v1"; + distance-matrix = <0 0 10>, + <0 1 20>, + <0 2 40>, + <0 3 20>, + <1 0 20>, + <1 1 10>, + <1 2 20>, + <1 3 40>, + <2 0 40>, + <2 1 20>, + <2 2 10>, + <2 3 20>, + <3 0 20>, + <3 1 40>, + <3 2 20>, + <3 3 10>; + }; + +============================================================================== +5 - Example dts ============================================================================== Dual socket system consists of 2 boards connected through ccn bus and -- 2.23.0 _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node 2021-09-27 6:41 ` Gavin Shan (?) @ 2021-10-04 18:13 ` Rob Herring -1 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, catalin.marinas, shan.gavin, will, rdunlap, maz, linux-efi, kvmarm, linux-arm-kernel, linux-kernel On Mon, 27 Sep 2021 14:41:18 +0800, Gavin Shan wrote: > The empty memory nodes, where no memory resides in, are allowed. > The NUMA node IDs are still valid and parsed, but memory may be > added to them through hotplug afterwards. Currently, QEMU fails > to boot when multiple empty memory nodes are specified. It's > caused by device-tree population failure and duplicated memory > node names. > > The device-tree specification doesn't provide how empty NUMA > nodes are handled. Besides, I finds difficulty to get where > this case is documented. So lets add a section for empty memory > nodes to cover it in NUMA binding document. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- > 1 file changed, 45 insertions(+), 1 deletion(-) > Applied, thanks! ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node @ 2021-10-04 18:13 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, catalin.marinas, shan.gavin, will, rdunlap, maz, linux-efi, kvmarm, linux-arm-kernel, linux-kernel On Mon, 27 Sep 2021 14:41:18 +0800, Gavin Shan wrote: > The empty memory nodes, where no memory resides in, are allowed. > The NUMA node IDs are still valid and parsed, but memory may be > added to them through hotplug afterwards. Currently, QEMU fails > to boot when multiple empty memory nodes are specified. It's > caused by device-tree population failure and duplicated memory > node names. > > The device-tree specification doesn't provide how empty NUMA > nodes are handled. Besides, I finds difficulty to get where > this case is documented. So lets add a section for empty memory > nodes to cover it in NUMA binding document. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- > 1 file changed, 45 insertions(+), 1 deletion(-) > Applied, thanks! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 1/2] Documentation, dt, numa: Add note to empty NUMA node @ 2021-10-04 18:13 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-efi, maz, rdunlap, linux-kernel, shan.gavin, catalin.marinas, will, kvmarm, linux-arm-kernel On Mon, 27 Sep 2021 14:41:18 +0800, Gavin Shan wrote: > The empty memory nodes, where no memory resides in, are allowed. > The NUMA node IDs are still valid and parsed, but memory may be > added to them through hotplug afterwards. Currently, QEMU fails > to boot when multiple empty memory nodes are specified. It's > caused by device-tree population failure and duplicated memory > node names. > > The device-tree specification doesn't provide how empty NUMA > nodes are handled. Besides, I finds difficulty to get where > this case is documented. So lets add a section for empty memory > nodes to cover it in NUMA binding document. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > Documentation/devicetree/bindings/numa.txt | 46 +++++++++++++++++++++- > 1 file changed, 45 insertions(+), 1 deletion(-) > Applied, thanks! _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-27 6:41 ` Gavin Shan (?) @ 2021-09-27 6:41 ` Gavin Shan -1 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin There is no device node for the empty NUMA node. However, the corresponding NUMA node ID and distance map is still valid in "numa-distance-map-v1" compatible device node. This fetches the NUMA node ID and distance map for these empty NUMA node from "numa-distance-map-v1" compatible device node. Signed-off-by: Gavin Shan <gshan@redhat.com> --- drivers/of/of_numa.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c index fe6b13608e51..5949829a1b00 100644 --- a/drivers/of/of_numa.c +++ b/drivers/of/of_numa.c @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) return -EINVAL; } + node_set(nodea, numa_nodes_parsed); + numa_set_distance(nodea, nodeb, distance); /* Set default distance of node B->A same as A->B */ -- 2.23.0 ^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: linux-kernel, linux-efi, linux-arm-kernel, kvmarm, robh, maz, rdunlap, catalin.marinas, will, shan.gavin There is no device node for the empty NUMA node. However, the corresponding NUMA node ID and distance map is still valid in "numa-distance-map-v1" compatible device node. This fetches the NUMA node ID and distance map for these empty NUMA node from "numa-distance-map-v1" compatible device node. Signed-off-by: Gavin Shan <gshan@redhat.com> --- drivers/of/of_numa.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c index fe6b13608e51..5949829a1b00 100644 --- a/drivers/of/of_numa.c +++ b/drivers/of/of_numa.c @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) return -EINVAL; } + node_set(nodea, numa_nodes_parsed); + numa_set_distance(nodea, nodeb, distance); /* Set default distance of node B->A same as A->B */ -- 2.23.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 6:41 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 6:41 UTC (permalink / raw) To: devicetree Cc: robh, linux-efi, maz, rdunlap, linux-kernel, shan.gavin, catalin.marinas, will, kvmarm, linux-arm-kernel There is no device node for the empty NUMA node. However, the corresponding NUMA node ID and distance map is still valid in "numa-distance-map-v1" compatible device node. This fetches the NUMA node ID and distance map for these empty NUMA node from "numa-distance-map-v1" compatible device node. Signed-off-by: Gavin Shan <gshan@redhat.com> --- drivers/of/of_numa.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c index fe6b13608e51..5949829a1b00 100644 --- a/drivers/of/of_numa.c +++ b/drivers/of/of_numa.c @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) return -EINVAL; } + node_set(nodea, numa_nodes_parsed); + numa_set_distance(nodea, nodeb, distance); /* Set default distance of node B->A same as A->B */ -- 2.23.0 _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-27 6:41 ` Gavin Shan (?) @ 2021-09-27 14:49 ` Rob Herring -1 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-27 14:49 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. This is much nicer. > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > index fe6b13608e51..5949829a1b00 100644 > --- a/drivers/of/of_numa.c > +++ b/drivers/of/of_numa.c > @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > return -EINVAL; > } > > + node_set(nodea, numa_nodes_parsed); > + With this, couldn't we remove of_numa_parse_cpu_nodes() as the only thing it does is node_set()? > numa_set_distance(nodea, nodeb, distance); > > /* Set default distance of node B->A same as A->B */ > -- > 2.23.0 > ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 14:49 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-27 14:49 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. This is much nicer. > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > index fe6b13608e51..5949829a1b00 100644 > --- a/drivers/of/of_numa.c > +++ b/drivers/of/of_numa.c > @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > return -EINVAL; > } > > + node_set(nodea, numa_nodes_parsed); > + With this, couldn't we remove of_numa_parse_cpu_nodes() as the only thing it does is node_set()? > numa_set_distance(nodea, nodeb, distance); > > /* Set default distance of node B->A same as A->B */ > -- > 2.23.0 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 14:49 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-27 14:49 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-efi, Marc Zyngier, Randy Dunlap, linux-kernel, shan.gavin, Catalin Marinas, Will Deacon, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), linux-arm-kernel On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. This is much nicer. > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > index fe6b13608e51..5949829a1b00 100644 > --- a/drivers/of/of_numa.c > +++ b/drivers/of/of_numa.c > @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > return -EINVAL; > } > > + node_set(nodea, numa_nodes_parsed); > + With this, couldn't we remove of_numa_parse_cpu_nodes() as the only thing it does is node_set()? > numa_set_distance(nodea, nodeb, distance); > > /* Set default distance of node B->A same as A->B */ > -- > 2.23.0 > _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-27 14:49 ` Rob Herring (?) @ 2021-09-27 23:59 ` Gavin Shan -1 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 23:59 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin Hi Rob, On 9/28/21 12:49 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >> >> There is no device node for the empty NUMA node. However, the >> corresponding NUMA node ID and distance map is still valid in >> "numa-distance-map-v1" compatible device node. >> >> This fetches the NUMA node ID and distance map for these empty >> NUMA node from "numa-distance-map-v1" compatible device node. > > This is much nicer. > Indeed, thanks for your suggestions :) >> Signed-off-by: Gavin Shan <gshan@redhat.com> >> --- >> drivers/of/of_numa.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >> index fe6b13608e51..5949829a1b00 100644 >> --- a/drivers/of/of_numa.c >> +++ b/drivers/of/of_numa.c >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >> return -EINVAL; >> } >> >> + node_set(nodea, numa_nodes_parsed); >> + > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > thing it does is node_set()? > I don't think so for couple of reasons: (1) With problematic device-tree, the distance map node might be missed or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. (2) @numa_nodes_parsed is also updated when the memory nodes are iterated in of_numa_parse_memory_nodes() and numa_add_memblk(). So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, is the gathering output of CPU nodes, memory nodes and distance map node. >> numa_set_distance(nodea, nodeb, distance); >> >> /* Set default distance of node B->A same as A->B */ Thanks, Gavin ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 23:59 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 23:59 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin Hi Rob, On 9/28/21 12:49 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >> >> There is no device node for the empty NUMA node. However, the >> corresponding NUMA node ID and distance map is still valid in >> "numa-distance-map-v1" compatible device node. >> >> This fetches the NUMA node ID and distance map for these empty >> NUMA node from "numa-distance-map-v1" compatible device node. > > This is much nicer. > Indeed, thanks for your suggestions :) >> Signed-off-by: Gavin Shan <gshan@redhat.com> >> --- >> drivers/of/of_numa.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >> index fe6b13608e51..5949829a1b00 100644 >> --- a/drivers/of/of_numa.c >> +++ b/drivers/of/of_numa.c >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >> return -EINVAL; >> } >> >> + node_set(nodea, numa_nodes_parsed); >> + > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > thing it does is node_set()? > I don't think so for couple of reasons: (1) With problematic device-tree, the distance map node might be missed or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. (2) @numa_nodes_parsed is also updated when the memory nodes are iterated in of_numa_parse_memory_nodes() and numa_add_memblk(). So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, is the gathering output of CPU nodes, memory nodes and distance map node. >> numa_set_distance(nodea, nodeb, distance); >> >> /* Set default distance of node B->A same as A->B */ Thanks, Gavin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-27 23:59 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-27 23:59 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-efi, Marc Zyngier, Randy Dunlap, linux-kernel, shan.gavin, Catalin Marinas, Will Deacon, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), linux-arm-kernel Hi Rob, On 9/28/21 12:49 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >> >> There is no device node for the empty NUMA node. However, the >> corresponding NUMA node ID and distance map is still valid in >> "numa-distance-map-v1" compatible device node. >> >> This fetches the NUMA node ID and distance map for these empty >> NUMA node from "numa-distance-map-v1" compatible device node. > > This is much nicer. > Indeed, thanks for your suggestions :) >> Signed-off-by: Gavin Shan <gshan@redhat.com> >> --- >> drivers/of/of_numa.c | 2 ++ >> 1 file changed, 2 insertions(+) >> >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >> index fe6b13608e51..5949829a1b00 100644 >> --- a/drivers/of/of_numa.c >> +++ b/drivers/of/of_numa.c >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >> return -EINVAL; >> } >> >> + node_set(nodea, numa_nodes_parsed); >> + > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > thing it does is node_set()? > I don't think so for couple of reasons: (1) With problematic device-tree, the distance map node might be missed or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. (2) @numa_nodes_parsed is also updated when the memory nodes are iterated in of_numa_parse_memory_nodes() and numa_add_memblk(). So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, is the gathering output of CPU nodes, memory nodes and distance map node. >> numa_set_distance(nodea, nodeb, distance); >> >> /* Set default distance of node B->A same as A->B */ Thanks, Gavin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-27 23:59 ` Gavin Shan (?) @ 2021-09-28 17:22 ` Rob Herring -1 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-28 17:22 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: > > Hi Rob, > > On 9/28/21 12:49 AM, Rob Herring wrote: > > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > >> > >> There is no device node for the empty NUMA node. However, the > >> corresponding NUMA node ID and distance map is still valid in > >> "numa-distance-map-v1" compatible device node. > >> > >> This fetches the NUMA node ID and distance map for these empty > >> NUMA node from "numa-distance-map-v1" compatible device node. > > > > This is much nicer. > > > > Indeed, thanks for your suggestions :) > > >> Signed-off-by: Gavin Shan <gshan@redhat.com> > >> --- > >> drivers/of/of_numa.c | 2 ++ > >> 1 file changed, 2 insertions(+) > >> > >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > >> index fe6b13608e51..5949829a1b00 100644 > >> --- a/drivers/of/of_numa.c > >> +++ b/drivers/of/of_numa.c > >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > >> return -EINVAL; > >> } > >> > >> + node_set(nodea, numa_nodes_parsed); > >> + > > > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > > thing it does is node_set()? > > > > I don't think so for couple of reasons: > > (1) With problematic device-tree, the distance map node might be missed > or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. It's not the kernel's job to validate the DT (if it was, it is doing a terrible job). I would suggest writing some checks for dtc if we're worried about correctness. (The schemas don't work too well for cross node checks.) > (2) @numa_nodes_parsed is also updated when the memory nodes are iterated > in of_numa_parse_memory_nodes() and numa_add_memblk(). > > So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, > is the gathering output of CPU nodes, memory nodes and distance map node. Is it valid to have node id's that are not in the distance map? Rob ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-28 17:22 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-28 17:22 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: > > Hi Rob, > > On 9/28/21 12:49 AM, Rob Herring wrote: > > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > >> > >> There is no device node for the empty NUMA node. However, the > >> corresponding NUMA node ID and distance map is still valid in > >> "numa-distance-map-v1" compatible device node. > >> > >> This fetches the NUMA node ID and distance map for these empty > >> NUMA node from "numa-distance-map-v1" compatible device node. > > > > This is much nicer. > > > > Indeed, thanks for your suggestions :) > > >> Signed-off-by: Gavin Shan <gshan@redhat.com> > >> --- > >> drivers/of/of_numa.c | 2 ++ > >> 1 file changed, 2 insertions(+) > >> > >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > >> index fe6b13608e51..5949829a1b00 100644 > >> --- a/drivers/of/of_numa.c > >> +++ b/drivers/of/of_numa.c > >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > >> return -EINVAL; > >> } > >> > >> + node_set(nodea, numa_nodes_parsed); > >> + > > > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > > thing it does is node_set()? > > > > I don't think so for couple of reasons: > > (1) With problematic device-tree, the distance map node might be missed > or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. It's not the kernel's job to validate the DT (if it was, it is doing a terrible job). I would suggest writing some checks for dtc if we're worried about correctness. (The schemas don't work too well for cross node checks.) > (2) @numa_nodes_parsed is also updated when the memory nodes are iterated > in of_numa_parse_memory_nodes() and numa_add_memblk(). > > So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, > is the gathering output of CPU nodes, memory nodes and distance map node. Is it valid to have node id's that are not in the distance map? Rob _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-28 17:22 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-09-28 17:22 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-efi, Marc Zyngier, Randy Dunlap, linux-kernel, shan.gavin, Catalin Marinas, Will Deacon, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), linux-arm-kernel On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: > > Hi Rob, > > On 9/28/21 12:49 AM, Rob Herring wrote: > > On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: > >> > >> There is no device node for the empty NUMA node. However, the > >> corresponding NUMA node ID and distance map is still valid in > >> "numa-distance-map-v1" compatible device node. > >> > >> This fetches the NUMA node ID and distance map for these empty > >> NUMA node from "numa-distance-map-v1" compatible device node. > > > > This is much nicer. > > > > Indeed, thanks for your suggestions :) > > >> Signed-off-by: Gavin Shan <gshan@redhat.com> > >> --- > >> drivers/of/of_numa.c | 2 ++ > >> 1 file changed, 2 insertions(+) > >> > >> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c > >> index fe6b13608e51..5949829a1b00 100644 > >> --- a/drivers/of/of_numa.c > >> +++ b/drivers/of/of_numa.c > >> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) > >> return -EINVAL; > >> } > >> > >> + node_set(nodea, numa_nodes_parsed); > >> + > > > > With this, couldn't we remove of_numa_parse_cpu_nodes() as the only > > thing it does is node_set()? > > > > I don't think so for couple of reasons: > > (1) With problematic device-tree, the distance map node might be missed > or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. It's not the kernel's job to validate the DT (if it was, it is doing a terrible job). I would suggest writing some checks for dtc if we're worried about correctness. (The schemas don't work too well for cross node checks.) > (2) @numa_nodes_parsed is also updated when the memory nodes are iterated > in of_numa_parse_memory_nodes() and numa_add_memblk(). > > So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, > is the gathering output of CPU nodes, memory nodes and distance map node. Is it valid to have node id's that are not in the distance map? Rob _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-28 17:22 ` Rob Herring (?) @ 2021-09-29 2:00 ` Gavin Shan -1 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-29 2:00 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On 9/29/21 3:22 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: >> On 9/28/21 12:49 AM, Rob Herring wrote: >>> On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >>>> >>>> There is no device node for the empty NUMA node. However, the >>>> corresponding NUMA node ID and distance map is still valid in >>>> "numa-distance-map-v1" compatible device node. >>>> >>>> This fetches the NUMA node ID and distance map for these empty >>>> NUMA node from "numa-distance-map-v1" compatible device node. >>> >>> This is much nicer. >>> >> >> Indeed, thanks for your suggestions :) >> >>>> Signed-off-by: Gavin Shan <gshan@redhat.com> >>>> --- >>>> drivers/of/of_numa.c | 2 ++ >>>> 1 file changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >>>> index fe6b13608e51..5949829a1b00 100644 >>>> --- a/drivers/of/of_numa.c >>>> +++ b/drivers/of/of_numa.c >>>> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >>>> return -EINVAL; >>>> } >>>> >>>> + node_set(nodea, numa_nodes_parsed); >>>> + >>> >>> With this, couldn't we remove of_numa_parse_cpu_nodes() as the only >>> thing it does is node_set()? >>> >> >> I don't think so for couple of reasons: >> >> (1) With problematic device-tree, the distance map node might be missed >> or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. > > It's not the kernel's job to validate the DT (if it was, it is doing a > terrible job). I would suggest writing some checks for dtc if we're > worried about correctness. (The schemas don't work too well for cross > node checks.) > I didn't look into dtc's code and not sure if dtc has this sort of validation. Besides, dtc is out of scope when QEMU is involved. The device-tree blob isn't produced by dtc in QEMU. >> (2) @numa_nodes_parsed is also updated when the memory nodes are iterated >> in of_numa_parse_memory_nodes() and numa_add_memblk(). >> >> So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, >> is the gathering output of CPU nodes, memory nodes and distance map node. > > Is it valid to have node id's that are not in the distance map? > Yes, it's valid from the kernel's perspective. The default distance matrix, where the local and remote distances are 10 and 20, is applied if the distance map is missed in device-tree. The code can be found from drivers/base/arch_numa.c::numa_alloc_distance() Besides, it's possible that the distance map isn't populated by QEMU. However, I'm going to improve the situation so the distance map will be populated unconditionally. The following option is supported by QEMU, to specify the distance between two NUMA nodes. However, it's not mandatory. The distance map in device-tree won't be populated if the option is missed. -numa dist,a=<src_numa_node>,b=<dst_numa_node>,val=<distance> Thanks, Gavin ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-29 2:00 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-29 2:00 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-kernel, linux-efi, linux-arm-kernel, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), Marc Zyngier, Randy Dunlap, Catalin Marinas, Will Deacon, shan.gavin On 9/29/21 3:22 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: >> On 9/28/21 12:49 AM, Rob Herring wrote: >>> On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >>>> >>>> There is no device node for the empty NUMA node. However, the >>>> corresponding NUMA node ID and distance map is still valid in >>>> "numa-distance-map-v1" compatible device node. >>>> >>>> This fetches the NUMA node ID and distance map for these empty >>>> NUMA node from "numa-distance-map-v1" compatible device node. >>> >>> This is much nicer. >>> >> >> Indeed, thanks for your suggestions :) >> >>>> Signed-off-by: Gavin Shan <gshan@redhat.com> >>>> --- >>>> drivers/of/of_numa.c | 2 ++ >>>> 1 file changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >>>> index fe6b13608e51..5949829a1b00 100644 >>>> --- a/drivers/of/of_numa.c >>>> +++ b/drivers/of/of_numa.c >>>> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >>>> return -EINVAL; >>>> } >>>> >>>> + node_set(nodea, numa_nodes_parsed); >>>> + >>> >>> With this, couldn't we remove of_numa_parse_cpu_nodes() as the only >>> thing it does is node_set()? >>> >> >> I don't think so for couple of reasons: >> >> (1) With problematic device-tree, the distance map node might be missed >> or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. > > It's not the kernel's job to validate the DT (if it was, it is doing a > terrible job). I would suggest writing some checks for dtc if we're > worried about correctness. (The schemas don't work too well for cross > node checks.) > I didn't look into dtc's code and not sure if dtc has this sort of validation. Besides, dtc is out of scope when QEMU is involved. The device-tree blob isn't produced by dtc in QEMU. >> (2) @numa_nodes_parsed is also updated when the memory nodes are iterated >> in of_numa_parse_memory_nodes() and numa_add_memblk(). >> >> So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, >> is the gathering output of CPU nodes, memory nodes and distance map node. > > Is it valid to have node id's that are not in the distance map? > Yes, it's valid from the kernel's perspective. The default distance matrix, where the local and remote distances are 10 and 20, is applied if the distance map is missed in device-tree. The code can be found from drivers/base/arch_numa.c::numa_alloc_distance() Besides, it's possible that the distance map isn't populated by QEMU. However, I'm going to improve the situation so the distance map will be populated unconditionally. The following option is supported by QEMU, to specify the distance between two NUMA nodes. However, it's not mandatory. The distance map in device-tree won't be populated if the option is missed. -numa dist,a=<src_numa_node>,b=<dst_numa_node>,val=<distance> Thanks, Gavin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-09-29 2:00 ` Gavin Shan 0 siblings, 0 replies; 27+ messages in thread From: Gavin Shan @ 2021-09-29 2:00 UTC (permalink / raw) To: Rob Herring Cc: devicetree, linux-efi, Marc Zyngier, Randy Dunlap, linux-kernel, shan.gavin, Catalin Marinas, Will Deacon, open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64), linux-arm-kernel On 9/29/21 3:22 AM, Rob Herring wrote: > On Mon, Sep 27, 2021 at 6:59 PM Gavin Shan <gshan@redhat.com> wrote: >> On 9/28/21 12:49 AM, Rob Herring wrote: >>> On Mon, Sep 27, 2021 at 1:42 AM Gavin Shan <gshan@redhat.com> wrote: >>>> >>>> There is no device node for the empty NUMA node. However, the >>>> corresponding NUMA node ID and distance map is still valid in >>>> "numa-distance-map-v1" compatible device node. >>>> >>>> This fetches the NUMA node ID and distance map for these empty >>>> NUMA node from "numa-distance-map-v1" compatible device node. >>> >>> This is much nicer. >>> >> >> Indeed, thanks for your suggestions :) >> >>>> Signed-off-by: Gavin Shan <gshan@redhat.com> >>>> --- >>>> drivers/of/of_numa.c | 2 ++ >>>> 1 file changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c >>>> index fe6b13608e51..5949829a1b00 100644 >>>> --- a/drivers/of/of_numa.c >>>> +++ b/drivers/of/of_numa.c >>>> @@ -111,6 +111,8 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map) >>>> return -EINVAL; >>>> } >>>> >>>> + node_set(nodea, numa_nodes_parsed); >>>> + >>> >>> With this, couldn't we remove of_numa_parse_cpu_nodes() as the only >>> thing it does is node_set()? >>> >> >> I don't think so for couple of reasons: >> >> (1) With problematic device-tree, the distance map node might be missed >> or incomplete. In this case, of_numa_parse_cpu_nodes() still helps. > > It's not the kernel's job to validate the DT (if it was, it is doing a > terrible job). I would suggest writing some checks for dtc if we're > worried about correctness. (The schemas don't work too well for cross > node checks.) > I didn't look into dtc's code and not sure if dtc has this sort of validation. Besides, dtc is out of scope when QEMU is involved. The device-tree blob isn't produced by dtc in QEMU. >> (2) @numa_nodes_parsed is also updated when the memory nodes are iterated >> in of_numa_parse_memory_nodes() and numa_add_memblk(). >> >> So @numa_nodes_parsed, which is synchronized to @node_possible_map afterwards, >> is the gathering output of CPU nodes, memory nodes and distance map node. > > Is it valid to have node id's that are not in the distance map? > Yes, it's valid from the kernel's perspective. The default distance matrix, where the local and remote distances are 10 and 20, is applied if the distance map is missed in device-tree. The code can be found from drivers/base/arch_numa.c::numa_alloc_distance() Besides, it's possible that the distance map isn't populated by QEMU. However, I'm going to improve the situation so the distance map will be populated unconditionally. The following option is supported by QEMU, to specify the distance between two NUMA nodes. However, it's not mandatory. The distance map in device-tree won't be populated if the option is missed. -numa dist,a=<src_numa_node>,b=<dst_numa_node>,val=<distance> Thanks, Gavin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map 2021-09-27 6:41 ` Gavin Shan (?) @ 2021-10-04 18:13 ` Rob Herring -1 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, maz, linux-efi, linux-kernel, shan.gavin, catalin.marinas, rdunlap, kvmarm, will, linux-arm-kernel On Mon, 27 Sep 2021 14:41:19 +0800, Gavin Shan wrote: > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > Applied, thanks! ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-10-04 18:13 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, maz, linux-efi, linux-kernel, shan.gavin, catalin.marinas, rdunlap, kvmarm, will, linux-arm-kernel On Mon, 27 Sep 2021 14:41:19 +0800, Gavin Shan wrote: > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > Applied, thanks! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map @ 2021-10-04 18:13 ` Rob Herring 0 siblings, 0 replies; 27+ messages in thread From: Rob Herring @ 2021-10-04 18:13 UTC (permalink / raw) To: Gavin Shan Cc: devicetree, linux-efi, catalin.marinas, rdunlap, linux-kernel, shan.gavin, maz, will, kvmarm, linux-arm-kernel On Mon, 27 Sep 2021 14:41:19 +0800, Gavin Shan wrote: > There is no device node for the empty NUMA node. However, the > corresponding NUMA node ID and distance map is still valid in > "numa-distance-map-v1" compatible device node. > > This fetches the NUMA node ID and distance map for these empty > NUMA node from "numa-distance-map-v1" compatible device node. > > Signed-off-by: Gavin Shan <gshan@redhat.com> > --- > drivers/of/of_numa.c | 2 ++ > 1 file changed, 2 insertions(+) > Applied, thanks! _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm ^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2021-10-04 18:15 UTC | newest] Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-27 6:41 [PATCH v2 0/2] Specify empty NUMA node Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-09-27 6:41 ` [PATCH v2 1/2] Documentation, dt, numa: Add note to " Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-10-04 18:13 ` Rob Herring 2021-10-04 18:13 ` Rob Herring 2021-10-04 18:13 ` Rob Herring 2021-09-27 6:41 ` [PATCH v2 2/2] of, numa: Fetch empty NUMA node ID from distance map Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-09-27 6:41 ` Gavin Shan 2021-09-27 14:49 ` Rob Herring 2021-09-27 14:49 ` Rob Herring 2021-09-27 14:49 ` Rob Herring 2021-09-27 23:59 ` Gavin Shan 2021-09-27 23:59 ` Gavin Shan 2021-09-27 23:59 ` Gavin Shan 2021-09-28 17:22 ` Rob Herring 2021-09-28 17:22 ` Rob Herring 2021-09-28 17:22 ` Rob Herring 2021-09-29 2:00 ` Gavin Shan 2021-09-29 2:00 ` Gavin Shan 2021-09-29 2:00 ` Gavin Shan 2021-10-04 18:13 ` Rob Herring 2021-10-04 18:13 ` Rob Herring 2021-10-04 18:13 ` Rob Herring
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.