All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 10:17 ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 10:17 UTC (permalink / raw)
  To: robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon,
	anshuman.khandual, peterz

Currently the NUMA distance map parsing does not validate the distance
table for the distance-matrix rules 1-2 in [1].

However the arch NUMA code may enforce some of these rules, but not all.
Such is the case for the arm64 port, which does not enforce the rule that
the distance between separates nodes cannot equal LOCAL_DISTANCE.

The patch adds the following rules validation:
- distance of node to self equals LOCAL_DISTANCE
- distance of separate nodes > LOCAL_DISTANCE

This change avoids a yet-unresolved crash reported in [2].

A note on dealing with symmetrical distances between nodes:

Validating symmetrical distances between nodes is difficult. If it were
mandated in the bindings that every distance must be recorded in the
table, then it would be easy. However, it isn't.

In addition to this, it is also possible to record [b, a] distance only
(and not [a, b]). So, when processing the table for [b, a], we cannot
assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
distance may not be present in the table and current distance would be
default at REMOTE_DISTANCE.

As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
for b > a. This policy is different to kernel ACPI SLIT validation, which
allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
the distance debug message is dropped as it may be misleading (for a distance
which is later overwritten).

Some final notes on semantics:

- It is implied that it is the responsibility of the arch NUMA code to
  reset the NUMA distance map for an error in distance map parsing.

- It is the responsibility of the FW NUMA topology parsing (whether OF or
  ACPI) to enforce NUMA distance rules, and not arch NUMA code.

[1] Documents/devicetree/bindings/numa.txt
[2] https://www.spinics.net/lists/arm-kernel/msg683304.html

Cc: stable@vger.kernel.org # 4.7
Signed-off-by: John Garry <john.garry@huawei.com>
Acked-by: Will Deacon <will.deacon@arm.com>
---
Changes from v1:
- Add note on masking crash reported
- Add Will's tag
- Add cc stable

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 35c64a4295e0..fe6b13608e51 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
 		distance = of_read_number(matrix, 1);
 		matrix++;
 
+		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
+		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
+			pr_err("Invalid distance[node%d -> node%d] = %d\n",
+			       nodea, nodeb, distance);
+			return -EINVAL;
+		}
+
 		numa_set_distance(nodea, nodeb, distance);
-		pr_debug("distance[node%d -> node%d] = %d\n",
-			 nodea, nodeb, distance);
 
 		/* Set default distance of node B->A same as A->B */
 		if (nodeb > nodea)
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 10:17 ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 10:17 UTC (permalink / raw)
  To: robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon,
	anshuman.khandual, peterz

Currently the NUMA distance map parsing does not validate the distance
table for the distance-matrix rules 1-2 in [1].

However the arch NUMA code may enforce some of these rules, but not all.
Such is the case for the arm64 port, which does not enforce the rule that
the distance between separates nodes cannot equal LOCAL_DISTANCE.

The patch adds the following rules validation:
- distance of node to self equals LOCAL_DISTANCE
- distance of separate nodes > LOCAL_DISTANCE

This change avoids a yet-unresolved crash reported in [2].

A note on dealing with symmetrical distances between nodes:

Validating symmetrical distances between nodes is difficult. If it were
mandated in the bindings that every distance must be recorded in the
table, then it would be easy. However, it isn't.

In addition to this, it is also possible to record [b, a] distance only
(and not [a, b]). So, when processing the table for [b, a], we cannot
assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
distance may not be present in the table and current distance would be
default at REMOTE_DISTANCE.

As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
for b > a. This policy is different to kernel ACPI SLIT validation, which
allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
the distance debug message is dropped as it may be misleading (for a distance
which is later overwritten).

Some final notes on semantics:

- It is implied that it is the responsibility of the arch NUMA code to
  reset the NUMA distance map for an error in distance map parsing.

- It is the responsibility of the FW NUMA topology parsing (whether OF or
  ACPI) to enforce NUMA distance rules, and not arch NUMA code.

[1] Documents/devicetree/bindings/numa.txt
[2] https://www.spinics.net/lists/arm-kernel/msg683304.html

Cc: stable@vger.kernel.org # 4.7
Signed-off-by: John Garry <john.garry@huawei.com>
Acked-by: Will Deacon <will.deacon@arm.com>
---
Changes from v1:
- Add note on masking crash reported
- Add Will's tag
- Add cc stable

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 35c64a4295e0..fe6b13608e51 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
 		distance = of_read_number(matrix, 1);
 		matrix++;
 
+		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
+		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
+			pr_err("Invalid distance[node%d -> node%d] = %d\n",
+			       nodea, nodeb, distance);
+			return -EINVAL;
+		}
+
 		numa_set_distance(nodea, nodeb, distance);
-		pr_debug("distance[node%d -> node%d] = %d\n",
-			 nodea, nodeb, distance);
 
 		/* Set default distance of node B->A same as A->B */
 		if (nodeb > nodea)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH v2] of, numa: Validate some distance map rules
  2018-11-08 10:17 ` John Garry
  (?)
@ 2018-11-08 13:08 ` Rob Herring
  -1 siblings, 0 replies; 10+ messages in thread
From: Rob Herring @ 2018-11-08 13:08 UTC (permalink / raw)
  To: John Garry
  Cc: frowand.list, devicetree, linux-kernel, linuxarm, will.deacon,
	anshuman.khandual, peterz

On Thu, Nov 08, 2018 at 06:17:03PM +0800, John Garry wrote:
> Currently the NUMA distance map parsing does not validate the distance
> table for the distance-matrix rules 1-2 in [1].
> 
> However the arch NUMA code may enforce some of these rules, but not all.
> Such is the case for the arm64 port, which does not enforce the rule that
> the distance between separates nodes cannot equal LOCAL_DISTANCE.
> 
> The patch adds the following rules validation:
> - distance of node to self equals LOCAL_DISTANCE
> - distance of separate nodes > LOCAL_DISTANCE
> 
> This change avoids a yet-unresolved crash reported in [2].
> 
> A note on dealing with symmetrical distances between nodes:
> 
> Validating symmetrical distances between nodes is difficult. If it were
> mandated in the bindings that every distance must be recorded in the
> table, then it would be easy. However, it isn't.
> 
> In addition to this, it is also possible to record [b, a] distance only
> (and not [a, b]). So, when processing the table for [b, a], we cannot
> assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
> distance may not be present in the table and current distance would be
> default at REMOTE_DISTANCE.
> 
> As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
> for b > a. This policy is different to kernel ACPI SLIT validation, which
> allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
> the distance debug message is dropped as it may be misleading (for a distance
> which is later overwritten).
> 
> Some final notes on semantics:
> 
> - It is implied that it is the responsibility of the arch NUMA code to
>   reset the NUMA distance map for an error in distance map parsing.
> 
> - It is the responsibility of the FW NUMA topology parsing (whether OF or
>   ACPI) to enforce NUMA distance rules, and not arch NUMA code.
> 
> [1] Documents/devicetree/bindings/numa.txt
> [2] https://www.spinics.net/lists/arm-kernel/msg683304.html
> 
> Cc: stable@vger.kernel.org # 4.7
> Signed-off-by: John Garry <john.garry@huawei.com>
> Acked-by: Will Deacon <will.deacon@arm.com>
> ---

Applied, thanks.

Rob

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2] of, numa: Validate some distance map rules
  2018-11-08 10:17 ` John Garry
  (?)
  (?)
@ 2018-11-08 13:41 ` Anshuman Khandual
  2018-11-08 15:01     ` John Garry
  -1 siblings, 1 reply; 10+ messages in thread
From: Anshuman Khandual @ 2018-11-08 13:41 UTC (permalink / raw)
  To: John Garry, robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon, peterz

On 11/08/2018 03:47 PM, John Garry wrote:
> Currently the NUMA distance map parsing does not validate the distance
> table for the distance-matrix rules 1-2 in [1].

Right. 

> 
> However the arch NUMA code may enforce some of these rules, but not all.

Either it does enforce all these rules again or it does not. What is the
purpose it serves when enforcing parts of it ! Rather it should focus on
looking for boundary conditions (e.g maximum number of nodes supported
on a platform against whats present on the table) which are important
from kernel point of view.

> Such is the case for the arm64 port, which does not enforce the rule that
> the distance between separates nodes cannot equal LOCAL_DISTANCE.
> 
> The patch adds the following rules validation:
> - distance of node to self equals LOCAL_DISTANCE
> - distance of separate nodes > LOCAL_DISTANCE

Which exactly corresponds to first two rules as mentioned in the DT doc
(Documents/devicetree/bindings/numa.txt)

        1. Each entry represents distance from first node to second node.
        The distances are equal in either direction.
        2. The distance from a node to self (local distance) is represented
        with value 10 and all internode distance should be represented with
        a value greater than 10.

> 
> This change avoids a yet-unresolved crash reported in [2].
> 
> A note on dealing with symmetrical distances between nodes:
> 
> Validating symmetrical distances between nodes is difficult. If it were
> mandated in the bindings that every distance must be recorded in the
> table, then it would be easy. However, it isn't.

But if [a, b] and [b, a] both are present, they must be equal.
 
> 
> In addition to this, it is also possible to record [b, a] distance only
> (and not [a, b]). So, when processing the table for [b, a], we cannot
> assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
> distance may not be present in the table and current distance would be
> default at REMOTE_DISTANCE.

Falling back to REMOTE_DISTANCE is right in case one of the symmetrical
distances is not present. But it must abort if they are present and not
identical. That rule should be enforced here while processing DT.

> 
> As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
> for b > a. This policy is different to kernel ACPI SLIT validation, which
> allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
> the distance debug message is dropped as it may be misleading (for a distance
> which is later overwritten).

We could override but if the DT passed table is inconsistent then it must
abort as non-symmetrical distance is not supported. Hence overriding the
distance should happen only in case the inverse set is absent (probably
outside the first loop).

> 
> Some final notes on semantics:
> 
> - It is implied that it is the responsibility of the arch NUMA code to
>   reset the NUMA distance map for an error in distance map parsing.

Right.

> 
> - It is the responsibility of the FW NUMA topology parsing (whether OF or
>   ACPI) to enforce NUMA distance rules, and not arch NUMA code.

This is where we still have some inconsistencies. Arch NUMA code on arm64
should either enforce rule 1 and 2 as stated above or should not at all. As
discussed previously numa_set_distance() enforces first part of rule 2
(self distance should be 10) but not the second part of rule 2 (internode
distance > 10). So either the following condition should be dropped from

(from == to && distance != LOCAL_DISTANCE)

or the following condition should be added to

(from != to && distance <= LOCAL_DISTANCE)

numa_set_distance() in order to conform to the semantics described above.
But thats for a separate discussion how much and what should be enforced
in any given DT compatible arch NUMA node. May not be part of this patch.

> 
> [1] Documents/devicetree/bindings/numa.txt
> [2] https://www.spinics.net/lists/arm-kernel/msg683304.html
> 
> Cc: stable@vger.kernel.org # 4.7
> Signed-off-by: John Garry <john.garry@huawei.com>
> Acked-by: Will Deacon <will.deacon@arm.com>
> ---
> Changes from v1:
> - Add note on masking crash reported
> - Add Will's tag
> - Add cc stable
> 
> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
> index 35c64a4295e0..fe6b13608e51 100644
> --- a/drivers/of/of_numa.c
> +++ b/drivers/of/of_numa.c
> @@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
>  		distance = of_read_number(matrix, 1);
>  		matrix++;
>  
> +		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
> +		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
> +			pr_err("Invalid distance[node%d -> node%d] = %d\n",
> +			       nodea, nodeb, distance);
> +			return -EINVAL;
> +		}
> +
>  		numa_set_distance(nodea, nodeb, distance);
> -		pr_debug("distance[node%d -> node%d] = %d\n",
> -			 nodea, nodeb, distance);

It makes sense to go through the table and print the finalized values
(being passed to arch NUMA) out side of this loop after any identical
replacement is done.

>  
>  		/* Set default distance of node B->A same as A->B */
>  		if (nodeb > nodea)
> 

As mentioned before it should invalidate the table [a, b] != [b, a] in
case they are present and not equal.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2] of, numa: Validate some distance map rules
  2018-11-08 13:41 ` Anshuman Khandual
@ 2018-11-08 15:01     ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 15:01 UTC (permalink / raw)
  To: Anshuman Khandual, robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon, peterz

On 08/11/2018 13:41, Anshuman Khandual wrote:
> On 11/08/2018 03:47 PM, John Garry wrote:
>> Currently the NUMA distance map parsing does not validate the distance
>> table for the distance-matrix rules 1-2 in [1].
>
> Right.
>
>>
>> However the arch NUMA code may enforce some of these rules, but not all.
>
> Either it does enforce all these rules again or it does not. What is the
> purpose it serves when enforcing parts of it ! Rather it should focus on
> looking for boundary conditions (e.g maximum number of nodes supported
> on a platform against whats present on the table) which are important
> from kernel point of view.
>
>> Such is the case for the arm64 port, which does not enforce the rule that
>> the distance between separates nodes cannot equal LOCAL_DISTANCE.
>>
>> The patch adds the following rules validation:
>> - distance of node to self equals LOCAL_DISTANCE
>> - distance of separate nodes > LOCAL_DISTANCE
>
> Which exactly corresponds to first two rules as mentioned in the DT doc
> (Documents/devicetree/bindings/numa.txt)
>
>         1. Each entry represents distance from first node to second node.
>         The distances are equal in either direction.
>         2. The distance from a node to self (local distance) is represented
>         with value 10 and all internode distance should be represented with
>         a value greater than 10.
>
>>

Not exactly. As you know, it does not add validation of distance 
equality in either direction (for values present in the table).

>> This change avoids a yet-unresolved crash reported in [2].
>>
>> A note on dealing with symmetrical distances between nodes:
>>
>> Validating symmetrical distances between nodes is difficult. If it were
>> mandated in the bindings that every distance must be recorded in the
>> table, then it would be easy. However, it isn't.
>
> But if [a, b] and [b, a] both are present, they must be equal.
>
>>
>> In addition to this, it is also possible to record [b, a] distance only
>> (and not [a, b]). So, when processing the table for [b, a], we cannot
>> assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
>> distance may not be present in the table and current distance would be
>> default at REMOTE_DISTANCE.
>
> Falling back to REMOTE_DISTANCE is right in case one of the symmetrical
> distances is not present. But it must abort if they are present and not
> identical. That rule should be enforced here while processing DT.

Do you really think it's worth the effort?

For this, we would need something considerably more complicated than 
what we have today. Maybe similar to the arm64 NUMA code does, in 
allocating a memblock and then doing some 2-phase table parsing. But 
then we're just replicated this same functionality.

On this point, I will also note that there is another problem with the 
DT NUMA bindings (first problem being that both distance entries are not 
required to be present in the table).

That is, it's inconsistency with ACPI SLIT. ACPI SLIT allows 
asymmetrical distance. So that's why I'm not too hung up on this issue 
of validating equality of distances.

This patch does not change the current kernel policy: if the table has 
entries with unequal distances, then it's not rejected nor fixed up.

>
>>
>> As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
>> for b > a. This policy is different to kernel ACPI SLIT validation, which
>> allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
>> the distance debug message is dropped as it may be misleading (for a distance
>> which is later overwritten).
>
> We could override but if the DT passed table is inconsistent then it must
> abort as non-symmetrical distance is not supported. Hence overriding the
> distance should happen only in case the inverse set is absent (probably
> outside the first loop).
>
>>
>> Some final notes on semantics:
>>
>> - It is implied that it is the responsibility of the arch NUMA code to
>>   reset the NUMA distance map for an error in distance map parsing.
>
> Right.
>
>>
>> - It is the responsibility of the FW NUMA topology parsing (whether OF or
>>   ACPI) to enforce NUMA distance rules, and not arch NUMA code.
>
> This is where we still have some inconsistencies. Arch NUMA code on arm64
> should either enforce rule 1 and 2 as stated above or should not at all.

It was implied that any validation can be later dropped from the arch 
NUMA code.

  As
> discussed previously numa_set_distance() enforces first part of rule 2
> (self distance should be 10) but not the second part of rule 2 (internode
> distance > 10). So either the following condition should be dropped from
>
> (from == to && distance != LOCAL_DISTANCE)
>
> or the following condition should be added to
>
> (from != to && distance <= LOCAL_DISTANCE)
>
> numa_set_distance() in order to conform to the semantics described above.
> But thats for a separate discussion how much and what should be enforced
> in any given DT compatible arch NUMA node. May not be part of this patch.
>
>>
>> [1] Documents/devicetree/bindings/numa.txt
>> [2] https://www.spinics.net/lists/arm-kernel/msg683304.html
>>
>> Cc: stable@vger.kernel.org # 4.7
>> Signed-off-by: John Garry <john.garry@huawei.com>
>> Acked-by: Will Deacon <will.deacon@arm.com>
>> ---
>> Changes from v1:
>> - Add note on masking crash reported
>> - Add Will's tag
>> - Add cc stable
>>
>> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
>> index 35c64a4295e0..fe6b13608e51 100644
>> --- a/drivers/of/of_numa.c
>> +++ b/drivers/of/of_numa.c
>> @@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
>>  		distance = of_read_number(matrix, 1);
>>  		matrix++;
>>
>> +		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
>> +		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
>> +			pr_err("Invalid distance[node%d -> node%d] = %d\n",
>> +			       nodea, nodeb, distance);
>> +			return -EINVAL;
>> +		}
>> +
>>  		numa_set_distance(nodea, nodeb, distance);
>> -		pr_debug("distance[node%d -> node%d] = %d\n",
>> -			 nodea, nodeb, distance);
>
> It makes sense to go through the table and print the finalized values
> (being passed to arch NUMA) out side of this loop after any identical
> replacement is done.
>>
>>  		/* Set default distance of node B->A same as A->B */
>>  		if (nodeb > nodea)
>>
>
> As mentioned before it should invalidate the table [a, b] != [b, a] in
> case they are present and not equal.
>

I think that most of these comments are related to earlier points.

> .
>

Thanks,
John



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 15:01     ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 15:01 UTC (permalink / raw)
  To: Anshuman Khandual, robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon, peterz

On 08/11/2018 13:41, Anshuman Khandual wrote:
> On 11/08/2018 03:47 PM, John Garry wrote:
>> Currently the NUMA distance map parsing does not validate the distance
>> table for the distance-matrix rules 1-2 in [1].
>
> Right.
>
>>
>> However the arch NUMA code may enforce some of these rules, but not all.
>
> Either it does enforce all these rules again or it does not. What is the
> purpose it serves when enforcing parts of it ! Rather it should focus on
> looking for boundary conditions (e.g maximum number of nodes supported
> on a platform against whats present on the table) which are important
> from kernel point of view.
>
>> Such is the case for the arm64 port, which does not enforce the rule that
>> the distance between separates nodes cannot equal LOCAL_DISTANCE.
>>
>> The patch adds the following rules validation:
>> - distance of node to self equals LOCAL_DISTANCE
>> - distance of separate nodes > LOCAL_DISTANCE
>
> Which exactly corresponds to first two rules as mentioned in the DT doc
> (Documents/devicetree/bindings/numa.txt)
>
>         1. Each entry represents distance from first node to second node.
>         The distances are equal in either direction.
>         2. The distance from a node to self (local distance) is represented
>         with value 10 and all internode distance should be represented with
>         a value greater than 10.
>
>>

Not exactly. As you know, it does not add validation of distance 
equality in either direction (for values present in the table).

>> This change avoids a yet-unresolved crash reported in [2].
>>
>> A note on dealing with symmetrical distances between nodes:
>>
>> Validating symmetrical distances between nodes is difficult. If it were
>> mandated in the bindings that every distance must be recorded in the
>> table, then it would be easy. However, it isn't.
>
> But if [a, b] and [b, a] both are present, they must be equal.
>
>>
>> In addition to this, it is also possible to record [b, a] distance only
>> (and not [a, b]). So, when processing the table for [b, a], we cannot
>> assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
>> distance may not be present in the table and current distance would be
>> default at REMOTE_DISTANCE.
>
> Falling back to REMOTE_DISTANCE is right in case one of the symmetrical
> distances is not present. But it must abort if they are present and not
> identical. That rule should be enforced here while processing DT.

Do you really think it's worth the effort?

For this, we would need something considerably more complicated than 
what we have today. Maybe similar to the arm64 NUMA code does, in 
allocating a memblock and then doing some 2-phase table parsing. But 
then we're just replicated this same functionality.

On this point, I will also note that there is another problem with the 
DT NUMA bindings (first problem being that both distance entries are not 
required to be present in the table).

That is, it's inconsistency with ACPI SLIT. ACPI SLIT allows 
asymmetrical distance. So that's why I'm not too hung up on this issue 
of validating equality of distances.

This patch does not change the current kernel policy: if the table has 
entries with unequal distances, then it's not rejected nor fixed up.

>
>>
>> As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
>> for b > a. This policy is different to kernel ACPI SLIT validation, which
>> allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
>> the distance debug message is dropped as it may be misleading (for a distance
>> which is later overwritten).
>
> We could override but if the DT passed table is inconsistent then it must
> abort as non-symmetrical distance is not supported. Hence overriding the
> distance should happen only in case the inverse set is absent (probably
> outside the first loop).
>
>>
>> Some final notes on semantics:
>>
>> - It is implied that it is the responsibility of the arch NUMA code to
>>   reset the NUMA distance map for an error in distance map parsing.
>
> Right.
>
>>
>> - It is the responsibility of the FW NUMA topology parsing (whether OF or
>>   ACPI) to enforce NUMA distance rules, and not arch NUMA code.
>
> This is where we still have some inconsistencies. Arch NUMA code on arm64
> should either enforce rule 1 and 2 as stated above or should not at all.

It was implied that any validation can be later dropped from the arch 
NUMA code.

  As
> discussed previously numa_set_distance() enforces first part of rule 2
> (self distance should be 10) but not the second part of rule 2 (internode
> distance > 10). So either the following condition should be dropped from
>
> (from == to && distance != LOCAL_DISTANCE)
>
> or the following condition should be added to
>
> (from != to && distance <= LOCAL_DISTANCE)
>
> numa_set_distance() in order to conform to the semantics described above.
> But thats for a separate discussion how much and what should be enforced
> in any given DT compatible arch NUMA node. May not be part of this patch.
>
>>
>> [1] Documents/devicetree/bindings/numa.txt
>> [2] https://www.spinics.net/lists/arm-kernel/msg683304.html
>>
>> Cc: stable@vger.kernel.org # 4.7
>> Signed-off-by: John Garry <john.garry@huawei.com>
>> Acked-by: Will Deacon <will.deacon@arm.com>
>> ---
>> Changes from v1:
>> - Add note on masking crash reported
>> - Add Will's tag
>> - Add cc stable
>>
>> diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
>> index 35c64a4295e0..fe6b13608e51 100644
>> --- a/drivers/of/of_numa.c
>> +++ b/drivers/of/of_numa.c
>> @@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
>>  		distance = of_read_number(matrix, 1);
>>  		matrix++;
>>
>> +		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
>> +		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
>> +			pr_err("Invalid distance[node%d -> node%d] = %d\n",
>> +			       nodea, nodeb, distance);
>> +			return -EINVAL;
>> +		}
>> +
>>  		numa_set_distance(nodea, nodeb, distance);
>> -		pr_debug("distance[node%d -> node%d] = %d\n",
>> -			 nodea, nodeb, distance);
>
> It makes sense to go through the table and print the finalized values
> (being passed to arch NUMA) out side of this loop after any identical
> replacement is done.
>>
>>  		/* Set default distance of node B->A same as A->B */
>>  		if (nodeb > nodea)
>>
>
> As mentioned before it should invalidate the table [a, b] != [b, a] in
> case they are present and not equal.
>

I think that most of these comments are related to earlier points.

> .
>

Thanks,
John

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 10:16 ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 10:16 UTC (permalink / raw)
  To: robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon,
	anshuman.khandual, peterz

Currently the NUMA distance map parsing does not validate the distance
table for the distance-matrix rules 1-2 in [1].

However the arch NUMA code may enforce some of these rules, but not all.
Such is the case for the arm64 port, which does not enforce the rule that
the distance between separates nodes cannot equal LOCAL_DISTANCE.

The patch adds the following rules validation:
- distance of node to self equals LOCAL_DISTANCE
- distance of separate nodes > LOCAL_DISTANCE

This change avoids a yet-unresolved crash reported in [2].

A note on dealing with symmetrical distances between nodes:

Validating symmetrical distances between nodes is difficult. If it were
mandated in the bindings that every distance must be recorded in the
table, then it would be easy. However, it isn't.

In addition to this, it is also possible to record [b, a] distance only
(and not [a, b]). So, when processing the table for [b, a], we cannot
assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
distance may not be present in the table and current distance would be
default at REMOTE_DISTANCE.

As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
for b > a. This policy is different to kernel ACPI SLIT validation, which
allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
the distance debug message is dropped as it may be misleading (for a distance
which is later overwritten).

Some final notes on semantics:

- It is implied that it is the responsibility of the arch NUMA code to
  reset the NUMA distance map for an error in distance map parsing.

- It is the responsibility of the FW NUMA topology parsing (whether OF or
  ACPI) to enforce NUMA distance rules, and not arch NUMA code.

[1] Documents/devicetree/bindings/numa.txt
[2] https://www.spinics.net/lists/arm-kernel/msg683304.html

Cc: stable@vger.kernel.org # 4.7
Signed-off-by: John Garry <john.garry@huawei.com>
Acked-by: Will Deacon <will.deacon@arm.com>
---
Changes from v1:
- Add note on masking crash reported
- Add Will's tag
- Add cc stable

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 35c64a4295e0..fe6b13608e51 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
 		distance = of_read_number(matrix, 1);
 		matrix++;
 
+		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
+		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
+			pr_err("Invalid distance[node%d -> node%d] = %d\n",
+			       nodea, nodeb, distance);
+			return -EINVAL;
+		}
+
 		numa_set_distance(nodea, nodeb, distance);
-		pr_debug("distance[node%d -> node%d] = %d\n",
-			 nodea, nodeb, distance);
 
 		/* Set default distance of node B->A same as A->B */
 		if (nodeb > nodea)
-- 
1.9.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 10:16 ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 10:16 UTC (permalink / raw)
  To: robh+dt, frowand.list
  Cc: devicetree, linux-kernel, linuxarm, will.deacon,
	anshuman.khandual, peterz

Currently the NUMA distance map parsing does not validate the distance
table for the distance-matrix rules 1-2 in [1].

However the arch NUMA code may enforce some of these rules, but not all.
Such is the case for the arm64 port, which does not enforce the rule that
the distance between separates nodes cannot equal LOCAL_DISTANCE.

The patch adds the following rules validation:
- distance of node to self equals LOCAL_DISTANCE
- distance of separate nodes > LOCAL_DISTANCE

This change avoids a yet-unresolved crash reported in [2].

A note on dealing with symmetrical distances between nodes:

Validating symmetrical distances between nodes is difficult. If it were
mandated in the bindings that every distance must be recorded in the
table, then it would be easy. However, it isn't.

In addition to this, it is also possible to record [b, a] distance only
(and not [a, b]). So, when processing the table for [b, a], we cannot
assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
distance may not be present in the table and current distance would be
default at REMOTE_DISTANCE.

As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
for b > a. This policy is different to kernel ACPI SLIT validation, which
allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
the distance debug message is dropped as it may be misleading (for a distance
which is later overwritten).

Some final notes on semantics:

- It is implied that it is the responsibility of the arch NUMA code to
  reset the NUMA distance map for an error in distance map parsing.

- It is the responsibility of the FW NUMA topology parsing (whether OF or
  ACPI) to enforce NUMA distance rules, and not arch NUMA code.

[1] Documents/devicetree/bindings/numa.txt
[2] https://www.spinics.net/lists/arm-kernel/msg683304.html

Cc: stable@vger.kernel.org # 4.7
Signed-off-by: John Garry <john.garry@huawei.com>
Acked-by: Will Deacon <will.deacon@arm.com>
---
Changes from v1:
- Add note on masking crash reported
- Add Will's tag
- Add cc stable

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 35c64a4295e0..fe6b13608e51 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
 		distance = of_read_number(matrix, 1);
 		matrix++;
 
+		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
+		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
+			pr_err("Invalid distance[node%d -> node%d] = %d\n",
+			       nodea, nodeb, distance);
+			return -EINVAL;
+		}
+
 		numa_set_distance(nodea, nodeb, distance);
-		pr_debug("distance[node%d -> node%d] = %d\n",
-			 nodea, nodeb, distance);
 
 		/* Set default distance of node B->A same as A->B */
 		if (nodeb > nodea)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH v2] of, numa: Validate some distance map rules
  2018-11-08 10:05 John Garry
@ 2018-11-08 10:10 ` John Garry
  0 siblings, 0 replies; 10+ messages in thread
From: John Garry @ 2018-11-08 10:10 UTC (permalink / raw)
  Cc: stable

On 08/11/2018 10:05, John Garry wrote:
> Currently the NUMA distance map parsing does not validate the distance
> table for the distance-matrix rules 1-2 in [1].
>

Please disregard this.

John

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH v2] of, numa: Validate some distance map rules
@ 2018-11-08 10:05 John Garry
  2018-11-08 10:10 ` John Garry
  0 siblings, 1 reply; 10+ messages in thread
From: John Garry @ 2018-11-08 10:05 UTC (permalink / raw)
  To: john.garry2; +Cc: John Garry, stable

Currently the NUMA distance map parsing does not validate the distance
table for the distance-matrix rules 1-2 in [1].

However the arch NUMA code may enforce some of these rules, but not all.
Such is the case for the arm64 port, which does not enforce the rule that
the distance between separates nodes cannot equal LOCAL_DISTANCE.

The patch adds the following rules validation:
- distance of node to self equals LOCAL_DISTANCE
- distance of separate nodes > LOCAL_DISTANCE

This change avoids a yet-unresolved crash reported in [2].

A note on dealing with symmetrical distances between nodes:

Validating symmetrical distances between nodes is difficult. If it were
mandated in the bindings that every distance must be recorded in the
table, then it would be easy. However, it isn't.

In addition to this, it is also possible to record [b, a] distance only
(and not [a, b]). So, when processing the table for [b, a], we cannot
assert that current distance of [a, b] != [b, a] as invalid, as [a, b]
distance may not be present in the table and current distance would be
default at REMOTE_DISTANCE.

As such, we maintain the policy that we overwrite distance [a, b] = [b, a]
for b > a. This policy is different to kernel ACPI SLIT validation, which
allows non-symmetrical distances (ACPI spec SLIT rules allow it). However,
the distance debug message is dropped as it may be misleading (for a distance
which is later overwritten).

Some final notes on semantics:

- It is implied that it is the responsibility of the arch NUMA code to
  reset the NUMA distance map for an error in distance map parsing.

- It is the responsibility of the FW NUMA topology parsing (whether OF or
  ACPI) to enforce NUMA distance rules, and not arch NUMA code.

[1] Documents/devicetree/bindings/numa.txt
[2] https://www.spinics.net/lists/arm-kernel/msg683304.html

Cc: stable@vger.kernel.org
Signed-off-by: John Garry <john.garry@huawei.com>
Acked-by: Will Deacon <will.deacon@arm.com>
---
Change from v1:
- A note of masking crash reported
- Add Will's tag
- Add cc stable

diff --git a/drivers/of/of_numa.c b/drivers/of/of_numa.c
index 35c64a4295e0..fe6b13608e51 100644
--- a/drivers/of/of_numa.c
+++ b/drivers/of/of_numa.c
@@ -104,9 +104,14 @@ static int __init of_numa_parse_distance_map_v1(struct device_node *map)
 		distance = of_read_number(matrix, 1);
 		matrix++;
 
+		if ((nodea == nodeb && distance != LOCAL_DISTANCE) ||
+		    (nodea != nodeb && distance <= LOCAL_DISTANCE)) {
+			pr_err("Invalid distance[node%d -> node%d] = %d\n",
+			       nodea, nodeb, distance);
+			return -EINVAL;
+		}
+
 		numa_set_distance(nodea, nodeb, distance);
-		pr_debug("distance[node%d -> node%d] = %d\n",
-			 nodea, nodeb, distance);
 
 		/* Set default distance of node B->A same as A->B */
 		if (nodeb > nodea)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2018-11-08 19:45 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-08 10:17 [PATCH v2] of, numa: Validate some distance map rules John Garry
2018-11-08 10:17 ` John Garry
2018-11-08 13:08 ` Rob Herring
2018-11-08 13:41 ` Anshuman Khandual
2018-11-08 15:01   ` John Garry
2018-11-08 15:01     ` John Garry
  -- strict thread matches above, loose matches on Subject: below --
2018-11-08 10:16 John Garry
2018-11-08 10:16 ` John Garry
2018-11-08 10:05 John Garry
2018-11-08 10:10 ` John Garry

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.