[PATCH RFC] driver core: ensure a device has valid node id in device

linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH RFC] driver core: ensure a device has valid node id in device_add()
@ 2019-09-05  1:33 Yunsheng Lin
  2019-09-05  5:57 ` Greg KH
  0 siblings, 1 reply; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-05  1:33 UTC (permalink / raw)
  To: gregkh, rafael; +Cc: linux-kernel, peterz, mingo, mhocko, linuxarm

Currently a device does not belong to any of the numa nodes
(dev->numa_node is NUMA_NO_NODE) when the FW does not provide
the node id and the device has not no parent device.

According to discussion in [1]:
Even if a device's numa node is not set by fw, the device
really does belong to a node.

This patch sets the device node to node 0 in device_add() if
the fw has not specified the node id and it either has no
parent device, or the parent device also does not have a valid
node id.

There may be explicit handling out there relying on NUMA_NO_NODE,
like in nvme_probe().

[1] https://lkml.org/lkml/2019/9/2/466

Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
---
 drivers/base/core.c  | 17 ++++++++++++++---
 include/linux/numa.h |  2 ++
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/base/core.c b/drivers/base/core.c
index 1669d41..466b8ff 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
 	if (kobj)
 		dev->kobj.parent = kobj;
 
-	/* use parent numa_node */
-	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
-		set_dev_node(dev, dev_to_node(parent));
+	/* use parent numa_node or default node 0 */
+	if (!numa_node_valid(dev_to_node(dev))) {
+		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
+
+		if (numa_node_valid(nid)) {
+			set_dev_node(dev, nid);
+		} else {
+			if (nr_node_ids > 1U)
+				pr_err("device: '%s': has invalid NUMA node(%d)\n",
+				       dev_name(dev), dev_to_node(dev));
+
+			set_dev_node(dev, 0);
+		}
+	}
 
 	/* first, register with generic layer. */
 	/* we require the name to be set before, and pass NULL */
diff --git a/include/linux/numa.h b/include/linux/numa.h
index 110b0e5..eccc757 100644
--- a/include/linux/numa.h
+++ b/include/linux/numa.h
@@ -13,4 +13,6 @@
 
 #define	NUMA_NO_NODE	(-1)
 
+#define numa_node_valid(node)	((unsigned int)(node) < nr_node_ids)
+
 #endif /* _LINUX_NUMA_H */
-- 
2.8.1


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  1:33 [PATCH RFC] driver core: ensure a device has valid node id in device_add() Yunsheng Lin
@ 2019-09-05  5:57 ` Greg KH
  2019-09-05  6:48   ` Yunsheng Lin
  0 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2019-09-05  5:57 UTC (permalink / raw)
  To: Yunsheng Lin; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
> Currently a device does not belong to any of the numa nodes
> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
> the node id and the device has not no parent device.
> 
> According to discussion in [1]:
> Even if a device's numa node is not set by fw, the device
> really does belong to a node.
> 
> This patch sets the device node to node 0 in device_add() if
> the fw has not specified the node id and it either has no
> parent device, or the parent device also does not have a valid
> node id.
> 
> There may be explicit handling out there relying on NUMA_NO_NODE,
> like in nvme_probe().
> 
> [1] https://lkml.org/lkml/2019/9/2/466
> 
> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> ---
>  drivers/base/core.c  | 17 ++++++++++++++---
>  include/linux/numa.h |  2 ++
>  2 files changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 1669d41..466b8ff 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>  	if (kobj)
>  		dev->kobj.parent = kobj;
>  
> -	/* use parent numa_node */
> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
> -		set_dev_node(dev, dev_to_node(parent));
> +	/* use parent numa_node or default node 0 */
> +	if (!numa_node_valid(dev_to_node(dev))) {
> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;

Can you expand this to be a "real" if statement please?

> +
> +		if (numa_node_valid(nid)) {
> +			set_dev_node(dev, nid);
> +		} else {
> +			if (nr_node_ids > 1U)
> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
> +				       dev_name(dev), dev_to_node(dev));

dev_err() will show you the exact device properly, instead of having to
rely on dev_name().

And what is a user to do if this message happens?  How do they fix this?
If they can not, what good is this error message?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  5:57 ` Greg KH
@ 2019-09-05  6:48   ` Yunsheng Lin
  2019-09-05  7:33     ` Greg KH
  0 siblings, 1 reply; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-05  6:48 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/5 13:57, Greg KH wrote:
> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>> Currently a device does not belong to any of the numa nodes
>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>> the node id and the device has not no parent device.
>>
>> According to discussion in [1]:
>> Even if a device's numa node is not set by fw, the device
>> really does belong to a node.
>>
>> This patch sets the device node to node 0 in device_add() if
>> the fw has not specified the node id and it either has no
>> parent device, or the parent device also does not have a valid
>> node id.
>>
>> There may be explicit handling out there relying on NUMA_NO_NODE,
>> like in nvme_probe().
>>
>> [1] https://lkml.org/lkml/2019/9/2/466
>>
>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>> ---
>>  drivers/base/core.c  | 17 ++++++++++++++---
>>  include/linux/numa.h |  2 ++
>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>> index 1669d41..466b8ff 100644
>> --- a/drivers/base/core.c
>> +++ b/drivers/base/core.c
>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>  	if (kobj)
>>  		dev->kobj.parent = kobj;
>>  
>> -	/* use parent numa_node */
>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>> -		set_dev_node(dev, dev_to_node(parent));
>> +	/* use parent numa_node or default node 0 */
>> +	if (!numa_node_valid(dev_to_node(dev))) {
>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
> 
> Can you expand this to be a "real" if statement please?

Sure. May I ask why "? :" is not appropriate here?

> 
>> +
>> +		if (numa_node_valid(nid)) {
>> +			set_dev_node(dev, nid);
>> +		} else {
>> +			if (nr_node_ids > 1U)
>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>> +				       dev_name(dev), dev_to_node(dev));
> 
> dev_err() will show you the exact device properly, instead of having to
> rely on dev_name().
> 
> And what is a user to do if this message happens?  How do they fix this?
> If they can not, what good is this error message?

If user know about their system's topology well enough and node 0
is not the nearest node to the device, maybe user can readjust that by
writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
if not, then maybe user need to contact the vendor for info or updates.

Maybe print error message as below:

dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
	dev_to_node(dev));

> 
> thanks,
> 
> greg k-h
> 
> .
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  6:48   ` Yunsheng Lin
@ 2019-09-05  7:33     ` Greg KH
  2019-09-05  8:57       ` Yunsheng Lin
  2019-09-06  6:41       ` Yunsheng Lin
  0 siblings, 2 replies; 12+ messages in thread
From: Greg KH @ 2019-09-05  7:33 UTC (permalink / raw)
  To: Yunsheng Lin; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
> On 2019/9/5 13:57, Greg KH wrote:
> > On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
> >> Currently a device does not belong to any of the numa nodes
> >> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
> >> the node id and the device has not no parent device.
> >>
> >> According to discussion in [1]:
> >> Even if a device's numa node is not set by fw, the device
> >> really does belong to a node.
> >>
> >> This patch sets the device node to node 0 in device_add() if
> >> the fw has not specified the node id and it either has no
> >> parent device, or the parent device also does not have a valid
> >> node id.
> >>
> >> There may be explicit handling out there relying on NUMA_NO_NODE,
> >> like in nvme_probe().
> >>
> >> [1] https://lkml.org/lkml/2019/9/2/466
> >>
> >> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> >> ---
> >>  drivers/base/core.c  | 17 ++++++++++++++---
> >>  include/linux/numa.h |  2 ++
> >>  2 files changed, 16 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/drivers/base/core.c b/drivers/base/core.c
> >> index 1669d41..466b8ff 100644
> >> --- a/drivers/base/core.c
> >> +++ b/drivers/base/core.c
> >> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
> >>  	if (kobj)
> >>  		dev->kobj.parent = kobj;
> >>  
> >> -	/* use parent numa_node */
> >> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
> >> -		set_dev_node(dev, dev_to_node(parent));
> >> +	/* use parent numa_node or default node 0 */
> >> +	if (!numa_node_valid(dev_to_node(dev))) {
> >> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
> > 
> > Can you expand this to be a "real" if statement please?
> 
> Sure. May I ask why "? :" is not appropriate here?

Because it is a pain to read, just spell it out and make it obvious what
is happening.  You write code for developers first, and the compiler
second, and in this case, either way is identical to the compiler.

> >> +
> >> +		if (numa_node_valid(nid)) {
> >> +			set_dev_node(dev, nid);
> >> +		} else {
> >> +			if (nr_node_ids > 1U)
> >> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
> >> +				       dev_name(dev), dev_to_node(dev));
> > 
> > dev_err() will show you the exact device properly, instead of having to
> > rely on dev_name().
> > 
> > And what is a user to do if this message happens?  How do they fix this?
> > If they can not, what good is this error message?
> 
> If user know about their system's topology well enough and node 0
> is not the nearest node to the device, maybe user can readjust that by
> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
> if not, then maybe user need to contact the vendor for info or updates.
> 
> Maybe print error message as below:
> 
> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
> 	dev_to_node(dev));

FW_BUG?

Anyway, if you make this change, how many machines start reporting this
error?  You should also say something like "default node of 0 now
selected" or something like that, right?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  7:33     ` Greg KH
@ 2019-09-05  8:57       ` Yunsheng Lin
  2019-09-05  9:02         ` Greg KH
  2019-09-06  6:41       ` Yunsheng Lin
  1 sibling, 1 reply; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-05  8:57 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/5 15:33, Greg KH wrote:
> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
>> On 2019/9/5 13:57, Greg KH wrote:
>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>>>> Currently a device does not belong to any of the numa nodes
>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>>>> the node id and the device has not no parent device.
>>>>
>>>> According to discussion in [1]:
>>>> Even if a device's numa node is not set by fw, the device
>>>> really does belong to a node.
>>>>
>>>> This patch sets the device node to node 0 in device_add() if
>>>> the fw has not specified the node id and it either has no
>>>> parent device, or the parent device also does not have a valid
>>>> node id.
>>>>
>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
>>>> like in nvme_probe().
>>>>
>>>> [1] https://lkml.org/lkml/2019/9/2/466
>>>>
>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>>>> ---
>>>>  drivers/base/core.c  | 17 ++++++++++++++---
>>>>  include/linux/numa.h |  2 ++
>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>>>> index 1669d41..466b8ff 100644
>>>> --- a/drivers/base/core.c
>>>> +++ b/drivers/base/core.c
>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>>>  	if (kobj)
>>>>  		dev->kobj.parent = kobj;
>>>>  
>>>> -	/* use parent numa_node */
>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>>>> -		set_dev_node(dev, dev_to_node(parent));
>>>> +	/* use parent numa_node or default node 0 */
>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
>>>
>>> Can you expand this to be a "real" if statement please?
>>
>> Sure. May I ask why "? :" is not appropriate here?
> 
> Because it is a pain to read, just spell it out and make it obvious what
> is happening.  You write code for developers first, and the compiler
> second, and in this case, either way is identical to the compiler.
> 
>>>> +
>>>> +		if (numa_node_valid(nid)) {
>>>> +			set_dev_node(dev, nid);
>>>> +		} else {
>>>> +			if (nr_node_ids > 1U)
>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>>>> +				       dev_name(dev), dev_to_node(dev));
>>>
>>> dev_err() will show you the exact device properly, instead of having to
>>> rely on dev_name().
>>>
>>> And what is a user to do if this message happens?  How do they fix this?
>>> If they can not, what good is this error message?
>>
>> If user know about their system's topology well enough and node 0
>> is not the nearest node to the device, maybe user can readjust that by
>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
>> if not, then maybe user need to contact the vendor for info or updates.
>>
>> Maybe print error message as below:
>>
>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
>> 	dev_to_node(dev));
> 
> FW_BUG?

The sysfs numa_node writing interface does print FW_BUG error.
Maybe it is a way of telling the user to contact the vendors, which
pushing the vendors to update the FW.

If FW_BUG is too much, there is FW_WARN or FW_INFO.

> 
> Anyway, if you make this change, how many machines start reporting this
> error?  You should also say something like "default node of 0 now
> selected" or something like that, right?

Yes.

> 
> thanks,
> 
> greg k-h
> 
> .
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  8:57       ` Yunsheng Lin
@ 2019-09-05  9:02         ` Greg KH
  2019-09-05  9:07           ` Yunsheng Lin
  0 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2019-09-05  9:02 UTC (permalink / raw)
  To: Yunsheng Lin; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On Thu, Sep 05, 2019 at 04:57:00PM +0800, Yunsheng Lin wrote:
> On 2019/9/5 15:33, Greg KH wrote:
> > On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
> >> On 2019/9/5 13:57, Greg KH wrote:
> >>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
> >>>> Currently a device does not belong to any of the numa nodes
> >>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
> >>>> the node id and the device has not no parent device.
> >>>>
> >>>> According to discussion in [1]:
> >>>> Even if a device's numa node is not set by fw, the device
> >>>> really does belong to a node.
> >>>>
> >>>> This patch sets the device node to node 0 in device_add() if
> >>>> the fw has not specified the node id and it either has no
> >>>> parent device, or the parent device also does not have a valid
> >>>> node id.
> >>>>
> >>>> There may be explicit handling out there relying on NUMA_NO_NODE,
> >>>> like in nvme_probe().
> >>>>
> >>>> [1] https://lkml.org/lkml/2019/9/2/466
> >>>>
> >>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> >>>> ---
> >>>>  drivers/base/core.c  | 17 ++++++++++++++---
> >>>>  include/linux/numa.h |  2 ++
> >>>>  2 files changed, 16 insertions(+), 3 deletions(-)
> >>>>
> >>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
> >>>> index 1669d41..466b8ff 100644
> >>>> --- a/drivers/base/core.c
> >>>> +++ b/drivers/base/core.c
> >>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
> >>>>  	if (kobj)
> >>>>  		dev->kobj.parent = kobj;
> >>>>  
> >>>> -	/* use parent numa_node */
> >>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
> >>>> -		set_dev_node(dev, dev_to_node(parent));
> >>>> +	/* use parent numa_node or default node 0 */
> >>>> +	if (!numa_node_valid(dev_to_node(dev))) {
> >>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
> >>>
> >>> Can you expand this to be a "real" if statement please?
> >>
> >> Sure. May I ask why "? :" is not appropriate here?
> > 
> > Because it is a pain to read, just spell it out and make it obvious what
> > is happening.  You write code for developers first, and the compiler
> > second, and in this case, either way is identical to the compiler.
> > 
> >>>> +
> >>>> +		if (numa_node_valid(nid)) {
> >>>> +			set_dev_node(dev, nid);
> >>>> +		} else {
> >>>> +			if (nr_node_ids > 1U)
> >>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
> >>>> +				       dev_name(dev), dev_to_node(dev));
> >>>
> >>> dev_err() will show you the exact device properly, instead of having to
> >>> rely on dev_name().
> >>>
> >>> And what is a user to do if this message happens?  How do they fix this?
> >>> If they can not, what good is this error message?
> >>
> >> If user know about their system's topology well enough and node 0
> >> is not the nearest node to the device, maybe user can readjust that by
> >> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
> >> if not, then maybe user need to contact the vendor for info or updates.
> >>
> >> Maybe print error message as below:
> >>
> >> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
> >> 	dev_to_node(dev));
> > 
> > FW_BUG?
> 
> The sysfs numa_node writing interface does print FW_BUG error.
> Maybe it is a way of telling the user to contact the vendors, which
> pushing the vendors to update the FW.

But is this always going to be caused by a firmware bug?  If so, ok, if
not, and it's a driver/bus kernel issue, we should not say this.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  9:02         ` Greg KH
@ 2019-09-05  9:07           ` Yunsheng Lin
  0 siblings, 0 replies; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-05  9:07 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/5 17:02, Greg KH wrote:
> On Thu, Sep 05, 2019 at 04:57:00PM +0800, Yunsheng Lin wrote:
>> On 2019/9/5 15:33, Greg KH wrote:
>>> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
>>>> On 2019/9/5 13:57, Greg KH wrote:
>>>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>>>>>> Currently a device does not belong to any of the numa nodes
>>>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>>>>>> the node id and the device has not no parent device.
>>>>>>
>>>>>> According to discussion in [1]:
>>>>>> Even if a device's numa node is not set by fw, the device
>>>>>> really does belong to a node.
>>>>>>
>>>>>> This patch sets the device node to node 0 in device_add() if
>>>>>> the fw has not specified the node id and it either has no
>>>>>> parent device, or the parent device also does not have a valid
>>>>>> node id.
>>>>>>
>>>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
>>>>>> like in nvme_probe().
>>>>>>
>>>>>> [1] https://lkml.org/lkml/2019/9/2/466
>>>>>>
>>>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>>>>>> ---
>>>>>>  drivers/base/core.c  | 17 ++++++++++++++---
>>>>>>  include/linux/numa.h |  2 ++
>>>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>>>>>> index 1669d41..466b8ff 100644
>>>>>> --- a/drivers/base/core.c
>>>>>> +++ b/drivers/base/core.c
>>>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>>>>>  	if (kobj)
>>>>>>  		dev->kobj.parent = kobj;
>>>>>>  
>>>>>> -	/* use parent numa_node */
>>>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>>>>>> -		set_dev_node(dev, dev_to_node(parent));
>>>>>> +	/* use parent numa_node or default node 0 */
>>>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
>>>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
>>>>>
>>>>> Can you expand this to be a "real" if statement please?
>>>>
>>>> Sure. May I ask why "? :" is not appropriate here?
>>>
>>> Because it is a pain to read, just spell it out and make it obvious what
>>> is happening.  You write code for developers first, and the compiler
>>> second, and in this case, either way is identical to the compiler.
>>>
>>>>>> +
>>>>>> +		if (numa_node_valid(nid)) {
>>>>>> +			set_dev_node(dev, nid);
>>>>>> +		} else {
>>>>>> +			if (nr_node_ids > 1U)
>>>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>>>>>> +				       dev_name(dev), dev_to_node(dev));
>>>>>
>>>>> dev_err() will show you the exact device properly, instead of having to
>>>>> rely on dev_name().
>>>>>
>>>>> And what is a user to do if this message happens?  How do they fix this?
>>>>> If they can not, what good is this error message?
>>>>
>>>> If user know about their system's topology well enough and node 0
>>>> is not the nearest node to the device, maybe user can readjust that by
>>>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
>>>> if not, then maybe user need to contact the vendor for info or updates.
>>>>
>>>> Maybe print error message as below:
>>>>
>>>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
>>>> 	dev_to_node(dev));
>>>
>>> FW_BUG?
>>
>> The sysfs numa_node writing interface does print FW_BUG error.
>> Maybe it is a way of telling the user to contact the vendors, which
>> pushing the vendors to update the FW.
> 
> But is this always going to be caused by a firmware bug?  If so, ok, if
> not, and it's a driver/bus kernel issue, we should not say this.

Ok, Make sense. Will not add the FW_BUG printing.

> 
> thanks,
> 
> greg k-h
> 
> .
> 


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-05  7:33     ` Greg KH
  2019-09-05  8:57       ` Yunsheng Lin
@ 2019-09-06  6:41       ` Yunsheng Lin
  2019-09-06  6:52         ` Greg KH
  1 sibling, 1 reply; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-06  6:41 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/5 15:33, Greg KH wrote:
> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
>> On 2019/9/5 13:57, Greg KH wrote:
>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>>>> Currently a device does not belong to any of the numa nodes
>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>>>> the node id and the device has not no parent device.
>>>>
>>>> According to discussion in [1]:
>>>> Even if a device's numa node is not set by fw, the device
>>>> really does belong to a node.
>>>>
>>>> This patch sets the device node to node 0 in device_add() if
>>>> the fw has not specified the node id and it either has no
>>>> parent device, or the parent device also does not have a valid
>>>> node id.
>>>>
>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
>>>> like in nvme_probe().
>>>>
>>>> [1] https://lkml.org/lkml/2019/9/2/466
>>>>
>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>>>> ---
>>>>  drivers/base/core.c  | 17 ++++++++++++++---
>>>>  include/linux/numa.h |  2 ++
>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>>>> index 1669d41..466b8ff 100644
>>>> --- a/drivers/base/core.c
>>>> +++ b/drivers/base/core.c
>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>>>  	if (kobj)
>>>>  		dev->kobj.parent = kobj;
>>>>  
>>>> -	/* use parent numa_node */
>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>>>> -		set_dev_node(dev, dev_to_node(parent));
>>>> +	/* use parent numa_node or default node 0 */
>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
>>>
>>> Can you expand this to be a "real" if statement please?
>>
>> Sure. May I ask why "? :" is not appropriate here?
> 
> Because it is a pain to read, just spell it out and make it obvious what
> is happening.  You write code for developers first, and the compiler
> second, and in this case, either way is identical to the compiler.
> 
>>>> +
>>>> +		if (numa_node_valid(nid)) {
>>>> +			set_dev_node(dev, nid);
>>>> +		} else {
>>>> +			if (nr_node_ids > 1U)
>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>>>> +				       dev_name(dev), dev_to_node(dev));
>>>
>>> dev_err() will show you the exact device properly, instead of having to
>>> rely on dev_name().
>>>
>>> And what is a user to do if this message happens?  How do they fix this?
>>> If they can not, what good is this error message?
>>
>> If user know about their system's topology well enough and node 0
>> is not the nearest node to the device, maybe user can readjust that by
>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
>> if not, then maybe user need to contact the vendor for info or updates.
>>
>> Maybe print error message as below:
>>
>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
>> 	dev_to_node(dev));
> 
> FW_BUG?
> 
> Anyway, if you make this change, how many machines start reporting this
> error? 

Any machines with more than one numa node will start reporting this error.

1) many virtual deivces maybe do not set the node id before calling
   device_register(), such as vfio, tun, etc.

2) struct cpu has a dev, but does not set the dev' node according to
   cpu_to_node().

3) Many platform Device also do not have a node id provided by FW.

Below is my system with 4 nodes:

14.327031]  platform: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.387712]  cpu: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.456242]  container: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.525992]  workqueue: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.785572] vtconsole vtcon0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.870664]  node: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
14.991627] dmi id: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.061682] node node0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.136323] node node1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.210963] node node2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.285602] node node3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.360241] cpu cpu0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.434180] cpu cpu1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.508116] cpu cpu2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.582050] cpu cpu3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.655985] cpu cpu4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.729920] cpu cpu5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.803856] cpu cpu6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.877792] cpu cpu7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
15.951726] cpu cpu8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.025660] cpu cpu9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.099599] cpu cpu10: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.173621] cpu cpu11: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.247644] cpu cpu12: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.321668] cpu cpu13: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.395690] cpu cpu14: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.469711] cpu cpu15: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.543732] cpu cpu16: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.617756] cpu cpu17: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.691777] cpu cpu18: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.765799] cpu cpu19: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.839822] cpu cpu20: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.913846] cpu cpu21: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
16.987953] cpu cpu22: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.062453] cpu cpu23: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.136907] cpu cpu24: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.211365] cpu cpu25: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.285822] cpu cpu26: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.360273] cpu cpu27: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.434755] cpu cpu28: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.508784] cpu cpu29: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.582807] cpu cpu30: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.656832] cpu cpu31: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.730855] cpu cpu32: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.804876] cpu cpu33: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.878902] cpu cpu34: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
17.952924] cpu cpu35: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.026946] cpu cpu36: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.100968] cpu cpu37: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.174992] cpu cpu38: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.249013] cpu cpu39: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.323037] cpu cpu40: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.397151] cpu cpu41: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.471178] cpu cpu42: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.545198] cpu cpu43: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.619219] cpu cpu44: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.693239] cpu cpu45: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.767263] cpu cpu46: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.841287] cpu cpu47: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.915309] cpu cpu48: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
18.989333] cpu cpu49: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.063356] cpu cpu50: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.137379] cpu cpu51: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.211402] cpu cpu52: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.285425] cpu cpu53: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.359448] cpu cpu54: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.433470] cpu cpu55: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.507493] cpu cpu56: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.581516] cpu cpu57: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.655540] cpu cpu58: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.729562] cpu cpu59: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.803587] cpu cpu60: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.877611] cpu cpu61: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
19.951634] cpu cpu62: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.025656] cpu cpu63: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.099681] cpu cpu64: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.173703] cpu cpu65: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.247726] cpu cpu66: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.321749] cpu cpu67: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.395773] cpu cpu68: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.469796] cpu cpu69: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.543818] cpu cpu70: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.617840] cpu cpu71: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.691864] cpu cpu72: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.765887] cpu cpu73: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.839909] cpu cpu74: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.913932] cpu cpu75: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
20.987955] cpu cpu76: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.062013] cpu cpu77: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.136486] cpu cpu78: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.210941] cpu cpu79: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.285403] cpu cpu80: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.359855] cpu cpu81: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.434323] cpu cpu82: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.508809] cpu cpu83: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.582842] cpu cpu84: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.656868] cpu cpu85: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.730891] cpu cpu86: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.804915] cpu cpu87: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.878942] cpu cpu88: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
21.952967] cpu cpu89: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.026990] cpu cpu90: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.101011] cpu cpu91: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.175034] cpu cpu92: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.249060] cpu cpu93: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.323084] cpu cpu94: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.397105] cpu cpu95: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.471130] cpu cpu96: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.545155] cpu cpu97: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.619181] cpu cpu98: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.693206] cpu cpu99: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.767232] cpu cpu100: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.841343] cpu cpu101: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.915455] cpu cpu102: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
22.989566] cpu cpu103: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.063678] cpu cpu104: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.137790] cpu cpu105: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.211902] cpu cpu106: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.286011] cpu cpu107: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.360122] cpu cpu108: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.434232] cpu cpu109: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.508343] cpu cpu110: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.582453] cpu cpu111: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.656565] cpu cpu112: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.730675] cpu cpu113: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.804784] cpu cpu114: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.878895] cpu cpu115: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
23.953079] cpu cpu116: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.027190] cpu cpu117: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.101301] cpu cpu118: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.175416] cpu cpu119: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.249529] cpu cpu120: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.323641] cpu cpu121: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.397751] cpu cpu122: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.471864] cpu cpu123: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.545974] cpu cpu124: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.620085] cpu cpu125: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.694193] cpu cpu126: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.768305] cpu cpu127: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
24.844988] workqueue writeback: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
26.128405] graphics fbcon: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
26.279506] acpi LNXSYSTM:00: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
26.470076]  pci0000:00: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
26.815436]  pci0000:7b: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
27.004447]  pci0000:7a: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
27.236797]  pci0000:78: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
27.505833]  pci0000:7c: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
28.056452]  pci0000:74: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
28.470018]  pci0000:80: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
28.726411]  pci0000:bb: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
28.916656]  pci0000:ba: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
29.152839]  pci0000:b8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
29.425808]  pci0000:bc: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
29.718593]  pci0000:b4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.032859] misc vga_arbiter: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.185572]  edac: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.263437] edac mc: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.353062] net lo: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.600675]  pnp0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.679539] mem mem: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.753818] mem null: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.828180] mem port: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.902538] mem zero: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
30.976892] mem full: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.051331] mem random: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.125860] mem urandom: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.200476] mem kmsg: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.274833] tty tty: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.352315] tty console: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.431276] tty tty0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.513192] vc vcs: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.597014] vc vcsu: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.680923] vc vcsa: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.764834] vc vcs1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.852390] vc vcsu1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
31.940030] vc vcsa1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.027739] tty tty1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.112960] tty tty2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.198173] tty tty3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.283383] tty tty4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.368590] tty tty5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.453805] tty tty6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.539015] tty tty7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.624230] tty tty8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.709440] tty tty9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.794653] tty tty10: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.879958] tty tty11: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
32.965258] tty tty12: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.050621] tty tty13: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.135925] tty tty14: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.221225] tty tty15: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.306525] tty tty16: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.391827] tty tty17: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.477128] tty tty18: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.562434] tty tty19: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.647735] tty tty20: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.733037] tty tty21: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.818340] tty tty22: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.903640] tty tty23: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
33.988998] tty tty24: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.074301] tty tty25: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.159605] tty tty26: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.244918] tty tty27: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.330223] tty tty28: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.415531] tty tty29: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.500831] tty tty30: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.586131] tty tty31: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.671428] tty tty32: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.756726] tty tty33: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.842025] tty tty34: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
34.927323] tty tty35: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.012701] tty tty36: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.097999] tty tty37: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.183296] tty tty38: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.268593] tty tty39: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.353890] tty tty40: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.439188] tty tty41: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.524483] tty tty42: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.609779] tty tty43: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.695079] tty tty44: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.780375] tty tty45: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.865670] tty tty46: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
35.950968] tty tty47: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.036324] tty tty48: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.121624] tty tty49: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.206922] tty tty50: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.292231] tty tty51: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.377530] tty tty52: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.462827] tty tty53: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.548123] tty tty54: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.633424] tty tty55: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.718724] tty tty56: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.804024] tty tty57: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.889324] tty tty58: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
36.974624] tty tty59: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.059986] tty tty60: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.145290] tty tty61: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.230590] tty tty62: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.315893] tty tty63: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.401809] workqueue ib-comp-wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
37.481505] workqueue ib-comp-unb-wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.214619] misc kvm: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.311549] misc snapshot: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.391755]  clocksource: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.471340] clocksource clocksource0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.543678]  clockevents: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.623339] clockevents clockevent0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.695264] clockevents clockevent1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.767187] clockevents clockevent2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.839108] clockevents clockevent3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.911025] clockevents clockevent4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
39.982945] clockevents clockevent5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.054863] clockevents clockevent6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.126780] clockevents clockevent7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.198696] clockevents clockevent8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.270612] clockevents clockevent9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.342531] clockevents clockevent10: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.414534] clockevents clockevent11: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.486537] clockevents clockevent12: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.558539] clockevents clockevent13: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.630548] clockevents clockevent14: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.702553] clockevents clockevent15: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.774554] clockevents clockevent16: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.846555] clockevents clockevent17: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.918557] clockevents clockevent18: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
40.990560] clockevents clockevent19: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.062562] clockevents clockevent20: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.134563] clockevents clockevent21: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.206569] clockevents clockevent22: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.278571] clockevents clockevent23: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.350572] clockevents clockevent24: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.422575] clockevents clockevent25: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.494582] clockevents clockevent26: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.566583] clockevents clockevent27: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.638583] clockevents clockevent28: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.710584] clockevents clockevent29: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.782592] clockevents clockevent30: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.854596] clockevents clockevent31: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.926601] clockevents clockevent32: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
41.998604] clockevents clockevent33: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.070609] clockevents clockevent34: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.142615] clockevents clockevent35: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.214620] clockevents clockevent36: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.286628] clockevents clockevent37: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.358635] clockevents clockevent38: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.430641] clockevents clockevent39: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.502650] clockevents clockevent40: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.574655] clockevents clockevent41: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.646662] clockevents clockevent42: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.718669] clockevents clockevent43: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.790677] clockevents clockevent44: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.862685] clockevents clockevent45: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
42.934692] clockevents clockevent46: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.006698] clockevents clockevent47: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.078705] clockevents clockevent48: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.150712] clockevents clockevent49: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.222720] clockevents clockevent50: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.294726] clockevents clockevent51: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.366732] clockevents clockevent52: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.438741] clockevents clockevent53: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.510746] clockevents clockevent54: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.582762] clockevents clockevent55: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.654768] clockevents clockevent56: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.726773] clockevents clockevent57: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.798778] clockevents clockevent58: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.870784] clockevents clockevent59: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
43.942790] clockevents clockevent60: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.014796] clockevents clockevent61: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.086801] clockevents clockevent62: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.158806] clockevents clockevent63: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.230812] clockevents clockevent64: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.302819] clockevents clockevent65: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.374824] clockevents clockevent66: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.446829] clockevents clockevent67: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.518834] clockevents clockevent68: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.590838] clockevents clockevent69: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.662847] clockevents clockevent70: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.734851] clockevents clockevent71: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.806855] clockevents clockevent72: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.878859] clockevents clockevent73: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
44.950862] clockevents clockevent74: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.022868] clockevents clockevent75: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.094873] clockevents clockevent76: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.166877] clockevents clockevent77: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.238880] clockevents clockevent78: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.310883] clockevents clockevent79: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.382888] clockevents clockevent80: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.454892] clockevents clockevent81: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.526896] clockevents clockevent82: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.598900] clockevents clockevent83: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.670909] clockevents clockevent84: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.742915] clockevents clockevent85: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.814925] clockevents clockevent86: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.886932] clockevents clockevent87: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
45.958940] clockevents clockevent88: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.030947] clockevents clockevent89: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.102955] clockevents clockevent90: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.174961] clockevents clockevent91: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.246967] clockevents clockevent92: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.318974] clockevents clockevent93: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.390976] clockevents clockevent94: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.462983] clockevents clockevent95: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.534990] clockevents clockevent96: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.606997] clockevents clockevent97: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.679003] clockevents clockevent98: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.751008] clockevents clockevent99: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.823016] clockevents clockevent100: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.895109] clockevents clockevent101: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
46.967201] clockevents clockevent102: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.039293] clockevents clockevent103: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.111386] clockevents clockevent104: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.183480] clockevents clockevent105: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.255573] clockevents clockevent106: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.327666] clockevents clockevent107: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.399760] clockevents clockevent108: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.471853] clockevents clockevent109: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.543946] clockevents clockevent110: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.616038] clockevents clockevent111: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.688132] clockevents clockevent112: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.760233] clockevents clockevent113: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.832325] clockevents clockevent114: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.904419] clockevents clockevent115: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
47.976511] clockevents clockevent116: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.048603] clockevents clockevent117: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.120695] clockevents clockevent118: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.192787] clockevents clockevent119: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.264880] clockevents clockevent120: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.336972] clockevents clockevent121: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.409064] clockevents clockevent122: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.481157] clockevents clockevent123: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.553248] clockevents clockevent124: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.625348] clockevents clockevent125: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.697441] clockevents clockevent126: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.769530] clockevents clockevent127: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.841623] clockevents broadcast: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.913502] event_source armv8_pmuv3_0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
48.985462] event_source breakpoint: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.057120] event_source uprobe: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.128431] event_source kprobe: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.199742] event_source tracepoint: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.271399] event_source software: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.382131] misc autofs: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
49.514840] gpio gpiochip0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.352095] ipmi ipmi0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.504825] tty ptyp0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.583473] tty ptyp1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.662113] tty ptyp2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.740743] tty ptyp3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.819376] tty ptyp4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.898009] tty ptyp5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
50.976635] tty ptyp6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.055274] tty ptyp7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.133901] tty ptyp8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.212544] tty ptyp9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.291170] tty ptypa: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.369794] tty ptypb: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.448422] tty ptypc: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.527047] tty ptypd: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.605672] tty ptype: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.684295] tty ptypf: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.762921] tty ttyp0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.841550] tty ttyp1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.920177] tty ttyp2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
51.998802] tty ttyp3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.077428] tty ttyp4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.156058] tty ttyp5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.234693] tty ttyp6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.313319] tty ttyp7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.391954] tty ttyp8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.470578] tty ttyp9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.549200] tty ttypa: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.627824] tty ttypb: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.706452] tty ttypc: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.785075] tty ttypd: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.863701] tty ttype: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
52.942322] tty ttypf: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.020951] tty ptmx: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.139972] drm ttm: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.256246] vtconsole vtcon1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.458305] misc loop-control: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.538270] bdi 7:0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.635500] block loop0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.714423] bdi 7:1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.811666] block loop1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.890518] bdi 7:2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
53.987741] block loop2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.066649] bdi 7:3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.163875] block loop3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.242734] bdi 7:4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.339960] block loop4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.418789] bdi 7:5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.516011] block loop5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.594892] bdi 7:6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.692155] block loop6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.771105] bdi 7:7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.868240] block loop7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
54.950348] bdi 43:0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.047759] block nbd0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.126764] bdi 43:32: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.224242] block nbd1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.303101] bdi 43:64: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.400575] block nbd2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.479446] bdi 43:96: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.576919] block nbd3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.655754] bdi 43:128: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.753319] block nbd4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.832148] bdi 43:160: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
55.929710] block nbd5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.008551] bdi 43:192: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.106113] block nbd6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.184970] bdi 43:224: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.282534] block nbd7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.361357] bdi 43:256: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.458917] block nbd8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.537756] bdi 43:288: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.635317] block nbd9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.714287] bdi 43:320: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.811867] block nbd10: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.890748] bdi 43:352: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
56.988310] block nbd11: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.067250] bdi 43:384: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.164814] block nbd12: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.243730] bdi 43:416: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.341294] block nbd13: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.420210] bdi 43:448: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.517773] block nbd14: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.596654] bdi 43:480: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.694216] block nbd15: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.781918] misc mpt3ctl: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
57.865419] misc mpt2ctl: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
62.079202] bdi 8:0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
64.720183] workqueue nvme-wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
64.799501] workqueue nvme-reset-wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
64.879295] workqueue nvme-delete-wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
65.017555] bdi mtd-0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
65.100098] spi_master spi0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
65.238160] misc tun: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
65.441705] misc vfio: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.015188] workqueue raid5wq: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.132941] misc rdma_cm: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.208914] misc efi_capsule_loader: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.309719] misc vhost-net: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.397213] event_source hisi_sccl3_l3c0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.533004] event_source has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.674077] event_source hisi_sccl3_l3c2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.830922] event_source hisi_sccl3_l3c3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
66.951928] event_source hisi_sccl3_l3c4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.072943] event_source hisi_sccl3_l3c5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.193933] event_source hisi_sccl3_l3c6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.315076] event_source hisi_sccl3_l3c7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.436231] event_source hisi_sccl1_l3c8: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.557346] event_source hisi_sccl1_l3c9: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.678501] event_source hisi_sccl1_l3c10: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.799688] event_source hisi_sccl1_l3c11: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
67.921022] event_source hisi_sccl1_l3c12: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.042216] event_source hisi_sccl1_l3c13: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.163424] event_source hisi_sccl1_l3c14: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.284651] event_source hisi_sccl1_l3c15: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.405842] event_source hisi_sccl7_l3c16: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.527047] event_source hisi_sccl7_l3c17: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.648238] event_source hisi_sccl7_l3c18: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.769387] event_source hisi_sccl7_l3c19: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
68.890555] event_source hisi_sccl7_l3c20: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.011720] event_source hisi_sccl7_l3c21: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.132887] event_source hisi_sccl7_l3c22: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.254115] event_source hisi_sccl7_l3c23: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.375316] event_source hisi_sccl5_l3c24: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.496492] event_source hisi_sccl5_l3c25: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.617652] event_source hisi_sccl5_l3c26: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.738795] event_source hisi_sccl5_l3c27: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.859972] event_source hisi_sccl5_l3c28: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
69.981156] event_source hisi_sccl5_l3c29: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.102377] event_source hisi_sccl5_l3c30: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.223592] event_source hisi_sccl5_l3c31: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.345082] event_source hisi_sccl3_hha0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.466185] event_source hisi_sccl3_hha1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.587322] event_source hisi_sccl1_hha2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.708441] event_source hisi_sccl1_hha3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.829546] event_source hisi_sccl7_hha4: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
70.950649] event_source hisi_sccl7_hha5: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.071759] event_source hisi_sccl5_hha6: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.192863] event_source hisi_sccl5_hha7: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.314101] event_source hisi_sccl3_ddrc0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.435545] event_source hisi_sccl3_ddrc1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.556955] event_source hisi_sccl3_ddrc2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.678327] event_source hisi_sccl3_ddrc3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.799699] event_source hisi_sccl1_ddrc0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
71.921061] event_source hisi_sccl1_ddrc1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.042438] event_source hisi_sccl1_ddrc2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.163816] event_source hisi_sccl1_ddrc3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.285219] event_source hisi_sccl7_ddrc0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.406588] event_source hisi_sccl7_ddrc1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.527963] event_source hisi_sccl7_ddrc2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.649343] event_source hisi_sccl7_ddrc3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.770693] event_source hisi_sccl5_ddrc0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
72.892045] event_source hisi_sccl5_ddrc1: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.013423] event_source hisi_sccl5_ddrc2: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.134751] event_source hisi_sccl5_ddrc3: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.255559] sound timer: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.357023] net sit0: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.481434] misc cpu_dma_latency: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.562171] misc network_latency: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.642890] misc network_throughput: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.
73.723876] misc memory_bandwidth: has invalid NUMA node(-1), default node of 0 now selected. Readjust it by writing to sysfs numa_node or contact your vendor for updates.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-06  6:41       ` Yunsheng Lin
@ 2019-09-06  6:52         ` Greg KH
  2019-09-06  8:21           ` Yunsheng Lin
  0 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2019-09-06  6:52 UTC (permalink / raw)
  To: Yunsheng Lin; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On Fri, Sep 06, 2019 at 02:41:36PM +0800, Yunsheng Lin wrote:
> On 2019/9/5 15:33, Greg KH wrote:
> > On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
> >> On 2019/9/5 13:57, Greg KH wrote:
> >>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
> >>>> Currently a device does not belong to any of the numa nodes
> >>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
> >>>> the node id and the device has not no parent device.
> >>>>
> >>>> According to discussion in [1]:
> >>>> Even if a device's numa node is not set by fw, the device
> >>>> really does belong to a node.
> >>>>
> >>>> This patch sets the device node to node 0 in device_add() if
> >>>> the fw has not specified the node id and it either has no
> >>>> parent device, or the parent device also does not have a valid
> >>>> node id.
> >>>>
> >>>> There may be explicit handling out there relying on NUMA_NO_NODE,
> >>>> like in nvme_probe().
> >>>>
> >>>> [1] https://lkml.org/lkml/2019/9/2/466
> >>>>
> >>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> >>>> ---
> >>>>  drivers/base/core.c  | 17 ++++++++++++++---
> >>>>  include/linux/numa.h |  2 ++
> >>>>  2 files changed, 16 insertions(+), 3 deletions(-)
> >>>>
> >>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
> >>>> index 1669d41..466b8ff 100644
> >>>> --- a/drivers/base/core.c
> >>>> +++ b/drivers/base/core.c
> >>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
> >>>>  	if (kobj)
> >>>>  		dev->kobj.parent = kobj;
> >>>>  
> >>>> -	/* use parent numa_node */
> >>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
> >>>> -		set_dev_node(dev, dev_to_node(parent));
> >>>> +	/* use parent numa_node or default node 0 */
> >>>> +	if (!numa_node_valid(dev_to_node(dev))) {
> >>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
> >>>
> >>> Can you expand this to be a "real" if statement please?
> >>
> >> Sure. May I ask why "? :" is not appropriate here?
> > 
> > Because it is a pain to read, just spell it out and make it obvious what
> > is happening.  You write code for developers first, and the compiler
> > second, and in this case, either way is identical to the compiler.
> > 
> >>>> +
> >>>> +		if (numa_node_valid(nid)) {
> >>>> +			set_dev_node(dev, nid);
> >>>> +		} else {
> >>>> +			if (nr_node_ids > 1U)
> >>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
> >>>> +				       dev_name(dev), dev_to_node(dev));
> >>>
> >>> dev_err() will show you the exact device properly, instead of having to
> >>> rely on dev_name().
> >>>
> >>> And what is a user to do if this message happens?  How do they fix this?
> >>> If they can not, what good is this error message?
> >>
> >> If user know about their system's topology well enough and node 0
> >> is not the nearest node to the device, maybe user can readjust that by
> >> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
> >> if not, then maybe user need to contact the vendor for info or updates.
> >>
> >> Maybe print error message as below:
> >>
> >> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
> >> 	dev_to_node(dev));
> > 
> > FW_BUG?
> > 
> > Anyway, if you make this change, how many machines start reporting this
> > error? 
> 
> Any machines with more than one numa node will start reporting this error.
> 
> 1) many virtual deivces maybe do not set the node id before calling
>    device_register(), such as vfio, tun, etc.
> 
> 2) struct cpu has a dev, but does not set the dev' node according to
>    cpu_to_node().
> 
> 3) Many platform Device also do not have a node id provided by FW.

Then this patch is not ok, as you are flooding the kernel log saying the
system is "broken" when this is just what it always has been like.  How
is anyone going to "fix" things?

You can adjust the default node to 0 as isn't that what always has
happened, but you can not claim it is a "error" that this is happening
because it is not an error, it's just the default operation.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-06  6:52         ` Greg KH
@ 2019-09-06  8:21           ` Yunsheng Lin
  2019-09-06 14:00             ` Greg KH
  0 siblings, 1 reply; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-06  8:21 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/6 14:52, Greg KH wrote:
> On Fri, Sep 06, 2019 at 02:41:36PM +0800, Yunsheng Lin wrote:
>> On 2019/9/5 15:33, Greg KH wrote:
>>> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
>>>> On 2019/9/5 13:57, Greg KH wrote:
>>>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>>>>>> Currently a device does not belong to any of the numa nodes
>>>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>>>>>> the node id and the device has not no parent device.
>>>>>>
>>>>>> According to discussion in [1]:
>>>>>> Even if a device's numa node is not set by fw, the device
>>>>>> really does belong to a node.
>>>>>>
>>>>>> This patch sets the device node to node 0 in device_add() if
>>>>>> the fw has not specified the node id and it either has no
>>>>>> parent device, or the parent device also does not have a valid
>>>>>> node id.
>>>>>>
>>>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
>>>>>> like in nvme_probe().
>>>>>>
>>>>>> [1] https://lkml.org/lkml/2019/9/2/466
>>>>>>
>>>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>>>>>> ---
>>>>>>  drivers/base/core.c  | 17 ++++++++++++++---
>>>>>>  include/linux/numa.h |  2 ++
>>>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>>>>>> index 1669d41..466b8ff 100644
>>>>>> --- a/drivers/base/core.c
>>>>>> +++ b/drivers/base/core.c
>>>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>>>>>  	if (kobj)
>>>>>>  		dev->kobj.parent = kobj;
>>>>>>  
>>>>>> -	/* use parent numa_node */
>>>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>>>>>> -		set_dev_node(dev, dev_to_node(parent));
>>>>>> +	/* use parent numa_node or default node 0 */
>>>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
>>>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
>>>>>
>>>>> Can you expand this to be a "real" if statement please?
>>>>
>>>> Sure. May I ask why "? :" is not appropriate here?
>>>
>>> Because it is a pain to read, just spell it out and make it obvious what
>>> is happening.  You write code for developers first, and the compiler
>>> second, and in this case, either way is identical to the compiler.
>>>
>>>>>> +
>>>>>> +		if (numa_node_valid(nid)) {
>>>>>> +			set_dev_node(dev, nid);
>>>>>> +		} else {
>>>>>> +			if (nr_node_ids > 1U)
>>>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>>>>>> +				       dev_name(dev), dev_to_node(dev));
>>>>>
>>>>> dev_err() will show you the exact device properly, instead of having to
>>>>> rely on dev_name().
>>>>>
>>>>> And what is a user to do if this message happens?  How do they fix this?
>>>>> If they can not, what good is this error message?
>>>>
>>>> If user know about their system's topology well enough and node 0
>>>> is not the nearest node to the device, maybe user can readjust that by
>>>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
>>>> if not, then maybe user need to contact the vendor for info or updates.
>>>>
>>>> Maybe print error message as below:
>>>>
>>>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
>>>> 	dev_to_node(dev));
>>>
>>> FW_BUG?
>>>
>>> Anyway, if you make this change, how many machines start reporting this
>>> error? 
>>
>> Any machines with more than one numa node will start reporting this error.
>>
>> 1) many virtual deivces maybe do not set the node id before calling
>>    device_register(), such as vfio, tun, etc.
>>
>> 2) struct cpu has a dev, but does not set the dev' node according to
>>    cpu_to_node().
>>
>> 3) Many platform Device also do not have a node id provided by FW.
> 
> Then this patch is not ok, as you are flooding the kernel log saying the
> system is "broken" when this is just what it always has been like.  How
> is anyone going to "fix" things?

cpu->node_id does not seem to be used, maybe we can fix the cpu device:

diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index cc37511d..ad0a841 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -41,7 +41,7 @@ static void change_cpu_under_node(struct cpu *cpu,
        int cpuid = cpu->dev.id;
        unregister_cpu_under_node(cpuid, from_nid);
        register_cpu_under_node(cpuid, to_nid);
-       cpu->node_id = to_nid;
+       set_dev_node(&cpu->dev, to_nid);
 }

 static int cpu_subsys_online(struct device *dev)
@@ -367,7 +367,7 @@ int register_cpu(struct cpu *cpu, int num)
 {
        int error;

-       cpu->node_id = cpu_to_node(num);
+       set_dev_node(&cpu->dev, cpu_to_node(num));
        memset(&cpu->dev, 0x00, sizeof(struct device));
        cpu->dev.id = num;
        cpu->dev.bus = &cpu_subsys;
diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index fcb1386..9a6fc51 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -24,7 +24,6 @@ struct device_node;
 struct attribute_group;

 struct cpu {
-       int node_id;            /* The node which contains the CPU */
        int hotpluggable;       /* creates sysfs control file if hotpluggable */
        struct device dev;
 };


> 
> You can adjust the default node to 0 as isn't that what always has
> happened, but you can not claim it is a "error" that this is happening
> because it is not an error, it's just the default operation.

You are right, will remove the error log.


^ permalink raw reply related	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-06  8:21           ` Yunsheng Lin
@ 2019-09-06 14:00             ` Greg KH
  2019-09-07  2:10               ` Yunsheng Lin
  0 siblings, 1 reply; 12+ messages in thread
From: Greg KH @ 2019-09-06 14:00 UTC (permalink / raw)
  To: Yunsheng Lin; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On Fri, Sep 06, 2019 at 04:21:47PM +0800, Yunsheng Lin wrote:
> On 2019/9/6 14:52, Greg KH wrote:
> > On Fri, Sep 06, 2019 at 02:41:36PM +0800, Yunsheng Lin wrote:
> >> On 2019/9/5 15:33, Greg KH wrote:
> >>> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
> >>>> On 2019/9/5 13:57, Greg KH wrote:
> >>>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
> >>>>>> Currently a device does not belong to any of the numa nodes
> >>>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
> >>>>>> the node id and the device has not no parent device.
> >>>>>>
> >>>>>> According to discussion in [1]:
> >>>>>> Even if a device's numa node is not set by fw, the device
> >>>>>> really does belong to a node.
> >>>>>>
> >>>>>> This patch sets the device node to node 0 in device_add() if
> >>>>>> the fw has not specified the node id and it either has no
> >>>>>> parent device, or the parent device also does not have a valid
> >>>>>> node id.
> >>>>>>
> >>>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
> >>>>>> like in nvme_probe().
> >>>>>>
> >>>>>> [1] https://lkml.org/lkml/2019/9/2/466
> >>>>>>
> >>>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
> >>>>>> ---
> >>>>>>  drivers/base/core.c  | 17 ++++++++++++++---
> >>>>>>  include/linux/numa.h |  2 ++
> >>>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
> >>>>>>
> >>>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
> >>>>>> index 1669d41..466b8ff 100644
> >>>>>> --- a/drivers/base/core.c
> >>>>>> +++ b/drivers/base/core.c
> >>>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
> >>>>>>  	if (kobj)
> >>>>>>  		dev->kobj.parent = kobj;
> >>>>>>  
> >>>>>> -	/* use parent numa_node */
> >>>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
> >>>>>> -		set_dev_node(dev, dev_to_node(parent));
> >>>>>> +	/* use parent numa_node or default node 0 */
> >>>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
> >>>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
> >>>>>
> >>>>> Can you expand this to be a "real" if statement please?
> >>>>
> >>>> Sure. May I ask why "? :" is not appropriate here?
> >>>
> >>> Because it is a pain to read, just spell it out and make it obvious what
> >>> is happening.  You write code for developers first, and the compiler
> >>> second, and in this case, either way is identical to the compiler.
> >>>
> >>>>>> +
> >>>>>> +		if (numa_node_valid(nid)) {
> >>>>>> +			set_dev_node(dev, nid);
> >>>>>> +		} else {
> >>>>>> +			if (nr_node_ids > 1U)
> >>>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
> >>>>>> +				       dev_name(dev), dev_to_node(dev));
> >>>>>
> >>>>> dev_err() will show you the exact device properly, instead of having to
> >>>>> rely on dev_name().
> >>>>>
> >>>>> And what is a user to do if this message happens?  How do they fix this?
> >>>>> If they can not, what good is this error message?
> >>>>
> >>>> If user know about their system's topology well enough and node 0
> >>>> is not the nearest node to the device, maybe user can readjust that by
> >>>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
> >>>> if not, then maybe user need to contact the vendor for info or updates.
> >>>>
> >>>> Maybe print error message as below:
> >>>>
> >>>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
> >>>> 	dev_to_node(dev));
> >>>
> >>> FW_BUG?
> >>>
> >>> Anyway, if you make this change, how many machines start reporting this
> >>> error? 
> >>
> >> Any machines with more than one numa node will start reporting this error.
> >>
> >> 1) many virtual deivces maybe do not set the node id before calling
> >>    device_register(), such as vfio, tun, etc.
> >>
> >> 2) struct cpu has a dev, but does not set the dev' node according to
> >>    cpu_to_node().
> >>
> >> 3) Many platform Device also do not have a node id provided by FW.
> > 
> > Then this patch is not ok, as you are flooding the kernel log saying the
> > system is "broken" when this is just what it always has been like.  How
> > is anyone going to "fix" things?
> 
> cpu->node_id does not seem to be used, maybe we can fix the cpu device:
> 
> diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
> index cc37511d..ad0a841 100644
> --- a/drivers/base/cpu.c
> +++ b/drivers/base/cpu.c
> @@ -41,7 +41,7 @@ static void change_cpu_under_node(struct cpu *cpu,
>         int cpuid = cpu->dev.id;
>         unregister_cpu_under_node(cpuid, from_nid);
>         register_cpu_under_node(cpuid, to_nid);
> -       cpu->node_id = to_nid;
> +       set_dev_node(&cpu->dev, to_nid);
>  }
> 
>  static int cpu_subsys_online(struct device *dev)
> @@ -367,7 +367,7 @@ int register_cpu(struct cpu *cpu, int num)
>  {
>         int error;
> 
> -       cpu->node_id = cpu_to_node(num);
> +       set_dev_node(&cpu->dev, cpu_to_node(num));
>         memset(&cpu->dev, 0x00, sizeof(struct device));
>         cpu->dev.id = num;
>         cpu->dev.bus = &cpu_subsys;
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index fcb1386..9a6fc51 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -24,7 +24,6 @@ struct device_node;
>  struct attribute_group;
> 
>  struct cpu {
> -       int node_id;            /* The node which contains the CPU */
>         int hotpluggable;       /* creates sysfs control file if hotpluggable */
>         struct device dev;
>  };

I have no idea what you are trying to do here, it feels like you are
flailing around trying to set something that almost no bios/firmware
sets or cares about.

If setting the proper node is a requirement, make sure your firmware
does this and then all should be fine.  Otherwise just use the default
node like what happens today, right?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [PATCH RFC] driver core: ensure a device has valid node id in device_add()
  2019-09-06 14:00             ` Greg KH
@ 2019-09-07  2:10               ` Yunsheng Lin
  0 siblings, 0 replies; 12+ messages in thread
From: Yunsheng Lin @ 2019-09-07  2:10 UTC (permalink / raw)
  To: Greg KH; +Cc: rafael, linux-kernel, peterz, mingo, mhocko, linuxarm

On 2019/9/6 22:00, Greg KH wrote:
> On Fri, Sep 06, 2019 at 04:21:47PM +0800, Yunsheng Lin wrote:
>> On 2019/9/6 14:52, Greg KH wrote:
>>> On Fri, Sep 06, 2019 at 02:41:36PM +0800, Yunsheng Lin wrote:
>>>> On 2019/9/5 15:33, Greg KH wrote:
>>>>> On Thu, Sep 05, 2019 at 02:48:24PM +0800, Yunsheng Lin wrote:
>>>>>> On 2019/9/5 13:57, Greg KH wrote:
>>>>>>> On Thu, Sep 05, 2019 at 09:33:50AM +0800, Yunsheng Lin wrote:
>>>>>>>> Currently a device does not belong to any of the numa nodes
>>>>>>>> (dev->numa_node is NUMA_NO_NODE) when the FW does not provide
>>>>>>>> the node id and the device has not no parent device.
>>>>>>>>
>>>>>>>> According to discussion in [1]:
>>>>>>>> Even if a device's numa node is not set by fw, the device
>>>>>>>> really does belong to a node.
>>>>>>>>
>>>>>>>> This patch sets the device node to node 0 in device_add() if
>>>>>>>> the fw has not specified the node id and it either has no
>>>>>>>> parent device, or the parent device also does not have a valid
>>>>>>>> node id.
>>>>>>>>
>>>>>>>> There may be explicit handling out there relying on NUMA_NO_NODE,
>>>>>>>> like in nvme_probe().
>>>>>>>>
>>>>>>>> [1] https://lkml.org/lkml/2019/9/2/466
>>>>>>>>
>>>>>>>> Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
>>>>>>>> ---
>>>>>>>>  drivers/base/core.c  | 17 ++++++++++++++---
>>>>>>>>  include/linux/numa.h |  2 ++
>>>>>>>>  2 files changed, 16 insertions(+), 3 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/drivers/base/core.c b/drivers/base/core.c
>>>>>>>> index 1669d41..466b8ff 100644
>>>>>>>> --- a/drivers/base/core.c
>>>>>>>> +++ b/drivers/base/core.c
>>>>>>>> @@ -2107,9 +2107,20 @@ int device_add(struct device *dev)
>>>>>>>>  	if (kobj)
>>>>>>>>  		dev->kobj.parent = kobj;
>>>>>>>>  
>>>>>>>> -	/* use parent numa_node */
>>>>>>>> -	if (parent && (dev_to_node(dev) == NUMA_NO_NODE))
>>>>>>>> -		set_dev_node(dev, dev_to_node(parent));
>>>>>>>> +	/* use parent numa_node or default node 0 */
>>>>>>>> +	if (!numa_node_valid(dev_to_node(dev))) {
>>>>>>>> +		int nid = parent ? dev_to_node(parent) : NUMA_NO_NODE;
>>>>>>>
>>>>>>> Can you expand this to be a "real" if statement please?
>>>>>>
>>>>>> Sure. May I ask why "? :" is not appropriate here?
>>>>>
>>>>> Because it is a pain to read, just spell it out and make it obvious what
>>>>> is happening.  You write code for developers first, and the compiler
>>>>> second, and in this case, either way is identical to the compiler.
>>>>>
>>>>>>>> +
>>>>>>>> +		if (numa_node_valid(nid)) {
>>>>>>>> +			set_dev_node(dev, nid);
>>>>>>>> +		} else {
>>>>>>>> +			if (nr_node_ids > 1U)
>>>>>>>> +				pr_err("device: '%s': has invalid NUMA node(%d)\n",
>>>>>>>> +				       dev_name(dev), dev_to_node(dev));
>>>>>>>
>>>>>>> dev_err() will show you the exact device properly, instead of having to
>>>>>>> rely on dev_name().
>>>>>>>
>>>>>>> And what is a user to do if this message happens?  How do they fix this?
>>>>>>> If they can not, what good is this error message?
>>>>>>
>>>>>> If user know about their system's topology well enough and node 0
>>>>>> is not the nearest node to the device, maybe user can readjust that by
>>>>>> writing the nearest node to /sys/class/pci_bus/XXXX/device/numa_node,
>>>>>> if not, then maybe user need to contact the vendor for info or updates.
>>>>>>
>>>>>> Maybe print error message as below:
>>>>>>
>>>>>> dev_err(dev, FW_BUG "has invalid NUMA node(%d). Readjust it by writing to sysfs numa_node or contact your vendor for updates.\n",
>>>>>> 	dev_to_node(dev));
>>>>>
>>>>> FW_BUG?
>>>>>
>>>>> Anyway, if you make this change, how many machines start reporting this
>>>>> error? 
>>>>
>>>> Any machines with more than one numa node will start reporting this error.
>>>>
>>>> 1) many virtual deivces maybe do not set the node id before calling
>>>>    device_register(), such as vfio, tun, etc.
>>>>
>>>> 2) struct cpu has a dev, but does not set the dev' node according to
>>>>    cpu_to_node().
>>>>
>>>> 3) Many platform Device also do not have a node id provided by FW.
>>>
>>> Then this patch is not ok, as you are flooding the kernel log saying the
>>> system is "broken" when this is just what it always has been like.  How
>>> is anyone going to "fix" things?
>>
>> cpu->node_id does not seem to be used, maybe we can fix the cpu device:
>>
>> diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
>> index cc37511d..ad0a841 100644
>> --- a/drivers/base/cpu.c
>> +++ b/drivers/base/cpu.c
>> @@ -41,7 +41,7 @@ static void change_cpu_under_node(struct cpu *cpu,
>>         int cpuid = cpu->dev.id;
>>         unregister_cpu_under_node(cpuid, from_nid);
>>         register_cpu_under_node(cpuid, to_nid);
>> -       cpu->node_id = to_nid;
>> +       set_dev_node(&cpu->dev, to_nid);
>>  }
>>
>>  static int cpu_subsys_online(struct device *dev)
>> @@ -367,7 +367,7 @@ int register_cpu(struct cpu *cpu, int num)
>>  {
>>         int error;
>>
>> -       cpu->node_id = cpu_to_node(num);
>> +       set_dev_node(&cpu->dev, cpu_to_node(num));
>>         memset(&cpu->dev, 0x00, sizeof(struct device));
>>         cpu->dev.id = num;
>>         cpu->dev.bus = &cpu_subsys;
>> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
>> index fcb1386..9a6fc51 100644
>> --- a/include/linux/cpu.h
>> +++ b/include/linux/cpu.h
>> @@ -24,7 +24,6 @@ struct device_node;
>>  struct attribute_group;
>>
>>  struct cpu {
>> -       int node_id;            /* The node which contains the CPU */
>>         int hotpluggable;       /* creates sysfs control file if hotpluggable */
>>         struct device dev;
>>  };
> 
> I have no idea what you are trying to do here, it feels like you are
> flailing around trying to set something that almost no bios/firmware
> sets or cares about.

The above isn't related to my problem really.

It just that there may be three fields that can indicate
the node id of a cpu:
per_cpu(numa_node), cpu->node_id and cpu->dev.numa_node

The per_cpu(numa_node) may be used for the fast path, and
cpu->node_id does not seems to be used, so it can be removed
when cpu->dev.numa_node is there.

Anyway, this is different problem here, maybe a separate patch
to "fix" it or clean it up if the above makes sense to you.

Sorry for the confusion.

> 
> If setting the proper node is a requirement, make sure your firmware
> does this and then all should be fine.  Otherwise just use the default
> node like what happens today, right?

Yes


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-09-07  2:11 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-05  1:33 [PATCH RFC] driver core: ensure a device has valid node id in device_add() Yunsheng Lin
2019-09-05  5:57 ` Greg KH
2019-09-05  6:48   ` Yunsheng Lin
2019-09-05  7:33     ` Greg KH
2019-09-05  8:57       ` Yunsheng Lin
2019-09-05  9:02         ` Greg KH
2019-09-05  9:07           ` Yunsheng Lin
2019-09-06  6:41       ` Yunsheng Lin
2019-09-06  6:52         ` Greg KH
2019-09-06  8:21           ` Yunsheng Lin
2019-09-06 14:00             ` Greg KH
2019-09-07  2:10               ` Yunsheng Lin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).