RE: [RFC PATCH 1/2] sched/topology: Get rid of NUMA overlapping groups

From: Valentin Schneider <valentin.schneider@arm.com>
To: "Song Bao Hua \(Barry Song\)" <song.bao.hua@hisilicon.com>,
	"linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>
Cc: "vincent.guittot\@linaro.org" <vincent.guittot@linaro.org>,
	"mgorman\@suse.de" <mgorman@suse.de>,
	"mingo\@kernel.org" <mingo@kernel.org>,
	"peterz\@infradead.org" <peterz@infradead.org>,
	"dietmar.eggemann\@arm.com" <dietmar.eggemann@arm.com>,
	"morten.rasmussen\@arm.com" <morten.rasmussen@arm.com>,
	"linuxarm\@openeuler.org" <linuxarm@openeuler.org>,
	"xuwei \(O\)" <xuwei5@huawei.com>,
	"Liguozhu \(Kenneth\)" <liguozhu@hisilicon.com>,
	"tiantao \(H\)" <tiantao6@hisilicon.com>,
	wanghuiqiang <wanghuiqiang@huawei.com>,
	"Zengtao \(B\)" <prime.zeng@hisilicon.com>,
	Jonathan Cameron <jonathan.cameron@huawei.com>,
	"guodong.xu\@linaro.org" <guodong.xu@linaro.org>,
	Meelis Roos <mroos@linux.ee>
Subject: RE: [RFC PATCH 1/2] sched/topology: Get rid of NUMA overlapping groups
Date: Tue, 09 Feb 2021 10:41:45 +0000	[thread overview]
Message-ID: <jhj5z31v7yu.mognet@arm.com> (raw)
In-Reply-To: <9619692afa1d4571b59c8a1da5d1076a@hisilicon.com>

On 09/02/21 00:12, Song Bao Hua (Barry Song) wrote:
>> -----Original Message-----
>> From: Valentin Schneider [mailto:valentin.schneider@arm.com]
>>
>> Yes; let's take your topology for instance:
>>
>> node   0   1   2   3
>>     0:  10  12  20  22
>>     1:  12  10  22  24
>>     2:  20  22  10  12
>>     3:  22  24  12  10
>>
>>       2       10      2
>>   0 <---> 1 <---> 2 <---> 3
>
> Guess you actually mean
>        2       10      2
>    1 <---> 0 <---> 2 <---> 3
>

Yeah, you're right, sorry about that!

>>
>>
>> Domains for node1 will look like (before any fixes are applied):
>>
>> NUMA<=10: span=1   groups=(1)
>> NUMA<=12: span=0-1 groups=(1)->(0)
>> NUMA<=20: span=0-1 groups=(0,1)
>> NUMA<=22: span=0-2 groups=(0,1)->(0,2-3)
>> NUMA<=24: span=0-3 groups=(0-2)->(0,2-3)
>>
>> As you can see, the domain representing distance <= 20 will be degenerated
>> (it has a single group). If we were to e.g. add some more nodes to the left
>> of node0, then we would trigger the "grandchildren logic" for node1 and
>> would end up creating a reference to node1 NUMA<=20's sgc, which is a
>> mistake: that domain will be degenerated, and that sgc will never be
>> updated. The right thing to do here would be reference node1 NUMA<=12's
>> sgc, which the above snippet does.
>
> Guess I got your point even though the diagram is not correct :-)
>

Good!

> If the topology is as below(add a node left to node1 rather than
> node0):
>
>     9       2       10      2
> A <---> 1 <---> 0 <---> 2 <---> 3
>
> For nodeA,
> NUMA<=10: span=A   groups=(A)
> NUMA<=12: span= A groups= (A)
> NUMA<=19: span=A-1 groups=(A),(1)
> NUMA<=20: span=A-1 groups=(A,1)
> *1 NUMA<=21: span=A-1-0 groups=(A,1), node1's numa<=20
>
> For node0,
> NUMA<=10: span=9   groups=(0)
> #3 NUMA<=12: span=0-1 groups=(0)->(1)
> #2 NUMA<=19: span=0-1 groups=(0,1)
> #1 NUMA<=20: span=0-1-2 groups=(0,1),....
>
> *1 will firstly try #1, and it finds 2 is outside the A-1-0,
> then it will try #2. Finally #2 will be degenerated, so we
> should actually use #3. Amazing!
>

Bingo!

>>
>> >> +
>> >> +	return parent;
>> >> +}
>> >> +
>
> Thanks
> Barry