From: Darren Hart <darren@os.amperecomputing.com>
To: Yicong Yang <yangyicong@huawei.com>
Cc: yangyicong@hisilicon.com, Sudeep Holla <sudeep.holla@arm.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
"D . Scott Phillips" <scott@os.amperecomputing.com>,
Ilkka Koskinen <ilkka@os.amperecomputing.com>,
stable@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
Linux Arm <linux-arm-kernel@lists.infradead.org>,
Ionela Voinescu <ionela.voinescu@arm.com>,
Barry Song <21cnbao@gmail.com>,
Jonathan Cameron <jonathan.cameron@huawei.com>
Subject: Re: [PATCH v5] topology: make core_mask include at least cluster_siblings
Date: Thu, 15 Sep 2022 10:56:02 -0700 [thread overview]
Message-ID: <YyNnMmtoOrdexLoy@fedora> (raw)
In-Reply-To: <eee69d10-11d0-be2d-69f6-34089947311e@huawei.com>
On Thu, Sep 15, 2022 at 08:01:18PM +0800, Yicong Yang wrote:
> Hi Darren,
>
Hi Yicong,
...
> > diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
> > index 1d6636ebaac5..5497c5ab7318 100644
> > --- a/drivers/base/arch_topology.c
> > +++ b/drivers/base/arch_topology.c
> > @@ -667,6 +667,15 @@ const struct cpumask *cpu_coregroup_mask(int cpu)
> > core_mask = &cpu_topology[cpu].llc_sibling;
> > }
> >
> > + /*
> > + * For systems with no shared cpu-side LLC but with clusters defined,
> > + * extend core_mask to cluster_siblings. The sched domain builder will
> > + * then remove MC as redundant with CLS if SCHED_CLUSTER is enabled.
> > + */
> > + if (IS_ENABLED(CONFIG_SCHED_CLUSTER) &&
> > + cpumask_subset(core_mask, &cpu_topology[cpu].cluster_sibling))
> > + core_mask = &cpu_topology[cpu].cluster_sibling;
> > +
> > return core_mask;
> > }
> >
>
> Is this patch still necessary for Ampere after Ionela's patch [1], which
> will limit the cluster's span within coregroup's span.
Yes, see:
https://lore.kernel.org/lkml/YshYAyEWhE4z%2FKpB@fedora/
Both patches work together to accomplish the desired sched domains for the
Ampere Altra family.
>
> I found an issue that the NUMA domains are not built on qemu with:
>
> qemu-system-aarch64 \
> -kernel ${Image} \
> -smp 8 \
> -cpu cortex-a72 \
> -m 32G \
> -object memory-backend-ram,id=node0,size=8G \
> -object memory-backend-ram,id=node1,size=8G \
> -object memory-backend-ram,id=node2,size=8G \
> -object memory-backend-ram,id=node3,size=8G \
> -numa node,memdev=node0,cpus=0-1,nodeid=0 \
> -numa node,memdev=node1,cpus=2-3,nodeid=1 \
> -numa node,memdev=node2,cpus=4-5,nodeid=2 \
> -numa node,memdev=node3,cpus=6-7,nodeid=3 \
> -numa dist,src=0,dst=1,val=12 \
> -numa dist,src=0,dst=2,val=20 \
> -numa dist,src=0,dst=3,val=22 \
> -numa dist,src=1,dst=2,val=22 \
> -numa dist,src=1,dst=3,val=24 \
> -numa dist,src=2,dst=3,val=12 \
> -machine virt,iommu=smmuv3 \
> -net none \
> -initrd ${Rootfs} \
> -nographic \
> -bios QEMU_EFI.fd \
> -append "rdinit=/init console=ttyAMA0 earlycon=pl011,0x9000000 sched_verbose loglevel=8"
>
> I can see the schedule domain build stops at MC level since we reach all the
> cpus in the system:
>
> [ 2.141316] CPU0 attaching sched-domain(s):
> [ 2.142558] domain-0: span=0-7 level=MC
> [ 2.145364] groups: 0:{ span=0 cap=964 }, 1:{ span=1 cap=914 }, 2:{ span=2 cap=921 }, 3:{ span=3 cap=964 }, 4:{ span=4 cap=925 }, 5:{ span=5 cap=964 }, 6:{ span=6 cap=967 }, 7:{ span=7 cap=967 }
> [ 2.158357] CPU1 attaching sched-domain(s):
> [ 2.158964] domain-0: span=0-7 level=MC
> [...]
>
> Without this the NUMA domains are built correctly:
>
Without which? My patch, Ionela's patch, or both?
> [ 2.008885] CPU0 attaching sched-domain(s):
> [ 2.009764] domain-0: span=0-1 level=MC
> [ 2.012654] groups: 0:{ span=0 cap=962 }, 1:{ span=1 cap=925 }
> [ 2.016532] domain-1: span=0-3 level=NUMA
> [ 2.017444] groups: 0:{ span=0-1 cap=1887 }, 2:{ span=2-3 cap=1871 }
> [ 2.019354] domain-2: span=0-5 level=NUMA
I'm not following this topology - what in the description above should result in
a domain with span=0-5?
> [ 2.019983] groups: 0:{ span=0-3 cap=3758 }, 4:{ span=4-5 cap=1935 }
> [ 2.021527] domain-3: span=0-7 level=NUMA
> [ 2.022516] groups: 0:{ span=0-5 mask=0-1 cap=5693 }, 6:{ span=4-7 mask=6-7 cap=3978 }
> [...]
>
> Hope to see your comments since I have no Ampere machine and I don't know
> how to emulate its topology on qemu.
>
> [1] bfcc4397435d ("arch_topology: Limit span of cpu_clustergroup_mask()")
>
> Thanks,
> Yicong
Thanks,
--
Darren Hart
Ampere Computing / OS and Kernel
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-09-15 17:57 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-11 20:53 [PATCH v5] topology: make core_mask include at least cluster_siblings Darren Hart
2022-09-15 12:01 ` Yicong Yang
2022-09-15 17:56 ` Darren Hart [this message]
2022-09-16 7:59 ` Yicong Yang
2022-09-16 16:14 ` Ionela Voinescu
2022-09-16 17:46 ` Darren Hart
2022-09-16 17:41 ` Darren Hart
2022-09-19 13:22 ` Yicong Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YyNnMmtoOrdexLoy@fedora \
--to=darren@os.amperecomputing.com \
--cc=21cnbao@gmail.com \
--cc=catalin.marinas@arm.com \
--cc=dietmar.eggemann@arm.com \
--cc=gregkh@linuxfoundation.org \
--cc=ilkka@os.amperecomputing.com \
--cc=ionela.voinescu@arm.com \
--cc=jonathan.cameron@huawei.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rafael@kernel.org \
--cc=scott@os.amperecomputing.com \
--cc=stable@vger.kernel.org \
--cc=sudeep.holla@arm.com \
--cc=vincent.guittot@linaro.org \
--cc=will@kernel.org \
--cc=yangyicong@hisilicon.com \
--cc=yangyicong@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).