From: "Zengtao (B)" <prime.zeng@hisilicon.com>
To: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Sudeep Holla <sudeep.holla@arm.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Linuxarm <linuxarm@huawei.com>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
"Rafael J. Wysocki" <rafael@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH] cpu-topology: warn if NUMA configurations conflicts with lower layer
Date: Thu, 9 Jan 2020 12:58:44 +0000 [thread overview]
Message-ID: <678F3D1BB717D949B966B68EAEB446ED340BEDD6@dggemm526-mbx.china.huawei.com> (raw)
In-Reply-To: <20200109104306.GA10914@e105550-lin.cambridge.arm.com>
> -----Original Message-----
> From: Morten Rasmussen [mailto:morten.rasmussen@arm.com]
> Sent: Thursday, January 09, 2020 6:43 PM
> To: Zengtao (B)
> Cc: Sudeep Holla; Valentin Schneider; Linuxarm; Greg Kroah-Hartman;
> Rafael J. Wysocki; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations conflicts
> with lower layer
>
> On Mon, Jan 06, 2020 at 01:37:59AM +0000, Zengtao (B) wrote:
> > > -----Original Message-----
> > > From: Sudeep Holla [mailto:sudeep.holla@arm.com]
> > > Sent: Friday, January 03, 2020 7:40 PM
> > > To: Zengtao (B)
> > > Cc: Valentin Schneider; Linuxarm; Greg Kroah-Hartman; Rafael J.
> Wysocki;
> > > linux-kernel@vger.kernel.org; Morten Rasmussen; Sudeep Holla
> > > Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations
> conflicts
> > > with lower layer
> > >
> > > On Fri, Jan 03, 2020 at 04:24:04AM +0000, Zengtao (B) wrote:
> > > > > -----Original Message-----
> > > > > From: Valentin Schneider [mailto:valentin.schneider@arm.com]
> > > > > Sent: Thursday, January 02, 2020 9:22 PM
> > > > > To: Zengtao (B); Sudeep Holla
> > > > > Cc: Linuxarm; Greg Kroah-Hartman; Rafael J. Wysocki;
> > > > > linux-kernel@vger.kernel.org; Morten Rasmussen
> > > > > Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations
> > > conflicts
> > > > > with lower layer
> > > > >
> > >
> > > [...]
> > >
> > > > >
> > > > > Right, and that is checked when you have sched_debug on the
> cmdline
> > > > > (or write 1 to /sys/kernel/debug/sched_debug & regenerate the
> sched
> > > > > domains)
> > > > >
> > > >
> > > > No, here I think you don't get my issue, please try to understand my
> > > example
> > > > First:.
> > > >
> > > > *************************************
> > > > NUMA: 0-2, 3-7
> > > > core_siblings: 0-3, 4-7
> > > > *************************************
> > > > When we are building the sched domain, per the current code:
> > > > (1) For core 3
> > > > MC sched domain fallbacks to 3~7
> > > > DIE sched domain is 3~7
> > > > (2) For core 4:
> > > > MC sched domain is 4~7
> > > > DIE sched domain is 3~7
> > > >
> > > > When we are build sched groups for the MC level:
> > > > (1). core3's sched groups chain is built like as: 3->4->5->6->7->3
> > > > (2). core4's sched groups chain is built like as: 4->5->6->7->4
> > > > so after (2),
> > > > core3's sched groups is overlapped, and it's not a chain any more.
> > > > In the afterwards usecase of core3's sched groups, deadloop
> happens.
> > > >
> > > > And it's difficult for the scheduler to find out such errors,
> > > > that is why I think a warning is necessary here.
> > > >
> > >
> > > We can figure out a way to warn if it's absolutely necessary, but I
> > > would like to understand the system topology here. You haven't
> answered
> > > my query on cache topology. Please give more description on why the
> > > NUMA configuration is like the above example with specific hardware
> > > design details. Is this just a case where user can specify anything
> > > they wish ?
> > >
> >
> > Sorry for the late response, In fact, it's a VM usecase, you can simply
> > understand it as a test case. It's a corner case, but it will hang the
> kernel,
> > that is why I suggest a warning is needed.
> >
> > I think we need an sanity check or just simply warning, either in the
> scheduler
> > or arch topology parsing.
>
> IIUC, the problem is that virt can set up a broken topology in some
> cases where MPIDR doesn't line up correctly with the defined NUMA
> nodes.
>
> We could argue that it is a qemu/virt problem, but it would be nice if
> we could at least detect it. The proposed patch isn't really the right
> solution as it warns on some valid topologies as Sudeep already pointed
> out.
>
> It sounds more like we need a mask subset check in the sched_domain
> building code, if there isn't already one?
Currently no, it's a bit complex to do the check in the sched_domain building code,
I need to take a think of that.
Suggestion welcomed.
Thanks
Zengtao
>
> Morten
next prev parent reply other threads:[~2020-01-09 12:59 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-23 8:16 [PATCH] cpu-topology: warn if NUMA configurations conflicts with lower layer z00214469
2019-12-31 16:40 ` Sudeep Holla
2020-01-02 3:05 ` Zengtao (B)
2020-01-02 11:29 ` Sudeep Holla
2020-01-02 12:47 ` Zengtao (B)
2020-01-02 13:22 ` Valentin Schneider
2020-01-02 19:30 ` Dietmar Eggemann
2020-01-03 4:24 ` Zengtao (B)
2020-01-03 10:57 ` Valentin Schneider
2020-01-03 12:14 ` Valentin Schneider
2020-01-03 17:20 ` Dietmar Eggemann
2020-01-06 1:48 ` Zengtao (B)
2020-01-06 14:31 ` Dietmar Eggemann
2020-01-08 2:19 ` Zengtao (B)
2020-01-09 11:05 ` Morten Rasmussen
2020-01-09 12:07 ` Dietmar Eggemann
2020-01-06 1:52 ` Zengtao (B)
2020-01-03 11:40 ` Sudeep Holla
2020-01-06 1:37 ` Zengtao (B)
2020-01-09 10:43 ` Morten Rasmussen
2020-01-09 12:58 ` Zengtao (B) [this message]
2020-01-11 20:56 ` Valentin Schneider
2020-01-13 6:51 ` Zengtao (B)
2020-01-13 11:16 ` Valentin Schneider
2020-01-13 12:08 ` Zengtao (B)
2020-01-13 12:22 ` Dietmar Eggemann
2020-01-13 14:49 ` Dietmar Eggemann
2020-01-13 15:15 ` Valentin Schneider
2020-01-09 10:52 ` Morten Rasmussen
2020-01-12 13:22 ` Valentin Schneider
2020-01-13 13:22 ` Morten Rasmussen
2020-01-02 13:59 ` Sudeep Holla
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=678F3D1BB717D949B966B68EAEB446ED340BEDD6@dggemm526-mbx.china.huawei.com \
--to=prime.zeng@hisilicon.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=morten.rasmussen@arm.com \
--cc=rafael@kernel.org \
--cc=sudeep.holla@arm.com \
--cc=valentin.schneider@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).