From: Mel Gorman <mgorman@techsingularity.net>
To: "Nayak, KPrateek (K Prateek)" <kprateek.nayak@amd.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Valentin Schneider <valentin.schneider@arm.com>,
Aubrey Li <aubrey.li@linux.intel.com>,
Barry Song <song.bao.hua@hisilicon.com>,
Mike Galbraith <efault@gmx.de>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
Gautham Shenoy <gautham.shenoy@amd.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans multiple LLCs
Date: Fri, 4 Feb 2022 16:45:57 +0000 [thread overview]
Message-ID: <20220204164556.GN3366@techsingularity.net> (raw)
In-Reply-To: <9d7e8fe1-d9d7-90df-0f30-cf82b82e7f1f@amd.com>
On Fri, Feb 04, 2022 at 08:37:53PM +0530, Nayak, KPrateek (K Prateek) wrote:
> On 2/3/2022 8:16 PM, Mel Gorman wrote:
> > @@ -9003,10 +9006,9 @@ static bool update_pick_idlest(struct sched_group *idlest,
> > * This is an approximation as the number of running tasks may not be
> > * related to the number of busy CPUs due to sched_setaffinity.
> > */
> > -static inline bool
> > -allow_numa_imbalance(unsigned int running, unsigned int weight)
> > +static inline bool allow_numa_imbalance(int running, int imb_numa_nr)
> > {
> > - return (running < (weight >> 2));
> > + return running < imb_numa_nr;
> > }
> >
> > /*
> > @@ -9146,7 +9148,7 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p, int this_cpu)
> > * allowed. If there is a real need of migration,
> > * periodic load balance will take care of it.
> > */
> > - if (allow_numa_imbalance(local_sgs.sum_nr_running + 1, local_sgs.group_weight))
> > + if (allow_numa_imbalance(local_sgs.sum_nr_running + 1, sd->imb_numa_nr))
>
> Could you please clarify why are we adding 1 to local_sgs.sum_nr_running while allowing imbalance?
To account for the new task similar to what task_numa_find_cpu before
calling adjust_numa_imbalance.
> allow_numa_imbalance allows the imbalance based on the following inequality:
>
> running < imb_numa_nr
>
> Consider on a Zen3 CPU with 8 LLCs in the sched group of the NUMA domain.
> Assume the group is running 7 task and we are finding the idlest group for the 8th task:
>
> sd->imb_numa_nr = 8
> local_sgs.sum_nr_running = 7
>
> In this case, local_sgs.sum_nr_running + 1 is equal to sd->imb_numa_nr and if we allow NUMA imbalance
> and place the task in the same group, each task can be given one LLC.
> However, allow_numa_imbalance returns 0 for the above case and can lead to task being placed on a different
> NUMA group.
>
> In case of Gautham's suggested fix (https://lore.kernel.org/lkml/YcHs37STv71n4erJ@BLR-5CG11610CF.amd.com/),
> the v4 patch in question (https://lore.kernel.org/lkml/20211210093307.31701-3-mgorman@techsingularity.net/)
> used the inequality "<=" to allow NUMA imbalance where we needed to consider the additional load CPU had
> to bear. However that doesn't seem to be the case here.
>
I failed to change < to <= in allow_numa_imbalance, I'll fix and retest.
--
Mel Gorman
SUSE Labs
next prev parent reply other threads:[~2022-02-04 16:46 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-03 14:46 [PATCH v5 0/2] Adjust NUMA imbalance for multiple LLCs Mel Gorman
2022-02-03 14:46 ` [PATCH 1/2] sched/fair: Improve consistency of allowed NUMA balance calculations Mel Gorman
2022-02-03 14:46 ` [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans multiple LLCs Mel Gorman
2022-02-04 7:06 ` Srikar Dronamraju
2022-02-04 9:04 ` Mel Gorman
2022-02-04 15:07 ` Nayak, KPrateek (K Prateek)
2022-02-04 16:45 ` Mel Gorman [this message]
-- strict thread matches above, loose matches on Subject: below --
2022-02-08 9:43 [PATCH v6 0/2] Adjust NUMA imbalance for " Mel Gorman
2022-02-08 9:43 ` [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans " Mel Gorman
2022-02-08 16:19 ` Gautham R. Shenoy
2022-02-09 5:10 ` K Prateek Nayak
2022-02-09 10:33 ` Mel Gorman
2022-02-11 19:02 ` Jirka Hladky
2022-02-14 10:27 ` Srikar Dronamraju
2022-02-14 11:03 ` Vincent Guittot
2021-12-10 9:33 [PATCH v4 0/2] Adjust NUMA imbalance for " Mel Gorman
2021-12-10 9:33 ` [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans " Mel Gorman
2021-12-13 8:28 ` Gautham R. Shenoy
2021-12-13 13:01 ` Mel Gorman
2021-12-13 14:47 ` Gautham R. Shenoy
2021-12-15 11:52 ` Gautham R. Shenoy
2021-12-15 12:25 ` Mel Gorman
2021-12-16 18:33 ` Gautham R. Shenoy
2021-12-20 11:12 ` Mel Gorman
2021-12-21 15:03 ` Gautham R. Shenoy
2021-12-21 17:13 ` Vincent Guittot
2021-12-22 8:52 ` Jirka Hladky
2022-01-04 19:52 ` Jirka Hladky
2022-01-05 10:42 ` Mel Gorman
2022-01-05 10:49 ` Mel Gorman
2022-01-10 15:53 ` Vincent Guittot
2022-01-12 10:24 ` Mel Gorman
2021-12-17 19:54 ` Gautham R. Shenoy
2021-12-01 15:18 [PATCH v3 0/2] Adjust NUMA imbalance for " Mel Gorman
2021-12-01 15:18 ` [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans " Mel Gorman
2021-12-03 8:15 ` Barry Song
2021-12-03 10:50 ` Mel Gorman
2021-12-03 11:14 ` Barry Song
2021-12-03 13:27 ` Mel Gorman
2021-12-04 10:40 ` Peter Zijlstra
2021-12-06 8:48 ` Gautham R. Shenoy
2021-12-06 14:51 ` Peter Zijlstra
2021-12-06 15:12 ` Mel Gorman
2021-12-09 14:23 ` Valentin Schneider
2021-12-09 15:43 ` Mel Gorman
2021-11-25 15:19 [PATCH 0/2] Adjust NUMA imbalance for " Mel Gorman
2021-11-25 15:19 ` [PATCH 2/2] sched/fair: Adjust the allowed NUMA imbalance when SD_NUMA spans " Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220204164556.GN3366@techsingularity.net \
--to=mgorman@techsingularity.net \
--cc=aubrey.li@linux.intel.com \
--cc=efault@gmx.de \
--cc=gautham.shenoy@amd.com \
--cc=kprateek.nayak@amd.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=song.bao.hua@hisilicon.com \
--cc=srikar@linux.vnet.ibm.com \
--cc=valentin.schneider@arm.com \
--cc=vincent.guittot@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).