From: Vincent Guittot <vincent.guittot@linaro.org>
To: Mel Gorman <mgorman@techsingularity.net>
Cc: Peter Zijlstra <peterz@infradead.org>,
LKML <linux-kernel@vger.kernel.org>,
Ingo Molnar <mingo@kernel.org>,
Valentin Schneider <valentin.schneider@arm.com>,
Juri Lelli <juri.lelli@redhat.com>
Subject: Re: [PATCH 3/3] sched/numa: Limit the amount of imbalance that can exist at fork time
Date: Tue, 17 Nov 2020 16:53:10 +0100 [thread overview]
Message-ID: <CAKfTPtD27L0Epg7wPzw7G2zDL8XgdVbB45dZEZEsLmuwRp5gcg@mail.gmail.com> (raw)
In-Reply-To: <20201117151740.GB3371@techsingularity.net>
On Tue, 17 Nov 2020 at 16:17, Mel Gorman <mgorman@techsingularity.net> wrote:
>
> On Tue, Nov 17, 2020 at 03:31:19PM +0100, Vincent Guittot wrote:
> > On Tue, 17 Nov 2020 at 15:18, Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Tue, Nov 17, 2020 at 01:42:22PM +0000, Mel Gorman wrote:
> > > > - if (local_sgs.idle_cpus)
> > > > + if (local_sgs.idle_cpus >= (sd->span_weight >> 2))
> > > > return NULL;
> > >
> > > Is that the same 25% ?
> >
> > same question for me
>
> It's the same 25%. It's in the comment as -- utilisation is not too high
utilization is misleading, it usually refers to pelt utilization
whereas whet you check is the number of busy cpus
> where "high" is related to adjust_numa_imbalance.
>
> > could we encapsulate this 25% allowed imbalance like for adjust_numa_imbalance
>
> Using adjust_numa_imbalance() directly in this context would be awkward
Would be good to use the same function to say if we allow or not the imbalance
something like numa_allow_imbalance(sg_lb_stats * group_stats)
which would return how much margin remains available before not
allowing the imbalance
and use the same metrics in all cases
> but the threshold could be shared with something like the additional
> diff below. Is that what you had in mind or something different?
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index adfab218a498..49ef3484c56c 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5878,6 +5878,8 @@ static int wake_affine(struct sched_domain *sd, struct task_struct *p,
> static struct sched_group *
> find_idlest_group(struct sched_domain *sd, struct task_struct *p, int this_cpu);
>
> +static inline int numa_imbalance_threshold(int weight);
> +
> /*
> * find_idlest_group_cpu - find the idlest CPU among the CPUs in the group.
> */
> @@ -8894,7 +8896,7 @@ find_idlest_group(struct sched_domain *sd, struct task_struct *p, int this_cpu)
> * If there is a real need of migration, periodic load
> * balance will take care of it.
> */
> - if (local_sgs.idle_cpus >= (sd->span_weight >> 2))
also here you use idle_cpus and on the other part you use nr_running.
Can't we use the same metrics on both sides?
> + if (local_sgs.idle_cpus >= numa_imbalance_threshold(sd->span_weight))
> return NULL;
> }
>
> @@ -8998,6 +9000,14 @@ static inline void update_sd_lb_stats(struct lb_env *env, struct sd_lb_stats *sd
>
> #define NUMA_IMBALANCE_MIN 2
>
> +/* Allows imbalances until active CPUs hits 25% of a domain */
> +#define IMBALANCE_THRESHOLD_SHIFT 2
> +
> +static inline int numa_imbalance_threshold(int weight)
> +{
> + return weight >> IMBALANCE_THRESHOLD_SHIFT;
> +}
> +
> static inline long adjust_numa_imbalance(int imbalance,
> int dst_running, int dst_weight)
> {
> @@ -9006,8 +9016,10 @@ static inline long adjust_numa_imbalance(int imbalance,
> * when the destination is lightly loaded so that pairs of
> * communicating tasks may remain local.
> */
> - if (dst_running < (dst_weight >> 2) && imbalance <= NUMA_IMBALANCE_MIN)
> + if (dst_running < numa_imbalance_threshold(dst_weight) &&
> + imbalance <= NUMA_IMBALANCE_MIN) {
> return 0;
> + }
>
> return imbalance;
> }
>
> --
> Mel Gorman
> SUSE Labs
next prev parent reply other threads:[~2020-11-17 15:53 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-17 13:42 [RFC PATCH 0/3] Revisit NUMA imbalance tolerance and fork balancing Mel Gorman
2020-11-17 13:42 ` [PATCH 1/3] sched/numa: Rename nr_running and break out the magic number Mel Gorman
2020-11-17 13:42 ` [PATCH 2/3] sched/numa: Allow a floating imbalance between NUMA nodes Mel Gorman
2020-11-17 14:16 ` Peter Zijlstra
2020-11-17 14:43 ` Mel Gorman
2020-11-17 14:24 ` Vincent Guittot
2020-11-17 14:44 ` Mel Gorman
2020-11-17 13:42 ` [PATCH 3/3] sched/numa: Limit the amount of imbalance that can exist at fork time Mel Gorman
2020-11-17 14:18 ` Peter Zijlstra
2020-11-17 14:31 ` Vincent Guittot
2020-11-17 15:17 ` Mel Gorman
2020-11-17 15:53 ` Vincent Guittot [this message]
2020-11-17 17:28 ` Mel Gorman
2020-11-18 16:06 ` Vincent Guittot
2020-11-18 16:50 ` Mel Gorman
2020-11-22 15:04 ` [sched/numa] e7f28850ea: unixbench.score 1.5% improvement kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKfTPtD27L0Epg7wPzw7G2zDL8XgdVbB45dZEZEsLmuwRp5gcg@mail.gmail.com \
--to=vincent.guittot@linaro.org \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=valentin.schneider@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).