* [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
@ 2022-04-11 6:16 Kuyo Chang
2022-04-11 8:25 ` Peter Zijlstra
2022-04-11 8:39 ` Vincent Guittot
0 siblings, 2 replies; 7+ messages in thread
From: Kuyo Chang @ 2022-04-11 6:16 UTC (permalink / raw)
To: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot,
Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger
Cc: wsd_upstream, kuyo chang, linux-kernel, linux-arm-kernel, linux-mediatek
From: kuyo chang <kuyo.chang@mediatek.com>
I meet the warning message at cfs_rq_is_decayed at below code.
SCHED_WARN_ON(cfs_rq->avg.load_avg ||
cfs_rq->avg.util_avg ||
cfs_rq->avg.runnable_avg)
Following is the calltrace.
Call trace:
__update_blocked_fair
update_blocked_averages
newidle_balance
pick_next_task_fair
__schedule
schedule
pipe_read
vfs_read
ksys_read
After code analyzing and some debug messages, I found it exits a corner
case at attach_entity_load_avg which will cause load_sum is zero and
load_avg is not.
Consider se_weight is 88761 according by sched_prio_to_weight table.
And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
By the calculating for se->avg.load_sum as following will become zero
as following.
se->avg.load_sum =
div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
se->avg.load_sum = 1*47742/88761 = 0.
After enqueue_load_avg code as below.
cfs_rq->avg.load_avg += se->avg.load_avg;
cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
Then the load_sum for cfs_rq will be 1 while the load_sum for cfs_rq is 0.
So it will hit the warning message.
After all, I refer the following commit patch to do the similar thing at
enqueue_load_avg.
sched/pelt: Relax the sync of load_sum with load_avg
After long time testing, the kernel warning was gone and the system runs
as well as before.
Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
---
kernel/sched/fair.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d4bd299d67ab..30d8b6dba249 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se)
static inline void
enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
{
- cfs_rq->avg.load_avg += se->avg.load_avg;
- cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
+ add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
+ add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
+ cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
+ cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
}
static inline void
--
2.18.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-11 6:16 [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method Kuyo Chang
@ 2022-04-11 8:25 ` Peter Zijlstra
2022-04-11 10:14 ` Vincent Guittot
2022-04-11 8:39 ` Vincent Guittot
1 sibling, 1 reply; 7+ messages in thread
From: Peter Zijlstra @ 2022-04-11 8:25 UTC (permalink / raw)
To: Kuyo Chang
Cc: Ingo Molnar, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
On Mon, Apr 11, 2022 at 02:16:56PM +0800, Kuyo Chang wrote:
> From: kuyo chang <kuyo.chang@mediatek.com>
>
> I meet the warning message at cfs_rq_is_decayed at below code.
>
> SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> cfs_rq->avg.util_avg ||
> cfs_rq->avg.runnable_avg)
>
> Following is the calltrace.
>
> Call trace:
> __update_blocked_fair
> update_blocked_averages
> newidle_balance
> pick_next_task_fair
> __schedule
> schedule
> pipe_read
> vfs_read
> ksys_read
>
> After code analyzing and some debug messages, I found it exits a corner
> case at attach_entity_load_avg which will cause load_sum is zero and
> load_avg is not.
> Consider se_weight is 88761 according by sched_prio_to_weight table.
> And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
> By the calculating for se->avg.load_sum as following will become zero
> as following.
> se->avg.load_sum =
> div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
> se->avg.load_sum = 1*47742/88761 = 0.
>
> After enqueue_load_avg code as below.
> cfs_rq->avg.load_avg += se->avg.load_avg;
> cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
>
> Then the load_sum for cfs_rq will be 1 while the load_sum for cfs_rq is 0.
> So it will hit the warning message.
>
> After all, I refer the following commit patch to do the similar thing at
> enqueue_load_avg.
> sched/pelt: Relax the sync of load_sum with load_avg
2d02fa8cc21a ("sched/pelt: Relax the sync of load_sum with load_avg")
>
> After long time testing, the kernel warning was gone and the system runs
> as well as before.
>
Could probably do with a Fixes: tag too
> Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> ---
> kernel/sched/fair.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d4bd299d67ab..30d8b6dba249 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se)
> static inline void
> enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> - cfs_rq->avg.load_avg += se->avg.load_avg;
> - cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
> + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> + cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> }
Yes,. that looks right. Looking at this again though, I wonder if we
should perhaps clean is all up a bit.
Does something like the below on top make sense?
---
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -3071,23 +3071,37 @@ account_entity_dequeue(struct cfs_rq *cf
} while (0)
#ifdef CONFIG_SMP
+
+/*
+ * Because of rounding, se->util_sum might ends up being +1 more than
+ * cfs->util_sum. Although this is not a problem by itself, detaching
+ * a lot of tasks with the rounding problem between 2 updates of
+ * util_avg (~1ms) can make cfs->util_sum becoming null whereas
+ * cfs_util_avg is not.
+ *
+ * Check that util_sum is still above its lower bound for the new
+ * util_avg. Given that period_contrib might have moved since the last
+ * sync, we are only sure that util_sum must be above or equal to
+ * util_avg * minimum possible divider
+ */
+#define __update_sa(sa, name, delta_avg, delta_sum) do { \
+ add_positive(&(sa)->name##_avg, delta_avg); \
+ add_positive(&(sa)->name##_sum, delta_sum); \
+ (sa)->name##_sum = max_t(typeof((sa)->name##_sum), \
+ (sa)->name##_sum, \
+ (sa)->name##_avg * PELT_MIN_DIVIDER); \
+} while (0)
+
static inline void
enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
{
- add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
- add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
- cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
- cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, load, se->avg.load_avg, se->avg.load_sum);
}
static inline void
dequeue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
{
- sub_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
- sub_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
- cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, load, -se->avg.load_avg, -se->avg.load_sum);
}
#else
static inline void
@@ -3529,12 +3543,7 @@ update_tg_cfs_util(struct cfs_rq *cfs_rq
se->avg.util_sum = new_sum;
/* Update parent cfs_rq utilization */
- add_positive(&cfs_rq->avg.util_avg, delta_avg);
- add_positive(&cfs_rq->avg.util_sum, delta_sum);
-
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.util_sum = max_t(u32, cfs_rq->avg.util_sum,
- cfs_rq->avg.util_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, util, delta_avg, delta_sum);
}
static inline void
@@ -3560,11 +3569,7 @@ update_tg_cfs_runnable(struct cfs_rq *cf
se->avg.runnable_sum = new_sum;
/* Update parent cfs_rq runnable */
- add_positive(&cfs_rq->avg.runnable_avg, delta_avg);
- add_positive(&cfs_rq->avg.runnable_sum, delta_sum);
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.runnable_sum = max_t(u32, cfs_rq->avg.runnable_sum,
- cfs_rq->avg.runnable_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, runnable, delta_avg, delta_sum);
}
static inline void
@@ -3628,11 +3633,7 @@ update_tg_cfs_load(struct cfs_rq *cfs_rq
se->avg.load_sum = runnable_sum;
se->avg.load_avg = load_avg;
- add_positive(&cfs_rq->avg.load_avg, delta_avg);
- add_positive(&cfs_rq->avg.load_sum, delta_sum);
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
- cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, load, delta_avg, delta_sum);
}
static inline void add_tg_cfs_propagate(struct cfs_rq *cfs_rq, long runnable_sum)
@@ -3747,33 +3748,13 @@ update_cfs_rq_load_avg(u64 now, struct c
raw_spin_unlock(&cfs_rq->removed.lock);
r = removed_load;
- sub_positive(&sa->load_avg, r);
- sub_positive(&sa->load_sum, r * divider);
- /* See sa->util_sum below */
- sa->load_sum = max_t(u32, sa->load_sum, sa->load_avg * PELT_MIN_DIVIDER);
+ __update_sa(sa, load, r, r*divider);
r = removed_util;
- sub_positive(&sa->util_avg, r);
- sub_positive(&sa->util_sum, r * divider);
- /*
- * Because of rounding, se->util_sum might ends up being +1 more than
- * cfs->util_sum. Although this is not a problem by itself, detaching
- * a lot of tasks with the rounding problem between 2 updates of
- * util_avg (~1ms) can make cfs->util_sum becoming null whereas
- * cfs_util_avg is not.
- * Check that util_sum is still above its lower bound for the new
- * util_avg. Given that period_contrib might have moved since the last
- * sync, we are only sure that util_sum must be above or equal to
- * util_avg * minimum possible divider
- */
- sa->util_sum = max_t(u32, sa->util_sum, sa->util_avg * PELT_MIN_DIVIDER);
+ __update_sa(sa, util, r, r*divider);
r = removed_runnable;
- sub_positive(&sa->runnable_avg, r);
- sub_positive(&sa->runnable_sum, r * divider);
- /* See sa->util_sum above */
- sa->runnable_sum = max_t(u32, sa->runnable_sum,
- sa->runnable_avg * PELT_MIN_DIVIDER);
+ __update_sa(sa, runnable, r, r*divider);
/*
* removed_runnable is the unweighted version of removed_load so we
@@ -3861,17 +3842,8 @@ static void attach_entity_load_avg(struc
static void detach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
{
dequeue_load_avg(cfs_rq, se);
- sub_positive(&cfs_rq->avg.util_avg, se->avg.util_avg);
- sub_positive(&cfs_rq->avg.util_sum, se->avg.util_sum);
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.util_sum = max_t(u32, cfs_rq->avg.util_sum,
- cfs_rq->avg.util_avg * PELT_MIN_DIVIDER);
-
- sub_positive(&cfs_rq->avg.runnable_avg, se->avg.runnable_avg);
- sub_positive(&cfs_rq->avg.runnable_sum, se->avg.runnable_sum);
- /* See update_cfs_rq_load_avg() */
- cfs_rq->avg.runnable_sum = max_t(u32, cfs_rq->avg.runnable_sum,
- cfs_rq->avg.runnable_avg * PELT_MIN_DIVIDER);
+ __update_sa(&cfs_rq->avg, util, se->avg.util_avg, se->avg.util_sum);
+ __update_sa(&cfs_rq->avg, runnable, se->avg.runnable_avg, se->avg.runnable_sum);
add_tg_cfs_propagate(cfs_rq, -se->avg.load_sum);
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-11 6:16 [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method Kuyo Chang
2022-04-11 8:25 ` Peter Zijlstra
@ 2022-04-11 8:39 ` Vincent Guittot
2022-04-12 2:51 ` Kuyo Chang
1 sibling, 1 reply; 7+ messages in thread
From: Vincent Guittot @ 2022-04-11 8:39 UTC (permalink / raw)
To: Kuyo Chang
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
On Mon, 11 Apr 2022 at 08:17, Kuyo Chang <kuyo.chang@mediatek.com> wrote:
>
> From: kuyo chang <kuyo.chang@mediatek.com>
>
> I meet the warning message at cfs_rq_is_decayed at below code.
>
> SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> cfs_rq->avg.util_avg ||
> cfs_rq->avg.runnable_avg)
>
> Following is the calltrace.
>
> Call trace:
> __update_blocked_fair
> update_blocked_averages
> newidle_balance
> pick_next_task_fair
> __schedule
> schedule
> pipe_read
> vfs_read
> ksys_read
>
> After code analyzing and some debug messages, I found it exits a corner
> case at attach_entity_load_avg which will cause load_sum is zero and
> load_avg is not.
> Consider se_weight is 88761 according by sched_prio_to_weight table.
> And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
> By the calculating for se->avg.load_sum as following will become zero
> as following.
> se->avg.load_sum =
> div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
> se->avg.load_sum = 1*47742/88761 = 0.
The root problem is there, se->avg.load_sum must not be null if
se->avg.load_avg is not null because the correct relation between _avg
and _sum is:
load_avg = weight * load_sum / divider.
so the fix should be attach_entity_load_avg() and probably the below is enough
se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum,
se_weight(se)) + 1;
>
> After enqueue_load_avg code as below.
> cfs_rq->avg.load_avg += se->avg.load_avg;
> cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
>
> Then the load_sum for cfs_rq will be 1 while the load_sum for cfs_rq is 0.
> So it will hit the warning message.
>
> After all, I refer the following commit patch to do the similar thing at
> enqueue_load_avg.
> sched/pelt: Relax the sync of load_sum with load_avg
>
> After long time testing, the kernel warning was gone and the system runs
> as well as before.
>
> Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> ---
> kernel/sched/fair.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d4bd299d67ab..30d8b6dba249 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se)
> static inline void
> enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> - cfs_rq->avg.load_avg += se->avg.load_avg;
> - cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
> + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> + cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> }
>
> static inline void
> --
> 2.18.0
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-11 8:25 ` Peter Zijlstra
@ 2022-04-11 10:14 ` Vincent Guittot
0 siblings, 0 replies; 7+ messages in thread
From: Vincent Guittot @ 2022-04-11 10:14 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Kuyo Chang, Ingo Molnar, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
On Mon, 11 Apr 2022 at 10:25, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Apr 11, 2022 at 02:16:56PM +0800, Kuyo Chang wrote:
> > From: kuyo chang <kuyo.chang@mediatek.com>
> >
> > I meet the warning message at cfs_rq_is_decayed at below code.
> >
> > SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> > cfs_rq->avg.util_avg ||
> > cfs_rq->avg.runnable_avg)
> >
> > Following is the calltrace.
> >
> > Call trace:
> > __update_blocked_fair
> > update_blocked_averages
> > newidle_balance
> > pick_next_task_fair
> > __schedule
> > schedule
> > pipe_read
> > vfs_read
> > ksys_read
> >
> > After code analyzing and some debug messages, I found it exits a corner
> > case at attach_entity_load_avg which will cause load_sum is zero and
> > load_avg is not.
> > Consider se_weight is 88761 according by sched_prio_to_weight table.
> > And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
> > By the calculating for se->avg.load_sum as following will become zero
> > as following.
> > se->avg.load_sum =
> > div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
> > se->avg.load_sum = 1*47742/88761 = 0.
> >
> > After enqueue_load_avg code as below.
> > cfs_rq->avg.load_avg += se->avg.load_avg;
> > cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> >
> > Then the load_sum for cfs_rq will be 1 while the load_sum for cfs_rq is 0.
> > So it will hit the warning message.
> >
> > After all, I refer the following commit patch to do the similar thing at
> > enqueue_load_avg.
> > sched/pelt: Relax the sync of load_sum with load_avg
>
> 2d02fa8cc21a ("sched/pelt: Relax the sync of load_sum with load_avg")
>
> >
> > After long time testing, the kernel warning was gone and the system runs
> > as well as before.
> >
>
> Could probably do with a Fixes: tag too
>
> > Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> > ---
> > kernel/sched/fair.c | 6 ++++--
> > 1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index d4bd299d67ab..30d8b6dba249 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq *cfs_rq, struct sched_entity *se)
> > static inline void
> > enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> > {
> > - cfs_rq->avg.load_avg += se->avg.load_avg;
> > - cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> > + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> > + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
> > + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> > + cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> > }
>
> Yes,. that looks right. Looking at this again though, I wonder if we
> should perhaps clean is all up a bit.
adding se->load_avg|sum in cfs_rq should not create a situation where
cfs->load_sum is null but not cfs->load_avg unless se->load_avg|sum
are wrong from the beginning which seems to be the case here
>
> Does something like the below on top make sense?
>
>
> ---
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3071,23 +3071,37 @@ account_entity_dequeue(struct cfs_rq *cf
> } while (0)
>
> #ifdef CONFIG_SMP
> +
> +/*
> + * Because of rounding, se->util_sum might ends up being +1 more than
> + * cfs->util_sum. Although this is not a problem by itself, detaching
> + * a lot of tasks with the rounding problem between 2 updates of
> + * util_avg (~1ms) can make cfs->util_sum becoming null whereas
> + * cfs_util_avg is not.
> + *
> + * Check that util_sum is still above its lower bound for the new
> + * util_avg. Given that period_contrib might have moved since the last
> + * sync, we are only sure that util_sum must be above or equal to
> + * util_avg * minimum possible divider
> + */
> +#define __update_sa(sa, name, delta_avg, delta_sum) do { \
> + add_positive(&(sa)->name##_avg, delta_avg); \
> + add_positive(&(sa)->name##_sum, delta_sum); \
> + (sa)->name##_sum = max_t(typeof((sa)->name##_sum), \
> + (sa)->name##_sum, \
> + (sa)->name##_avg * PELT_MIN_DIVIDER); \
> +} while (0)
> +
> static inline void
> enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> - add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> - add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
> - cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> - cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, load, se->avg.load_avg, se->avg.load_sum);
> }
>
> static inline void
> dequeue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> - sub_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> - sub_positive(&cfs_rq->avg.load_sum, se_weight(se) * se->avg.load_sum);
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> - cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, load, -se->avg.load_avg, -se->avg.load_sum);
> }
> #else
> static inline void
> @@ -3529,12 +3543,7 @@ update_tg_cfs_util(struct cfs_rq *cfs_rq
> se->avg.util_sum = new_sum;
>
> /* Update parent cfs_rq utilization */
> - add_positive(&cfs_rq->avg.util_avg, delta_avg);
> - add_positive(&cfs_rq->avg.util_sum, delta_sum);
> -
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.util_sum = max_t(u32, cfs_rq->avg.util_sum,
> - cfs_rq->avg.util_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, util, delta_avg, delta_sum);
> }
>
> static inline void
> @@ -3560,11 +3569,7 @@ update_tg_cfs_runnable(struct cfs_rq *cf
> se->avg.runnable_sum = new_sum;
>
> /* Update parent cfs_rq runnable */
> - add_positive(&cfs_rq->avg.runnable_avg, delta_avg);
> - add_positive(&cfs_rq->avg.runnable_sum, delta_sum);
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.runnable_sum = max_t(u32, cfs_rq->avg.runnable_sum,
> - cfs_rq->avg.runnable_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, runnable, delta_avg, delta_sum);
> }
>
> static inline void
> @@ -3628,11 +3633,7 @@ update_tg_cfs_load(struct cfs_rq *cfs_rq
>
> se->avg.load_sum = runnable_sum;
> se->avg.load_avg = load_avg;
> - add_positive(&cfs_rq->avg.load_avg, delta_avg);
> - add_positive(&cfs_rq->avg.load_sum, delta_sum);
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> - cfs_rq->avg.load_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, load, delta_avg, delta_sum);
> }
>
> static inline void add_tg_cfs_propagate(struct cfs_rq *cfs_rq, long runnable_sum)
> @@ -3747,33 +3748,13 @@ update_cfs_rq_load_avg(u64 now, struct c
> raw_spin_unlock(&cfs_rq->removed.lock);
>
> r = removed_load;
> - sub_positive(&sa->load_avg, r);
> - sub_positive(&sa->load_sum, r * divider);
> - /* See sa->util_sum below */
> - sa->load_sum = max_t(u32, sa->load_sum, sa->load_avg * PELT_MIN_DIVIDER);
> + __update_sa(sa, load, r, r*divider);
>
> r = removed_util;
> - sub_positive(&sa->util_avg, r);
> - sub_positive(&sa->util_sum, r * divider);
> - /*
> - * Because of rounding, se->util_sum might ends up being +1 more than
> - * cfs->util_sum. Although this is not a problem by itself, detaching
> - * a lot of tasks with the rounding problem between 2 updates of
> - * util_avg (~1ms) can make cfs->util_sum becoming null whereas
> - * cfs_util_avg is not.
> - * Check that util_sum is still above its lower bound for the new
> - * util_avg. Given that period_contrib might have moved since the last
> - * sync, we are only sure that util_sum must be above or equal to
> - * util_avg * minimum possible divider
> - */
> - sa->util_sum = max_t(u32, sa->util_sum, sa->util_avg * PELT_MIN_DIVIDER);
> + __update_sa(sa, util, r, r*divider);
>
> r = removed_runnable;
> - sub_positive(&sa->runnable_avg, r);
> - sub_positive(&sa->runnable_sum, r * divider);
> - /* See sa->util_sum above */
> - sa->runnable_sum = max_t(u32, sa->runnable_sum,
> - sa->runnable_avg * PELT_MIN_DIVIDER);
> + __update_sa(sa, runnable, r, r*divider);
>
> /*
> * removed_runnable is the unweighted version of removed_load so we
> @@ -3861,17 +3842,8 @@ static void attach_entity_load_avg(struc
> static void detach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> dequeue_load_avg(cfs_rq, se);
> - sub_positive(&cfs_rq->avg.util_avg, se->avg.util_avg);
> - sub_positive(&cfs_rq->avg.util_sum, se->avg.util_sum);
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.util_sum = max_t(u32, cfs_rq->avg.util_sum,
> - cfs_rq->avg.util_avg * PELT_MIN_DIVIDER);
> -
> - sub_positive(&cfs_rq->avg.runnable_avg, se->avg.runnable_avg);
> - sub_positive(&cfs_rq->avg.runnable_sum, se->avg.runnable_sum);
> - /* See update_cfs_rq_load_avg() */
> - cfs_rq->avg.runnable_sum = max_t(u32, cfs_rq->avg.runnable_sum,
> - cfs_rq->avg.runnable_avg * PELT_MIN_DIVIDER);
> + __update_sa(&cfs_rq->avg, util, se->avg.util_avg, se->avg.util_sum);
> + __update_sa(&cfs_rq->avg, runnable, se->avg.runnable_avg, se->avg.runnable_sum);
>
> add_tg_cfs_propagate(cfs_rq, -se->avg.load_sum);
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-11 8:39 ` Vincent Guittot
@ 2022-04-12 2:51 ` Kuyo Chang
2022-04-12 8:58 ` Vincent Guittot
0 siblings, 1 reply; 7+ messages in thread
From: Kuyo Chang @ 2022-04-12 2:51 UTC (permalink / raw)
To: Vincent Guittot
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
On Mon, 2022-04-11 at 10:39 +0200, Vincent Guittot wrote:
> On Mon, 11 Apr 2022 at 08:17, Kuyo Chang <kuyo.chang@mediatek.com>
> wrote:
> >
> > From: kuyo chang <kuyo.chang@mediatek.com>
> >
> > I meet the warning message at cfs_rq_is_decayed at below code.
> >
> > SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> > cfs_rq->avg.util_avg ||
> > cfs_rq->avg.runnable_avg)
> >
> > Following is the calltrace.
> >
> > Call trace:
> > __update_blocked_fair
> > update_blocked_averages
> > newidle_balance
> > pick_next_task_fair
> > __schedule
> > schedule
> > pipe_read
> > vfs_read
> > ksys_read
> >
> > After code analyzing and some debug messages, I found it exits a
> > corner
> > case at attach_entity_load_avg which will cause load_sum is zero
> > and
> > load_avg is not.
> > Consider se_weight is 88761 according by sched_prio_to_weight
> > table.
> > And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
> > By the calculating for se->avg.load_sum as following will become
> > zero
> > as following.
> > se->avg.load_sum =
> > div_u64(se->avg.load_avg * se->avg.load_sum,
> > se_weight(se));
> > se->avg.load_sum = 1*47742/88761 = 0.
>
> The root problem is there, se->avg.load_sum must not be null if
> se->avg.load_avg is not null because the correct relation between
> _avg
> and _sum is:
>
> load_avg = weight * load_sum / divider.
>
> so the fix should be attach_entity_load_avg() and probably the below
> is enough
>
> se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum,
> se_weight(se)) + 1;
Thanks for your kindly suggestion.
+1 would make the calcuation for load_sum may be overestimate?
How about the below code make sense for fix the corner case?
---
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -3832,7 +3832,8 @@ static void attach_entity_load_avg(struct cfs_rq
*cfs_rq, struct sched_entity *s
se->avg.load_sum = divider;
if (se_weight(se)) {
se->avg.load_sum =
- div_u64(se->avg.load_avg * se->avg.load_sum,
se_weight(se));
+ (se->avg.load_avg * se->avg.load_sum >
se_weight(se)) ?
+ div_u64(se->avg.load_avg * se->avg.load_sum,
se_weight(se)) : 1;
}
enqueue_load_avg(cfs_rq, se);
--
2.18.0
> >
> > After enqueue_load_avg code as below.
> > cfs_rq->avg.load_avg += se->avg.load_avg;
> > cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> >
> > Then the load_sum for cfs_rq will be 1 while the load_sum for
> > cfs_rq is 0.
> > So it will hit the warning message.
> >
> > After all, I refer the following commit patch to do the similar
> > thing at
> > enqueue_load_avg.
> > sched/pelt: Relax the sync of load_sum with load_avg
> >
> > After long time testing, the kernel warning was gone and the system
> > runs
> > as well as before.
> >
> > Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> > ---
> > kernel/sched/fair.c | 6 ++++--
> > 1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index d4bd299d67ab..30d8b6dba249 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq
> > *cfs_rq, struct sched_entity *se)
> > static inline void
> > enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> > {
> > - cfs_rq->avg.load_avg += se->avg.load_avg;
> > - cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> > + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> > + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se-
> > >avg.load_sum);
> > + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> > + cfs_rq->avg.load_avg *
> > PELT_MIN_DIVIDER);
> > }
> >
> > static inline void
> > --
> > 2.18.0
> >
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-12 2:51 ` Kuyo Chang
@ 2022-04-12 8:58 ` Vincent Guittot
2022-04-12 9:28 ` Kuyo Chang
0 siblings, 1 reply; 7+ messages in thread
From: Vincent Guittot @ 2022-04-12 8:58 UTC (permalink / raw)
To: Kuyo Chang
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
Le mardi 12 avril 2022 à 10:51:23 (+0800), Kuyo Chang a écrit :
> On Mon, 2022-04-11 at 10:39 +0200, Vincent Guittot wrote:
> > On Mon, 11 Apr 2022 at 08:17, Kuyo Chang <kuyo.chang@mediatek.com>
> > wrote:
> > >
> > > From: kuyo chang <kuyo.chang@mediatek.com>
> > >
> > > I meet the warning message at cfs_rq_is_decayed at below code.
> > >
> > > SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> > > cfs_rq->avg.util_avg ||
> > > cfs_rq->avg.runnable_avg)
> > >
> > > Following is the calltrace.
> > >
> > > Call trace:
> > > __update_blocked_fair
> > > update_blocked_averages
> > > newidle_balance
> > > pick_next_task_fair
> > > __schedule
> > > schedule
> > > pipe_read
> > > vfs_read
> > > ksys_read
> > >
> > > After code analyzing and some debug messages, I found it exits a
> > > corner
> > > case at attach_entity_load_avg which will cause load_sum is zero
> > > and
> > > load_avg is not.
> > > Consider se_weight is 88761 according by sched_prio_to_weight
> > > table.
> > > And assume the get_pelt_divider() is 47742, se->avg.load_avg is 1.
> > > By the calculating for se->avg.load_sum as following will become
> > > zero
> > > as following.
> > > se->avg.load_sum =
> > > div_u64(se->avg.load_avg * se->avg.load_sum,
> > > se_weight(se));
> > > se->avg.load_sum = 1*47742/88761 = 0.
> >
> > The root problem is there, se->avg.load_sum must not be null if
> > se->avg.load_avg is not null because the correct relation between
> > _avg
> > and _sum is:
> >
> > load_avg = weight * load_sum / divider.
> >
> > so the fix should be attach_entity_load_avg() and probably the below
> > is enough
> >
> > se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum,
> > se_weight(se)) + 1;
>
> Thanks for your kindly suggestion.
> +1 would make the calcuation for load_sum may be overestimate?
> How about the below code make sense for fix the corner case?
>
> ---
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3832,7 +3832,8 @@ static void attach_entity_load_avg(struct cfs_rq
> *cfs_rq, struct sched_entity *s
> se->avg.load_sum = divider;
> if (se_weight(se)) {
> se->avg.load_sum =
> - div_u64(se->avg.load_avg * se->avg.load_sum,
> se_weight(se));
> + (se->avg.load_avg * se->avg.load_sum >
> se_weight(se)) ?
> + div_u64(se->avg.load_avg * se->avg.load_sum,
> se_weight(se)) : 1;
> }
>
> enqueue_load_avg(cfs_rq, se);
> --
> 2.18.0
In this case, the below is easier to read
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 1658a9428d96..2c685474db23 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -3836,10 +3836,12 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s
se->avg.runnable_sum = se->avg.runnable_avg * divider;
- se->avg.load_sum = divider;
- if (se_weight(se)) {
+ se->avg.load_sum = se->avg.load_avg * divider;
+ if (se_weight(se) < se->avg.load_sum) {
se->avg.load_sum =
- div_u64(se->avg.load_avg * se->avg.load_sum, se_weight(se));
+ div_u64(se->avg.load_sum, se_weight(se));
+ } else {
+ se->avg.load_sum = 1;
}
enqueue_load_avg(cfs_rq, se);
>
>
> > >
> > > After enqueue_load_avg code as below.
> > > cfs_rq->avg.load_avg += se->avg.load_avg;
> > > cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> > >
> > > Then the load_sum for cfs_rq will be 1 while the load_sum for
> > > cfs_rq is 0.
> > > So it will hit the warning message.
> > >
> > > After all, I refer the following commit patch to do the similar
> > > thing at
> > > enqueue_load_avg.
> > > sched/pelt: Relax the sync of load_sum with load_avg
> > >
> > > After long time testing, the kernel warning was gone and the system
> > > runs
> > > as well as before.
> > >
> > > Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> > > ---
> > > kernel/sched/fair.c | 6 ++++--
> > > 1 file changed, 4 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index d4bd299d67ab..30d8b6dba249 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq
> > > *cfs_rq, struct sched_entity *se)
> > > static inline void
> > > enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> > > {
> > > - cfs_rq->avg.load_avg += se->avg.load_avg;
> > > - cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> > > + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> > > + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se-
> > > >avg.load_sum);
> > > + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> > > + cfs_rq->avg.load_avg *
> > > PELT_MIN_DIVIDER);
> > > }
> > >
> > > static inline void
> > > --
> > > 2.18.0
> > >
>
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method
2022-04-12 8:58 ` Vincent Guittot
@ 2022-04-12 9:28 ` Kuyo Chang
0 siblings, 0 replies; 7+ messages in thread
From: Kuyo Chang @ 2022-04-12 9:28 UTC (permalink / raw)
To: Vincent Guittot
Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Dietmar Eggemann,
Steven Rostedt, Ben Segall, Mel Gorman,
Daniel Bristot de Oliveira, Matthias Brugger, wsd_upstream,
linux-kernel, linux-arm-kernel, linux-mediatek
On Tue, 2022-04-12 at 10:58 +0200, Vincent Guittot wrote:
> Le mardi 12 avril 2022 à 10:51:23 (+0800), Kuyo Chang a écrit :
> > On Mon, 2022-04-11 at 10:39 +0200, Vincent Guittot wrote:
> > > On Mon, 11 Apr 2022 at 08:17, Kuyo Chang <kuyo.chang@mediatek.com
> > > >
> > > wrote:
> > > >
> > > > From: kuyo chang <kuyo.chang@mediatek.com>
> > > >
> > > > I meet the warning message at cfs_rq_is_decayed at below code.
> > > >
> > > > SCHED_WARN_ON(cfs_rq->avg.load_avg ||
> > > > cfs_rq->avg.util_avg ||
> > > > cfs_rq->avg.runnable_avg)
> > > >
> > > > Following is the calltrace.
> > > >
> > > > Call trace:
> > > > __update_blocked_fair
> > > > update_blocked_averages
> > > > newidle_balance
> > > > pick_next_task_fair
> > > > __schedule
> > > > schedule
> > > > pipe_read
> > > > vfs_read
> > > > ksys_read
> > > >
> > > > After code analyzing and some debug messages, I found it exits
> > > > a
> > > > corner
> > > > case at attach_entity_load_avg which will cause load_sum is
> > > > zero
> > > > and
> > > > load_avg is not.
> > > > Consider se_weight is 88761 according by sched_prio_to_weight
> > > > table.
> > > > And assume the get_pelt_divider() is 47742, se->avg.load_avg is
> > > > 1.
> > > > By the calculating for se->avg.load_sum as following will
> > > > become
> > > > zero
> > > > as following.
> > > > se->avg.load_sum =
> > > > div_u64(se->avg.load_avg * se->avg.load_sum,
> > > > se_weight(se));
> > > > se->avg.load_sum = 1*47742/88761 = 0.
> > >
> > > The root problem is there, se->avg.load_sum must not be null if
> > > se->avg.load_avg is not null because the correct relation between
> > > _avg
> > > and _sum is:
> > >
> > > load_avg = weight * load_sum / divider.
> > >
> > > so the fix should be attach_entity_load_avg() and probably the
> > > below
> > > is enough
> > >
> > > se->avg.load_sum = div_u64(se->avg.load_avg * se->avg.load_sum,
> > > se_weight(se)) + 1;
> >
> > Thanks for your kindly suggestion.
> > +1 would make the calcuation for load_sum may be overestimate?
> > How about the below code make sense for fix the corner case?
> >
> > ---
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -3832,7 +3832,8 @@ static void attach_entity_load_avg(struct
> > cfs_rq
> > *cfs_rq, struct sched_entity *s
> > se->avg.load_sum = divider;
> > if (se_weight(se)) {
> > se->avg.load_sum =
> > - div_u64(se->avg.load_avg * se->avg.load_sum,
> > se_weight(se));
> > + (se->avg.load_avg * se->avg.load_sum >
> > se_weight(se)) ?
> > + div_u64(se->avg.load_avg * se->avg.load_sum,
> > se_weight(se)) : 1;
> > }
> >
> > enqueue_load_avg(cfs_rq, se);
> > --
> > 2.18.0
>
> In this case, the below is easier to read
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 1658a9428d96..2c685474db23 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3836,10 +3836,12 @@ static void attach_entity_load_avg(struct
> cfs_rq *cfs_rq, struct sched_entity *s
>
> se->avg.runnable_sum = se->avg.runnable_avg * divider;
>
> - se->avg.load_sum = divider;
> - if (se_weight(se)) {
> + se->avg.load_sum = se->avg.load_avg * divider;
> + if (se_weight(se) < se->avg.load_sum) {
> se->avg.load_sum =
> - div_u64(se->avg.load_avg * se->avg.load_sum,
> se_weight(se));
> + div_u64(se->avg.load_sum, se_weight(se));
> + } else {
> + se->avg.load_sum = 1;
> }
>
> enqueue_load_avg(cfs_rq, se);
It really easier to read.
Thanks for your kindly suggestion.
>
>
> >
> >
> > > >
> > > > After enqueue_load_avg code as below.
> > > > cfs_rq->avg.load_avg += se->avg.load_avg;
> > > > cfs_rq->avg.load_sum += se_weight(se) * se->avg.load_sum;
> > > >
> > > > Then the load_sum for cfs_rq will be 1 while the load_sum for
> > > > cfs_rq is 0.
> > > > So it will hit the warning message.
> > > >
> > > > After all, I refer the following commit patch to do the similar
> > > > thing at
> > > > enqueue_load_avg.
> > > > sched/pelt: Relax the sync of load_sum with load_avg
> > > >
> > > > After long time testing, the kernel warning was gone and the
> > > > system
> > > > runs
> > > > as well as before.
> > > >
> > > > Signed-off-by: kuyo chang <kuyo.chang@mediatek.com>
> > > > ---
> > > > kernel/sched/fair.c | 6 ++++--
> > > > 1 file changed, 4 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > > index d4bd299d67ab..30d8b6dba249 100644
> > > > --- a/kernel/sched/fair.c
> > > > +++ b/kernel/sched/fair.c
> > > > @@ -3074,8 +3074,10 @@ account_entity_dequeue(struct cfs_rq
> > > > *cfs_rq, struct sched_entity *se)
> > > > static inline void
> > > > enqueue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity
> > > > *se)
> > > > {
> > > > - cfs_rq->avg.load_avg += se->avg.load_avg;
> > > > - cfs_rq->avg.load_sum += se_weight(se) * se-
> > > > >avg.load_sum;
> > > > + add_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> > > > + add_positive(&cfs_rq->avg.load_sum, se_weight(se) * se-
> > > > > avg.load_sum);
> > > >
> > > > + cfs_rq->avg.load_sum = max_t(u32, cfs_rq->avg.load_sum,
> > > > + cfs_rq->avg.load_avg
> > > > *
> > > > PELT_MIN_DIVIDER);
> > > > }
> > > >
> > > > static inline void
> > > > --
> > > > 2.18.0
> > > >
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-04-12 10:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-11 6:16 [PATCH 1/1] sched/pelt: Refine the enqueue_load_avg calculate method Kuyo Chang
2022-04-11 8:25 ` Peter Zijlstra
2022-04-11 10:14 ` Vincent Guittot
2022-04-11 8:39 ` Vincent Guittot
2022-04-12 2:51 ` Kuyo Chang
2022-04-12 8:58 ` Vincent Guittot
2022-04-12 9:28 ` Kuyo Chang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).