From: K Prateek Nayak <kprateek.nayak@amd.com>
To: Honglei Wang <wanghonglei@didichuxing.com>,
Chen Yu <yu.c.chen@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Vincent Guittot <vincent.guittot@linaro.org>,
Tim Chen <tim.c.chen@intel.com>,
Mel Gorman <mgorman@techsingularity.net>,
Juri Lelli <juri.lelli@redhat.com>,
Rik van Riel <riel@surriel.com>, Aaron Lu <aaron.lu@intel.com>,
Abel Wu <wuyun.abel@bytedance.com>,
Yicong Yang <yangyicong@hisilicon.com>,
"Gautham R . Shenoy" <gautham.shenoy@amd.com>,
Ingo Molnar <mingo@redhat.com>,
Dietmar Eggemann <dietmar.eggemann@arm.com>,
Steven Rostedt <rostedt@goodmis.org>,
Ben Segall <bsegall@google.com>,
Daniel Bristot de Oliveira <bristot@redhat.com>,
Valentin Schneider <vschneid@redhat.com>,
linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH] sched/fair: Choose the CPU where short task is running during wake up
Date: Thu, 29 Sep 2022 23:04:50 +0530 [thread overview]
Message-ID: <2c50baa4-beef-54b9-74fe-1cbf6e8f8dbd@amd.com> (raw)
In-Reply-To: <a6e38a33-0003-d3ea-de9b-cf805aef647f@didichuxing.com>
Hello Honglei,
Thank you for looking into this.
On 9/29/2022 12:29 PM, Honglei Wang wrote:
>
> [..snip..]
>
>>>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>>>> index 914096c5b1ae..7519ab5b911c 100644
>>>> --- a/kernel/sched/fair.c
>>>> +++ b/kernel/sched/fair.c
>>>> @@ -6020,6 +6020,19 @@ static int wake_wide(struct task_struct *p)
>>>> return 1;
>>>> }
>>>> +/*
>>>> + * If a task switches in and then voluntarily relinquishes the
>>>> + * CPU quickly, it is regarded as a short running task.
>>>> + * sysctl_sched_min_granularity is chosen as the threshold,
>>>> + * as this value is the minimal slice if there are too many
>>>> + * runnable tasks, see __sched_period().
>>>> + */
>>>> +static int is_short_task(struct task_struct *p)
>>>> +{
>>>> + return (p->se.sum_exec_runtime <=
>>>> + (p->nvcsw * sysctl_sched_min_granularity));
>>>> +}
>>>> +
>>>> /*
>>>> * The purpose of wake_affine() is to quickly determine on which CPU we can run
>>>> * soonest. For the purpose of speed we only consider the waking and previous
>>>> @@ -6050,7 +6063,8 @@ wake_affine_idle(int this_cpu, int prev_cpu, int sync)
>>>> if (available_idle_cpu(this_cpu) && cpus_share_cache(this_cpu, prev_cpu))
>>>> return available_idle_cpu(prev_cpu) ? prev_cpu : this_cpu;
>>>> - if (sync && cpu_rq(this_cpu)->nr_running == 1)
>>>> + if ((sync && cpu_rq(this_cpu)->nr_running == 1) ||
>>>> + is_short_task(cpu_curr(this_cpu)))
>
> Seems it a bit breaks idle (or will be idle) purpose of wake_affine_idle() here. Maybe we can do it something like this?
>
> if ((sync || is_short_task(cpu_curr(this_cpu))) && cpu_rq(this_cpu)->nr_running == 1)
I believe this will still cause performance degradation on split-LLC
system for Stream like workloads. Based on the logs below, we can
have a situation as follows:
stream-4135 [029] d..2. 353.580957: select_task_rq_fair: wake_affine_idle: Select this_cpu: sync(0) rq->nr_running(1) is_short_task(1)
Where sync is 0 but is_short_task() may return 1 and the
current_rq->nr_running is 1. This will lead to two Stream threads
getting placed on same LLC during wakeup which will cause cache
contention and performance degradation.
>
> Thanks,
> Honglei
>
>>>
>>> This change seems to optimize for affine wakeup which benefits
>>> tasks with producer-consumer pattern but is not ideal for Stream.
>>> Currently the logic ends will do an affine wakeup even if sync
>>> flag is not set:
>>>
>>> stream-4135 [029] d..2. 353.580953: sched_waking: comm=stream pid=4129 prio=120 target_cpu=082
>>> stream-4135 [029] d..2. 353.580957: select_task_rq_fair: wake_affine_idle: Select this_cpu: sync(0) rq->nr_running(1) is_short_task(1)
>>> stream-4135 [029] d..2. 353.580960: sched_migrate_task: comm=stream pid=4129 prio=120 orig_cpu=82 dest_cpu=30
>>> <idle>-0 [030] dNh2. 353.580993: sched_wakeup: comm=stream pid=4129 prio=120 target_cpu=030
This is the exact situation observed during our testing.
>>>
>>> [..snip..]
>>>
--
Thanks and Regards,
Prateek
next prev parent reply other threads:[~2022-09-29 17:35 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-15 16:54 [RFC PATCH] sched/fair: Choose the CPU where short task is running during wake up Chen Yu
2022-09-15 17:10 ` Tim Chen
2022-09-16 10:49 ` Chen Yu
2022-09-16 11:45 ` Peter Zijlstra
2022-09-17 13:55 ` Chen Yu
2022-09-16 11:47 ` Peter Zijlstra
2022-09-17 14:15 ` Chen Yu
2022-09-26 5:50 ` K Prateek Nayak
2022-09-26 14:39 ` Gautham R. Shenoy
2022-09-29 16:58 ` K Prateek Nayak
2022-09-30 17:26 ` Chen Yu
2022-09-29 5:25 ` Chen Yu
2022-09-29 6:59 ` Honglei Wang
2022-09-29 17:34 ` K Prateek Nayak [this message]
2022-09-30 0:58 ` Honglei Wang
2022-09-30 16:03 ` Chen Yu
2022-09-29 17:19 ` K Prateek Nayak
2022-09-29 8:00 ` Vincent Guittot
2022-09-30 16:53 ` Chen Yu
2022-10-03 12:42 ` Vincent Guittot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2c50baa4-beef-54b9-74fe-1cbf6e8f8dbd@amd.com \
--to=kprateek.nayak@amd.com \
--cc=aaron.lu@intel.com \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dietmar.eggemann@arm.com \
--cc=gautham.shenoy@amd.com \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@surriel.com \
--cc=rostedt@goodmis.org \
--cc=tim.c.chen@intel.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=wanghonglei@didichuxing.com \
--cc=wuyun.abel@bytedance.com \
--cc=yangyicong@hisilicon.com \
--cc=yu.c.chen@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).