From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752863AbbE0VWr (ORCPT ); Wed, 27 May 2015 17:22:47 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:1952 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751064AbbE0VWq (ORCPT ); Wed, 27 May 2015 17:22:46 -0400 From: Josef Bacik To: , , , Subject: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE Date: Wed, 27 May 2015 17:22:16 -0400 Message-ID: <1432761736-22093-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 2.1.0 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [192.168.52.123] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151,1.0.33,0.0.0000 definitions=2015-05-27_07:2015-05-27,2015-05-27,1970-01-01 signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [ sorry if you get this twice, it seems like the first submission got lost ] At Facebook we have a pretty heavily multi-threaded application that is sensitive to latency. We have been pulling forward the old SD_WAKE_IDLE code because it gives us a pretty significant performance gain (like 20%). It turns out this is because there are cases where the scheduler puts our task on a busy CPU when there are idle CPU's in the system. We verify this by reading the cpu_delay_req_avg_us from the scheduler netlink stuff. With our crappy patch we get much lower numbers vs baseline. SD_BALANCE_WAKE is supposed to find us an idle cpu to run on, however it is just looking for an idle sibling, preferring affinity over all else. This is not helpful in all cases, and SD_BALANCE_WAKE's job is to find us an idle cpu, not garuntee affinity. Fix this by first trying to find an idle sibling, and then if the cpu is not idle fall through to the logic to find an idle cpu. With this patch we get slightly better performance than with our forward port of SD_WAKE_IDLE. Thanks, Signed-off-by: Josef Bacik Acked-by: Rik van Riel --- kernel/sched/fair.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 241213b..03dafa3 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4766,7 +4766,8 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f if (sd_flag & SD_BALANCE_WAKE) { new_cpu = select_idle_sibling(p, prev_cpu); - goto unlock; + if (idle_cpu(new_cpu)) + goto unlock; } while (sd) { -- 1.8.1