From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752597AbdEDK1K (ORCPT ); Thu, 4 May 2017 06:27:10 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:37730 "EHLO mail-wm0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751609AbdEDK1C (ORCPT ); Thu, 4 May 2017 06:27:02 -0400 Date: Thu, 4 May 2017 12:26:57 +0200 From: Vincent Guittot To: Peter Zijlstra Cc: Tejun Heo , Ingo Molnar , linux-kernel@vger.kernel.org, Linus Torvalds , Mike Galbraith , Paul Turner , Chris Mason , kernel-team@fb.com Subject: Re: [PATCH v2 1/2] sched/fair: Fix how load gets propagated from cfs_rq to its sched_entity Message-ID: <20170504102657.GA26960@linaro.org> References: <20170424201344.GA14169@wtj.duckdns.org> <20170424201415.GB14169@wtj.duckdns.org> <20170424213324.GA23619@wtj.duckdns.org> <20170503180028.ejf73et3pc4meqji@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170503180028.ejf73et3pc4meqji@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Le Wednesday 03 May 2017 à 20:00:28 (+0200), Peter Zijlstra a écrit : > [snip] > > Just FUDGE2 on its own seems to be the best on my system and is a change > that makes sense (and something Paul recently pointed out as well). > > The implementation isn't particularly pretty or fast, but should > illustrate the idea. > > Poking at the whole update_tg_cfs_load() thing only makes it worse after > that. And while I agree that that code is mind bending; it seems to work > OK-ish. > > Tejun, Vincent, could you guys have a poke? I have added below patch on to of your: --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -3191,7 +3191,7 @@ update_tg_cfs_load(struct cfs_rq *cfs_rq, struct sched_entity *se) } } } else { - load = calc_cfs_shares(gcfs_rq, prop_type); + load = scale_load_down(calc_cfs_shares(gcfs_rq, prop_type)); } delta = load - se->avg.load_avg; -- The results for each configurations are : ** Config 1** linaro@linaro-developer:~/schbench$cat /sys/module/fair/parameters/prop_type 1 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD NO_FUDGE NO_FUDGE2 Latency percentiles (usec) 50.0000th: 252 75.0000th: 346 90.0000th: 438 95.0000th: 485 *99.0000th: 537 99.5000th: 581 99.9000th: 5768 min=0, max=14202 ** Config 2 ** linaro@linaro-developer:~/schbench$ cat /sys/module/fair/parameters/prop_type 1 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD FUDGE NO_FUDGE2 Latency percentiles (usec) 50.0000th: 261 75.0000th: 374 90.0000th: 457 95.0000th: 490 *99.0000th: 533 99.5000th: 585 99.9000th: 9392 min=0, max=13295 **Config 3** linaro@linaro-developer:~/schbench$ cat /sys/module/fair/parameters/prop_type 1 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD NO_FUDGE FUDGE2 Latency percentiles (usec) 50.0000th: 233 75.0000th: 309 90.0000th: 456 95.0000th: 498 *99.0000th: 5272 99.5000th: 8184 99.9000th: 12368 min=0, max=14865 I have run several time the test, few were correct but most of them were like above **Config 4** linaro@linaro-developer:~/schbench$ cat /sys/module/fair/parameters/prop_type 1 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD FUDGE FUDGE2 Latency percentiles (usec) 50.0000th: 211 75.0000th: 290 90.0000th: 380 95.0000th: 451 *99.0000th: 1778 99.5000th: 5048 99.9000th: 12752 min=0, max=15090 I have run several time the test, few were correct but most of them were like above ** Config 5 ** linaro@linaro-developer:~/schbench$ cat /sys/module/fair/parameters/prop_type 0 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD FUDGE FUDGE2 Latency percentiles (usec) 50.0000th: 216 75.0000th: 297 90.0000th: 430 95.0000th: 487 *99.0000th: 2748 99.5000th: 7432 99.9000th: 12912 min=0, max=15046 I have run several time the test, few were correct but most of them were like above ** Config 6 ** linaro@linaro-developer:~/schbench$ cat /sys/module/fair/parameters/prop_type 0 linaro@linaro-developer:~/schbench$ sudo cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE NO_SIS_AVG_CPU RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD FUDGE NO_FUDGE2 Latency percentiles (usec) 50.0000th: 245 75.0000th: 339 90.0000th: 444 95.0000th: 492 *99.0000th: 3036 99.5000th: 9104 99.9000th: 12496 min=0, max=14098 I have run several time the test, few were correct but most of them were like above Config 1 and 2 are the two configuraton which have stable and good results Regards, Vincent