From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753454AbbCaI6h (ORCPT ); Tue, 31 Mar 2015 04:58:37 -0400 Received: from e33.co.us.ibm.com ([32.97.110.151]:54738 "EHLO e33.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751879AbbCaI6d (ORCPT ); Tue, 31 Mar 2015 04:58:33 -0400 Message-ID: <551A61A9.6020009@linux.vnet.ibm.com> Date: Tue, 31 Mar 2015 14:28:17 +0530 From: Preeti U Murthy User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Morten Rasmussen , Peter Zijlstra CC: "mingo@kernel.org" , "riel@redhat.com" , "daniel.lezcano@linaro.org" , "vincent.guittot@linaro.org" , "srikar@linux.vnet.ibm.com" , "pjt@google.com" , "benh@kernel.crashing.org" , "efault@gmx.de" , "linux-kernel@vger.kernel.org" , "iamjoonsoo.kim@lge.com" , "svaidy@linux.vnet.ibm.com" , "tim.c.chen@linux.intel.com" , "jason.low2@hp.com" Subject: Re: [PATCH V2] sched: Improve load balancing in the presence of idle CPUs References: <20150326130014.21532.17158.stgit@preeti.in.ibm.com> <20150327143839.GO18994@e105550-lin.cambridge.arm.com> <55158966.4050300@linux.vnet.ibm.com> <20150327175651.GR18994@e105550-lin.cambridge.arm.com> <20150330110632.GT23123@twins.programming.kicks-ass.net> <20150330120302.GT18994@e105550-lin.cambridge.arm.com> In-Reply-To: <20150330120302.GT18994@e105550-lin.cambridge.arm.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15033108-0009-0000-0000-000009CA879E Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/30/2015 05:33 PM, Morten Rasmussen wrote: > On Mon, Mar 30, 2015 at 12:06:32PM +0100, Peter Zijlstra wrote: >> On Fri, Mar 27, 2015 at 05:56:51PM +0000, Morten Rasmussen wrote: >> >>> I agree that it is hard to predict how many additional cpus you need, >>> but I don't think you necessarily need that information as long as you >>> start by filling up the cpu that was kicked to do the >>> nohz_idle_balance() first. >> >>> Reducing unnecessary wakeups is quite important for energy consumption >>> and something a lot of effort is put into. You really don't want to wake >>> up another cluster/package unnecessarily just because there was only one >>> nohz-idle cpu left in the previous one which could have handled the >>> additional load. It gets even worse if the other cluster is less >>> energy-efficient (big.LITTLE). >> >> So the only way to get tasks to cross your cluster is by balancing that >> domain. At this point we'll compute sg stats for either group >> (=cluster). >> >> The only thing we need to ensure is that it doesn't view the small >> cluster as overloaded (as long as it really isn't of course), as long as >> its not viewed as overloaded it will not pull tasks from it into the big >> cluster, no matter how many ILBs we run before the ILB duty cpu's >> rebalance_domains() call. >> >> I'm really not seeing the problem here. > > I see. The group_classify() should take care of it in all cases of > balancing across clusters. You would be iterating over all cpus in the > other cluster running rebalance_domains() if the balancer cpu happens to > be the last one in the little cluster though. However, within the > cluster (in case you have 2 or more nohz-idle cpus) you still take a > double hit. No? Morten, I am a bit confused about the problem you are pointing to. nohz_idle_balance() does not kick the idle CPUs into action unless there is work to be done. So there are no redundant wakeups. Hence I see no problem here. The ILB CPU is woken up to do the nohz idle balancing, but with this patch, may end up with no work for itself at the end of nohz_idle_balance() and return to sleep. That is one wakeup for merely doing idle load balancing, but this wakeup is needed and worthwhile for spreading the load. So I find no problem here too. I am unable to see the issue. What is it that I am missing ? Regards Preeti U Murthy >