From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758631AbcGKLD1 (ORCPT ); Mon, 11 Jul 2016 07:03:27 -0400 Received: from foss.arm.com ([217.140.101.70]:52184 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758532AbcGKLD0 (ORCPT ); Mon, 11 Jul 2016 07:03:26 -0400 Date: Mon, 11 Jul 2016 12:04:58 +0100 From: Morten Rasmussen To: Peter Zijlstra Cc: mingo@redhat.com, dietmar.eggemann@arm.com, yuyang.du@intel.com, vincent.guittot@linaro.org, mgalbraith@suse.de, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 05/13] sched: Enable SD_BALANCE_WAKE for asymmetric capacity systems Message-ID: <20160711110458.GH12540@e105550-lin.cambridge.arm.com> References: <1466615004-3503-1-git-send-email-morten.rasmussen@arm.com> <1466615004-3503-6-git-send-email-morten.rasmussen@arm.com> <20160711100449.GM30909@twins.programming.kicks-ass.net> <20160711103717.GF12540@e105550-lin.cambridge.arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160711103717.GF12540@e105550-lin.cambridge.arm.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jul 11, 2016 at 11:37:18AM +0100, Morten Rasmussen wrote: > On Mon, Jul 11, 2016 at 12:04:49PM +0200, Peter Zijlstra wrote: > > On Wed, Jun 22, 2016 at 06:03:16PM +0100, Morten Rasmussen wrote: > > > Systems with the SD_ASYM_CPUCAPACITY flag set indicate that sched_groups > > > at this level or below do not include cpus of all capacities available > > > (e.g. group containing little-only or big-only cpus in big.LITTLE > > > systems). It is therefore necessary to put in more effort in finding an > > > appropriate cpu at task wake-up by enabling balancing at wake-up > > > (SD_BALANCE_WAKE). > > > > > --- a/kernel/sched/core.c > > > +++ b/kernel/sched/core.c > > > @@ -6397,6 +6397,9 @@ sd_init(struct sched_domain_topology_level *tl, int cpu) > > > * Convert topological properties into behaviour. > > > */ > > > > > > + if (sd->flags & SD_ASYM_CPUCAPACITY) > > > + sd->flags |= SD_BALANCE_WAKE; > > > + > > > > So I'm a bit confused on the exact requirements for this; as also per > > the previous patch. > > > > Should all sched domains get BALANCE_WAKE if one (typically the top) > > domain has ASYM_CAP set? > > > > The previous patch set it on the actual asym one and one below that, but > > what if there's more levels below that? Imagine ARM gaining SMT or > > somesuch. Should not then that level also get BALANCE_WAKE in order to > > 'correctly' place light/heavy tasks? > > > > IOW, are you trying to fudge the behaviour semantics by creating 'weird' > > ASYM_CAP rules instead of having a more complex behaviour rule here? > > That is one possible way of describing it :-) > > The proposed semantic is to set ASYM_CAP at all levels starting from the > bottom up until you have sched_groups containing all types of cpus > available in the system, or reach the top level. > > The fundamental reason for this weird semantics is that we somehow need > to know at the lower levels, which may be capacity symmetric, if we need > to consider balancing at a higher level to see the asymmetry or not. > > If the flag isn't set bottom up we need some other way of knowing if the > system is asymmetric, or we would have to go look for the flag further > up the sched_domain hierarchy each time. > > I'm not saying this is the perfect solution, I'm happy to discuss > alternatives. One alternative to setting ASYM_CAP bottom up would be to set it only where the asymmetry can be observed, and instead come up with a more complicated way of setting BALANCE_WAKE bottom up until and including the first level having the ASYM_CAP. I looked at it briefly an realized that I couldn't find a clean way of implementing it as I don't think we have visibility of which flags that will be set at higher levels in the sched_domain hierarchy when the lower levels are initialized. IOW, we have behavioural flags settings depend on topology flags settings at a different level.