From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S934757AbcECSWT (ORCPT <rfc822;w@1wt.eu>);
	Tue, 3 May 2016 14:22:19 -0400
Received: from bombadil.infradead.org ([198.137.202.9]:38965 "EHLO
	bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S934676AbcECSWN (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Tue, 3 May 2016 14:22:13 -0400
Date: Tue, 3 May 2016 20:22:07 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Ingo Molnar <mingo@kernel.org>
Cc: Mike Galbraith <mgalbraith@suse.de>, Chris Mason <clm@fb.com>,
        Matt Fleming <matt@codeblueprint.co.uk>, linux-kernel@vger.kernel.org
Subject: Re: sched: tweak select_idle_sibling to look for idle threads
Message-ID: <20160503182207.GU24771@twins.programming.kicks-ass.net>
References: <20160405180822.tjtyyc3qh4leflfj@floor.thefacebook.com>
 <20160409190554.honue3gtian2p6vr@floor.thefacebook.com>
 <20160430124731.GE2975@worktop.cust.blueprintrf.com>
 <1462086753.9717.29.camel@suse.de>
 <20160502084615.GB3430@twins.programming.kicks-ass.net>
 <1462200604.3736.42.camel@suse.de>
 <20160502150156.GX3408@twins.programming.kicks-ass.net>
 <20160502160403.GA15153@gmail.com>
 <20160503113131.GF3448@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20160503113131.GF3448@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, May 03, 2016 at 01:31:31PM +0200, Peter Zijlstra wrote:
> Then flip on the last_idle tracking in select_idle_core():
> 
> root@ivb-ep:~/bench/sysbench# for i in NO_OLD_IDLE NO_ORDER_IDLE IDLE_CORE NO_FORCE_CORE IDLE IDLE_SMT IDLE_LAST NO_IDLE_FIRST ; do echo $i > /debug/sched_features ; done ; ./doit-psql 30 2 5 10 20 40 80
>   2: [30 secs]     transactions:                        54355  (1811.78 per sec.)
>   5: [30 secs]     transactions:                        122609 (4086.81 per sec.)
>  10: [30 secs]     transactions:                        238738 (7957.66 per sec.)
>  20: [30 secs]     transactions:                        354693 (11822.49 per sec.)
>  40: [30 secs]     transactions:                        421807 (14059.32 per sec.)
>  80: [30 secs]     transactions:                        427088 (14234.25 per sec.)
> 
> 
> And see the top end collapse..

> +	if (sched_feat(IDLE_CORE)) {
> +		if (test_idle_cores(target)) {
> +			i = select_idle_core(p, start);
> +			if ((unsigned)i < nr_cpumask_bits)
> +				return i;
>  
>  			/*
> +			 * Failed to find an idle core; stop looking for one.
>  			 */
> +			clear_idle_cores(target);
> +		}
>  	}

'Obvious' bug there; if we start returning idle threads; we'll not get
to clear_idle_cores() and the thing remains active and badness happens.