From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754069AbcEBPCT (ORCPT <rfc822;w@1wt.eu>);
	Mon, 2 May 2016 11:02:19 -0400
Received: from merlin.infradead.org ([205.233.59.134]:51203 "EHLO
	merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751903AbcEBPCH (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Mon, 2 May 2016 11:02:07 -0400
Date: Mon, 2 May 2016 17:01:56 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Mike Galbraith <mgalbraith@suse.de>
Cc: Chris Mason <clm@fb.com>, Ingo Molnar <mingo@kernel.org>,
        Matt Fleming <matt@codeblueprint.co.uk>, linux-kernel@vger.kernel.org
Subject: Re: sched: tweak select_idle_sibling to look for idle threads
Message-ID: <20160502150156.GX3408@twins.programming.kicks-ass.net>
References: <20160405180822.tjtyyc3qh4leflfj@floor.thefacebook.com>
 <20160409190554.honue3gtian2p6vr@floor.thefacebook.com>
 <20160430124731.GE2975@worktop.cust.blueprintrf.com>
 <1462086753.9717.29.camel@suse.de>
 <20160502084615.GB3430@twins.programming.kicks-ass.net>
 <1462200604.3736.42.camel@suse.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1462200604.3736.42.camel@suse.de>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, May 02, 2016 at 04:50:04PM +0200, Mike Galbraith wrote:
> On Mon, 2016-05-02 at 10:46 +0200, Peter Zijlstra wrote:
> 5226         /*
> 5227          * If there are idle cores to be had, go find one.
> 5228          */
> 5229         if (sched_feat(IDLE_CORE) && test_idle_cores(target)) {
> 5230                 i = select_idle_core(p, target);
> 5231                 if ((unsigned)i < nr_cpumask_bits)
> 5232                         return i;
> 5233  
> 5234                 /*
> 5235                  * Failed to find an idle core; stop looking for one.
> 5236                  */
> 5237                 clear_idle_cores(target);
> 5238         }
> 5239 #if 1
> 5240         for_each_cpu(i, cpu_smt_mask(target)) {
> 5241                 if (idle_cpu(i))
> 5242                         return i;
> 5243         }
> 5244  
> 5245         return target;
> 5246 #endif

And yes, I have a variant of that, that does indeed work way better than
scanning the whole LLC domain for idle threads.

If you want a laugh, modify select_idle_core() to remember the last idle
thread it encounters and have it return that when it fails to find an
idle core.. I'm still stumped to explain why it behaves the way it does.