From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758183AbcEFHZs (ORCPT ); Fri, 6 May 2016 03:25:48 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:35619 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753180AbcEFHZq (ORCPT ); Fri, 6 May 2016 03:25:46 -0400 Date: Fri, 6 May 2016 09:25:41 +0200 From: Peter Zijlstra To: Chris Mason , Mike Galbraith , Ingo Molnar , Matt Fleming , linux-kernel@vger.kernel.org Subject: Re: sched: tweak select_idle_sibling to look for idle threads Message-ID: <20160506072541.GN3448@twins.programming.kicks-ass.net> References: <20160405180822.tjtyyc3qh4leflfj@floor.thefacebook.com> <20160409190554.honue3gtian2p6vr@floor.thefacebook.com> <20160430124731.GE2975@worktop.cust.blueprintrf.com> <1462086753.9717.29.camel@suse.de> <20160502084615.GB3430@twins.programming.kicks-ass.net> <1462200604.3736.42.camel@suse.de> <20160502145817.GW3408@twins.programming.kicks-ass.net> <20160502154725.ckiewczbdubudyc7@floor.masoncoding.com> <20160503143225.GG3448@twins.programming.kicks-ass.net> <20160503151153.wp6jcnjadmw2ypmx@floor.masoncoding.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160503151153.wp6jcnjadmw2ypmx@floor.masoncoding.com> User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, May 03, 2016 at 11:11:53AM -0400, Chris Mason wrote: > # pick a single core, in my case cpus 0,20 are the same core > # cpu_hog is any program that spins > # > taskset -c 20 cpu_hog & > > # schbench -p 4 means message passing mode with 4 byte messages (like > # pipe test), no sleeps, just bouncing as fast as it can. > # > # make the scheduler choose between the sibling of the hog and cpu 1 > # > taskset -c 0,1 schbench -p 4 -m 1 -t 1 > > Current mainline will stuff both schbench threads onto CPU 1, leaving > CPU 0 100% idle. My first patch with the minimal task_hot() checks > would sometimes pick CPU 0. My second patch that just directly calls > task_hot sticks to cpu1, which is ~3x faster than spreading it. Ok, with the thing fixed, my current patch seems to DTRT. If I trace sched_migrate_task() I get: $ grep schbench trace doit-schbench-2-4042 [004] d..3 144541.309747: sched_migrate_task: comm=doit-schbench-2 pid=4042 prio=120 orig_cpu=4 dest_cpu=4 doit-schbench-2-4042 [004] d..2 144541.309772: sched_migrate_task: comm=doit-schbench-2 pid=4043 prio=120 orig_cpu=4 dest_cpu=11 doit-schbench-2-4042 [004] d..3 144541.309855: sched_migrate_task: comm=doit-schbench-2 pid=4042 prio=120 orig_cpu=4 dest_cpu=4 doit-schbench-2-4042 [004] d..2 144541.309882: sched_migrate_task: comm=doit-schbench-2 pid=4044 prio=120 orig_cpu=4 dest_cpu=5 migration/11-77 [011] d..4 144541.309974: sched_migrate_task: comm=doit-schbench-2 pid=4043 prio=120 orig_cpu=11 dest_cpu=12 migration/5-40 [005] d..4 144541.310013: sched_migrate_task: comm=doit-schbench-2 pid=4044 prio=120 orig_cpu=5 dest_cpu=6 schbench-4044 [001] d..3 144541.310995: sched_migrate_task: comm=schbench pid=4044 prio=120 orig_cpu=1 dest_cpu=1 schbench-4044 [001] d..2 144541.310999: sched_migrate_task: comm=schbench pid=4045 prio=120 orig_cpu=1 dest_cpu=1 schbench-4045 [001] d..3 144541.311232: sched_migrate_task: comm=schbench pid=4045 prio=120 orig_cpu=1 dest_cpu=1 schbench-4045 [001] d..2 144541.311234: sched_migrate_task: comm=schbench pid=4046 prio=120 orig_cpu=1 dest_cpu=1 So the thing gets put on cpu1 and never leaves.