From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754686AbcEBRaS (ORCPT ); Mon, 2 May 2016 13:30:18 -0400 Received: from mx2.suse.de ([195.135.220.15]:53935 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754224AbcEBRaO (ORCPT ); Mon, 2 May 2016 13:30:14 -0400 Message-ID: <1462210213.3621.33.camel@suse.de> Subject: Re: sched: tweak select_idle_sibling to look for idle threads From: Mike Galbraith To: Peter Zijlstra Cc: Chris Mason , Ingo Molnar , Matt Fleming , linux-kernel@vger.kernel.org Date: Mon, 02 May 2016 19:30:13 +0200 In-Reply-To: <20160502145817.GW3408@twins.programming.kicks-ass.net> References: <20160405180822.tjtyyc3qh4leflfj@floor.thefacebook.com> <20160409190554.honue3gtian2p6vr@floor.thefacebook.com> <20160430124731.GE2975@worktop.cust.blueprintrf.com> <1462086753.9717.29.camel@suse.de> <20160502084615.GB3430@twins.programming.kicks-ass.net> <1462200604.3736.42.camel@suse.de> <20160502145817.GW3408@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2016-05-02 at 16:58 +0200, Peter Zijlstra wrote: > On Mon, May 02, 2016 at 04:50:04PM +0200, Mike Galbraith wrote: > > Oh btw, did you know single socket boxen have no sd_busy? That > > doesn't > > look right. > > I suspected; didn't bother looking at yet. The 'problem' is that the > LLC > domain is the top-most, so it doesn't have a parent domain. I'm sure > we > can come up with something if we can get this all working right. > > And yes, I can get gains on various workloads with various options, I > can even break all workloads, but I've so far completely failed on > getting a win for everyone :/ > > In particular low count sysbench-psql (oltp test) vs tbench > client==nr_cores is having me flummoxed for a bit. When I was doing a lot of that, there was always considerable overlap to reclaim with db stuff, and often other players. With loalhost tbench only a wee bit of overlap and two players, it's pretty close to really being the synchronous load it advertises. For tbench (and ilk), the best thing you can do is completely kill select_idle_sibling() once past ramp. Hell, if you watch TCP_RR, pipe-test, whatever with much less overlap, you'll ask yourself why the hell select_idle_sibling() even exists for anything other than the original shared L2 case it was born to serve. Unfortunately, the real world spoiler is that a sync wakeup means jack diddly spit, a waker may very well suddenly be awakened on it's way to sleepy land.. so much for sync wakeup, oh crap, we need to scale. The problem ain't simple, but the scheduler has to be to perform. Catch22. -Mike