From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758981AbcDMNsK (ORCPT ); Wed, 13 Apr 2016 09:48:10 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:50830 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754132AbcDMNsI (ORCPT ); Wed, 13 Apr 2016 09:48:08 -0400 Date: Wed, 13 Apr 2016 09:44:30 -0400 From: Chris Mason To: Mike Galbraith CC: Peter Zijlstra , Ingo Molnar , Matt Fleming , Subject: Re: sched: tweak select_idle_sibling to look for idle threads Message-ID: <20160413134430.3s2w4dodocgislpb@floor.thefacebook.com> Mail-Followup-To: Chris Mason , Mike Galbraith , Peter Zijlstra , Ingo Molnar , Matt Fleming , linux-kernel@vger.kernel.org References: <20160409190554.honue3gtian2p6vr@floor.thefacebook.com> <1460282661.4251.44.camel@suse.de> <20160410195543.fp2tpixaafsts5x3@floor.thefacebook.com> <1460350461.3870.36.camel@suse.de> <20160412003044.smr24xzuom3locvo@floor.thefacebook.com> <1460436248.3839.80.camel@suse.de> <20160412132758.7apgqqwl2c2wksy6@floor.thefacebook.com> <1460484977.5617.32.camel@suse.de> <20160412200728.GA41928@clm-mbp.thefacebook.com> <1460517531.3780.29.camel@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <1460517531.3780.29.camel@suse.de> User-Agent: Mutt/1.5.23.1 (2014-03-12) X-Originating-IP: [192.168.52.123] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-04-13_08:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 13, 2016 at 05:18:51AM +0200, Mike Galbraith wrote: > On Tue, 2016-04-12 at 16:07 -0400, Chris Mason wrote: > > > I think that if we're worried about the cost of the idle scan for this > > workload, find_idlest_group() is either going to hurt much more, or not > > search enough CPUs to find the idle one. > > find_idlest_group()? No no no, that's not what I mean at all. > > wake_wide() identifies loads that really want to spread out, thus turns > off affine wakeups. We still call select_idle_sibling(), only > difference being that target is the original cpu, not the waking cpu. Ah ok, I see what you mean now. > Given making that wide connection bidirectional helped FB's load, it > seems reasonable that passing wide information to select_idle_sibling() > would have a good chance of hitting the candidate that stands to gain > from a full socket scan, while also keeping that cache scrambling scan > far away from the rest. > > > But I'm happy to try patches or other ideas, I have a fixed version of > > the bitmap one going through production benchmarks now. [ benchmarks say it needs more fixing, ick ] > > Making that wide/full search cheap is still good, because wake_wide() > also identifies interrupt sources that are waking many, so cheap wide > search should increase utilization there as well. The thought was to > just make the wide thing have a tad wider effect on what it already > does affect.. and hope that doesn't demolish anything. So you're interested in numbers where we pass the wake_wide decision into select_idle_sibling(), and then use that instead of (or in addition to?) my should_scan_idle() function? I agree we may need to tweak wake_wide, since most of our wakeups now are failed affine wakeups. But, the differences are in p99, so I'll probably need to get some better metrics. -chris