From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757055AbdDFJXx (ORCPT ); Thu, 6 Apr 2017 05:23:53 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:49717 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757019AbdDFJXf (ORCPT ); Thu, 6 Apr 2017 05:23:35 -0400 Date: Thu, 6 Apr 2017 11:23:29 +0200 From: Peter Zijlstra To: Michal Hocko Cc: Srikar Dronamraju , Ingo Molnar , LKML , Mel Gorman , Rik van Riel Subject: Re: [PATCH] sched: Fix numabalancing to work with isolated cpus Message-ID: <20170406092329.52zzkxlkvitnh44c@hirez.programming.kicks-ass.net> References: <1491326848-5748-1-git-send-email-srikar@linux.vnet.ibm.com> <20170405125743.GB7258@dhcp22.suse.cz> <20170405152215.GA6019@linux.vnet.ibm.com> <20170405164437.GT6035@dhcp22.suse.cz> <20170406071950.GA5843@linux.vnet.ibm.com> <20170406073436.GD5497@dhcp22.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170406073436.GD5497@dhcp22.suse.cz> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Apr 06, 2017 at 09:34:36AM +0200, Michal Hocko wrote: > On Thu 06-04-17 12:49:50, Srikar Dronamraju wrote: > > Similar example that I gave in my reply to Mel. > > > > Lets consider 2 node, 24 core with 12 cores in each node. > > Cores 0-11 in Node 1 and cores 12-23 in the other node. > > Lets also disable smt/hyperthreading, enable isolcpus from core > > 6-11,12-17. Lets run 48 thread ebizzy workload and give it a cpu list > > of say 11,12-17 using taskset. > > > > Now all the 48 ebizzy threads will only run on core 11. It will never > > spread to other cores even in the same node(or in the same node/but > > isolated cpus) or to the different nodes. i.e even if numabalancing is > > running or not, even if my fix is around or not, all threads will be > > confined to core 11, even though the cpus_allowed is 11,12-17. Argh, why such a convoluted example :-( > Isn't that a bug in isolcpus implementation? It is certainly an > unexpected behavior I would say. Is this documented anywhere? Without engaging the brain too much to decipher the example; it does look right. isolcpus will have no balancing. > Isn't sched_setaffinity the only way how to actually make it possible to > run on isolcpus? Think so.. Personally I hate isolcpus and never use it. > I would really like to see it confirmed by the scheduler maintainers and > documented properly as well. What you are claiming here is rather > surprising to my understanding of what isolcpus acutally is. isolcpus gets you a set of fully partitioned CPUs. What's surprising about that?