From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753517AbdDNQ7P (ORCPT ); Fri, 14 Apr 2017 12:59:15 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:36243 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753294AbdDNQ7M (ORCPT ); Fri, 14 Apr 2017 12:59:12 -0400 Date: Fri, 14 Apr 2017 18:58:57 +0200 From: Peter Zijlstra To: Lauro Ramos Venancio Cc: linux-kernel@vger.kernel.org, lwang@redhat.com, riel@redhat.com, Mike Galbraith , Thomas Gleixner , Ingo Molnar Subject: Re: [RFC 2/3] sched/topology: fix sched groups on NUMA machines with mesh topology Message-ID: <20170414165857.7n75lxk4usfsbjaq@hirez.programming.kicks-ass.net> References: <1492091769-19879-1-git-send-email-lvenanci@redhat.com> <1492091769-19879-3-git-send-email-lvenanci@redhat.com> <20170414113813.vktcpsrsuu2st2fm@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170414113813.vktcpsrsuu2st2fm@hirez.programming.kicks-ass.net> User-Agent: NeoMutt/20170113 (1.7.2) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Apr 14, 2017 at 01:38:13PM +0200, Peter Zijlstra wrote: > On Thu, Apr 13, 2017 at 10:56:08AM -0300, Lauro Ramos Venancio wrote: > > This patch constructs the sched groups from each CPU perspective. So, on > > a 4 nodes machine with ring topology, while nodes 0 and 2 keep the same > > groups as before [(3, 0, 1)(1, 2, 3)], nodes 1 and 3 have new groups > > [(0, 1, 2)(2, 3, 0)]. This allows moving tasks between any node 2-hops > > apart. > > Ah,.. so after drawing pictures I see what went wrong; duh :-( > > An equivalent patch would be (if for_each_cpu_wrap() were exposed): > > @@ -521,11 +588,11 @@ build_overlap_sched_groups(struct sched_domain *sd, int cpu) > struct cpumask *covered = sched_domains_tmpmask; > struct sd_data *sdd = sd->private; > struct sched_domain *sibling; > - int i; > + int i, wrap; > > cpumask_clear(covered); > > - for_each_cpu(i, span) { > + for_each_cpu_wrap(i, span, cpu, wrap) { > struct cpumask *sg_span; > > if (cpumask_test_cpu(i, covered)) > > > We need to start iterating at @cpu, not start at 0 every time. > > OK, please have a look here: https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=sched/core