From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1752117AbdGaMVx (ORCPT <rfc822;w@1wt.eu>);
        Mon, 31 Jul 2017 08:21:53 -0400
Received: from mail-yw0-f176.google.com ([209.85.161.176]:35669 "EHLO
        mail-yw0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1750977AbdGaMVw (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 31 Jul 2017 08:21:52 -0400
Date: Mon, 31 Jul 2017 12:21:50 +0000
From: Josef Bacik <josef@toxicpanda.com>
To: Joel Fernandes <joelaf@google.com>
Cc: Mike Galbraith <umgwanakikbuti@gmail.com>,
        Josef Bacik <josef@toxicpanda.com>,
        Peter Zijlstra <peterz@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>, Juri Lelli <Juri.Lelli@arm.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Patrick Bellasi <patrick.bellasi@arm.com>,
        Brendan Jackman <brendan.jackman@arm.com>,
        Chris Redpath <Chris.Redpath@arm.com>,
        Michael Wang <wangyun@linux.vnet.ibm.com>,
        Matt Fleming <matt@codeblueprint.co.uk>
Subject: Re: wake_wide mechanism clarification
Message-ID: <20170731122149.GA7539@li70-116.members.linode.com>
References: <CAJWu+oo0_zAWLjObRZp-r3VH5kKKb4aFVnRevtbYh+uH_tCNag@mail.gmail.com>
 <20170630004912.GA2457@destiny>
 <CAJWu+opFTvnGnYyRo9eYZJUxiHdoMCMoadUvnX3p=6ypwWX_Jw@mail.gmail.com>
 <20170630142815.GA9743@destiny>
 <1498842140.15161.66.camel@gmail.com>
 <CAJWu+oo7AY+PYqdwSoW6ToRe2o40MhT1wSCrRU8nkQuH0cFO4w@mail.gmail.com>
 <1501340845.7706.168.camel@gmail.com>
 <CAJWu+ooaqWVTad8JNc4gnCAjkgZj4toCb7RrCKNQfm=x3tD1Gw@mail.gmail.com>
 <CAJWu+ore7pfzMw4dz=y4YMi5YGs_b5DwjA6=GfXFsqqJN10gWQ@mail.gmail.com>
 <CAJWu+ookYPvXrEF=bDW_aui30NV91hMBawy=+GZn_xEiriU0Ag@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJWu+ookYPvXrEF=bDW_aui30NV91hMBawy=+GZn_xEiriU0Ag@mail.gmail.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Jul 29, 2017 at 03:41:56PM -0700, Joel Fernandes wrote:
> On Sat, Jul 29, 2017 at 3:28 PM, Joel Fernandes <joelaf@google.com> wrote:
> <snip>
> >>>> Again I didn't follow why the second condition couldn't just be:
> >>>> waker->nr_wakee_switch > factor, or, (waker->nr_wakee_switch +
> >>>> wakee->nr_wakee_switch) > factor, based on the above explanation from
> >>>> Micheal Wang that I quoted.
> >>>> and why he's instead doing the whole multiplication thing there that I
> >>>> was talking about earlier: "factor * wakee->nr_wakee_switch".
> >>>>
> >>>> Rephrasing my question in another way, why are we talking the ratio of
> >>>> master/slave instead of the sum when comparing if its > factor? I am
> >>>> surely missing something here.
> >>>
> >>> Because the heuristic tries to not demolish 1:1 buddies.  Big partner
> >>> flip delta means the pair are unlikely to be a communicating pair,
> >>> perhaps at high frequency where misses hurt like hell.
> >>
> >> But it does seem to me to demolish the N:N communicating pairs from a
> >> latency/load balancing standpoint. For he case of N readers and N
> >> writers, the ratio (master/slave) comes down to 1:1 and we wake
> >> affine. Hopefully I didn't miss something too obvious about that.
> >
> > I think wake_affine() should correctly handle the case (of
> > overloading) I bring up here where wake_wide() is too conservative and
> > does affine a lot, (I don't have any data for this though, this just
> > from code reading), so I take this comment back for this reason.
> 
> aargh, nope :( it still runs select_idle_sibling although on the
> previous CPU even if want_affine is 0 (and doesn't do the wider
> wakeup..), so the comment still applies.. its easy to get lost into
> the code with so many if statements :-\  sorry about the noise :)
> 

I've been working in this area recently because of a cpu imbalance problem.
Wake_wide() definitely makes it so we're waking affine way too often, but I
think messing with wake_waide to solve that problem is the wrong solution.  This
is just a heuristic to see if we should wake affine, the simpler the better.  I
solved the problem of waking affine too often like this

https://marc.info/?l=linux-kernel&m=150003849602535&w=2

So why do you care about wake_wide() anyway?  Are you observing some problem
that you suspect is affected by the affine wakeup stuff?  Or are you just trying
to understand what is going on for fun?  Cause if you are just doing this for
fun you are a very strange person, thanks,

Josef