From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58DD2C04AB5 for ; Thu, 6 Jun 2019 08:34:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 30C712089E for ; Thu, 6 Jun 2019 08:34:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727318AbfFFIew (ORCPT ); Thu, 6 Jun 2019 04:34:52 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:42744 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725267AbfFFIev (ORCPT ); Thu, 6 Jun 2019 04:34:51 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1827A341; Thu, 6 Jun 2019 01:34:51 -0700 (PDT) Received: from [0.0.0.0] (e107985-lin.cambridge.arm.com [10.1.194.38]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C70A63F690; Thu, 6 Jun 2019 01:34:47 -0700 (PDT) Subject: Re: [PATCH] arm64: dts: sdm845: Add CPU topology To: Vincent Guittot , Quentin Perret Cc: Stephen Boyd , Amit Kucheria , Andy Gross , Matthias Kaehlcke , David Brown , Rob Herring , Mark Rutland , linux-arm-msm , "open list:ARM/QUALCOMM SUPPORT" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , Douglas Anderson , Rajendra Nayak , Morten Rasmussen References: <20190114184255.258318-1-mka@chromium.org> <155786856719.14659.2902538189660269078@swboyd.mtv.corp.google.com> <5cdf2dc8.1c69fb81.521c8.9339@mx.google.com> <20190605172048.ahzusevvdxrpnebk@queper01-ThinkPad-T460s> <20190606074921.43mbinemk3j565yu@queper01-ThinkPad-T460s> From: Dietmar Eggemann Message-ID: <9267b9ed-89b0-7b71-88a2-ca1894d4c497@arm.com> Date: Thu, 6 Jun 2019 10:34:45 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org On 6/6/19 10:20 AM, Vincent Guittot wrote: > On Thu, 6 Jun 2019 at 09:49, Quentin Perret wrote: >> >> Hi Vincent, >> >> On Thursday 06 Jun 2019 at 09:05:16 (+0200), Vincent Guittot wrote: >>> Hi Quentin, >>> >>> On Wed, 5 Jun 2019 at 19:21, Quentin Perret wrote: >>>> >>>> On Friday 17 May 2019 at 14:55:19 (-0700), Stephen Boyd wrote: >>>>> Quoting Amit Kucheria (2019-05-16 04:54:45) >>>>>> (cc'ing Andy's correct email address) >>>>>> >>>>>> On Wed, May 15, 2019 at 2:46 AM Stephen Boyd wrote: >>>>>>> >>>>>>> Quoting Amit Kucheria (2019-05-13 04:54:12) >>>>>>>> On Mon, May 13, 2019 at 4:31 PM Amit Kucheria wrote: >>>>>>>>> >>>>>>>>> On Tue, Jan 15, 2019 at 12:13 AM Matthias Kaehlcke wrote: >>>>>>>>>> >>>>>>>>>> The 8 CPU cores of the SDM845 are organized in two clusters of 4 big >>>>>>>>>> ("gold") and 4 little ("silver") cores. Add a cpu-map node to the DT >>>>>>>>>> that describes this topology. >>>>>>>>> >>>>>>>>> This is partly true. There are two groups of gold and silver cores, >>>>>>>>> but AFAICT they are in a single cluster, not two separate ones. SDM845 >>>>>>>>> is one of the early examples of ARM's Dynamiq architecture. >>>>>>>>> >>>>>>>>>> Signed-off-by: Matthias Kaehlcke >>>>>>>>> >>>>>>>>> I noticed that this patch sneaked through for this merge window but >>>>>>>>> perhaps we can whip up a quick fix for -rc2? >>>>>>>>> >>>>>>>> >>>>>>>> And please find attached a patch to fix this up. Andy, since this >>>>>>>> hasn't landed yet (can we still squash this into the original patch?), >>>>>>>> I couldn't add a Fixes tag. >>>>>>>> >>>>>>> >>>>>>> I had the same concern. Thanks for catching this. I suspect this must >>>>>>> cause some problem for IPA given that it can't discern between the big >>>>>>> and little "power clusters"? >>>>>> >>>>>> Both EAS and IPA, I believe. It influences the scheduler's view of the >>>>>> the topology. >>>>> >>>>> And EAS and IPA are OK with the real topology? I'm just curious if >>>>> changing the topology to reflect reality will be a problem for those >>>>> two. >>>> >>>> FWIW, neither EAS nor IPA depends on this. Not the upstream version of >>>> EAS at least (which is used in recent Android kernels -- 4.19+). >>>> >>>> But doing this is still required for other things in the scheduler (the >>>> so-called 'capacity-awareness' code). So until we have a better >>>> solution, this patch is doing the right thing. >>> >>> I'm not sure to catch what you mean ? >>> Which so-called 'capacity-awareness' code are you speaking about ? and >>> what is the problem ? >> >> I'm talking about the wake-up path. ATM select_idle_sibling() is totally >> unaware of capacity differences. In its current form, this function >> basically assumes that all CPUs in a given sd_llc have the same >> capacity, which would be wrong if we had a single MC level for SDM845. >> So, until select_idle_sibling() is 'fixed' to be capacity-aware, we need >> two levels of sd for asymetric systems (including DynamIQ) so the >> wake_cap() story actually works. >> >> I hope that clarifies it :) > > hmm... does this justifies this wrong topology ? > select_idle_sibling() is called only when system is overloaded and > scheduler disables the EAS path > In this case, the scheduler looks either for an idle cpu or for evenly > spreading the loads > This is maybe not always optimal and should probably be fixed but > doesn't justifies a wrong topology description IMHO The big/Little cluster detection in wake_cap() doesn't work anymore with DynamIQ w/o Phanton (DIE) domain. So the decision of going sis() or slow path is IMHO broken. But I support the idea of not introducing Phantom Domains in device tree and fix wake_cap() instead. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dietmar Eggemann Subject: Re: [PATCH] arm64: dts: sdm845: Add CPU topology Date: Thu, 6 Jun 2019 10:34:45 +0200 Message-ID: <9267b9ed-89b0-7b71-88a2-ca1894d4c497@arm.com> References: <20190114184255.258318-1-mka@chromium.org> <155786856719.14659.2902538189660269078@swboyd.mtv.corp.google.com> <5cdf2dc8.1c69fb81.521c8.9339@mx.google.com> <20190605172048.ahzusevvdxrpnebk@queper01-ThinkPad-T460s> <20190606074921.43mbinemk3j565yu@queper01-ThinkPad-T460s> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-GB Sender: linux-kernel-owner@vger.kernel.org To: Vincent Guittot , Quentin Perret Cc: Stephen Boyd , Amit Kucheria , Andy Gross , Matthias Kaehlcke , David Brown , Rob Herring , Mark Rutland , linux-arm-msm , "open list:ARM/QUALCOMM SUPPORT" , "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , LKML , Douglas Anderson , Rajendra Nayak , Morten Rasmussen List-Id: devicetree@vger.kernel.org On 6/6/19 10:20 AM, Vincent Guittot wrote: > On Thu, 6 Jun 2019 at 09:49, Quentin Perret wrote: >> >> Hi Vincent, >> >> On Thursday 06 Jun 2019 at 09:05:16 (+0200), Vincent Guittot wrote: >>> Hi Quentin, >>> >>> On Wed, 5 Jun 2019 at 19:21, Quentin Perret wrote: >>>> >>>> On Friday 17 May 2019 at 14:55:19 (-0700), Stephen Boyd wrote: >>>>> Quoting Amit Kucheria (2019-05-16 04:54:45) >>>>>> (cc'ing Andy's correct email address) >>>>>> >>>>>> On Wed, May 15, 2019 at 2:46 AM Stephen Boyd wrote: >>>>>>> >>>>>>> Quoting Amit Kucheria (2019-05-13 04:54:12) >>>>>>>> On Mon, May 13, 2019 at 4:31 PM Amit Kucheria wrote: >>>>>>>>> >>>>>>>>> On Tue, Jan 15, 2019 at 12:13 AM Matthias Kaehlcke wrote: >>>>>>>>>> >>>>>>>>>> The 8 CPU cores of the SDM845 are organized in two clusters of 4 big >>>>>>>>>> ("gold") and 4 little ("silver") cores. Add a cpu-map node to the DT >>>>>>>>>> that describes this topology. >>>>>>>>> >>>>>>>>> This is partly true. There are two groups of gold and silver cores, >>>>>>>>> but AFAICT they are in a single cluster, not two separate ones. SDM845 >>>>>>>>> is one of the early examples of ARM's Dynamiq architecture. >>>>>>>>> >>>>>>>>>> Signed-off-by: Matthias Kaehlcke >>>>>>>>> >>>>>>>>> I noticed that this patch sneaked through for this merge window but >>>>>>>>> perhaps we can whip up a quick fix for -rc2? >>>>>>>>> >>>>>>>> >>>>>>>> And please find attached a patch to fix this up. Andy, since this >>>>>>>> hasn't landed yet (can we still squash this into the original patch?), >>>>>>>> I couldn't add a Fixes tag. >>>>>>>> >>>>>>> >>>>>>> I had the same concern. Thanks for catching this. I suspect this must >>>>>>> cause some problem for IPA given that it can't discern between the big >>>>>>> and little "power clusters"? >>>>>> >>>>>> Both EAS and IPA, I believe. It influences the scheduler's view of the >>>>>> the topology. >>>>> >>>>> And EAS and IPA are OK with the real topology? I'm just curious if >>>>> changing the topology to reflect reality will be a problem for those >>>>> two. >>>> >>>> FWIW, neither EAS nor IPA depends on this. Not the upstream version of >>>> EAS at least (which is used in recent Android kernels -- 4.19+). >>>> >>>> But doing this is still required for other things in the scheduler (the >>>> so-called 'capacity-awareness' code). So until we have a better >>>> solution, this patch is doing the right thing. >>> >>> I'm not sure to catch what you mean ? >>> Which so-called 'capacity-awareness' code are you speaking about ? and >>> what is the problem ? >> >> I'm talking about the wake-up path. ATM select_idle_sibling() is totally >> unaware of capacity differences. In its current form, this function >> basically assumes that all CPUs in a given sd_llc have the same >> capacity, which would be wrong if we had a single MC level for SDM845. >> So, until select_idle_sibling() is 'fixed' to be capacity-aware, we need >> two levels of sd for asymetric systems (including DynamIQ) so the >> wake_cap() story actually works. >> >> I hope that clarifies it :) > > hmm... does this justifies this wrong topology ? > select_idle_sibling() is called only when system is overloaded and > scheduler disables the EAS path > In this case, the scheduler looks either for an idle cpu or for evenly > spreading the loads > This is maybe not always optimal and should probably be fixed but > doesn't justifies a wrong topology description IMHO The big/Little cluster detection in wake_cap() doesn't work anymore with DynamIQ w/o Phanton (DIE) domain. So the decision of going sis() or slow path is IMHO broken. But I support the idea of not introducing Phantom Domains in device tree and fix wake_cap() instead.