From mboxrd@z Thu Jan 1 00:00:00 1970 From: Lorenzo Pieralisi Subject: Re: [PATCH v8 07/26] PM / Domains: Add genpd governor for CPUs Date: Fri, 14 Sep 2018 13:30:37 +0100 Message-ID: <20180914123037.GA22926@e107981-ln.cambridge.arm.com> References: <20180620172226.15012-1-ulf.hansson@linaro.org> <20180809153925.GA20329@red-moon> <5398488.CyAMIAYSYI@aspire.rjw.lan> <20180914104431.GA20567@e107981-ln.cambridge.arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org To: "Rafael J. Wysocki" Cc: "Rafael J. Wysocki" , Ulf Hansson , Sudeep Holla , Mark Rutland , Linux PM , Kevin Hilman , Lina Iyer , Lina Iyer , Rob Herring , Daniel Lezcano , Thomas Gleixner , Vincent Guittot , Stephen Boyd , Juri Lelli , Geert Uytterhoeven , Linux ARM , linux-arm-msm , Linux Kernel Mailing List , Frederic Weisbecker List-Id: linux-arm-msm@vger.kernel.org On Fri, Sep 14, 2018 at 01:34:14PM +0200, Rafael J. Wysocki wrote: [...] > > > So for example, if your logical CPU has an idle state A that may trigger an > > > idle state X at the cluster level (if the other logical CPUs happen to be in > > > the right states and so on), then the worst-case exit latency for that > > > is the one of state X. > > > > I will provide an example: > > > > IDLE STATE A (affects CPU {0,1}): exit latency 1ms, min-residency 1.5ms > > > > CPU 0 is about to enter IDLE state A since its "next-event" fulfill the > > residency requirements and exit latency constraints. > > > > CPU 1 is in idle state A (given that CPU 0 is ON, some of the common > > logic shared between CPU {0,1} is still ON, but, as soon as CPU 0 > > enters idle state A CPU {0,1} can enter the "full" idle state A > > power savings mode). > > > > The current CPUidle governor does not check the "next-event" for CPU 1, > > that it may wake up in, say, 10us. > > Right. > > > Requesting IDLE STATE A is a waste of power (if firmware or hardware > > does not demote it since it does peek at CPU 1 next-event and actually > > demote CPU 0 request). > > OK, I see. > > That's because the state is "collaborative" so to speak. But was't > that supposed to be covered by the "coupled" thing? The coupled idle states code was merged because on some early SMP ARM platforms CPUs must enter cluster idle states orderly otherwise the system would break; "coupled" as-in "syncronized idle state entry". Basically coupled idle code fixed a HW bug. This series code instead applies to all arches where an idle state may span multiple CPUs (x86 inclusive, but as I mentioned it is probably not needed since FW/HW behind mwait is capable of detecting whether that's wortwhile to shut down, say, a package. PSCI, whether OSI or PC mode can work the same way). Entering an idle state spanning multiple cpus need not be synchronized but a sort of cpumask aware governor may help optimize idle state selection. I hope this makes the whole point clearer. Cheers, Lorenzo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.3 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 227A3ECDFD0 for ; Fri, 14 Sep 2018 12:30:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C028720853 for ; Fri, 14 Sep 2018 12:30:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C028720853 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728139AbeINRpJ (ORCPT ); Fri, 14 Sep 2018 13:45:09 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:60746 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726966AbeINRpI (ORCPT ); Fri, 14 Sep 2018 13:45:08 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D30C18A; Fri, 14 Sep 2018 05:30:52 -0700 (PDT) Received: from e107981-ln.cambridge.arm.com (e107981-ln.Emea.Arm.com [10.4.13.117]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C1B303F703; Fri, 14 Sep 2018 05:30:48 -0700 (PDT) Date: Fri, 14 Sep 2018 13:30:37 +0100 From: Lorenzo Pieralisi To: "Rafael J. Wysocki" Cc: "Rafael J. Wysocki" , Ulf Hansson , Sudeep Holla , Mark Rutland , Linux PM , Kevin Hilman , Lina Iyer , Lina Iyer , Rob Herring , Daniel Lezcano , Thomas Gleixner , Vincent Guittot , Stephen Boyd , Juri Lelli , Geert Uytterhoeven , Linux ARM , linux-arm-msm , Linux Kernel Mailing List , Frederic Weisbecker , Ingo Molnar Subject: Re: [PATCH v8 07/26] PM / Domains: Add genpd governor for CPUs Message-ID: <20180914123037.GA22926@e107981-ln.cambridge.arm.com> References: <20180620172226.15012-1-ulf.hansson@linaro.org> <20180809153925.GA20329@red-moon> <5398488.CyAMIAYSYI@aspire.rjw.lan> <20180914104431.GA20567@e107981-ln.cambridge.arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 14, 2018 at 01:34:14PM +0200, Rafael J. Wysocki wrote: [...] > > > So for example, if your logical CPU has an idle state A that may trigger an > > > idle state X at the cluster level (if the other logical CPUs happen to be in > > > the right states and so on), then the worst-case exit latency for that > > > is the one of state X. > > > > I will provide an example: > > > > IDLE STATE A (affects CPU {0,1}): exit latency 1ms, min-residency 1.5ms > > > > CPU 0 is about to enter IDLE state A since its "next-event" fulfill the > > residency requirements and exit latency constraints. > > > > CPU 1 is in idle state A (given that CPU 0 is ON, some of the common > > logic shared between CPU {0,1} is still ON, but, as soon as CPU 0 > > enters idle state A CPU {0,1} can enter the "full" idle state A > > power savings mode). > > > > The current CPUidle governor does not check the "next-event" for CPU 1, > > that it may wake up in, say, 10us. > > Right. > > > Requesting IDLE STATE A is a waste of power (if firmware or hardware > > does not demote it since it does peek at CPU 1 next-event and actually > > demote CPU 0 request). > > OK, I see. > > That's because the state is "collaborative" so to speak. But was't > that supposed to be covered by the "coupled" thing? The coupled idle states code was merged because on some early SMP ARM platforms CPUs must enter cluster idle states orderly otherwise the system would break; "coupled" as-in "syncronized idle state entry". Basically coupled idle code fixed a HW bug. This series code instead applies to all arches where an idle state may span multiple CPUs (x86 inclusive, but as I mentioned it is probably not needed since FW/HW behind mwait is capable of detecting whether that's wortwhile to shut down, say, a package. PSCI, whether OSI or PC mode can work the same way). Entering an idle state spanning multiple cpus need not be synchronized but a sort of cpumask aware governor may help optimize idle state selection. I hope this makes the whole point clearer. Cheers, Lorenzo From mboxrd@z Thu Jan 1 00:00:00 1970 From: lorenzo.pieralisi@arm.com (Lorenzo Pieralisi) Date: Fri, 14 Sep 2018 13:30:37 +0100 Subject: [PATCH v8 07/26] PM / Domains: Add genpd governor for CPUs In-Reply-To: References: <20180620172226.15012-1-ulf.hansson@linaro.org> <20180809153925.GA20329@red-moon> <5398488.CyAMIAYSYI@aspire.rjw.lan> <20180914104431.GA20567@e107981-ln.cambridge.arm.com> Message-ID: <20180914123037.GA22926@e107981-ln.cambridge.arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Fri, Sep 14, 2018 at 01:34:14PM +0200, Rafael J. Wysocki wrote: [...] > > > So for example, if your logical CPU has an idle state A that may trigger an > > > idle state X at the cluster level (if the other logical CPUs happen to be in > > > the right states and so on), then the worst-case exit latency for that > > > is the one of state X. > > > > I will provide an example: > > > > IDLE STATE A (affects CPU {0,1}): exit latency 1ms, min-residency 1.5ms > > > > CPU 0 is about to enter IDLE state A since its "next-event" fulfill the > > residency requirements and exit latency constraints. > > > > CPU 1 is in idle state A (given that CPU 0 is ON, some of the common > > logic shared between CPU {0,1} is still ON, but, as soon as CPU 0 > > enters idle state A CPU {0,1} can enter the "full" idle state A > > power savings mode). > > > > The current CPUidle governor does not check the "next-event" for CPU 1, > > that it may wake up in, say, 10us. > > Right. > > > Requesting IDLE STATE A is a waste of power (if firmware or hardware > > does not demote it since it does peek at CPU 1 next-event and actually > > demote CPU 0 request). > > OK, I see. > > That's because the state is "collaborative" so to speak. But was't > that supposed to be covered by the "coupled" thing? The coupled idle states code was merged because on some early SMP ARM platforms CPUs must enter cluster idle states orderly otherwise the system would break; "coupled" as-in "syncronized idle state entry". Basically coupled idle code fixed a HW bug. This series code instead applies to all arches where an idle state may span multiple CPUs (x86 inclusive, but as I mentioned it is probably not needed since FW/HW behind mwait is capable of detecting whether that's wortwhile to shut down, say, a package. PSCI, whether OSI or PC mode can work the same way). Entering an idle state spanning multiple cpus need not be synchronized but a sort of cpumask aware governor may help optimize idle state selection. I hope this makes the whole point clearer. Cheers, Lorenzo