From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650BBC433E0 for ; Fri, 8 Jan 2021 15:53:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C0FB23884 for ; Fri, 8 Jan 2021 15:53:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727859AbhAHPwq (ORCPT ); Fri, 8 Jan 2021 10:52:46 -0500 Received: from foss.arm.com ([217.140.110.172]:53368 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727182AbhAHPwp (ORCPT ); Fri, 8 Jan 2021 10:52:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA60E31B; Fri, 8 Jan 2021 07:51:59 -0800 (PST) Received: from localhost (unknown [10.1.198.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6D0283F70D; Fri, 8 Jan 2021 07:51:59 -0800 (PST) Date: Fri, 8 Jan 2021 15:51:58 +0000 From: Ionela Voinescu To: Viresh Kumar Cc: Catalin Marinas , Will Deacon , Vincent Guittot , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH V4 3/3] arm64: topology: Make AMUs work with modular cpufreq drivers Message-ID: <20210108155157.GA19903@arm.com> References: <89c1921334443e133c9c8791b4693607d65ed9f5.1610104461.git.viresh.kumar@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <89c1921334443e133c9c8791b4693607d65ed9f5.1610104461.git.viresh.kumar@linaro.org> User-Agent: Mutt/1.9.4 (2018-02-28) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Friday 08 Jan 2021 at 16:46:53 (+0530), Viresh Kumar wrote: > The AMU counters won't get used today if the cpufreq driver is built as > a module as the amu core requires everything to be ready by late init. > > Fix that properly by registering for cpufreq policy notifier. Note that > the amu core don't have any cpufreq dependency after the first time > CPUFREQ_CREATE_POLICY notifier is called for all the CPUs. And so we > don't need to do anything on the CPUFREQ_REMOVE_POLICY notifier. And for > the same reason we check if the CPUs are already parsed in the beginning > of amu_fie_setup() and skip if that is true. Alternatively we can shoot > a work from there to unregister the notifier instead, but that seemed > too much instead of this simple check. > > While at it, convert the print message to pr_debug instead of pr_info. > > Signed-off-by: Viresh Kumar > > --- > arch/arm64/kernel/topology.c | 92 +++++++++++++++++++----------------- > 1 file changed, 48 insertions(+), 44 deletions(-) > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c > index 57267d694495..e08a4126453a 100644 > --- a/arch/arm64/kernel/topology.c > +++ b/arch/arm64/kernel/topology.c > @@ -199,69 +199,38 @@ static int freq_inv_set_max_ratio(int cpu, u64 max_rate, u64 ref_rate) > return 0; > } > > -static inline void > -enable_policy_freq_counters(int cpu, cpumask_var_t valid_cpus) > -{ > - struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); > - > - if (!policy) { > - pr_debug("CPU%d: No cpufreq policy found.\n", cpu); > - return; > - } > - > - if (cpumask_subset(policy->related_cpus, valid_cpus)) > - cpumask_or(amu_fie_cpus, policy->related_cpus, > - amu_fie_cpus); > - > - cpufreq_cpu_put(policy); > -} > - > static DEFINE_STATIC_KEY_FALSE(amu_fie_key); > #define amu_freq_invariant() static_branch_unlikely(&amu_fie_key) > > -static int __init init_amu_fie(void) > +static void amu_fie_setup(const struct cpumask *cpus) > { > - cpumask_var_t valid_cpus; > bool invariant; > - int ret = 0; > int cpu; > > - if (!zalloc_cpumask_var(&valid_cpus, GFP_KERNEL)) > - return -ENOMEM; > - > - if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) { > - ret = -ENOMEM; > - goto free_valid_mask; > - } > + /* We are already set since the last insmod of cpufreq driver */ > + if (unlikely(cpumask_subset(cpus, amu_fie_cpus))) > + return; > > - for_each_present_cpu(cpu) { > + for_each_cpu(cpu, cpus) { > if (!freq_counters_valid(cpu) || > freq_inv_set_max_ratio(cpu, > cpufreq_get_hw_max_freq(cpu) * 1000, > arch_timer_get_rate())) > - continue; > - > - cpumask_set_cpu(cpu, valid_cpus); > - enable_policy_freq_counters(cpu, valid_cpus); > + return; > } > > - /* Overwrite amu_fie_cpus if all CPUs support AMU */ > - if (cpumask_equal(valid_cpus, cpu_present_mask)) > - cpumask_copy(amu_fie_cpus, cpu_present_mask); > - > - if (cpumask_empty(amu_fie_cpus)) > - goto free_valid_mask; > + cpumask_or(amu_fie_cpus, amu_fie_cpus, cpus); > > invariant = topology_scale_freq_invariant(); > > /* We aren't fully invariant yet */ > if (!invariant && !cpumask_equal(amu_fie_cpus, cpu_present_mask)) > - goto free_valid_mask; > + return; > > static_branch_enable(&amu_fie_key); > > - pr_info("CPUs[%*pbl]: counters will be used for FIE.", > - cpumask_pr_args(amu_fie_cpus)); > + pr_debug("CPUs[%*pbl]: counters will be used for FIE.", > + cpumask_pr_args(cpus)); > > /* > * Task scheduler behavior depends on frequency invariance support, > @@ -271,13 +240,48 @@ static int __init init_amu_fie(void) > */ > if (!invariant) > rebuild_sched_domains_energy(); > +} > + > +static int init_amu_fie_callback(struct notifier_block *nb, unsigned long val, > + void *data) > +{ > + struct cpufreq_policy *policy = data; > + > + if (val == CPUFREQ_CREATE_POLICY) > + amu_fie_setup(policy->related_cpus); > + > + /* > + * We don't need to handle CPUFREQ_REMOVE_POLICY event as the AMU > + * counters don't have any dependency on cpufreq driver once we have > + * initialized AMU support and enabled invariance. The AMU counters will > + * keep on working just fine in the absence of the cpufreq driver, and > + * for the CPUs for which there are no counters available, the last set > + * value of freq_scale will remain valid as that is the frequency those > + * CPUs are running at. > + */ > + > + return 0; > +} > + > +static struct notifier_block init_amu_fie_notifier = { > + .notifier_call = init_amu_fie_callback, > +}; > + > +static int __init init_amu_fie(void) > +{ > + int ret; > + > + if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) > + return -ENOMEM; > > -free_valid_mask: > - free_cpumask_var(valid_cpus); > + ret = cpufreq_register_notifier(&init_amu_fie_notifier, > + CPUFREQ_POLICY_NOTIFIER); > + if (ret) > + free_cpumask_var(amu_fie_cpus); > > return ret; > } > -late_initcall_sync(init_amu_fie); > +core_initcall(init_amu_fie); > > bool arch_freq_counters_available(const struct cpumask *cpus) > { > -- > 2.25.0.rc1.19.g042ed3e048af > Reviewed-by: Ionela Voinescu Thanks, Ionela. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86C52C433DB for ; Fri, 8 Jan 2021 15:54:14 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 37EFE23884 for ; Fri, 8 Jan 2021 15:54:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37EFE23884 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ivjJZRThGgmkYqC0VY+CAG/Y1R3/xBhADDTq6z7nbzw=; b=i6WH0Uekec8iZADG1i4zS+2QY 0TZtwPSLZZSBWq2j7138HvH6hmvLIwUd1Fhhu1Uy268mpvMNvbfMzngNP09z1YNW7J15sfIy97tvp NrsGzNDSy20HIueNDyEZ3Rs1G8BApaogPpCzR/+NCGPx16LbA2S9d0eXRJgInkJibbIKZ4RW9IWOC kUiJUgVhUiipTc7QGDGUAPExZbjGh1ubRjO68JUO0lLNAl05XuT4RFi76Wp0PMVD0VVhxYEC/ToYm U7x7xqG0IPoOHqlCOrf+fGOogYFUo+Zx1NWRYXFZlit5j/AdFfdNS2+bfUs66IGxU4mc+Sj6STQPj LX1EHnr3g==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxu3s-0007QG-Ne; Fri, 08 Jan 2021 15:52:24 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxu3V-0007Pk-4w for linux-arm-kernel@lists.infradead.org; Fri, 08 Jan 2021 15:52:02 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA60E31B; Fri, 8 Jan 2021 07:51:59 -0800 (PST) Received: from localhost (unknown [10.1.198.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6D0283F70D; Fri, 8 Jan 2021 07:51:59 -0800 (PST) Date: Fri, 8 Jan 2021 15:51:58 +0000 From: Ionela Voinescu To: Viresh Kumar Subject: Re: [PATCH V4 3/3] arm64: topology: Make AMUs work with modular cpufreq drivers Message-ID: <20210108155157.GA19903@arm.com> References: <89c1921334443e133c9c8791b4693607d65ed9f5.1610104461.git.viresh.kumar@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <89c1921334443e133c9c8791b4693607d65ed9f5.1610104461.git.viresh.kumar@linaro.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210108_105201_328766_85E91288 X-CRM114-Status: GOOD ( 33.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Will Deacon , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Vincent Guittot Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Friday 08 Jan 2021 at 16:46:53 (+0530), Viresh Kumar wrote: > The AMU counters won't get used today if the cpufreq driver is built as > a module as the amu core requires everything to be ready by late init. > > Fix that properly by registering for cpufreq policy notifier. Note that > the amu core don't have any cpufreq dependency after the first time > CPUFREQ_CREATE_POLICY notifier is called for all the CPUs. And so we > don't need to do anything on the CPUFREQ_REMOVE_POLICY notifier. And for > the same reason we check if the CPUs are already parsed in the beginning > of amu_fie_setup() and skip if that is true. Alternatively we can shoot > a work from there to unregister the notifier instead, but that seemed > too much instead of this simple check. > > While at it, convert the print message to pr_debug instead of pr_info. > > Signed-off-by: Viresh Kumar > > --- > arch/arm64/kernel/topology.c | 92 +++++++++++++++++++----------------- > 1 file changed, 48 insertions(+), 44 deletions(-) > > diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c > index 57267d694495..e08a4126453a 100644 > --- a/arch/arm64/kernel/topology.c > +++ b/arch/arm64/kernel/topology.c > @@ -199,69 +199,38 @@ static int freq_inv_set_max_ratio(int cpu, u64 max_rate, u64 ref_rate) > return 0; > } > > -static inline void > -enable_policy_freq_counters(int cpu, cpumask_var_t valid_cpus) > -{ > - struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); > - > - if (!policy) { > - pr_debug("CPU%d: No cpufreq policy found.\n", cpu); > - return; > - } > - > - if (cpumask_subset(policy->related_cpus, valid_cpus)) > - cpumask_or(amu_fie_cpus, policy->related_cpus, > - amu_fie_cpus); > - > - cpufreq_cpu_put(policy); > -} > - > static DEFINE_STATIC_KEY_FALSE(amu_fie_key); > #define amu_freq_invariant() static_branch_unlikely(&amu_fie_key) > > -static int __init init_amu_fie(void) > +static void amu_fie_setup(const struct cpumask *cpus) > { > - cpumask_var_t valid_cpus; > bool invariant; > - int ret = 0; > int cpu; > > - if (!zalloc_cpumask_var(&valid_cpus, GFP_KERNEL)) > - return -ENOMEM; > - > - if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) { > - ret = -ENOMEM; > - goto free_valid_mask; > - } > + /* We are already set since the last insmod of cpufreq driver */ > + if (unlikely(cpumask_subset(cpus, amu_fie_cpus))) > + return; > > - for_each_present_cpu(cpu) { > + for_each_cpu(cpu, cpus) { > if (!freq_counters_valid(cpu) || > freq_inv_set_max_ratio(cpu, > cpufreq_get_hw_max_freq(cpu) * 1000, > arch_timer_get_rate())) > - continue; > - > - cpumask_set_cpu(cpu, valid_cpus); > - enable_policy_freq_counters(cpu, valid_cpus); > + return; > } > > - /* Overwrite amu_fie_cpus if all CPUs support AMU */ > - if (cpumask_equal(valid_cpus, cpu_present_mask)) > - cpumask_copy(amu_fie_cpus, cpu_present_mask); > - > - if (cpumask_empty(amu_fie_cpus)) > - goto free_valid_mask; > + cpumask_or(amu_fie_cpus, amu_fie_cpus, cpus); > > invariant = topology_scale_freq_invariant(); > > /* We aren't fully invariant yet */ > if (!invariant && !cpumask_equal(amu_fie_cpus, cpu_present_mask)) > - goto free_valid_mask; > + return; > > static_branch_enable(&amu_fie_key); > > - pr_info("CPUs[%*pbl]: counters will be used for FIE.", > - cpumask_pr_args(amu_fie_cpus)); > + pr_debug("CPUs[%*pbl]: counters will be used for FIE.", > + cpumask_pr_args(cpus)); > > /* > * Task scheduler behavior depends on frequency invariance support, > @@ -271,13 +240,48 @@ static int __init init_amu_fie(void) > */ > if (!invariant) > rebuild_sched_domains_energy(); > +} > + > +static int init_amu_fie_callback(struct notifier_block *nb, unsigned long val, > + void *data) > +{ > + struct cpufreq_policy *policy = data; > + > + if (val == CPUFREQ_CREATE_POLICY) > + amu_fie_setup(policy->related_cpus); > + > + /* > + * We don't need to handle CPUFREQ_REMOVE_POLICY event as the AMU > + * counters don't have any dependency on cpufreq driver once we have > + * initialized AMU support and enabled invariance. The AMU counters will > + * keep on working just fine in the absence of the cpufreq driver, and > + * for the CPUs for which there are no counters available, the last set > + * value of freq_scale will remain valid as that is the frequency those > + * CPUs are running at. > + */ > + > + return 0; > +} > + > +static struct notifier_block init_amu_fie_notifier = { > + .notifier_call = init_amu_fie_callback, > +}; > + > +static int __init init_amu_fie(void) > +{ > + int ret; > + > + if (!zalloc_cpumask_var(&amu_fie_cpus, GFP_KERNEL)) > + return -ENOMEM; > > -free_valid_mask: > - free_cpumask_var(valid_cpus); > + ret = cpufreq_register_notifier(&init_amu_fie_notifier, > + CPUFREQ_POLICY_NOTIFIER); > + if (ret) > + free_cpumask_var(amu_fie_cpus); > > return ret; > } > -late_initcall_sync(init_amu_fie); > +core_initcall(init_amu_fie); > > bool arch_freq_counters_available(const struct cpumask *cpus) > { > -- > 2.25.0.rc1.19.g042ed3e048af > Reviewed-by: Ionela Voinescu Thanks, Ionela. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel