From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28EF7C433EF for ; Fri, 1 Jul 2022 08:49:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234921AbiGAItt (ORCPT ); Fri, 1 Jul 2022 04:49:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236329AbiGAItN (ORCPT ); Fri, 1 Jul 2022 04:49:13 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF5EC120BB; Fri, 1 Jul 2022 01:48:59 -0700 (PDT) Date: Thu, 30 Jun 2022 15:10:34 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1656665337; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=txenhuAm1wdt22ieIO4TxKPIWhHKwWofbdvU0fYyNkQ=; b=UwXmHGTQvfXhC9Wf5wvQK5WYD7Z226mnG9Zo8ooNVLOd4FtV3s8h3unEJg7OHRTATjnNvy kt8NwD/Om5ZaPRqw3gCpLFCS66rHd+axfTykb+zOeEyZWwXWG7Y4i4Tsn/Jfz9rtqaIftr ZBF1a/arJwoeE+dW6tXhKA2B8ESbuOgVjY6Xhgc6jFWy8+vDsFrZZoJGH/BKCbOhJE4dRV W4X8TfsXQKePwwwfnsV0s8gzLESUHQwq4qSWeTOhb0DHU4fQi5qgTyHcPtYMV+vXQAD5v2 JkHSIm6Sh/EpQCo3wJY1YRKuKLz9YIjPLuTz5dlQeplr/Tg4icKcFaQ1Vd96FQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1656665337; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=txenhuAm1wdt22ieIO4TxKPIWhHKwWofbdvU0fYyNkQ=; b=JKRmT6QttBQXeVoP0OGEqWy/mZ6Zz9aOKgSKHF46X2AAUTqqCQY3JRMqbCfv2wpvbEmCiO bC/gNR8gOA+3L0Ag== From: "tip-bot2 for Dietmar Eggemann" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: sched/core] sched/fair: Use the same cpumask per-PD throughout find_energy_efficient_cpu() Cc: Dietmar Eggemann , "Peter Zijlstra (Intel)" , Vincent Guittot , Lukasz Luba , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20220621090414.433602-6-vdonnefort@google.com> References: <20220621090414.433602-6-vdonnefort@google.com> MIME-Version: 1.0 Message-ID: <165660183471.15455.10860273694768857101.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The following commit has been merged into the sched/core branch of tip: Commit-ID: 9b340131a4bcf6d0a282a2bdcd8ca268a74da709 Gitweb: https://git.kernel.org/tip/9b340131a4bcf6d0a282a2bdcd8ca268a74da709 Author: Dietmar Eggemann AuthorDate: Tue, 21 Jun 2022 10:04:12 +01:00 Committer: Peter Zijlstra CommitterDate: Tue, 28 Jun 2022 09:17:47 +02:00 sched/fair: Use the same cpumask per-PD throughout find_energy_efficient_cpu() The Perf Domain (PD) cpumask (struct em_perf_domain.cpus) stays invariant after Energy Model creation, i.e. it is not updated after CPU hotplug operations. That's why the PD mask is used in conjunction with the cpu_online_mask (or Sched Domain cpumask). Thereby the cpu_online_mask is fetched multiple times (in compute_energy()) during a run-queue selection for a task. cpu_online_mask may change during this time which can lead to wrong energy calculations. To be able to avoid this, use the select_rq_mask per-cpu cpumask to create a cpumask out of PD cpumask and cpu_online_mask and pass it through the function calls of the EAS run-queue selection path. The PD cpumask for max_spare_cap_cpu/compute_prev_delta selection (find_energy_efficient_cpu()) is now ANDed not only with the SD mask but also with the cpu_online_mask. This is fine since this cpumask has to be in syc with the one used for energy computation (compute_energy()). An exclusive cpuset setup with at least one asymmetric CPU capacity island (hence the additional AND with the SD cpumask) is the obvious exception here. Signed-off-by: Dietmar Eggemann Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Vincent Guittot Tested-by: Lukasz Luba Link: https://lkml.kernel.org/r/20220621090414.433602-6-vdonnefort@google.com --- kernel/sched/fair.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e3f7501..46d6692 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6709,14 +6709,14 @@ static unsigned long cpu_util_without(int cpu, struct task_struct *p) * task. */ static long -compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) +compute_energy(struct task_struct *p, int dst_cpu, struct cpumask *cpus, + struct perf_domain *pd) { - struct cpumask *pd_mask = perf_domain_span(pd); unsigned long max_util = 0, sum_util = 0, cpu_cap; int cpu; - cpu_cap = arch_scale_cpu_capacity(cpumask_first(pd_mask)); - cpu_cap -= arch_scale_thermal_pressure(cpumask_first(pd_mask)); + cpu_cap = arch_scale_cpu_capacity(cpumask_first(cpus)); + cpu_cap -= arch_scale_thermal_pressure(cpumask_first(cpus)); /* * The capacity state of CPUs of the current rd can be driven by CPUs @@ -6727,7 +6727,7 @@ compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) * If an entire pd is outside of the current rd, it will not appear in * its pd list and will not be accounted by compute_energy(). */ - for_each_cpu_and(cpu, pd_mask, cpu_online_mask) { + for_each_cpu(cpu, cpus) { unsigned long util_freq = cpu_util_next(cpu, p, dst_cpu); unsigned long cpu_util, util_running = util_freq; struct task_struct *tsk = NULL; @@ -6814,6 +6814,7 @@ compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) */ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) { + struct cpumask *cpus = this_cpu_cpumask_var_ptr(select_rq_mask); unsigned long prev_delta = ULONG_MAX, best_delta = ULONG_MAX; struct root_domain *rd = cpu_rq(smp_processor_id())->rd; int cpu, best_energy_cpu = prev_cpu, target = -1; @@ -6848,7 +6849,9 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) unsigned long base_energy_pd; int max_spare_cap_cpu = -1; - for_each_cpu_and(cpu, perf_domain_span(pd), sched_domain_span(sd)) { + cpumask_and(cpus, perf_domain_span(pd), cpu_online_mask); + + for_each_cpu_and(cpu, cpus, sched_domain_span(sd)) { if (!cpumask_test_cpu(cpu, p->cpus_ptr)) continue; @@ -6885,12 +6888,12 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) continue; /* Compute the 'base' energy of the pd, without @p */ - base_energy_pd = compute_energy(p, -1, pd); + base_energy_pd = compute_energy(p, -1, cpus, pd); base_energy += base_energy_pd; /* Evaluate the energy impact of using prev_cpu. */ if (compute_prev_delta) { - prev_delta = compute_energy(p, prev_cpu, pd); + prev_delta = compute_energy(p, prev_cpu, cpus, pd); if (prev_delta < base_energy_pd) goto unlock; prev_delta -= base_energy_pd; @@ -6899,7 +6902,8 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) /* Evaluate the energy impact of using max_spare_cap_cpu. */ if (max_spare_cap_cpu >= 0) { - cur_delta = compute_energy(p, max_spare_cap_cpu, pd); + cur_delta = compute_energy(p, max_spare_cap_cpu, cpus, + pd); if (cur_delta < base_energy_pd) goto unlock; cur_delta -= base_energy_pd;