From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C31D7C4332F for ; Mon, 19 Dec 2022 08:07:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231365AbiLSIHl (ORCPT ); Mon, 19 Dec 2022 03:07:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48576 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229615AbiLSIHf (ORCPT ); Mon, 19 Dec 2022 03:07:35 -0500 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6235F62DB for ; Mon, 19 Dec 2022 00:07:34 -0800 (PST) Received: by mail-qk1-x735.google.com with SMTP id a25so3377085qkl.12 for ; Mon, 19 Dec 2022 00:07:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=SEgPCMOuM3toHAMRnPEv0nKP/n56PHLERtNl5qSP27Y=; b=GeCE1XpHOPsjmhot1alfGm10UwZR8WweYcK7Ys0ml6q0SKl29iSB4V/YGoLSd4bF1D k/gNkETnjVkPJ3Mu1VpAF8eFG5GiSNeo69v3wu2KtdmT4uHcvtB1BF5MN9dPZFmdphob 1mLvk0O3KpQe0zjxF6uDeaTIgsHmR8OuHDTH08g4AUFwm1wAsHiRLxiWMjfR8b1Dt2bt iyWWQAJD+eRSch1JqaY7QuT/BfK6poPY9PQY9D0PW4C07QFqWjcRmkGZNLBGT1Q6HBOs oyx07AoRsJLtXSd2iqJrF0eHZPK97da/C0T3ywoEpIBiJqNzLY4mwLdMjx05xon0pZLw nANw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SEgPCMOuM3toHAMRnPEv0nKP/n56PHLERtNl5qSP27Y=; b=rGG2XOw/hF34DN1Tn4fl9TORsee7zNTwJrUSocEPGhzZTV9Yb94w2/F9M7hB38OtqZ nR8D8FVCoHQ9TpMBTSB88fo7ceAK9bvH487tm00FO6OM7mxNW6WQPnKifER4zprp62Ee X6mSKySUIvn673iUOQfiVwZUar5V7Fw6GAsjk9noPQAd+eZS+JZmpAjp+U1fQWmX252n QdkUJAZh1KBHucsvJ4zibri+kkYl6xIPPnnoa6MgNAoqAiQuIhGSzniD9bQyqgXx/yLG glIm6Jn27whoilMIuSm+nHUjKwPScKAoj62KiaaXoP/H0iIwYzd+oQi6jX+YHFbToVJt Czdg== X-Gm-Message-State: ANoB5pn0O/7g27km/F1SLg63a5lNEmVsNV6qFqC0mLiD602b6d2+zr3Z Eg8Y5UhcaYjCn2yB/qiLOu1IoNzLhmP66xJre42cUA== X-Google-Smtp-Source: AA0mqf4hYcWqPwq7UNurh/zcWd4YRTG/jTjM6/0NYYnoB+BRyABPbRkLCDZqXLnbNzo8v38MVmQrUBGL+qURaCEBFHY= X-Received: by 2002:a05:620a:8109:b0:6ff:a7f1:ff4e with SMTP id os9-20020a05620a810900b006ffa7f1ff4emr752607qkn.292.1671437253480; Mon, 19 Dec 2022 00:07:33 -0800 (PST) MIME-Version: 1.0 References: <20190719140000.31694-1-juri.lelli@redhat.com> <20190719140000.31694-4-juri.lelli@redhat.com> <20221216233501.gh6m75e7s66dmjgo@airbuntu> In-Reply-To: <20221216233501.gh6m75e7s66dmjgo@airbuntu> From: Vincent Guittot Date: Mon, 19 Dec 2022 09:07:21 +0100 Message-ID: Subject: Re: [PATCH v9 3/8] cpuset: Rebuild root domain deadline accounting information To: Qais Yousef Cc: Juri Lelli , peterz@infradead.org, mingo@redhat.com, rostedt@goodmis.org, tj@kernel.org, linux-kernel@vger.kernel.org, luca.abeni@santannapisa.it, claudio@evidence.eu.com, tommaso.cucinotta@santannapisa.it, bristot@redhat.com, mathieu.poirier@linaro.org, lizefan@huawei.com, longman@redhat.com, dietmar.eggemann@arm.com, cgroups@vger.kernel.org, Wei Wang , Rick Yiu , Quentin Perret Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, 17 Dec 2022 at 00:35, Qais Yousef wrote: > > Hi > > On 07/19/19 15:59, Juri Lelli wrote: > > When the topology of root domains is modified by CPUset or CPUhotplug > > operations information about the current deadline bandwidth held in the > > root domain is lost. > > > > This patch addresses the issue by recalculating the lost deadline > > bandwidth information by circling through the deadline tasks held in > > CPUsets and adding their current load to the root domain they are > > associated with. > > > > Signed-off-by: Mathieu Poirier > > Signed-off-by: Juri Lelli > > --- > > We see that rebuild_root_domain() can take 10+ ms (I get a max of 20ms quite > consistently) on suspend/resume. > > Do we actually need to rebuild_root_domain() if we're going through > a suspend/resume cycle? During suspend to ram, there are cpus hotplug operation but If you use suspend to idle, you will skip cpus hotplug operation and its associated rebuild. > > ie: would something like the below make sense? We'd skip this logic if > cpuhp_tasks_frozen is set which indicates it's not a real hotplug operation but > we're suspending/resuming. > > > Cheers > > -- > Qais Yousef > > > --->8--- > > > From 4cfd50960ad872c5eb810ad3038eaf840bab5182 Mon Sep 17 00:00:00 2001 > From: Qais Yousef > Date: Tue, 29 Nov 2022 19:01:52 +0000 > Subject: [PATCH] sched: cpuset: Don't rebuild sched domains on suspend-resume > > Commit f9a25f776d78 ("cpusets: Rebuild root domain deadline accounting information") > enabled rebuilding sched domain on cpuset and hotplug operations to > correct deadline accounting. > > Rebuilding sched domain is a slow operation and we see 10+ ms delays > on suspend-resume because of that. > > Since nothing is expected to change on suspend-resume operation; skip > rebuilding the sched domains to regain some of the time lost. > > Debugged-by: Rick Yiu > Signed-off-by: Qais Yousef (Google) > --- > kernel/cgroup/cpuset.c | 6 ++++++ > kernel/sched/deadline.c | 3 +++ > 2 files changed, 9 insertions(+) > > diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c > index b474289c15b8..2ff68d625b7b 100644 > --- a/kernel/cgroup/cpuset.c > +++ b/kernel/cgroup/cpuset.c > @@ -1067,6 +1067,9 @@ static void update_tasks_root_domain(struct cpuset *cs) > struct css_task_iter it; > struct task_struct *task; > > + if (cpuhp_tasks_frozen) > + return; > + > css_task_iter_start(&cs->css, 0, &it); > > while ((task = css_task_iter_next(&it))) > @@ -1084,6 +1087,9 @@ static void rebuild_root_domains(void) > lockdep_assert_cpus_held(); > lockdep_assert_held(&sched_domains_mutex); > > + if (cpuhp_tasks_frozen) > + return; > + > rcu_read_lock(); > > /* > diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c > index 0d97d54276cc..42c1143a3956 100644 > --- a/kernel/sched/deadline.c > +++ b/kernel/sched/deadline.c > @@ -2575,6 +2575,9 @@ void dl_clear_root_domain(struct root_domain *rd) > { > unsigned long flags; > > + if (cpuhp_tasks_frozen) > + return; > + > raw_spin_lock_irqsave(&rd->dl_bw.lock, flags); > rd->dl_bw.total_bw = 0; > raw_spin_unlock_irqrestore(&rd->dl_bw.lock, flags); > -- > 2.25.1 >