From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E014C1B0D8 for ; Tue, 8 Dec 2020 16:16:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2405A23B01 for ; Tue, 8 Dec 2020 16:16:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730052AbgLHQQF (ORCPT ); Tue, 8 Dec 2020 11:16:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729252AbgLHQQE (ORCPT ); Tue, 8 Dec 2020 11:16:04 -0500 Received: from mail-lj1-x244.google.com (mail-lj1-x244.google.com [IPv6:2a00:1450:4864:20::244]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 18ECBC061793 for ; Tue, 8 Dec 2020 08:15:18 -0800 (PST) Received: by mail-lj1-x244.google.com with SMTP id m13so4211830ljo.11 for ; Tue, 08 Dec 2020 08:15:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fssjOy+xvCiaGsz6X4+ho23ckT+FIMcNGo0Pr9iqsRc=; b=lE3FeoRaLN9AM9uOwBAUfaCJUPsO/mfmAE+fk4/pkTkOKQXJo8ciNVok+0uzUheAab 8fyAuLiblFn1siw2qQcRTJhq4a5EpPm8GJJkkk6hqLTmsamYR5y6257YPw2ofdn9uPA4 6htGOObXqTnRZg+ToE2SdpHWckPjCb2qUlnFgUEgXWt+LE1aV3ECH3uHjSNPFYLHsNEB /QLQg/U+gJRW2T/Ex2NEpQG8Wd5pS/1foxdLewG9cceQy1DZYqcXecdydSf5vweBe/i3 PshElw8kcAONOnEpaFg2IDvi7CbWYA3aGaaUtlZ8ZQfozbRzYZ9DHsRqLDSoN3Vp5E1O ibQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fssjOy+xvCiaGsz6X4+ho23ckT+FIMcNGo0Pr9iqsRc=; b=relssRF0P+9ocoP6gcF0hC6+rbTJlZKoK7otvpoSahe3U3E02q2syPThVvffvyTWSY 88N/HkTEZer6Khtms9YqVmXqMW5R+KrQ9lzNzn+0FGi8VLRXFcfwTuQRO4iedsp+0VMe Kvzs0nLHik84h7R73Am3CSf3F9qPCmZdwZErROx50Cb0FM76LsVvACeKOaOIMQd8Q8hS b7iW8m7YuxyvImexx4RfbX6dpttF6q+kg+YtjVhnz8ZXZ07pSrPbMWPKAHA/KWStkGqH pauxzorBfB8VE3j+auNUUO50cTrUbUU6nOBSlT8m2cqYivxu4+HBEm9jOVb4t3Q64pK3 108Q== X-Gm-Message-State: AOAM532NyWvOOSjHJLxOSySgxPmwnylQ3b/fXI6DHhq0mu3bz+D6yZu3 KUqP1tzBxWtagydAOJGbkm1zgij5mxhI2Sr4/TzaCA== X-Google-Smtp-Source: ABdhPJzX9acF/q0Vl4WXOYY/qadNsEYYoicjAQKDMqq+e3RkNXsqp3+XPl5sKIlOUcEzVoOcdSPzAa/xn+571tJnxTg= X-Received: by 2002:a2e:98da:: with SMTP id s26mr11257780ljj.445.1607444112719; Tue, 08 Dec 2020 08:15:12 -0800 (PST) MIME-Version: 1.0 References: <20201208153501.1467-1-mgorman@techsingularity.net> <20201208153501.1467-5-mgorman@techsingularity.net> In-Reply-To: <20201208153501.1467-5-mgorman@techsingularity.net> From: Vincent Guittot Date: Tue, 8 Dec 2020 17:15:01 +0100 Message-ID: Subject: Re: [PATCH 4/4] sched/fair: Return an idle cpu if one is found after a failed search for an idle core To: Mel Gorman Cc: Peter Ziljstra , Ingo Molnar , LKML , Aubrey Li , Barry Song , Juri Lelli , Valentin Schneider , Linux-ARM Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 8 Dec 2020 at 16:35, Mel Gorman wrote: > > select_idle_core is called when SMT is active and there is likely a free > core available. It may find idle CPUs but this information is simply > discarded and the scan starts over again with select_idle_cpu. > > This patch caches information on idle CPUs found during the search for > a core and uses one if no core is found. This is a tradeoff. There may > be a slight impact when utilisation is low and an idle core can be > found quickly. It provides improvements as the number of busy CPUs > approaches 50% of the domain size when SMT is enabled. > > With tbench on a 2-socket CascadeLake machine, 80 logical CPUs, HT enabled > > 5.10.0-rc6 5.10.0-rc6 > schedstat idlecandidate > Hmean 1 500.06 ( 0.00%) 505.67 * 1.12%* > Hmean 2 975.90 ( 0.00%) 974.06 * -0.19%* > Hmean 4 1902.95 ( 0.00%) 1904.43 * 0.08%* > Hmean 8 3761.73 ( 0.00%) 3721.02 * -1.08%* > Hmean 16 6713.93 ( 0.00%) 6769.17 * 0.82%* > Hmean 32 10435.31 ( 0.00%) 10312.58 * -1.18%* > Hmean 64 12325.51 ( 0.00%) 13792.01 * 11.90%* > Hmean 128 21225.21 ( 0.00%) 20963.44 * -1.23%* > Hmean 256 20532.83 ( 0.00%) 20335.62 * -0.96%* > Hmean 320 20334.81 ( 0.00%) 20147.25 * -0.92%* > > Note that there is a significant corner case. As the SMT scan may be > terminated early, not all CPUs have been visited and select_idle_cpu() > is still called for a full scan. This case is handled in the next > patch. > > Signed-off-by: Mel Gorman Reviewed-by: Vincent Guittot > --- > kernel/sched/fair.c | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 413d895bbbf8..ed6f45832d97 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6066,6 +6066,7 @@ void __update_idle_core(struct rq *rq) > */ > static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int target) > { > + int idle_candidate = -1; > struct cpumask *cpus = this_cpu_cpumask_var_ptr(select_idle_mask); > int core, cpu; > > @@ -6085,6 +6086,11 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int > idle = false; > break; > } > + > + if (idle_candidate == -1 && > + cpumask_test_cpu(cpu, p->cpus_ptr)) { > + idle_candidate = cpu; > + } > } > > if (idle) > @@ -6098,7 +6104,7 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int > */ > set_idle_cores(target, 0); > > - return -1; > + return idle_candidate; > } > > /* > -- > 2.26.2 > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72216C4361B for ; Tue, 8 Dec 2020 16:16:50 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3895920780 for ; Tue, 8 Dec 2020 16:16:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3895920780 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ENxfaEmKCR7lOoCDviPjlsCGzc26/s9nfVyP5qB8w6k=; b=octjRvO99oof6SZSD7DRjxtR1 FrV/j6hgL2uz2QHMDimAsh/YwSvRgHGJmiYkMBuXDB7bLaqkumlqAgWzshCcAwpRI3z6wP7TTa/Gd IOgteMudZqwSC1UUGieO3PF7sdxwJpV0LBmJo1SzxsgGKFV4/Npi89q5GvrOlHJ/y+Zej67g5jPd8 U7aGjgne0ed3HtX+XoggAW+7DRoKA8SrMx4/wHqWrCRqY9dFuG/0fxsZ4vjlII/16ZTDk/4k9ZoJM SXuUuaZe6yhXr8Uko5b/5rsnXrftdsRGVkD3Kpzst/B3BpasonLeyOP6iz0/KY3QPY2V7wLN10zHt 1/9FZ5Yjw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmfe6-0006qs-1P; Tue, 08 Dec 2020 16:15:22 +0000 Received: from mail-lj1-x243.google.com ([2a00:1450:4864:20::243]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kmfdx-0006nb-TO for linux-arm-kernel@lists.infradead.org; Tue, 08 Dec 2020 16:15:15 +0000 Received: by mail-lj1-x243.google.com with SMTP id t22so20399967ljk.0 for ; Tue, 08 Dec 2020 08:15:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fssjOy+xvCiaGsz6X4+ho23ckT+FIMcNGo0Pr9iqsRc=; b=lE3FeoRaLN9AM9uOwBAUfaCJUPsO/mfmAE+fk4/pkTkOKQXJo8ciNVok+0uzUheAab 8fyAuLiblFn1siw2qQcRTJhq4a5EpPm8GJJkkk6hqLTmsamYR5y6257YPw2ofdn9uPA4 6htGOObXqTnRZg+ToE2SdpHWckPjCb2qUlnFgUEgXWt+LE1aV3ECH3uHjSNPFYLHsNEB /QLQg/U+gJRW2T/Ex2NEpQG8Wd5pS/1foxdLewG9cceQy1DZYqcXecdydSf5vweBe/i3 PshElw8kcAONOnEpaFg2IDvi7CbWYA3aGaaUtlZ8ZQfozbRzYZ9DHsRqLDSoN3Vp5E1O ibQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fssjOy+xvCiaGsz6X4+ho23ckT+FIMcNGo0Pr9iqsRc=; b=hM6sVca8ObNaEmyl4L9dsJXTSb8GLp1NEUqUvxz6j4H+2WsMBeNRHjcT49w1KkGHPI /kSh3nV0T0aF9L3FxVcGl29+GcfeS2XJ5Yl4jncn8UjDs0AywF2f+mWkQ1ENKzXckL7f 8TOKc/0jtXHNz2J2Byc2baY+Kit/yMN6833qGvtnZinzIru5a0Di6/Nt4UKFhQxlCf2l LRVYwqnSGGt/OJfkIoK9taPLOgRNUTSdRgfs10p9OEgtpZQh6hZkzPYsklUUEaxQzXBV EI4qHQyijVBDKpXJEf+wHHmg0oocg0QCeyuB6OfSQ4x0l/t0cs8qQLJuD8ybMvFQEtXb 9g0w== X-Gm-Message-State: AOAM530bOSzKPYetRRPH4rn/i3CuPajXeU0Y0kZ1DDs75/4zjWm+MNYK /LRhe1v7ZrW1abn2zHxU6Wj0naYDdPMKb+nlFfURajOkwReSJw== X-Google-Smtp-Source: ABdhPJzX9acF/q0Vl4WXOYY/qadNsEYYoicjAQKDMqq+e3RkNXsqp3+XPl5sKIlOUcEzVoOcdSPzAa/xn+571tJnxTg= X-Received: by 2002:a2e:98da:: with SMTP id s26mr11257780ljj.445.1607444112719; Tue, 08 Dec 2020 08:15:12 -0800 (PST) MIME-Version: 1.0 References: <20201208153501.1467-1-mgorman@techsingularity.net> <20201208153501.1467-5-mgorman@techsingularity.net> In-Reply-To: <20201208153501.1467-5-mgorman@techsingularity.net> From: Vincent Guittot Date: Tue, 8 Dec 2020 17:15:01 +0100 Message-ID: Subject: Re: [PATCH 4/4] sched/fair: Return an idle cpu if one is found after a failed search for an idle core To: Mel Gorman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201208_111514_000965_465A4DD2 X-CRM114-Status: GOOD ( 24.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , Barry Song , Peter Ziljstra , Aubrey Li , LKML , Ingo Molnar , Valentin Schneider , Linux-ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 8 Dec 2020 at 16:35, Mel Gorman wrote: > > select_idle_core is called when SMT is active and there is likely a free > core available. It may find idle CPUs but this information is simply > discarded and the scan starts over again with select_idle_cpu. > > This patch caches information on idle CPUs found during the search for > a core and uses one if no core is found. This is a tradeoff. There may > be a slight impact when utilisation is low and an idle core can be > found quickly. It provides improvements as the number of busy CPUs > approaches 50% of the domain size when SMT is enabled. > > With tbench on a 2-socket CascadeLake machine, 80 logical CPUs, HT enabled > > 5.10.0-rc6 5.10.0-rc6 > schedstat idlecandidate > Hmean 1 500.06 ( 0.00%) 505.67 * 1.12%* > Hmean 2 975.90 ( 0.00%) 974.06 * -0.19%* > Hmean 4 1902.95 ( 0.00%) 1904.43 * 0.08%* > Hmean 8 3761.73 ( 0.00%) 3721.02 * -1.08%* > Hmean 16 6713.93 ( 0.00%) 6769.17 * 0.82%* > Hmean 32 10435.31 ( 0.00%) 10312.58 * -1.18%* > Hmean 64 12325.51 ( 0.00%) 13792.01 * 11.90%* > Hmean 128 21225.21 ( 0.00%) 20963.44 * -1.23%* > Hmean 256 20532.83 ( 0.00%) 20335.62 * -0.96%* > Hmean 320 20334.81 ( 0.00%) 20147.25 * -0.92%* > > Note that there is a significant corner case. As the SMT scan may be > terminated early, not all CPUs have been visited and select_idle_cpu() > is still called for a full scan. This case is handled in the next > patch. > > Signed-off-by: Mel Gorman Reviewed-by: Vincent Guittot > --- > kernel/sched/fair.c | 8 +++++++- > 1 file changed, 7 insertions(+), 1 deletion(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 413d895bbbf8..ed6f45832d97 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6066,6 +6066,7 @@ void __update_idle_core(struct rq *rq) > */ > static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int target) > { > + int idle_candidate = -1; > struct cpumask *cpus = this_cpu_cpumask_var_ptr(select_idle_mask); > int core, cpu; > > @@ -6085,6 +6086,11 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int > idle = false; > break; > } > + > + if (idle_candidate == -1 && > + cpumask_test_cpu(cpu, p->cpus_ptr)) { > + idle_candidate = cpu; > + } > } > > if (idle) > @@ -6098,7 +6104,7 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int > */ > set_idle_cores(target, 0); > > - return -1; > + return idle_candidate; > } > > /* > -- > 2.26.2 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel