From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BEE6C433ED for ; Wed, 19 May 2021 11:09:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F85D6101E for ; Wed, 19 May 2021 11:09:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346103AbhESLKm (ORCPT ); Wed, 19 May 2021 07:10:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240482AbhESLKk (ORCPT ); Wed, 19 May 2021 07:10:40 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2EE4C06175F for ; Wed, 19 May 2021 04:09:20 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id 69so6827021plc.5 for ; Wed, 19 May 2021 04:09:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=e62NCutqVCKY1HZkBPIYfPXud7+Npfa1OW39gmbpZNw=; b=AqRy7PoVVsGWyaOqAExW+UbJ5G2Hg8v5DwW0yaQa4vR2DJ62ZlwKffibyrBx7d6fdF noszNvJZm3l0ad+gbM9y/fbi8ZYxUIS2pJebrXsod087KBCpm/dHj+CHE7b0FpEdy1+d z4qlFFQ4qLxwVvk4TLd9wKaa9bZEnv7oeKsukZyS+5uFkZhb3QabrzUCGEXGkEFcBaLI 2YenGVjj5Y46IJblWrwPHdKCd5Sjbh0BdmEl/XTjMAqFL7p8BJ3G1j+qtXGcs3rkXetg OYYrogimMCxT+Sj4BvaJgp7HTqTEb/MhNU9FfgD7YhHdlX873H7AhdoYIR+9cfYWAfG/ YUkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=e62NCutqVCKY1HZkBPIYfPXud7+Npfa1OW39gmbpZNw=; b=q1WfaFCeMmgA9nmCg5jVgAzFKacO+8GlRTr0dt4474Md4uzTxl3QbJ+8AQV3bPAxwA pIFaNkig5wbYo3wxzxu1z4hDVt6MdcD8rC4JXGJFxo7IKfD1kCuIaOrFGYgPrKM1Mno6 xKp8i4ZRTDS4hpS6Ut+6LC1Q6wNk2rYezotiKpV8Dt1UFVy2lb4WJ7vtmmc9hjnxoxfl HRuVQy7XeOIVIkPmr3JTvNcriWOkQDcigNx5vF9sdJNPt1wrsdDRoUTKz5agNvkDhRd3 zGhHI/hB31dl5b3SmoOYX8JlnuXLkiaH+XFeiZF3/Go40QBWEn/mziNF4cGDqNvLMDdv iFOg== X-Gm-Message-State: AOAM531iL5rah6/B+JiIX6rNpRu/d0fy5Nm1MeyFO8JwD0LkldfZ3zPt 5jpbqGMlJgqfSmqucoSHZBs= X-Google-Smtp-Source: ABdhPJzf5y+PAB13kZ36zpA6LV/PtNoQ5/SsALHEpOBAG8UxoHuBIAaArifwvs7eStDb2yHlpm/PbA== X-Received: by 2002:a17:90a:a08c:: with SMTP id r12mr10540431pjp.204.1621422560590; Wed, 19 May 2021 04:09:20 -0700 (PDT) Received: from localhost (14-201-155-8.tpgi.com.au. [14.201.155.8]) by smtp.gmail.com with ESMTPSA id x13sm990257pfn.43.2021.05.19.04.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 04:09:20 -0700 (PDT) Date: Wed, 19 May 2021 21:09:14 +1000 From: Nicholas Piggin Subject: Re: [PATCH v3 5/6] sched/fair: Consider SMT in ASYM_PACKING load balance To: Peter Zijlstra , Ricardo Neri Cc: Aubrey Li , Aubrey Li , Daniel Bristot de Oliveira , Ben Segall , Dietmar Eggemann , "Joel Fernandes (Google)" , Juri Lelli , Len Brown , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Mel Gorman , Ingo Molnar , Quentin Perret , "Ravi V. Shankar" , Ricardo Neri , Steven Rostedt , Srinivas Pandruvada , Tim Chen , Vincent Guittot References: <20210513154909.6385-1-ricardo.neri-calderon@linux.intel.com> <20210513154909.6385-6-ricardo.neri-calderon@linux.intel.com> <20210515021415.GB14212@ranerica-svr.sc.intel.com> <20210518190740.GA15251@ranerica-svr.sc.intel.com> In-Reply-To: MIME-Version: 1.0 Message-Id: <1621422058.5rx5cxsjqx.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Excerpts from Peter Zijlstra's message of May 19, 2021 7:59 pm: > On Tue, May 18, 2021 at 12:07:40PM -0700, Ricardo Neri wrote: >> On Fri, May 14, 2021 at 07:14:15PM -0700, Ricardo Neri wrote: >> > On Fri, May 14, 2021 at 11:47:45AM +0200, Peter Zijlstra wrote: >=20 >> > > So I'm thinking that this is a property of having ASYM_PACKING at a = core >> > > level, rather than some arch special. Wouldn't something like this b= e >> > > more appropriate? >=20 >> > Thanks Peter for the quick review! This makes sense to me. The only >> > reason we proposed arch_asym_check_smt_siblings() is because we were >> > about breaking powerpc (I need to study how they set priorities for SM= T, >> > if applicable). If you think this is not an issue I can post a >> > v4 with this update. >>=20 >> As far as I can see, priorities in powerpc are set by the CPU number. >> However, I am not sure how CPUs are enumerated? If CPUs in brackets are >> SMT sibling, Does an enumeration looks like A) [0, 1], [2, 3] or B) [0, = 2], >> [1, 3]? I guess B is the right answer. Otherwise, both SMT siblings of a >> core would need to be busy before a new core is used. >>=20 >> Still, I think the issue described in the cover letter may be >> reproducible in powerpc as well. If CPU3 is offlined, and [0, 2] pulled >> tasks from [1, -] so that both CPU0 and CPU2 become busy, CPU1 would not= be >> able to help since CPU0 has the highest priority. >>=20 >> I am cc'ing the linuxppc list to get some feedback. >=20 > IIRC the concern with Power is that their Cores can go faster if the > higher SMT siblings are unused. >=20 > That is, suppose you have an SMT4 Core with only a single active task, > then if only SMT0 is used it can reach max performance, but if the > active sibling is SMT1 it can not reach max performance, and if the only > active sibling is SMT2 it goes slower still. >=20 > So they need to pack the tasks to the lowest SMT siblings, and have the > highest SMT siblings idle (where possible) in order to increase > performance. That's correct. Thanks, Nick From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCE86C433B4 for ; Wed, 19 May 2021 11:09:54 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A15626101E for ; Wed, 19 May 2021 11:09:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A15626101E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4FlVWh06Q0z3bV0 for ; Wed, 19 May 2021 21:09:52 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=AqRy7PoV; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::1034; helo=mail-pj1-x1034.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=AqRy7PoV; dkim-atps=neutral Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4FlVW90lLbz2xgL for ; Wed, 19 May 2021 21:09:23 +1000 (AEST) Received: by mail-pj1-x1034.google.com with SMTP id cu11-20020a17090afa8bb029015d5d5d2175so3294294pjb.3 for ; Wed, 19 May 2021 04:09:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=e62NCutqVCKY1HZkBPIYfPXud7+Npfa1OW39gmbpZNw=; b=AqRy7PoVVsGWyaOqAExW+UbJ5G2Hg8v5DwW0yaQa4vR2DJ62ZlwKffibyrBx7d6fdF noszNvJZm3l0ad+gbM9y/fbi8ZYxUIS2pJebrXsod087KBCpm/dHj+CHE7b0FpEdy1+d z4qlFFQ4qLxwVvk4TLd9wKaa9bZEnv7oeKsukZyS+5uFkZhb3QabrzUCGEXGkEFcBaLI 2YenGVjj5Y46IJblWrwPHdKCd5Sjbh0BdmEl/XTjMAqFL7p8BJ3G1j+qtXGcs3rkXetg OYYrogimMCxT+Sj4BvaJgp7HTqTEb/MhNU9FfgD7YhHdlX873H7AhdoYIR+9cfYWAfG/ YUkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=e62NCutqVCKY1HZkBPIYfPXud7+Npfa1OW39gmbpZNw=; b=AzUYrVIody7pKs/dXkj/F5BQGaYbSOYMPiY3B3XzzSyZY31pNY+ktnul46twFMXEdv YQVucOjwmNTll2Odo+UlCy31qz8kUN76HIt0ukRjHZuXRiycxa4mkLpJOONX1lkHAM7U TcSOsOiUyjKXluFEBc41L0GL0GxAR60gmlZXdJIks4bCa6mVcQGOWXXSNiFzpR+Tth6j M6GCNuDwKmHs9JBR3seXQiyhnPZncMOoG82zkONlQ0EzRWm4CUo6bz63zKk8Xebks1f+ Yln/ZlUmaMhfQ57aBwukQEXy9Xgq3GBUso2aijruEkMABQF5VU3AGA0R538Od6yDzTFE 4ooA== X-Gm-Message-State: AOAM533zwj9Siy4gTC56VXjt/z6bRE7HutzBXdLOuISUDUOkHBuBwUoo hOGhlRqoLtvU++CMk2D9BPI= X-Google-Smtp-Source: ABdhPJzf5y+PAB13kZ36zpA6LV/PtNoQ5/SsALHEpOBAG8UxoHuBIAaArifwvs7eStDb2yHlpm/PbA== X-Received: by 2002:a17:90a:a08c:: with SMTP id r12mr10540431pjp.204.1621422560590; Wed, 19 May 2021 04:09:20 -0700 (PDT) Received: from localhost (14-201-155-8.tpgi.com.au. [14.201.155.8]) by smtp.gmail.com with ESMTPSA id x13sm990257pfn.43.2021.05.19.04.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 May 2021 04:09:20 -0700 (PDT) Date: Wed, 19 May 2021 21:09:14 +1000 From: Nicholas Piggin Subject: Re: [PATCH v3 5/6] sched/fair: Consider SMT in ASYM_PACKING load balance To: Peter Zijlstra , Ricardo Neri References: <20210513154909.6385-1-ricardo.neri-calderon@linux.intel.com> <20210513154909.6385-6-ricardo.neri-calderon@linux.intel.com> <20210515021415.GB14212@ranerica-svr.sc.intel.com> <20210518190740.GA15251@ranerica-svr.sc.intel.com> In-Reply-To: MIME-Version: 1.0 Message-Id: <1621422058.5rx5cxsjqx.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , Len Brown , Ricardo Neri , Vincent Guittot , Tim Chen , "Ravi V. Shankar" , Quentin Perret , linuxppc-dev@lists.ozlabs.org, Aubrey Li , linux-kernel@vger.kernel.org, Steven Rostedt , Ingo Molnar , Ben Segall , Aubrey Li , Srinivas Pandruvada , "Joel Fernandes \(Google\)" , Daniel Bristot de Oliveira , Dietmar Eggemann , Mel Gorman Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Excerpts from Peter Zijlstra's message of May 19, 2021 7:59 pm: > On Tue, May 18, 2021 at 12:07:40PM -0700, Ricardo Neri wrote: >> On Fri, May 14, 2021 at 07:14:15PM -0700, Ricardo Neri wrote: >> > On Fri, May 14, 2021 at 11:47:45AM +0200, Peter Zijlstra wrote: >=20 >> > > So I'm thinking that this is a property of having ASYM_PACKING at a = core >> > > level, rather than some arch special. Wouldn't something like this b= e >> > > more appropriate? >=20 >> > Thanks Peter for the quick review! This makes sense to me. The only >> > reason we proposed arch_asym_check_smt_siblings() is because we were >> > about breaking powerpc (I need to study how they set priorities for SM= T, >> > if applicable). If you think this is not an issue I can post a >> > v4 with this update. >>=20 >> As far as I can see, priorities in powerpc are set by the CPU number. >> However, I am not sure how CPUs are enumerated? If CPUs in brackets are >> SMT sibling, Does an enumeration looks like A) [0, 1], [2, 3] or B) [0, = 2], >> [1, 3]? I guess B is the right answer. Otherwise, both SMT siblings of a >> core would need to be busy before a new core is used. >>=20 >> Still, I think the issue described in the cover letter may be >> reproducible in powerpc as well. If CPU3 is offlined, and [0, 2] pulled >> tasks from [1, -] so that both CPU0 and CPU2 become busy, CPU1 would not= be >> able to help since CPU0 has the highest priority. >>=20 >> I am cc'ing the linuxppc list to get some feedback. >=20 > IIRC the concern with Power is that their Cores can go faster if the > higher SMT siblings are unused. >=20 > That is, suppose you have an SMT4 Core with only a single active task, > then if only SMT0 is used it can reach max performance, but if the > active sibling is SMT1 it can not reach max performance, and if the only > active sibling is SMT2 it goes slower still. >=20 > So they need to pack the tasks to the lowest SMT siblings, and have the > highest SMT siblings idle (where possible) in order to increase > performance. That's correct. Thanks, Nick