From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22634C433F5 for ; Tue, 8 Mar 2022 17:51:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p3o/RrlF7lVT5ZSLaMYyuJLmS+247hwwp0eENnpW8EI=; b=UQmH4wvbn2KTyx raBQ0SN3Jqf4LZHbftUqSWoYYbVqoXL0fK2icbumrVcFWyEeeC65YZ+xCx1JTigusZD+5ziCqDK0r QUyMWYwszJ6XFs5M//wmgD+iWYK90imlZHMFIgYj7mTNePyqidOTv7PD3dl251WyuGfGL2Eubf1JW KxNxP2zrS5ovbDIO/8T7Kil+iHHohNzLvrPLch5gSAQco2ZvjClqFrYCXpia6uAPSdL3T+S2EPz3a nsAg4A5dOngFyq98LEJg8VemGUc0IIHAdRv8UtN1SAU1HFtkzl+NTmhVbaJtVJZHb/MD6Kif+e+Kf Bpi6HBn6U1oV93FeZaBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nRdyL-005ehb-2F; Tue, 08 Mar 2022 17:50:09 +0000 Received: from mail-dm6nam10on20726.outbound.protection.outlook.com ([2a01:111:f400:7e88::726] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nRdyD-005eew-Gl for linux-arm-kernel@lists.infradead.org; Tue, 08 Mar 2022 17:50:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m8JIUKIAAtMpwgIApYVmZbz0/JTyNg3yW6XzSdoeC6f/LHPUoCOIlEP9g7KMxxYW3HtvfVh7gdTeR3/OuWkkhvKbUPZS8QurPrIaEw43/0kP8U50hE4Zgf2qjvG10iD06ZxOAAOqprKfXOTaWdJyk7Mn6G0MyQcdMag24P52xDhDR8G3cBUZxaW81gXuxhAGU4PCOzNaJ3zMEHEPKIRJtrJtSGfz8+UQ+QmmvqSjtYWG6QjacPYPhV73vYrlbExRwriDt3OmFDYdvY8jqPttFSRjRY0nwWFtMM/TeK62RlagcA5iU2kBO/CApPaCkCgpNl7RnNMLygoSpf3zIp9YYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=e56urm4SC0V5+Ku0QylJtS5f5ctZQOSjnt/v4QZ/wAw=; b=PqxTrM5on0lOgDfjW2EpfMvXjayAWVSyw3C59fe9tI30jdlvhHAlLlbhBgjTW1eRxHTPIhBp7G554Nc8CP6FbLoClE9AxqLsHvLeDwFCNSdlUbWQL4AfWgoXuDV9P2xtGbB2aX/5WZaAJJCSSNj3nhUTOcM9XMxdjlpZh6xkSNC7t36uQb9UATOCV0IYO9j9g6gWbIn/a4drYlOebXwQFK0kF0dnLUkuKhmVFqoMhHONUCuqgU/FlRp9otl3RFjhkXqsJ31rhAXzvVQ28+1R+IzyPCr4m/CQrRbQsb9SlBjW3Mkz4R3Z9qCqSG4pBUEkKaWlS2zPQ959IM0rLhnFUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=e56urm4SC0V5+Ku0QylJtS5f5ctZQOSjnt/v4QZ/wAw=; b=BDC8YTjxDlvpwJYYBuDwf768g6/VK31bjQ4ZFnqKiOzJH5T9fWBKZHv74lMsZKL006y9PVLIQlZCD893hEnkIHrKYpyKkgz5GQnPc7yU3tLxSGrbr5N/mVRIiFXccY/iPW2gd535jV7twpUyWT7dVR/0RJe5uFd6qQ/TDCYXrJY= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from SA0PR01MB6329.prod.exchangelabs.com (2603:10b6:806:ee::12) by BL0PR01MB4641.prod.exchangelabs.com (2603:10b6:208:7f::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.14; Tue, 8 Mar 2022 17:49:55 +0000 Received: from SA0PR01MB6329.prod.exchangelabs.com ([fe80::f56a:e18f:b6c4:ddb5]) by SA0PR01MB6329.prod.exchangelabs.com ([fe80::f56a:e18f:b6c4:ddb5%7]) with mapi id 15.20.5038.027; Tue, 8 Mar 2022 17:49:55 +0000 Date: Tue, 8 Mar 2022 09:49:52 -0800 From: Darren Hart To: Dietmar Eggemann Cc: Vincent Guittot , Will Deacon , LKML , Linux Arm , Sudeep Holla , Greg Kroah-Hartman , "Rafael J. Wysocki" , Catalin Marinas , Peter Zijlstra , Barry Song , Valentin Schneider , "D . Scott Phillips" , Ilkka Koskinen , stable@vger.kernel.org Subject: Re: [PATCH v3] topology: make core_mask include at least cluster_siblings Message-ID: References: <20220308103012.GA31267@willie-the-truck> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: CH0P220CA0008.NAMP220.PROD.OUTLOOK.COM (2603:10b6:610:ef::6) To SA0PR01MB6329.prod.exchangelabs.com (2603:10b6:806:ee::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3dd5c922-5910-420c-0ac6-08da012c0d96 X-MS-TrafficTypeDiagnostic: BL0PR01MB4641:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VZ3XN7nNqvFXp3s5k8Jf5KvPnJEv+dSH4+uYqBOkyNszy7laJ4h9vTCw0glXZ8vD2myUQJZzG8EhboA1yZFlqSeEp3918gNwyoGbAicirFOf6InaDk9M+M1nqxiD94wJxz/YdqeNbLG3IjsH6jXm1w08veSWV9VU798yMKO+D8B9o9tcjoJuVG9UcsyRdpdkHOHv5H6RFSpoGQ9OJC6+U4AzDNspQVqGdL/kOcGPIo1JEpSqNA/hhPnb6gjuW2Eb31HIzy2nR0HUv8lg5Ks5v8O/jtZSuVE3GpA/aMrDM96NrtR9iNoP818A5NmBmv+UA0z6GSjaHqXzjrkPTc3VtezQf0Q14hr+27dRxOMPVThTQhrvbFjD8CMBDuwsCbpXZ2VHtPPO7w6Rsa3vMNJBLkGkBZi1CgtYT9tCX0NduyL3EvD6ufAOTj9+WZDpWQST6ecF7F11qS3g7EJYoolExPOIoI35cL++wE11c0/TxGgT4NHzHMYp3wSgNRhvhO050BTCf8MzKoJeYl0v4TM4qQjPwC5X+nCOgPKZ+cuAt7Golqe9OZXeR2Iaa0lJ2GV9EjUwEBRRNriJ3/BMnRUfAeBiJzb46j4MQmdz82Qv4DDawNM0Gw0DLJF64kxkKGIbHKecJenlCR6xr/g6HZs78P4tH5/GNa2CrcmNEp9tPZ6wYfqb2xz+SJrmU9bS47/6xbEbihXtT1xMtBiFwrcihaUWjOF1E2y9KAMeL5f+CwY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SA0PR01MB6329.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230001)(7916004)(4636009)(366004)(86362001)(38100700002)(38350700002)(83380400001)(26005)(186003)(66476007)(4326008)(8676002)(66556008)(66946007)(316002)(2906002)(7416002)(8936002)(5660300002)(6666004)(6506007)(53546011)(52116002)(9686003)(54906003)(6512007)(33716001)(508600001)(6486002)(6916009)(67856001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?vQaqx99v5E6H3S1SKS3Mtu6MB9sGRnXnWEju2Nf8MxB7VyTyoJDAcM5bCyGQ?= =?us-ascii?Q?Szo97IabF+6WBKcwHRY4408sb8KQO+27JxEfqW0ykWcUbw730adRDJx0YXqj?= =?us-ascii?Q?gIIWqCzh8QqnmwxG4JZCPjAEGD7cS7w4jykT7hjI3I5gDXn4XUME6oXDi99K?= =?us-ascii?Q?9SSY7ilvnviXn4PxAAzdNawZ547ss7CR0cY2Uk0cbU5OCLw4EVXOuDDzKOp/?= =?us-ascii?Q?roPl1KeOJfHvqo/uOweT03I5Li5P5BJ3PFih/Y+sq7BBQpkEqK08XjPuXUtU?= =?us-ascii?Q?9KaL65Cu3LH6jpI1fCFPMfoPeDDc8eR9p+TesCvGVZwDy6UGKxaFlnL7GZNr?= =?us-ascii?Q?OgNGoYWu7p0CnmqPUGL2Wd5+aBlLwGBaBk/pQiZSxKW1JN1TVEh4g7a0dFz5?= =?us-ascii?Q?VWvGPCGuNbnyipGzBug2nuum4jbTKQGRMdGdNCo/SvJT5qASsoPApAFSguAC?= =?us-ascii?Q?NoQ0DUnfEsz/QQvRam3jJDUUKjHfwEAb+OTgU0qYg618EjXNoUhXbWtNBYKe?= =?us-ascii?Q?IyAdwMPjyfquIJcDxp9AGfItcKdxILvG1BIk+sYdkXSgNHq5+ZCvtPnSSJcW?= =?us-ascii?Q?d2i+uZF6UzwCunQQO61+o8vNkpK8YERf2EwzIDOylOPXZCFVbF/piw17c2Qb?= =?us-ascii?Q?w2eTYxE/pDl0w9QoYcivrFb7tnsFAccimOvT7RhOzNLRsa7go5WreEHPk5x9?= =?us-ascii?Q?ZEpLBT/POiTTsp1hnaCZ/yzvaIs94y10LJBMy4XdCyFekm0xkmUTdcgsRTFn?= =?us-ascii?Q?W+vf3eGGc0FfgdBsrqMW6/MXksRcegx3XAYzbhEUNsOmGR+MYgYqRsvGnlrV?= =?us-ascii?Q?nsbnXKmcXKD15XIu0dY29/LbOGo2w1aj0GXXSVRfHZX6enf3EMRM/rzYZ00X?= =?us-ascii?Q?NQmeKCagTu7Sa3fCTKdOASBIaGBz6V2iJGeoIeHo9+tom7JxUBK/iaeCFYZG?= =?us-ascii?Q?B4j7g6ac/REuM6cEzkZco0ZEQvrQqFuts+Sg9A0DSbI9ddI+8kbx6xhaMlkg?= =?us-ascii?Q?E8QrRV+gnlYNWww/8i4wgQ2xXgt+2syEmvxN7mRGNfGunKOpFdJQmBArxXs8?= =?us-ascii?Q?HTCFzv6M4Eid0IFfa+Yck8f6uoHSzv3HjCrCZSfrgyAJlRhhskmDKkd2tUNe?= =?us-ascii?Q?JYewlJ2mud7R+2egtINhhZWGfooOKtqN8V2w4feoCQpMDjGP/3NNyWTPQCFv?= =?us-ascii?Q?h3WN53ka7zYcVCa38tmCNsDHOI51x6l8hsf2dSlCkc62xvh3dqSArLwc9+E/?= =?us-ascii?Q?WdC7ASj2XlK5GNy+cu1qTUWAeWPCoYxo2xzFyNSgibXbSjT4OlcoyXdLt8fM?= =?us-ascii?Q?scMYa5jZfkKb3oBTXtNiywXnbwmDZapkGXl9RBSZUeOIM6VAk/4r8jSagP+V?= =?us-ascii?Q?nQQ9Jk/DZx0Er1uLfgiT3Z0ra5YHR9uNSF/8HpAc91/uBd6ulIvG3crnSM4f?= =?us-ascii?Q?nhz2lRYqi21oxf/O+wCOSZqOUxSWwbVaQhXqzjV2fcmo2VVA1i4EqmFmbEhk?= =?us-ascii?Q?oA09PQKVe1uLCPptet4oEHeOx9HkPJCG3VEcyOmxQ4Dk3W1ccQ6IwmEtVSk/?= =?us-ascii?Q?SehacPEQnBKidJlzPZU69O4+kLmWiaigOB99wecoRrKYO+F5awZgZQhUpmOR?= =?us-ascii?Q?2+sUKj4NKd5aPl7bMFbSc9nlb6IwUTFHCzDrisa3AgNa4k/TJ8siiItPCrAd?= =?us-ascii?Q?alftv/gVFdF8C1T3z2HZLGTs67SU5koN1CdNT2FbK0iO479y0cJsD/bRFw5B?= =?us-ascii?Q?Z9TXbPRoBA=3D=3D?= X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3dd5c922-5910-420c-0ac6-08da012c0d96 X-MS-Exchange-CrossTenant-AuthSource: SA0PR01MB6329.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2022 17:49:54.8221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: A2m93Z49UguWFm3lO9V9DKMZ4w7Wt0npAS71IKog2mR5EGzouWMq2OU/OqQFVHz52E3WZhxFsuh5a18eaXxl6eac8UW6A21n1iRwlKfXWsWiuFmjwQNjNzrkjyjLX4j9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR01MB4641 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220308_095001_683619_86ADAEEC X-CRM114-Status: GOOD ( 30.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Mar 08, 2022 at 05:03:07PM +0100, Dietmar Eggemann wrote: > On 08/03/2022 12:04, Vincent Guittot wrote: > > On Tue, 8 Mar 2022 at 11:30, Will Deacon wrote: > > [...] > > >>> --- > >>> v1: Drop MC level if coregroup weight == 1 > >>> v2: New sd topo in arch/arm64/kernel/smp.c > >>> v3: No new topo, extend core_mask to cluster_siblings > >>> > >>> drivers/base/arch_topology.c | 8 ++++++++ > >>> 1 file changed, 8 insertions(+) > >>> > >>> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c > >>> index 976154140f0b..a96f45db928b 100644 > >>> --- a/drivers/base/arch_topology.c > >>> +++ b/drivers/base/arch_topology.c > >>> @@ -628,6 +628,14 @@ const struct cpumask *cpu_coregroup_mask(int cpu) > >>> core_mask = &cpu_topology[cpu].llc_sibling; > >>> } > >>> > >>> + /* > >>> + * For systems with no shared cpu-side LLC but with clusters defined, > >>> + * extend core_mask to cluster_siblings. The sched domain builder will > >>> + * then remove MC as redundant with CLS if SCHED_CLUSTER is enabled. > > IMHO, if core_mask weight is 1, MC will be removed/degenerated anyway. > > This is what I get on my Ampere Altra (I guess I don't have the ACPI > changes which would let to a CLS sched domain): > > # cat /sys/kernel/debug/sched/domains/cpu0/domain*/name > DIE > NUMA > root@oss-altra01:~# zcat /proc/config.gz | grep SCHED_CLUSTER > CONFIG_SCHED_CLUSTER=y I'd like to follow up on this. Would you share your dmidecode BIOS Information section? Which kernel version? > >>> + */ > >>> + if (cpumask_subset(core_mask, &cpu_topology[cpu].cluster_sibling)) > >>> + core_mask = &cpu_topology[cpu].cluster_sibling; > >>> + > >> > >> Sudeep, Vincent, are you happy with this now? > > > > I would not say that I'm happy because this solution skews the core > > cpu mask in order to abuse the scheduler so that it will remove a > > wrong but useless level when it will build its domains. > > But this works so as long as the maintainer are happy, I'm fine I did explore the other options and they added considerably more complexity without much benefit in my view. I prefer this option which maintains the cpu_topology as described by the platform, and maps it into something that suits the current scheduler abstraction. I agree there is more work to be done here and intend to continue with it. > I do not have any better idea than this tweak here either in case the > platform can't provide a cleaner setup. I'd argue The platform is describing itself accurately in ACPI PPTT terms. The topology doesn't fit nicely within the kernel abstractions today. This is an area where I hope to continue to improve things going forward. > Maybe the following is easier to read but then we use > '&cpu_topology[cpu].llc_sibling' in cpu_coregroup_mask() already ... > > @@ -617,6 +617,7 @@ EXPORT_SYMBOL_GPL(cpu_topology); > const struct cpumask *cpu_coregroup_mask(int cpu) > { > const cpumask_t *core_mask = cpumask_of_node(cpu_to_node(cpu)); > + const cpumask_t *cluster_mask = cpu_clustergroup_mask(cpu); > > /* Find the smaller of NUMA, core or LLC siblings */ > if (cpumask_subset(&cpu_topology[cpu].core_sibling, core_mask)) { > @@ -628,6 +629,9 @@ const struct cpumask *cpu_coregroup_mask(int cpu) > core_mask = &cpu_topology[cpu].llc_sibling; > } > > + if (cpumask_subset(core_mask, cluster_mask)) > + core_mask = cluster_mask; > + Either works for me. I felt the version I sent was parallel to the existing implementation, but have no preference either way. > return core_mask; > } > > Reviewed-by: Dietmar Eggemann > Thanks for the review Dietmar. -- Darren Hart Ampere Computing / OS and Kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel