From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C95C7C4707F for ; Tue, 25 May 2021 09:43:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A545E611C9 for ; Tue, 25 May 2021 09:43:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232670AbhEYJpW (ORCPT ); Tue, 25 May 2021 05:45:22 -0400 Received: from mail.kernel.org ([198.145.29.99]:42472 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230508AbhEYJpV (ORCPT ); Tue, 25 May 2021 05:45:21 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0E39B61409; Tue, 25 May 2021 09:43:48 +0000 (UTC) Date: Tue, 25 May 2021 10:43:46 +0100 From: Catalin Marinas To: Will Deacon Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 18/21] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Message-ID: <20210525094346.GB15564@arm.com> References: <20210518094725.7701-1-will@kernel.org> <20210518094725.7701-19-will@kernel.org> <20210524154657.GE14645@arm.com> <20210524203249.GD15545@willie-the-truck> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210524203249.GD15545@willie-the-truck> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 24, 2021 at 09:32:50PM +0100, Will Deacon wrote: > On Mon, May 24, 2021 at 04:46:58PM +0100, Catalin Marinas wrote: > > On Tue, May 18, 2021 at 10:47:22AM +0100, Will Deacon wrote: > > > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c > > > index 959442f76ed7..72efdc611b14 100644 > > > --- a/arch/arm64/kernel/cpufeature.c > > > +++ b/arch/arm64/kernel/cpufeature.c > > > @@ -2896,15 +2896,33 @@ void __init setup_cpu_features(void) > > > > > > static int enable_mismatched_32bit_el0(unsigned int cpu) > > > { > > > + static int lucky_winner = -1; > > > + > > > struct cpuinfo_arm64 *info = &per_cpu(cpu_data, cpu); > > > bool cpu_32bit = id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0); > > > > > > if (cpu_32bit) { > > > cpumask_set_cpu(cpu, cpu_32bit_el0_mask); > > > static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0); > > > - setup_elf_hwcaps(compat_elf_hwcaps); > > > } > > > > > > + if (cpumask_test_cpu(0, cpu_32bit_el0_mask) == cpu_32bit) > > > + return 0; > > > > I don't fully understand this early return. AFAICT, we still call > > setup_elf_hwcaps() via setup_cpu_features() if the system supports > > 32-bit EL0 (mismatched or not) at boot. For CPU hotplug, we can add the > > compat hwcaps later if we didn't set them up at boot. So this part is > > fine. > > > > However, if CPU0 is 32-bit-capable, it looks like we'd never disable the > > offlining on any of the 32-bit-capable CPUs and there's nothing that > > prevents offlining CPU0. > > That is also deferred until we actually detect the mismatch. For example, if > CPU0 is 32-bit capable but none of the others are, then when we online CPU1 > we will print: > > | CPU features: Asymmetric 32-bit EL0 support detected on CPU 1; CPU hot-unplug disabled on CPU 0 > > so the check above is really asking "Is the CPU being onlined mismatched wrt > the boot CPU?". If yes, then we need to make sure that we're keeping a > 32-bit-capable CPU around. Got it now, the offlining will only be disabled if we detected a mismatch. For this patch: Reviewed-by: Catalin Marinas From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 839DFC2B9F8 for ; Tue, 25 May 2021 09:45:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4633D61409 for ; Tue, 25 May 2021 09:45:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4633D61409 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oobEM/ZEz6Sfu5UqFAWpMBE5jykY9dvoDzslf5pAWrQ=; b=QkFXuHgwjlcH71 UAkhCAKhVttNSMO0PbakEbErWyKvBvQzSqtXgnQ44dW0WZlfd2jICpDJE/9VKOvkrBQ/H2Uv0mvgp YNEvXU55WWmhgoSQaCyv+NiHx0GVjlKcGmcL/8XEOzTYcVRVt16JhWDi2+4lKZZ1MeXXgKc0t6IpB qzXjVvRpIkoQ3tbkvkKSUBuwa9Hku4qZbVRJaxrS6PU6+NA5s/b02Q/8/4LyJAiSY+j8iLqgbvoiW 4EY8FmfRYff2jVjCusOftdkNttZPqgZMsfbjXW+33Aitz6YISwJpeztoYyt9hDahL3lmoamTk4p0b nd5WdkbeHSHxEUSMflCw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1llTbO-004Qzj-VY; Tue, 25 May 2021 09:43:55 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1llTbM-004Qys-6o for linux-arm-kernel@lists.infradead.org; Tue, 25 May 2021 09:43:53 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 0E39B61409; Tue, 25 May 2021 09:43:48 +0000 (UTC) Date: Tue, 25 May 2021 10:43:46 +0100 From: Catalin Marinas To: Will Deacon Cc: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Marc Zyngier , Greg Kroah-Hartman , Peter Zijlstra , Morten Rasmussen , Qais Yousef , Suren Baghdasaryan , Quentin Perret , Tejun Heo , Li Zefan , Johannes Weiner , Ingo Molnar , Juri Lelli , Vincent Guittot , "Rafael J. Wysocki" , kernel-team@android.com Subject: Re: [PATCH v6 18/21] arm64: Prevent offlining first CPU with 32-bit EL0 on mismatched system Message-ID: <20210525094346.GB15564@arm.com> References: <20210518094725.7701-1-will@kernel.org> <20210518094725.7701-19-will@kernel.org> <20210524154657.GE14645@arm.com> <20210524203249.GD15545@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210524203249.GD15545@willie-the-truck> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210525_024352_278306_7AD3C5A6 X-CRM114-Status: GOOD ( 26.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, May 24, 2021 at 09:32:50PM +0100, Will Deacon wrote: > On Mon, May 24, 2021 at 04:46:58PM +0100, Catalin Marinas wrote: > > On Tue, May 18, 2021 at 10:47:22AM +0100, Will Deacon wrote: > > > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c > > > index 959442f76ed7..72efdc611b14 100644 > > > --- a/arch/arm64/kernel/cpufeature.c > > > +++ b/arch/arm64/kernel/cpufeature.c > > > @@ -2896,15 +2896,33 @@ void __init setup_cpu_features(void) > > > > > > static int enable_mismatched_32bit_el0(unsigned int cpu) > > > { > > > + static int lucky_winner = -1; > > > + > > > struct cpuinfo_arm64 *info = &per_cpu(cpu_data, cpu); > > > bool cpu_32bit = id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0); > > > > > > if (cpu_32bit) { > > > cpumask_set_cpu(cpu, cpu_32bit_el0_mask); > > > static_branch_enable_cpuslocked(&arm64_mismatched_32bit_el0); > > > - setup_elf_hwcaps(compat_elf_hwcaps); > > > } > > > > > > + if (cpumask_test_cpu(0, cpu_32bit_el0_mask) == cpu_32bit) > > > + return 0; > > > > I don't fully understand this early return. AFAICT, we still call > > setup_elf_hwcaps() via setup_cpu_features() if the system supports > > 32-bit EL0 (mismatched or not) at boot. For CPU hotplug, we can add the > > compat hwcaps later if we didn't set them up at boot. So this part is > > fine. > > > > However, if CPU0 is 32-bit-capable, it looks like we'd never disable the > > offlining on any of the 32-bit-capable CPUs and there's nothing that > > prevents offlining CPU0. > > That is also deferred until we actually detect the mismatch. For example, if > CPU0 is 32-bit capable but none of the others are, then when we online CPU1 > we will print: > > | CPU features: Asymmetric 32-bit EL0 support detected on CPU 1; CPU hot-unplug disabled on CPU 0 > > so the check above is really asking "Is the CPU being onlined mismatched wrt > the boot CPU?". If yes, then we need to make sure that we're keeping a > 32-bit-capable CPU around. Got it now, the offlining will only be disabled if we detected a mismatch. For this patch: Reviewed-by: Catalin Marinas _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel