From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7A61C433E0 for ; Mon, 1 Feb 2021 12:48:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7EED464E9E for ; Mon, 1 Feb 2021 12:48:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231526AbhBAMs2 (ORCPT ); Mon, 1 Feb 2021 07:48:28 -0500 Received: from foss.arm.com ([217.140.110.172]:59038 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231477AbhBAMsR (ORCPT ); Mon, 1 Feb 2021 07:48:17 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A1E1ED1; Mon, 1 Feb 2021 04:47:31 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.41.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DE4193F66E; Mon, 1 Feb 2021 04:47:28 -0800 (PST) Date: Mon, 1 Feb 2021 12:47:20 +0000 From: Mark Rutland To: Giancarlo Ferrari Cc: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, rppt@kernel.org, penberg@kernel.org, geert@linux-m68k.org, giancarlo.ferrari@nokia.com Subject: Re: [PATCH] ARM: kexec: Fix panic after TLB are invalidated Message-ID: <20210201124720.GA66060@C02TD0UTHF1T.local> References: <1612140296-12546-1-git-send-email-giancarlo.ferrari89@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1612140296-12546-1-git-send-email-giancarlo.ferrari89@gmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Feb 01, 2021 at 12:44:56AM +0000, Giancarlo Ferrari wrote: > machine_kexec() need to set rw permission in text and rodata sections > to assign some variables (e.g. kexec_start_address). To do that at > the end (after flushing pdm in memory, etc.) it needs to invalidate > TLB [section] entries. It'd be worth noting explicitly that set_kernel_text_rw() alters current->active_mm... > If during the TLB invalidation an interrupt occours, which might cause > a context switch, there is the risk to inject invalid TLBs, with ro > permissions. ... which is why if there's a context switch things can go wrong, since active_mm isn't stable, and so it's possible that set_kernel_text_rw() updates multiple tables, none of which might be the active table at the point we try to make an access. It would be nice to spell that out rather than saying "invalid TLBs". We could disable preemption to prevent that, which is possibly better than disabling interrupts. Overall, it would be much better to avoid having to mess with the kernel page tables. So rather than going: 1. mark kernel RW 2. alter variables in reloc code 3. copy reloc code into buffer 4. branch to buffer ... we should be able to go: 1. copy reloc code into buffer 2. alter variables in copy of reloc code 3. branch to buffer ... which would avoid this class of problem too. Thanks, Mark. > When trying to assign .text labels, this lead to the following: > > Unable to handle kernel paging request at virtual address 80112f38 > pgd = fd7ef03e > [80112f38] *pgd=0001141e(bad) > Internal error: Oops: 80d [#1] PREEMPT SMP ARM > ... > > Signed-off-by: Giancarlo Ferrari > --- > arch/arm/kernel/machine_kexec.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/arch/arm/kernel/machine_kexec.c b/arch/arm/kernel/machine_kexec.c > index 5d84ad3..23e8816 100644 > --- a/arch/arm/kernel/machine_kexec.c > +++ b/arch/arm/kernel/machine_kexec.c > @@ -174,6 +174,13 @@ void machine_kexec(struct kimage *image) > > reboot_code_buffer = page_address(image->control_code_page); > > + /* > + * If below part is not atomic TLB entries might be corrupted after TLB > + * invalidation, which leads to Data Abort in .text variable assignment > + */ > + raw_local_irq_disable(); > + local_fiq_disable(); > + > /* Prepare parameters for reboot_code_buffer*/ > set_kernel_text_rw(); > kexec_start_address = image->start; > @@ -181,6 +188,9 @@ void machine_kexec(struct kimage *image) > kexec_mach_type = machine_arch_type; > kexec_boot_atags = image->arch.kernel_r2; > > + local_fiq_enable(); > + raw_local_irq_enable(); > + > /* copy our kernel relocation code to the control code page */ > reboot_entry = fncpy(reboot_code_buffer, > &relocate_new_kernel, > -- > 2.7.4 > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39663C433DB for ; Mon, 1 Feb 2021 12:49:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A691B64D9E for ; Mon, 1 Feb 2021 12:49:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A691B64D9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zU/aXihgs9mYiISUfIFSP68i6akTcchX9RH2qR82mZs=; b=iwdq8Aim3Trz3OjzahUr2s0Qe aHX7ima2UUUwQKYrJ12URIC50AsZe9VcyiX/ongflIz7JsHqHJvRXzewZrQZU1AtbP1Zk3py+OQ90 /g+FnhzFgsWqh7pU2rrj1Bgy+hxlYIUaapPbJDZ5BVgirGiQdrshH++ZugmNYwzNRbpIa+IUCsn2f IB4URU8JxWW2pDKyNwE8umRvVAuc3WT2FsXYjcsf9GNVbmnIGDPgGVwnGZ2NV56pwinE5f6+MRlUL 4ocOrYRC1UU57YxzfXCotJs9ReAxauAhSb1kHahcMaIUzXu4g4XXqARkh3areNvbaVPn9DV5IzbHN esMiAC2pA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6YcC-0004BB-DN; Mon, 01 Feb 2021 12:47:36 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6Yc9-0004Ai-7W for linux-arm-kernel@lists.infradead.org; Mon, 01 Feb 2021 12:47:34 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A1E1ED1; Mon, 1 Feb 2021 04:47:31 -0800 (PST) Received: from C02TD0UTHF1T.local (unknown [10.57.41.104]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DE4193F66E; Mon, 1 Feb 2021 04:47:28 -0800 (PST) Date: Mon, 1 Feb 2021 12:47:20 +0000 From: Mark Rutland To: Giancarlo Ferrari Subject: Re: [PATCH] ARM: kexec: Fix panic after TLB are invalidated Message-ID: <20210201124720.GA66060@C02TD0UTHF1T.local> References: <1612140296-12546-1-git-send-email-giancarlo.ferrari89@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1612140296-12546-1-git-send-email-giancarlo.ferrari89@gmail.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210201_074733_356109_6F8F3348 X-CRM114-Status: GOOD ( 26.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux@armlinux.org.uk, linux-kernel@vger.kernel.org, penberg@kernel.org, geert@linux-m68k.org, linux-arm-kernel@lists.infradead.org, akpm@linux-foundation.org, rppt@kernel.org, giancarlo.ferrari@nokia.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Feb 01, 2021 at 12:44:56AM +0000, Giancarlo Ferrari wrote: > machine_kexec() need to set rw permission in text and rodata sections > to assign some variables (e.g. kexec_start_address). To do that at > the end (after flushing pdm in memory, etc.) it needs to invalidate > TLB [section] entries. It'd be worth noting explicitly that set_kernel_text_rw() alters current->active_mm... > If during the TLB invalidation an interrupt occours, which might cause > a context switch, there is the risk to inject invalid TLBs, with ro > permissions. ... which is why if there's a context switch things can go wrong, since active_mm isn't stable, and so it's possible that set_kernel_text_rw() updates multiple tables, none of which might be the active table at the point we try to make an access. It would be nice to spell that out rather than saying "invalid TLBs". We could disable preemption to prevent that, which is possibly better than disabling interrupts. Overall, it would be much better to avoid having to mess with the kernel page tables. So rather than going: 1. mark kernel RW 2. alter variables in reloc code 3. copy reloc code into buffer 4. branch to buffer ... we should be able to go: 1. copy reloc code into buffer 2. alter variables in copy of reloc code 3. branch to buffer ... which would avoid this class of problem too. Thanks, Mark. > When trying to assign .text labels, this lead to the following: > > Unable to handle kernel paging request at virtual address 80112f38 > pgd = fd7ef03e > [80112f38] *pgd=0001141e(bad) > Internal error: Oops: 80d [#1] PREEMPT SMP ARM > ... > > Signed-off-by: Giancarlo Ferrari > --- > arch/arm/kernel/machine_kexec.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > diff --git a/arch/arm/kernel/machine_kexec.c b/arch/arm/kernel/machine_kexec.c > index 5d84ad3..23e8816 100644 > --- a/arch/arm/kernel/machine_kexec.c > +++ b/arch/arm/kernel/machine_kexec.c > @@ -174,6 +174,13 @@ void machine_kexec(struct kimage *image) > > reboot_code_buffer = page_address(image->control_code_page); > > + /* > + * If below part is not atomic TLB entries might be corrupted after TLB > + * invalidation, which leads to Data Abort in .text variable assignment > + */ > + raw_local_irq_disable(); > + local_fiq_disable(); > + > /* Prepare parameters for reboot_code_buffer*/ > set_kernel_text_rw(); > kexec_start_address = image->start; > @@ -181,6 +188,9 @@ void machine_kexec(struct kimage *image) > kexec_mach_type = machine_arch_type; > kexec_boot_atags = image->arch.kernel_r2; > > + local_fiq_enable(); > + raw_local_irq_enable(); > + > /* copy our kernel relocation code to the control code page */ > reboot_entry = fncpy(reboot_code_buffer, > &relocate_new_kernel, > -- > 2.7.4 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel