From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E85A529A2 for ; Wed, 30 Mar 2022 17:42:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 946C4C34112 for ; Wed, 30 Mar 2022 17:42:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648662163; bh=I2nwMsi0ASZuANmFO4D1q4EQjpFm+YKdWDV/S1qQdgI=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=FZ2l9bRDejt84c449bQ8hFmBugEh4ngaORwyNebdds5Vjs+hO4eIfYlYJldqQ3nNK QCwl50twfPNO3ziojiuMerBKvGxxIub990HcMr86mBlfRm1aBkVZiM2NPYYYOLhVBW aS8YJiKmZZTer9XXLLNYxigFLQRgD0PhyiC+4EcWIKcdCQUmTPtDpxcGKkEkEVcH5a TzLAsk//ijZ6Aa4F+8YixhuqkQJvq2/gCYnJfU2UhK7IAlBBJmk2w+5VxL5M2gKlUs n94ZlOCC2mcah6c0U0vFQ6di2PkqooUItM2L1jYnyokumZ/C4EhpE41PH9iZDL8mkS ZqBQAfZ3rjTkQ== Received: by mail-oa1-f46.google.com with SMTP id 586e51a60fabf-df26ea5bfbso5188518fac.1 for ; Wed, 30 Mar 2022 10:42:43 -0700 (PDT) X-Gm-Message-State: AOAM533ArbUa1Q2TA6sdVNzo8HEVrxz9T9C3Wtedb4AiLJRp7i0WX4T7 77Djk9kue4/hoFhtbOmOtT3x3NZmVTSBtd/z+8I= X-Google-Smtp-Source: ABdhPJxVwc1vcsNmifyLf6iCDsIRT/4G7CdnAH+8eeTpR1xBYRGvRayW8V+k/WnefL7MglY1+3MBxoJgQt3zkfQEiW8= X-Received: by 2002:a05:6870:b027:b0:de:7fcd:fabf with SMTP id y39-20020a056870b02700b000de7fcdfabfmr431384oae.126.1648662162730; Wed, 30 Mar 2022 10:42:42 -0700 (PDT) Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <10062923.nUPlyArG6x@localhost.localdomain> In-Reply-To: <10062923.nUPlyArG6x@localhost.localdomain> From: Ard Biesheuvel Date: Wed, 30 Mar 2022 19:42:31 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: CONFIG_THUMB2_KERNEL=y boot failure after Spectre BHB fixes To: Christian Eggers Cc: "Russell King (Oracle)" , Linux ARM , Catalin Marinas , Linux Kernel Mailing List , llvm@lists.linux.dev, Nathan Chancellor Content-Type: text/plain; charset="UTF-8" On Wed, 30 Mar 2022 at 19:33, Christian Eggers wrote: > > On Wednesday, 30 March 2022, 18:45:18 CEST, Ard Biesheuvel wrote: > > > On Wed, 30 Mar 2022 at 18:37, Russell King (Oracle) > > wrote: > > > > > > On Wed, Mar 30, 2022 at 06:34:25PM +0200, Ard Biesheuvel wrote: > > > > On Wed, 30 Mar 2022 at 18:12, Russell King (Oracle) > > > > wrote: > > > > > > > > > > On Tue, Mar 22, 2022 at 06:49:17PM +0100, Christian Eggers wrote: > > > > > > Hi Nathan, hi Russel, > > > > > > > > > > > > I stumbled today over the same problem (no output on serial console > > > > > > with v5.15.28-rt36). During `git bisect`, I had also some commits > > > > > > where a few lines of output were visible. > > > > > > > > > > > > At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of > > > > > > sections"), the system boots up to here: > > > > > > > > > > > > start_kernel() > > > > > > +--setup_arch() > > > > > > +--paging_init() > > > > > > +--devicemaps_init() > > > > > > +--eary_trap_init(vectors_base = 0xC7FFE000) > > > > > > +--copy_from_lma(vectors_base = 0xC7FFE000, __vectors_start=0x0, __vectors_end=0x20) > > > > > > +--__memcpy() > > > > > > > > > > > > copy_template.S:113 > > > > > > ldr8w r1, r3, r4, r5, r6, r7, r8, ip, lr, abort=20f > > > > > > r1 = 0 > > > > > > > > > > > > > > > > > > With the final v5.15.28-rt36 I found out that the system boots fine > > > > > > after disabling CONFIG_HARDEN_BRANCH_HISTORY. > > > > > > > > > > > > Is there anything else I could analyze? My SoC system is a NXP i.MX6LL > > > > > > with a ARMv7 core. I have access to a JTAG debugger. > > > > > > > > > > I think this is already fixed in mainline. Commit: > > > > > > > > > > 6c7cb60bff7a ("ARM: fix Thumb2 regression") > > > > > > > > > > > > > It's still broken - I sent a couple of patches on Monday, among which > > > > one to fix the boot issue with loop8 on Thumb2. The problem is 'b . + > > > > 4', which produces a narrow encoding, and so it skips the subsequent > > > > subs instruction and loops forever. > > > > > > And what's the current status? Sorry, I've way too much email from the > > > last 2.5 weeks to find it myself. > > > > > > > https://lore.kernel.org/linux-arm-kernel/20220328134714.205342-1-ardb@kernel.org/ > > > > Nobody bothered to respond yet, I can drop the first two in the patch > > tracker if you like. > > I just switched to v5.15.31-rt38 which already includes > 6c7cb60bff7a ("ARM: fix Thumb2 regression") > > This kernel boots fine now, even with CONFIG_HARDEN_BRANCH_HISTORY=y. After > applying the patch series from Ard, the system still boots fine. > > I haven't any understanding what these patches do. Is there anything I shall > test? > Thanks for confirming. The first fix affects all Thumb2 configurations, my patch only affects Thumb2 configurations that actually enable the loop8 mitigation for Spectre-BHB. What type of CPU are you booting on? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 824A1C433EF for ; Wed, 30 Mar 2022 17:44:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EAYMtHmCH6npSM8wiXeEZk6WX+ovZ31ZHuyWVJpux88=; b=fCw0oB3yNuOqEn xVfrfjtb95aULlp/2hFd6evaWYllwHF27hq+RpN6Oi4KLghhsXRqpe1dXgkEiRP1HlNZZA1RWQOoy Wp1KBw+3t55YwidMicUKv0XTgZtUSiziWpufnyLQMB+pDEdeRRxeYj3RQOnLwZTWXaPOeZv4hwVr0 8kj6tDarlvzVdO4PgHIRLjdbjoSa2ImOcCX5JD6uSIOnJeAdLS1PSwCGIoNeiXrYox0e6pbuBGVPC uyBfK19fk9bKVBAPR9d7HYCES5X/5DGkTFyl8ECVsRR5k2mR/jdZYt/dtHcO/yR2UOotumtdoJksL B0gOUdZMJj+zrsCoqs2A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZcLK-00H1Qc-IR; Wed, 30 Mar 2022 17:42:50 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nZcLG-00H1Pn-Fj for linux-arm-kernel@lists.infradead.org; Wed, 30 Mar 2022 17:42:48 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CC11BB81DA6 for ; Wed, 30 Mar 2022 17:42:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 979ECC36AE2 for ; Wed, 30 Mar 2022 17:42:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1648662163; bh=I2nwMsi0ASZuANmFO4D1q4EQjpFm+YKdWDV/S1qQdgI=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=FZ2l9bRDejt84c449bQ8hFmBugEh4ngaORwyNebdds5Vjs+hO4eIfYlYJldqQ3nNK QCwl50twfPNO3ziojiuMerBKvGxxIub990HcMr86mBlfRm1aBkVZiM2NPYYYOLhVBW aS8YJiKmZZTer9XXLLNYxigFLQRgD0PhyiC+4EcWIKcdCQUmTPtDpxcGKkEkEVcH5a TzLAsk//ijZ6Aa4F+8YixhuqkQJvq2/gCYnJfU2UhK7IAlBBJmk2w+5VxL5M2gKlUs n94ZlOCC2mcah6c0U0vFQ6di2PkqooUItM2L1jYnyokumZ/C4EhpE41PH9iZDL8mkS ZqBQAfZ3rjTkQ== Received: by mail-oa1-f44.google.com with SMTP id 586e51a60fabf-df22f50e0cso5519134fac.3 for ; Wed, 30 Mar 2022 10:42:43 -0700 (PDT) X-Gm-Message-State: AOAM533ERbDmssMPK/Lysf5D7/nh4kLDFlUCApCg/gaj03FKkgB+q+Is UukzXF4iRhv+ppNwHKZoOGwxRZMUVniBlgzXEoc= X-Google-Smtp-Source: ABdhPJxVwc1vcsNmifyLf6iCDsIRT/4G7CdnAH+8eeTpR1xBYRGvRayW8V+k/WnefL7MglY1+3MBxoJgQt3zkfQEiW8= X-Received: by 2002:a05:6870:b027:b0:de:7fcd:fabf with SMTP id y39-20020a056870b02700b000de7fcdfabfmr431384oae.126.1648662162730; Wed, 30 Mar 2022 10:42:42 -0700 (PDT) MIME-Version: 1.0 References: <10062923.nUPlyArG6x@localhost.localdomain> In-Reply-To: <10062923.nUPlyArG6x@localhost.localdomain> From: Ard Biesheuvel Date: Wed, 30 Mar 2022 19:42:31 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: CONFIG_THUMB2_KERNEL=y boot failure after Spectre BHB fixes To: Christian Eggers Cc: "Russell King (Oracle)" , Linux ARM , Catalin Marinas , Linux Kernel Mailing List , llvm@lists.linux.dev, Nathan Chancellor X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220330_104246_853562_91F898A7 X-CRM114-Status: GOOD ( 33.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 30 Mar 2022 at 19:33, Christian Eggers wrote: > > On Wednesday, 30 March 2022, 18:45:18 CEST, Ard Biesheuvel wrote: > > > On Wed, 30 Mar 2022 at 18:37, Russell King (Oracle) > > wrote: > > > > > > On Wed, Mar 30, 2022 at 06:34:25PM +0200, Ard Biesheuvel wrote: > > > > On Wed, 30 Mar 2022 at 18:12, Russell King (Oracle) > > > > wrote: > > > > > > > > > > On Tue, Mar 22, 2022 at 06:49:17PM +0100, Christian Eggers wrote: > > > > > > Hi Nathan, hi Russel, > > > > > > > > > > > > I stumbled today over the same problem (no output on serial console > > > > > > with v5.15.28-rt36). During `git bisect`, I had also some commits > > > > > > where a few lines of output were visible. > > > > > > > > > > > > At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of > > > > > > sections"), the system boots up to here: > > > > > > > > > > > > start_kernel() > > > > > > +--setup_arch() > > > > > > +--paging_init() > > > > > > +--devicemaps_init() > > > > > > +--eary_trap_init(vectors_base = 0xC7FFE000) > > > > > > +--copy_from_lma(vectors_base = 0xC7FFE000, __vectors_start=0x0, __vectors_end=0x20) > > > > > > +--__memcpy() > > > > > > > > > > > > copy_template.S:113 > > > > > > ldr8w r1, r3, r4, r5, r6, r7, r8, ip, lr, abort=20f > > > > > > r1 = 0 > > > > > > > > > > > > > > > > > > With the final v5.15.28-rt36 I found out that the system boots fine > > > > > > after disabling CONFIG_HARDEN_BRANCH_HISTORY. > > > > > > > > > > > > Is there anything else I could analyze? My SoC system is a NXP i.MX6LL > > > > > > with a ARMv7 core. I have access to a JTAG debugger. > > > > > > > > > > I think this is already fixed in mainline. Commit: > > > > > > > > > > 6c7cb60bff7a ("ARM: fix Thumb2 regression") > > > > > > > > > > > > > It's still broken - I sent a couple of patches on Monday, among which > > > > one to fix the boot issue with loop8 on Thumb2. The problem is 'b . + > > > > 4', which produces a narrow encoding, and so it skips the subsequent > > > > subs instruction and loops forever. > > > > > > And what's the current status? Sorry, I've way too much email from the > > > last 2.5 weeks to find it myself. > > > > > > > https://lore.kernel.org/linux-arm-kernel/20220328134714.205342-1-ardb@kernel.org/ > > > > Nobody bothered to respond yet, I can drop the first two in the patch > > tracker if you like. > > I just switched to v5.15.31-rt38 which already includes > 6c7cb60bff7a ("ARM: fix Thumb2 regression") > > This kernel boots fine now, even with CONFIG_HARDEN_BRANCH_HISTORY=y. After > applying the patch series from Ard, the system still boots fine. > > I haven't any understanding what these patches do. Is there anything I shall > test? > Thanks for confirming. The first fix affects all Thumb2 configurations, my patch only affects Thumb2 configurations that actually enable the loop8 mitigation for Spectre-BHB. What type of CPU are you booting on? _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel