From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A54BDC433EF for ; Mon, 11 Oct 2021 12:17:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 74B8060F92 for ; Mon, 11 Oct 2021 12:17:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 74B8060F92 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XgyS0RP/1NkBT5tdjCzCy2vxgxC69bboq1f5+AxJKFc=; b=v9UCdl0IPAOzPo S3+i4CMQ6s0RSTTgb+YaxBgFSPTYx//7gDT3LRuSsSjewXZPRD76Yx0jtYInZq5GBVoOHh+RfA86x NQAYx1wpbqldnxAyfg+dFoqEbD/78loClXUQ05LeqggDzbWccPAtqhCYkP4GBmwklfm36mu7CFo3o WnRdUtimBqhK3b+XJYCsdCvxC9ymzdi0QlfUCA8Eiw0QcUkDtv+Wi68FZLgASjpJiYQmbNbhAXxgt AS+TRyREngaTLfqpqRfuoe76+iewjO1Qk+FbzClz5rjlNMLrcs/0XkxbK4vdTU+T/Xp1q3omhBdxY m0e8YnlFDVgOXFk8PYcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mZuDY-009Hrx-0e; Mon, 11 Oct 2021 12:15:44 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mZuDT-009Hqn-0X for linux-arm-kernel@lists.infradead.org; Mon, 11 Oct 2021 12:15:40 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 52CDE1063; Mon, 11 Oct 2021 05:15:35 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.22.9]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6F3F53F70D; Mon, 11 Oct 2021 05:15:33 -0700 (PDT) Date: Mon, 11 Oct 2021 13:15:30 +0100 From: Mark Rutland To: Zhaoyang Huang Cc: Will Deacon , Catalin Marinas , Suzuki K Poulose , Ionela Voinescu , Quentin Perret , Vladimir Murzin , linux-arm-kernel@lists.infradead.org, Zhaoyang Huang , LKML , Ke Wang , ping.zhou1@unisoc.com Subject: Re: [RFC PATCH] arch: ARM64: add isb before enable pan Message-ID: <20211011121530.GC1421@C02TD0UTHF1T.local> References: <1633673269-15048-1-git-send-email-huangzhaoyang@gmail.com> <20211008080113.GA441@willie-the-truck> <20211011093803.GA1421@C02TD0UTHF1T.local> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211011_051539_172780_99D1E969 X-CRM114-Status: GOOD ( 35.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Oct 11, 2021 at 07:08:00PM +0800, Zhaoyang Huang wrote: > On Mon, Oct 11, 2021 at 5:38 PM Mark Rutland wrote: > > On Fri, Oct 08, 2021 at 04:34:12PM +0800, Zhaoyang Huang wrote: > > > On Fri, Oct 8, 2021 at 4:01 PM Will Deacon wrote: > > > > On Fri, Oct 08, 2021 at 02:07:49PM +0800, Huangzhaoyang wrote: > > > > > From: Zhaoyang Huang > > > > > > > > > > set_pstate_pan failure is observed in an ARM64 system occasionaly on a reboot > > > > > test, which can be work around by a msleep on the sw context. We assume > > > > > suspicious on disorder of previous instr of disabling SW_PAN and add an isb here. > > > > > > > > > > PS: > > > > > The bootup test failed with a invalid TTBR1_EL1 that equals 0x34000000, which is > > > > > alike racing between on chip PAN and SW_PAN. > > > > > > > > Sorry, but I'm struggling to understand the problem here. Please could you > > > > explain it in more detail? > > > > > > > > - Why does a TTBR1_EL1 value of `0x34000000` indicate a race? > > > > - Can you explain the race that you think might be occurring? > > > > - Why does an ISB prevent the race? > > > Please find panic logs[1], related codes[2], sample of debug patch[3] > > > below. TTBR1_EL1 equals 0x34000000 when panic > > > > Just to check, how do you know the value of TTBR1_EL1 was 0x34000000? > > That isn't in the log sample below -- was that from the output of > > show_pte(), an external debugger, or something else? > > > > I'm assuming from the "(ptrval)" bits below that can't have been from > > show_pte(). > > > > > and can NOT be captured > > > by the debug patch during retest (all entrances that msr ttbr1_el1 are > > > under watch) which should work. Adding ISB here to prevent race on > > > TTBR1 from previous access of sysregs which can affect the msr > > > result(the test is still ongoing). Could the race be > > > ARM64_HAS_PAN(automated by core) and SW_PAN. > > > > > > [1] > > > [ 0.348000] [0: migration/0: 11] Synchronous External Abort: > > > level 1 (translation table walk) (0x96000055) at 0xffffffc000e06004 > > > [ 0.352000] [0: migration/0: 11] Internal error: : 96000055 > > > [#1] PREEMPT SMP > > > [ 0.352000] [0: migration/0: 11] Modules linked in: > > > [ 0.352000] [0: migration/0: 11] Process migration/0 (pid: > > > 11, stack limit = 0x (ptrval)) > > > [ 0.352000] [0: migration/0: 11] CPU: 0 PID: 11 Comm: > > > migration/0 Tainted: G S > > > > Assuming I've read the `taint_flags` table correctly, that 'S' is > > `TAINT_CPU_OUT_OF_SPEC`, for which we should dump warnings for at boot > > time. The 'G' indicates the absence of proprietary modules. > > > > Can you provide a full dmesg for a failed boot, please? > > > > Have you made any changes to arch/arm64/kernel/cpufeature.c? > > > > Are you able to test with a mainline kernel? > > > > > 4.14.199-22631304-abA035FXXU0AUJ4_T4 #2 > > > > > > [ 0.352000] [0: migration/0: 11] Hardware name: Spreadtrum > > > UMS9230 1H10 SoC (DT) > > > [ 0.352000] [0: migration/0: 11] task: (ptrval) > > > task.stack: (ptrval) > > > [ 0.352000] [0: migration/0: 11] pc : patch_alternative+0x68/0x27c > > > [ 0.352000] [0: migration/0: 11] lr : > > > __apply_alternatives.llvm.7450387295891320208+0x60/0x160 > > > > > > [2] > > > __apply_alternatives > > > for() > > > patch_alternative <----panic here in the 2nd round of loop > > > after invoking flush_icache_range > > > flush_icache_range > > > > > > [3] > > > sub \tmp1, \tmp1, #SWAPPER_DIR_SIZE > > > + tst \tmp1, #0xffff80000000 // check ttbr1_el1 valid > > > + b.le . > > > > What are you trying to detect for here? This is testing both the ASID > > and BADDR[47] bits, so I don;t understand the rationale. > > > > Thanks, > > Mark. > this issue is fixed by the patch 'arm64: Avoid flush_icache_range() in > alternatives patching code(429388682dc266e7a693f9c27e3aabd341d55343)'. > thanks Ah, thanks for this. So this is because in arch/arm64/mm/cache.S we do (abbreviated): | ENTRY(flush_icache_range) | /* FALLTHROUGH */ | ENTRY(__flush_cache_user_range) | uaccess_ttbr0_enable x2, x3, x4 | | ... | | uaccess_ttbr0_disable x1, x2 | ret | ENDPROC(flush_icache_range) ... and even if I-caches are coherent and we don't execute junk for something we've patched but not invalidated, since we patch each alternative site in turn we can end up with unbalanced calls to enable/disable across a number of patches. It looks like 429388682dc266e7a693f9c27e3aabd341d55343 isn't in the current linux-4.14.y, so we should post a backported version. From local testing, that applies trivially. Who wants to post that to stable? I can do if people want. Thanks, Mark. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel