From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AADE2C4320A for ; Tue, 24 Aug 2021 13:27:41 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 36C3461220 for ; Tue, 24 Aug 2021 13:27:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 36C3461220 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6FEE44B0CD; Tue, 24 Aug 2021 09:27:40 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ODAgzvOSP825; Tue, 24 Aug 2021 09:27:35 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id B3E364B0F7; Tue, 24 Aug 2021 09:27:35 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6004C4B0E1 for ; Tue, 24 Aug 2021 09:27:34 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z9YmW1yBsS6F for ; Tue, 24 Aug 2021 09:27:29 -0400 (EDT) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 5D9F84B0CD for ; Tue, 24 Aug 2021 09:27:29 -0400 (EDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E408A1042; Tue, 24 Aug 2021 06:27:28 -0700 (PDT) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 929953F5A1; Tue, 24 Aug 2021 06:27:27 -0700 (PDT) Subject: Re: [PATCH] arm64: Do not trap PMSNEVFR_EL1 From: Alexandru Elisei To: maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, broonie@kernel.org, Catalin Marinas , Will Deacon References: <20210824132459.562923-1-alexandru.elisei@arm.com> Message-ID: <01117fe3-d4fa-a735-36e1-ac9003e7b5cb@arm.com> Date: Tue, 24 Aug 2021 14:28:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20210824132459.562923-1-alexandru.elisei@arm.com> Content-Language: en-US X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Errr... somehow I forgot to add the arm64 maintainers. Fixing that. On 8/24/21 2:24 PM, Alexandru Elisei wrote: > Commit 31c00d2aeaa2 ("arm64: Disable fine grained traps on boot") zeroed > the fine grained trap registers to prevent unwanted register traps from > occuring. However, for the PMSNEVFR_EL1 register, the corresponding > HDFGRTR_EL2.nPMSNEVFR_EL1 field must be 1 to disable trapping. Set the > field to 1 if FEAT_SPEv1p2 is detected. > > Fixes: 31c00d2aeaa2 ("arm64: Disable fine grained traps on boot") > Signed-off-by: Alexandru Elisei > --- > Based on v5.14-rc7. Also, we could write 1 << 62 to HDFGRTR_EL2 unconditionally > since the field is RAZ/WI if !FEAT_SPEv1p2. I don't have a strong preference for > either approaches, but I chose this implementation because it's clearer (even > though it's more verbose and it's one extra trap on NV). > > Tested on the model, using boot-wrapper built from commit 5cd6238ec4ef > ("aarch32: fix .globl replacement"). Without this patch, in NVHE mode, the model > freezes when I try to read PMSNEVFR_EL1. With this patch, the model doesn't hang > when I read the register, but it hangs when I write to it. I've gone throught > the pseudocode for reading and writing to PMSNEVFR_EL1 and from what I can tell > nothing should be trapping the accesses. On top of that, this is what I tried on > the model with this patch applied: > > 1. VHE mode, I can read and write to PMSNEVFR_EL1 without any issues, so the > hang is not caused by an incorrect EL3 configuration. > > 2. NVHE mode, I can read and write just fine to *PMSEVFR_EL1*, so the hang is > not caused by an EL2 trap that affects the rest of the profiling control > registers. I have tried printing the HDFGRTR_EL2 value in this situation using > semihosting, the value is what it is programmed by __init_el2_fgt (that is, > 1 << 62). > > At this point, I am inclined to think it's a model bug because reading works, > but writing causes a hang and that looks very suspicious to me. I'm going to > open a model bug internally and see what comes of it. > > arch/arm64/include/asm/el2_setup.h | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h > index b83fb24954b7..8a9adb2039fd 100644 > --- a/arch/arm64/include/asm/el2_setup.h > +++ b/arch/arm64/include/asm/el2_setup.h > @@ -149,7 +149,16 @@ > ubfx x1, x1, #ID_AA64MMFR0_FGT_SHIFT, #4 > cbz x1, .Lskip_fgt_\@ > > - msr_s SYS_HDFGRTR_EL2, xzr > + mov x0, xzr > + mrs x1, id_aa64dfr0_el1 > + ubfx x1, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 > + cmp x1, #3 > + b.lt .Lset_fgt_\@ > + /* Set HDFGRTR_EL2.nPMSNEVFR_EL1 to disable the register trap */ > + orr x0, x0, #(1 << 62) > + > +.Lset_fgt_\@: > + msr_s SYS_HDFGRTR_EL2, x0 > msr_s SYS_HDFGWTR_EL2, xzr > msr_s SYS_HFGRTR_EL2, xzr > msr_s SYS_HFGWTR_EL2, xzr _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C540AC4338F for ; Tue, 24 Aug 2021 13:28:58 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 870E760FD8 for ; Tue, 24 Aug 2021 13:28:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 870E760FD8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:References:To:From:Subject:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DN0Mf6oR5TnP62sN/z9zEiR5nk7R/RhEDx+WET1iE2I=; b=tBf6+adtaO15s+TBBZwgBjNb+4 ReRd9poQSSHtYBpq5l5AMDbSnVwcZ18oXLwsyL0Xg31nkycIZ/N+Dqeo69KMAd7I54FzfTGPfsDLC IuUaikRJDK01HO2iCVKopzMappSyyPqejzY5z6dLR3U6Wx8n3588RCP4bxhvNjT3yV+A75nU/bcgC +iF0oIuduyL5Uo3rsp792LJuZGhC72aIk1AmLpnzPO/Q/EY0lOukLHz2Yegxxca5qKQeMEr8Ulzz/ XNyWm2ImStRalnFAmCZf3utWf+WNRa0y3zU69y5YAzxJHK5JUgZWRwsL8WGpue8rI6yTELoH1pQne MXZnmLtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIWSn-003Ez8-Ri; Tue, 24 Aug 2021 13:27:38 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mIWSg-003Ews-Un for linux-arm-kernel@lists.infradead.org; Tue, 24 Aug 2021 13:27:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E408A1042; Tue, 24 Aug 2021 06:27:28 -0700 (PDT) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 929953F5A1; Tue, 24 Aug 2021 06:27:27 -0700 (PDT) Subject: Re: [PATCH] arm64: Do not trap PMSNEVFR_EL1 From: Alexandru Elisei To: maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, broonie@kernel.org, Catalin Marinas , Will Deacon References: <20210824132459.562923-1-alexandru.elisei@arm.com> Message-ID: <01117fe3-d4fa-a735-36e1-ac9003e7b5cb@arm.com> Date: Tue, 24 Aug 2021 14:28:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <20210824132459.562923-1-alexandru.elisei@arm.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210824_062731_144469_76817EB1 X-CRM114-Status: GOOD ( 31.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Errr... somehow I forgot to add the arm64 maintainers. Fixing that. On 8/24/21 2:24 PM, Alexandru Elisei wrote: > Commit 31c00d2aeaa2 ("arm64: Disable fine grained traps on boot") zeroed > the fine grained trap registers to prevent unwanted register traps from > occuring. However, for the PMSNEVFR_EL1 register, the corresponding > HDFGRTR_EL2.nPMSNEVFR_EL1 field must be 1 to disable trapping. Set the > field to 1 if FEAT_SPEv1p2 is detected. > > Fixes: 31c00d2aeaa2 ("arm64: Disable fine grained traps on boot") > Signed-off-by: Alexandru Elisei > --- > Based on v5.14-rc7. Also, we could write 1 << 62 to HDFGRTR_EL2 unconditionally > since the field is RAZ/WI if !FEAT_SPEv1p2. I don't have a strong preference for > either approaches, but I chose this implementation because it's clearer (even > though it's more verbose and it's one extra trap on NV). > > Tested on the model, using boot-wrapper built from commit 5cd6238ec4ef > ("aarch32: fix .globl replacement"). Without this patch, in NVHE mode, the model > freezes when I try to read PMSNEVFR_EL1. With this patch, the model doesn't hang > when I read the register, but it hangs when I write to it. I've gone throught > the pseudocode for reading and writing to PMSNEVFR_EL1 and from what I can tell > nothing should be trapping the accesses. On top of that, this is what I tried on > the model with this patch applied: > > 1. VHE mode, I can read and write to PMSNEVFR_EL1 without any issues, so the > hang is not caused by an incorrect EL3 configuration. > > 2. NVHE mode, I can read and write just fine to *PMSEVFR_EL1*, so the hang is > not caused by an EL2 trap that affects the rest of the profiling control > registers. I have tried printing the HDFGRTR_EL2 value in this situation using > semihosting, the value is what it is programmed by __init_el2_fgt (that is, > 1 << 62). > > At this point, I am inclined to think it's a model bug because reading works, > but writing causes a hang and that looks very suspicious to me. I'm going to > open a model bug internally and see what comes of it. > > arch/arm64/include/asm/el2_setup.h | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h > index b83fb24954b7..8a9adb2039fd 100644 > --- a/arch/arm64/include/asm/el2_setup.h > +++ b/arch/arm64/include/asm/el2_setup.h > @@ -149,7 +149,16 @@ > ubfx x1, x1, #ID_AA64MMFR0_FGT_SHIFT, #4 > cbz x1, .Lskip_fgt_\@ > > - msr_s SYS_HDFGRTR_EL2, xzr > + mov x0, xzr > + mrs x1, id_aa64dfr0_el1 > + ubfx x1, x1, #ID_AA64DFR0_PMSVER_SHIFT, #4 > + cmp x1, #3 > + b.lt .Lset_fgt_\@ > + /* Set HDFGRTR_EL2.nPMSNEVFR_EL1 to disable the register trap */ > + orr x0, x0, #(1 << 62) > + > +.Lset_fgt_\@: > + msr_s SYS_HDFGRTR_EL2, x0 > msr_s SYS_HDFGWTR_EL2, xzr > msr_s SYS_HFGRTR_EL2, xzr > msr_s SYS_HFGWTR_EL2, xzr _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel