From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84FA3C4338F for ; Thu, 29 Jul 2021 16:42:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 605DC60F23 for ; Thu, 29 Jul 2021 16:42:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229565AbhG2QmX (ORCPT ); Thu, 29 Jul 2021 12:42:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:45334 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229556AbhG2QmX (ORCPT ); Thu, 29 Jul 2021 12:42:23 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4CADD60EBD; Thu, 29 Jul 2021 16:42:18 +0000 (UTC) Date: Thu, 29 Jul 2021 17:42:15 +0100 From: Catalin Marinas To: Marc Zyngier Cc: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Will Deacon , James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com, Quentin Perret , stable@vger.kernel.org Subject: Re: [PATCH] KVM: arm64: Unregister HYP sections from kmemleak in protected mode Message-ID: <20210729164214.GB31848@arm.com> References: <20210729135016.3037277-1-maz@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210729135016.3037277-1-maz@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org On Thu, Jul 29, 2021 at 02:50:16PM +0100, Marc Zyngier wrote: > Booting a KVM host in protected mode with kmemleak quickly results > in a pretty bad crash, as kmemleak doesn't know that the HYP sections > have been taken away. > > Make the unregistration from kmemleak part of marking the sections > as HYP-private. The rest of the HYP-specific data is obtained via > the page allocator, which is not subjected to kmemleak. > > Fixes: 90134ac9cabb ("KVM: arm64: Protect the .hyp sections from the host") > Signed-off-by: Marc Zyngier > Cc: Quentin Perret > Cc: Catalin Marinas > Cc: stable@vger.kernel.org # 5.13 > --- > arch/arm64/kvm/arm.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index e9a2b8f27792..23f12e602878 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -15,6 +15,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1960,8 +1961,12 @@ static inline int pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) > } > > #define pkvm_mark_hyp_section(__section) \ > +({ \ > + u64 sz = __section##_end - __section##_start; \ > + kmemleak_free_part(__section##_start, sz); \ > pkvm_mark_hyp(__pa_symbol(__section##_start), \ > - __pa_symbol(__section##_end)) > + __pa_symbol(__section##_end)); \ > +}) Using kmemleak_free_part() is fine in principle as this is not a slab object. However, the above would call the function even for ranges that are not tracked at all by kmemleak (text, idmap). Luckily Kmemleak won't complain, unless you #define DEBUG in the file (initially I tried to warn all the time but I couldn't fix all the callbacks). If it was just the BSS, I would move the kmemleak_free_part() call to finalize_hyp_mode() but there's the __hyp_rodata section as well. I think we have some inconsistency with .hyp.rodata which falls under _sdata.._edata while the kernel's own .rodata goes immediately after _etext. Should we move __hyp_rodata outside _sdata.._edata as well? It would benefit from the RO NX marking (probably more useful without the protected mode). If this works, we'd only need to call kmemleak once for the BSS. -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED1D2C4338F for ; Thu, 29 Jul 2021 16:42:25 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 6B6BF60F23 for ; Thu, 29 Jul 2021 16:42:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6B6BF60F23 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E40AD4B0E4; Thu, 29 Jul 2021 12:42:24 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yO1vDatQoTyQ; Thu, 29 Jul 2021 12:42:23 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id CA1284B0E9; Thu, 29 Jul 2021 12:42:23 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 663FF4B0E4 for ; Thu, 29 Jul 2021 12:42:22 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id h0ebtMNNKNlj for ; Thu, 29 Jul 2021 12:42:21 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 480DB4B0DF for ; Thu, 29 Jul 2021 12:42:21 -0400 (EDT) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4CADD60EBD; Thu, 29 Jul 2021 16:42:18 +0000 (UTC) Date: Thu, 29 Jul 2021 17:42:15 +0100 From: Catalin Marinas To: Marc Zyngier Subject: Re: [PATCH] KVM: arm64: Unregister HYP sections from kmemleak in protected mode Message-ID: <20210729164214.GB31848@arm.com> References: <20210729135016.3037277-1-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210729135016.3037277-1-maz@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) Cc: kernel-team@android.com, kvm@vger.kernel.org, stable@vger.kernel.org, Will Deacon , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Thu, Jul 29, 2021 at 02:50:16PM +0100, Marc Zyngier wrote: > Booting a KVM host in protected mode with kmemleak quickly results > in a pretty bad crash, as kmemleak doesn't know that the HYP sections > have been taken away. > > Make the unregistration from kmemleak part of marking the sections > as HYP-private. The rest of the HYP-specific data is obtained via > the page allocator, which is not subjected to kmemleak. > > Fixes: 90134ac9cabb ("KVM: arm64: Protect the .hyp sections from the host") > Signed-off-by: Marc Zyngier > Cc: Quentin Perret > Cc: Catalin Marinas > Cc: stable@vger.kernel.org # 5.13 > --- > arch/arm64/kvm/arm.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index e9a2b8f27792..23f12e602878 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -15,6 +15,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1960,8 +1961,12 @@ static inline int pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) > } > > #define pkvm_mark_hyp_section(__section) \ > +({ \ > + u64 sz = __section##_end - __section##_start; \ > + kmemleak_free_part(__section##_start, sz); \ > pkvm_mark_hyp(__pa_symbol(__section##_start), \ > - __pa_symbol(__section##_end)) > + __pa_symbol(__section##_end)); \ > +}) Using kmemleak_free_part() is fine in principle as this is not a slab object. However, the above would call the function even for ranges that are not tracked at all by kmemleak (text, idmap). Luckily Kmemleak won't complain, unless you #define DEBUG in the file (initially I tried to warn all the time but I couldn't fix all the callbacks). If it was just the BSS, I would move the kmemleak_free_part() call to finalize_hyp_mode() but there's the __hyp_rodata section as well. I think we have some inconsistency with .hyp.rodata which falls under _sdata.._edata while the kernel's own .rodata goes immediately after _etext. Should we move __hyp_rodata outside _sdata.._edata as well? It would benefit from the RO NX marking (probably more useful without the protected mode). If this works, we'd only need to call kmemleak once for the BSS. -- Catalin _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA333C4338F for ; Thu, 29 Jul 2021 16:44:11 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9964860EBD for ; Thu, 29 Jul 2021 16:44:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9964860EBD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6hquTAHV1K3M9drpSKjPgKYlnU6UA3odfx8Xb/TtjFE=; b=Akl4rJxaThRs7L TsMFG5sYyqWprrVm9CaAQwQ88CIgN5X4AFBaG5ZFfmh4R1OAN6i75XBvnpAN+49nEKORjVp0+3s6p FDrcTLygi9FLjSeLz0eQW6bnRdcN+wMWzNyTlXivazXeT8JjrhB6ebb1cn6eqdLAR7yOH5Dec+ZWW FJuav+6/fUiG1SLjkbtAZz6MtW7V8DQhxEqxjMtKVKlB0lVQ5xQZ8Wd0+fWAbEHtv+evpP39tAVay IMR6IjbS3M4TxqYI73h8WUTgKgaqZdMFRY79LZ1XylaPh2ss6KsuMpdrDiQRBazdZqpNae2At3r30 gjnVZId02DMoHFAB5ALg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m9972-0057x0-Up; Thu, 29 Jul 2021 16:42:25 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m996y-0057wG-W6 for linux-arm-kernel@lists.infradead.org; Thu, 29 Jul 2021 16:42:22 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4CADD60EBD; Thu, 29 Jul 2021 16:42:18 +0000 (UTC) Date: Thu, 29 Jul 2021 17:42:15 +0100 From: Catalin Marinas To: Marc Zyngier Cc: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Will Deacon , James Morse , Suzuki K Poulose , Alexandru Elisei , kernel-team@android.com, Quentin Perret , stable@vger.kernel.org Subject: Re: [PATCH] KVM: arm64: Unregister HYP sections from kmemleak in protected mode Message-ID: <20210729164214.GB31848@arm.com> References: <20210729135016.3037277-1-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210729135016.3037277-1-maz@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210729_094221_096166_2710EE6A X-CRM114-Status: GOOD ( 22.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Jul 29, 2021 at 02:50:16PM +0100, Marc Zyngier wrote: > Booting a KVM host in protected mode with kmemleak quickly results > in a pretty bad crash, as kmemleak doesn't know that the HYP sections > have been taken away. > > Make the unregistration from kmemleak part of marking the sections > as HYP-private. The rest of the HYP-specific data is obtained via > the page allocator, which is not subjected to kmemleak. > > Fixes: 90134ac9cabb ("KVM: arm64: Protect the .hyp sections from the host") > Signed-off-by: Marc Zyngier > Cc: Quentin Perret > Cc: Catalin Marinas > Cc: stable@vger.kernel.org # 5.13 > --- > arch/arm64/kvm/arm.c | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index e9a2b8f27792..23f12e602878 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -15,6 +15,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -1960,8 +1961,12 @@ static inline int pkvm_mark_hyp(phys_addr_t start, phys_addr_t end) > } > > #define pkvm_mark_hyp_section(__section) \ > +({ \ > + u64 sz = __section##_end - __section##_start; \ > + kmemleak_free_part(__section##_start, sz); \ > pkvm_mark_hyp(__pa_symbol(__section##_start), \ > - __pa_symbol(__section##_end)) > + __pa_symbol(__section##_end)); \ > +}) Using kmemleak_free_part() is fine in principle as this is not a slab object. However, the above would call the function even for ranges that are not tracked at all by kmemleak (text, idmap). Luckily Kmemleak won't complain, unless you #define DEBUG in the file (initially I tried to warn all the time but I couldn't fix all the callbacks). If it was just the BSS, I would move the kmemleak_free_part() call to finalize_hyp_mode() but there's the __hyp_rodata section as well. I think we have some inconsistency with .hyp.rodata which falls under _sdata.._edata while the kernel's own .rodata goes immediately after _etext. Should we move __hyp_rodata outside _sdata.._edata as well? It would benefit from the RO NX marking (probably more useful without the protected mode). If this works, we'd only need to call kmemleak once for the BSS. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel