From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A5AEE28E8; Fri, 27 Jan 2023 11:40:20 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C11D16A3; Fri, 27 Jan 2023 03:41:02 -0800 (PST) Received: from ewhatever.cambridge.arm.com (ewhatever.cambridge.arm.com [10.1.197.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B29523F64C; Fri, 27 Jan 2023 03:40:17 -0800 (PST) From: Suzuki K Poulose To: kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: suzuki.poulose@arm.com, Alexandru Elisei , Andrew Jones , Christoffer Dall , Fuad Tabba , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , linux-coco@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [RFC kvmtool 11/31] arm64: Lock realm RAM in memory Date: Fri, 27 Jan 2023 11:39:12 +0000 Message-Id: <20230127113932.166089-12-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127113932.166089-1-suzuki.poulose@arm.com> References: <20230127112248.136810-1-suzuki.poulose@arm.com> <20230127113932.166089-1-suzuki.poulose@arm.com> Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Alexandru Elisei RMM doesn't yet support paging protected memory pages. Thus the VMM must pin the entire VM memory. Use mlock2 to keep the realm pages pinned in memory once they are faulted in. Use the MLOCK_ONFAULT flag to prevent pre-mapping the pages and maintain some semblance of on demand-paging for a realm VM. Signed-off-by: Alexandru Elisei Signed-off-by: Suzuki K Poulose --- arm/kvm.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/arm/kvm.c b/arm/kvm.c index d51cc15d..0e40b753 100644 --- a/arm/kvm.c +++ b/arm/kvm.c @@ -7,6 +7,8 @@ #include "arm-common/gic.h" +#include + #include #include #include @@ -24,6 +26,25 @@ bool kvm__arch_cpu_supports_vm(void) return true; } +static void try_increase_mlock_limit(struct kvm *kvm) +{ + u64 size = kvm->arch.ram_alloc_size; + struct rlimit mlock_limit, new_limit; + + if (getrlimit(RLIMIT_MEMLOCK, &mlock_limit)) { + perror("getrlimit(RLIMIT_MEMLOCK)"); + return; + } + + if (mlock_limit.rlim_cur > size) + return; + + new_limit.rlim_cur = size; + new_limit.rlim_max = max((rlim_t)size, mlock_limit.rlim_max); + /* Requires CAP_SYS_RESOURCE capability. */ + setrlimit(RLIMIT_MEMLOCK, &new_limit); +} + void kvm__init_ram(struct kvm *kvm) { u64 phys_start, phys_size; @@ -49,8 +70,27 @@ void kvm__init_ram(struct kvm *kvm) kvm->ram_start = (void *)ALIGN((unsigned long)kvm->arch.ram_alloc_start, SZ_2M); - madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, - MADV_MERGEABLE); + /* + * Do not merge pages if this is a Realm. + * a) We cannot replace a page in realm stage2 without export/import + * + * Pin the realm memory until we have export/import, due to the same + * reason as above. + * + * Use mlock2(,,MLOCK_ONFAULT) to allow faulting in pages and thus + * allowing to lazily populate the PAR. + */ + if (kvm->cfg.arch.is_realm) { + int ret; + + try_increase_mlock_limit(kvm); + ret = mlock2(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, + MLOCK_ONFAULT); + if (ret) + die_perror("mlock2"); + } else { + madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, MADV_MERGEABLE); + } madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, MADV_HUGEPAGE); -- 2.34.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5A002C38142 for ; Fri, 27 Jan 2023 12:05:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pkdZBb41T+zA4nwlv6NkzRGMAYw/32HtdH49wIHFMp8=; b=Y6ZgjKAjILjnp1 72sEoqzjdPvCzhLyHCGqaj7oq0GUV8iwwy3/HMObDJaXg9oALfYMdc+cS8otMs7w2smhkBHVJ4EGh YOCL88CGPO95adSTNECRoW+tpp1+Cf9Tf3B/AgDyFEix8x3WrUNoC1BRfhz+UJHOV6bZ2Ax+JuaEp gLz/5DAvtQHu/FJukq1SFIrfmH8iSH7SJlN+v/A10fABN80+cQoOB1M7rPaUfOEq8CdxTHtKev7iP V+tNnF6ngu2DzVPmazyuTtjTmVyisA2AO1BN82obonZ3ONo1A40dbhwYrOrYv2/1bbGu3T2YN7FB/ pNL4fz7ZuPy2kkZVRIbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pLNT7-00EO7Q-Ok; Fri, 27 Jan 2023 12:04:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pLN5h-00ECtp-01 for linux-arm-kernel@lists.infradead.org; Fri, 27 Jan 2023 11:40:24 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C11D16A3; Fri, 27 Jan 2023 03:41:02 -0800 (PST) Received: from ewhatever.cambridge.arm.com (ewhatever.cambridge.arm.com [10.1.197.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B29523F64C; Fri, 27 Jan 2023 03:40:17 -0800 (PST) From: Suzuki K Poulose To: kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: suzuki.poulose@arm.com, Alexandru Elisei , Andrew Jones , Christoffer Dall , Fuad Tabba , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , linux-coco@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [RFC kvmtool 11/31] arm64: Lock realm RAM in memory Date: Fri, 27 Jan 2023 11:39:12 +0000 Message-Id: <20230127113932.166089-12-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230127113932.166089-1-suzuki.poulose@arm.com> References: <20230127112248.136810-1-suzuki.poulose@arm.com> <20230127113932.166089-1-suzuki.poulose@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230127_034022_417411_C6B97705 X-CRM114-Status: GOOD ( 13.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Alexandru Elisei RMM doesn't yet support paging protected memory pages. Thus the VMM must pin the entire VM memory. Use mlock2 to keep the realm pages pinned in memory once they are faulted in. Use the MLOCK_ONFAULT flag to prevent pre-mapping the pages and maintain some semblance of on demand-paging for a realm VM. Signed-off-by: Alexandru Elisei Signed-off-by: Suzuki K Poulose --- arm/kvm.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/arm/kvm.c b/arm/kvm.c index d51cc15d..0e40b753 100644 --- a/arm/kvm.c +++ b/arm/kvm.c @@ -7,6 +7,8 @@ #include "arm-common/gic.h" +#include + #include #include #include @@ -24,6 +26,25 @@ bool kvm__arch_cpu_supports_vm(void) return true; } +static void try_increase_mlock_limit(struct kvm *kvm) +{ + u64 size = kvm->arch.ram_alloc_size; + struct rlimit mlock_limit, new_limit; + + if (getrlimit(RLIMIT_MEMLOCK, &mlock_limit)) { + perror("getrlimit(RLIMIT_MEMLOCK)"); + return; + } + + if (mlock_limit.rlim_cur > size) + return; + + new_limit.rlim_cur = size; + new_limit.rlim_max = max((rlim_t)size, mlock_limit.rlim_max); + /* Requires CAP_SYS_RESOURCE capability. */ + setrlimit(RLIMIT_MEMLOCK, &new_limit); +} + void kvm__init_ram(struct kvm *kvm) { u64 phys_start, phys_size; @@ -49,8 +70,27 @@ void kvm__init_ram(struct kvm *kvm) kvm->ram_start = (void *)ALIGN((unsigned long)kvm->arch.ram_alloc_start, SZ_2M); - madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, - MADV_MERGEABLE); + /* + * Do not merge pages if this is a Realm. + * a) We cannot replace a page in realm stage2 without export/import + * + * Pin the realm memory until we have export/import, due to the same + * reason as above. + * + * Use mlock2(,,MLOCK_ONFAULT) to allow faulting in pages and thus + * allowing to lazily populate the PAR. + */ + if (kvm->cfg.arch.is_realm) { + int ret; + + try_increase_mlock_limit(kvm); + ret = mlock2(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, + MLOCK_ONFAULT); + if (ret) + die_perror("mlock2"); + } else { + madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, MADV_MERGEABLE); + } madvise(kvm->arch.ram_alloc_start, kvm->arch.ram_alloc_size, MADV_HUGEPAGE); -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel