From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B25AC07E9B for ; Mon, 19 Jul 2021 10:48:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 62BD26101E for ; Mon, 19 Jul 2021 10:48:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236424AbhGSKHm (ORCPT ); Mon, 19 Jul 2021 06:07:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236735AbhGSKHX (ORCPT ); Mon, 19 Jul 2021 06:07:23 -0400 Received: from mail-ej1-x649.google.com (mail-ej1-x649.google.com [IPv6:2a00:1450:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0B7A3C061574 for ; Mon, 19 Jul 2021 02:55:59 -0700 (PDT) Received: by mail-ej1-x649.google.com with SMTP id t8-20020a1709063e48b0290501cd965554so5226660eji.8 for ; Mon, 19 Jul 2021 03:48:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=Zo9CxzDhUX2cZC2KYawro8tC3zo0gCGU9bsbwig5j0S3OAyeSDfj6KLCvhewOJXss3 ScGYWjjud+BfSjX1bw156hEet1J0T/Zqk+5fp1+CJaOunTaub7sgNqmoVkWv4/0g/rzU ZYfZpco3hJ67xUtZI9pTg9Vo5ayKvZcYbWYX32i7f9viXdeSPFpPtsmRiydpflNHKvaZ xJX3jC/HH6IpfzQrNolC63LcfYbtbE36mpB0gvl0+eobqmpRV69cepN1WIehO6m5AK3p /KS5RCENmYr5NttKLDRGsPdS7odJ5r7HBgi/UC8GW3gSGPXH/MBjxAGstC6Bk+7nxeXY xpPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=AJVZPDdLIcZw7ho5UZkg0FDcnrdtms+tKK6EL+Lado376aa9rnw7f8DSy6e2YPSVHe IWZj8nlH0x4fCAefYt+w5WruJPKP3UKTaPKFdyDDREeBInSfZ5CW+Pp/mx/nuwXzBTtS I7aQVcUeB9c2HlhcUlCN20xL5KczIQAZwlkgRT92x27+kS0tkYli0Yr7tQGnS5+9otCj OihoYOMNpCGvJEq95saOhfUsLNE87kU/Hl5uKdzGEyKcqbfmui6FGU8yhQWkkCmkKyBA Gjzz2mdciS/aUFWYfbiT6NY3KIU6boKqx/I3u8/IlODc5imfY2fKg2Exa48oWJaHUz4A sd+w== X-Gm-Message-State: AOAM532uDJDZZOwipClPl7HSfcx1LkPvM49wqtC/6BuzLgk0CKTqlL2Q C/EfTMhJm40C6oCcu/TV3bOqMBLD7Acg X-Google-Smtp-Source: ABdhPJyAyrN45gMOAlaStcGcdibOpYIOibhoQ9RmoK/yB/n96peiRQ6P/R0p8ef0yM8amQWGs/batNr68Iti X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1a96:a43f:6c2e:bb5a]) (user=qperret job=sendgmr) by 2002:a17:906:d8da:: with SMTP id re26mr26655364ejb.205.1626691681533; Mon, 19 Jul 2021 03:48:01 -0700 (PDT) Date: Mon, 19 Jul 2021 11:47:30 +0100 In-Reply-To: <20210719104735.3681732-1-qperret@google.com> Message-Id: <20210719104735.3681732-10-qperret@google.com> Mime-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.402.g57bb445576-goog Subject: [PATCH 09/14] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the page back with the host. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 ++- arch/arm64/kvm/hyp/nvhe/setup.c | 52 ++++++++++++++++--- 3 files changed, 51 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..b39047463075 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -23,6 +23,7 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index cdace80d3e28..6f28edf58407 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -235,6 +235,11 @@ static bool host_stage2_want_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot pro return prot != KVM_PGTABLE_PROT_RW; } +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot) +{ + return host_stage2_try(__host_stage2_idmap, start, end, prot); +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RW; @@ -250,7 +255,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..74dce83a6fad 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -83,10 +83,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +91,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +109,25 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownerhsip from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transtion requires matching changes in the host + * stage-2. This will done later (see finalize_mappings()) once the + * hyp_vmemmap is addressable. + */ + ret = pkvm_create_mappings(__start_rodata, __end_rodata, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + return 0; } @@ -148,6 +159,27 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_mappings(void) +{ + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RWX; + int ret; + + /* + * The host's .bss and .rodata sections are now conceptually owned by + * the hypervisor, so mark them as 'borrowed' in the host stage-2. We + * can safely use host_stage2_idmap_locked() at this point since the + * host stage-2 has not been enabled yet. + */ + prot |= KVM_PGTABLE_STATE_SHARED | KVM_PGTABLE_STATE_BORROWED; + ret = host_stage2_idmap_locked(__hyp_pa(__start_rodata), + __hyp_pa(__end_rodata), prot); + if (ret) + return ret; + + return host_stage2_idmap_locked(__hyp_pa(__hyp_bss_end), + __hyp_pa(__bss_stop), prot); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +199,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, -- 2.32.0.402.g57bb445576-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EECEDC07E9D for ; Mon, 19 Jul 2021 10:48:08 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id A46C161107 for ; Mon, 19 Jul 2021 10:48:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A46C161107 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 4666949E8C; Mon, 19 Jul 2021 06:48:08 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@google.com Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kiXqgm5rqKRo; Mon, 19 Jul 2021 06:48:06 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id DA8544B096; Mon, 19 Jul 2021 06:48:06 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 0686240808 for ; Mon, 19 Jul 2021 06:48:05 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2HjZQhJXHSWr for ; Mon, 19 Jul 2021 06:48:03 -0400 (EDT) Received: from mail-ej1-f74.google.com (mail-ej1-f74.google.com [209.85.218.74]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id AEEE24A1AF for ; Mon, 19 Jul 2021 06:48:02 -0400 (EDT) Received: by mail-ej1-f74.google.com with SMTP id r10-20020a170906350ab02904f12f4a8c69so5226774eja.12 for ; Mon, 19 Jul 2021 03:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=Zo9CxzDhUX2cZC2KYawro8tC3zo0gCGU9bsbwig5j0S3OAyeSDfj6KLCvhewOJXss3 ScGYWjjud+BfSjX1bw156hEet1J0T/Zqk+5fp1+CJaOunTaub7sgNqmoVkWv4/0g/rzU ZYfZpco3hJ67xUtZI9pTg9Vo5ayKvZcYbWYX32i7f9viXdeSPFpPtsmRiydpflNHKvaZ xJX3jC/HH6IpfzQrNolC63LcfYbtbE36mpB0gvl0+eobqmpRV69cepN1WIehO6m5AK3p /KS5RCENmYr5NttKLDRGsPdS7odJ5r7HBgi/UC8GW3gSGPXH/MBjxAGstC6Bk+7nxeXY xpPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=aT4NlVBmdMG0UO//sNUU+aRg3FsnEAwmXlR/LnuxaDR5AEUQoDwug4POrZNvuh4khr x28Z20tTycu4e5eVSAxHKRorRgYDGUa+zsDoaa2gSv71gQ9ru2O1pZcJzAyuXXLo7f4x C04Cksnek59N0q/L2+yr6f8gT9Q3ICmOrNrhdQIfifqEE7D0fiW1p2uz3dqngarmUIC3 XHxjKZkOuXl9ny/a1TYm8DFhnb8Z+D1HD3WKiIM0F0o4Ys8YvRscFY6gYLlyN+cJQWA0 0G4JxXYzXclp1nFg2lQhAfAVqJhIocsKhPwRPH7QAjXK2W8vRR4qjM2fcARFxgpvU/kR Rs4g== X-Gm-Message-State: AOAM530HhZF9cabdSp1WdZb14R/0R6Vax9oRQxeLepoz4EBtmHkLVQTX 7b54xGAanfcRMd4qXEr749XH6l1k1utc X-Google-Smtp-Source: ABdhPJyAyrN45gMOAlaStcGcdibOpYIOibhoQ9RmoK/yB/n96peiRQ6P/R0p8ef0yM8amQWGs/batNr68Iti X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1a96:a43f:6c2e:bb5a]) (user=qperret job=sendgmr) by 2002:a17:906:d8da:: with SMTP id re26mr26655364ejb.205.1626691681533; Mon, 19 Jul 2021 03:48:01 -0700 (PDT) Date: Mon, 19 Jul 2021 11:47:30 +0100 In-Reply-To: <20210719104735.3681732-1-qperret@google.com> Message-Id: <20210719104735.3681732-10-qperret@google.com> Mime-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.402.g57bb445576-goog Subject: [PATCH 09/14] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: qwandor@google.com, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kernel-team@android.com X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the page back with the host. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 ++- arch/arm64/kvm/hyp/nvhe/setup.c | 52 ++++++++++++++++--- 3 files changed, 51 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..b39047463075 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -23,6 +23,7 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index cdace80d3e28..6f28edf58407 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -235,6 +235,11 @@ static bool host_stage2_want_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot pro return prot != KVM_PGTABLE_PROT_RW; } +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot) +{ + return host_stage2_try(__host_stage2_idmap, start, end, prot); +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RW; @@ -250,7 +255,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..74dce83a6fad 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -83,10 +83,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +91,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +109,25 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownerhsip from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transtion requires matching changes in the host + * stage-2. This will done later (see finalize_mappings()) once the + * hyp_vmemmap is addressable. + */ + ret = pkvm_create_mappings(__start_rodata, __end_rodata, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + return 0; } @@ -148,6 +159,27 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_mappings(void) +{ + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RWX; + int ret; + + /* + * The host's .bss and .rodata sections are now conceptually owned by + * the hypervisor, so mark them as 'borrowed' in the host stage-2. We + * can safely use host_stage2_idmap_locked() at this point since the + * host stage-2 has not been enabled yet. + */ + prot |= KVM_PGTABLE_STATE_SHARED | KVM_PGTABLE_STATE_BORROWED; + ret = host_stage2_idmap_locked(__hyp_pa(__start_rodata), + __hyp_pa(__end_rodata), prot); + if (ret) + return ret; + + return host_stage2_idmap_locked(__hyp_pa(__hyp_bss_end), + __hyp_pa(__bss_stop), prot); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +199,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, -- 2.32.0.402.g57bb445576-goog _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51F98C07E9B for ; Mon, 19 Jul 2021 10:55:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1ED3E600EF for ; Mon, 19 Jul 2021 10:55:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1ED3E600EF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=pxwo4Vhde3TpNaPPY0gTAcc0JOGpBMuPm7LXIvqrZUM=; b=FbQ3jir0vPvkcEi9UTdoDiHyuM pNpa41WXRljBWIbQDJA0tfzCqLjMXK8d2ZsuWa6zBpNJ4Xt7ihpRvBev08gRtHiAk2817qRFu1WNX vasg/w9OvnXeXE9dq/GPiJo0eINQeexEPgJX/DU0ObjTN9V+tZpPZ2nptp022xz+JrzKLOxt3kKHL ZGFiXN6bj14elBmnbEpGWLlglhdLafkaTN4gKcHsYt39LHVE+pF2jtJ1CUWbUaiEoe/0iwY9Wkd9t v3Q6vR0D7BbdLh+k71G+AqT+glGXw35mEzpEEpQyXJw6DVCefHhrylWqS/jD0Y/eInhYyOarZXpeQ /QJSb3Jw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m5QtO-009MhI-RS; Mon, 19 Jul 2021 10:52:59 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m5Qod-009KZX-Vj for linux-arm-kernel@lists.infradead.org; Mon, 19 Jul 2021 10:48:05 +0000 Received: by mail-ed1-x54a.google.com with SMTP id f11-20020a0564021e8bb02903b46e290a49so2342053edf.17 for ; Mon, 19 Jul 2021 03:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=Zo9CxzDhUX2cZC2KYawro8tC3zo0gCGU9bsbwig5j0S3OAyeSDfj6KLCvhewOJXss3 ScGYWjjud+BfSjX1bw156hEet1J0T/Zqk+5fp1+CJaOunTaub7sgNqmoVkWv4/0g/rzU ZYfZpco3hJ67xUtZI9pTg9Vo5ayKvZcYbWYX32i7f9viXdeSPFpPtsmRiydpflNHKvaZ xJX3jC/HH6IpfzQrNolC63LcfYbtbE36mpB0gvl0+eobqmpRV69cepN1WIehO6m5AK3p /KS5RCENmYr5NttKLDRGsPdS7odJ5r7HBgi/UC8GW3gSGPXH/MBjxAGstC6Bk+7nxeXY xpPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dNi42ehjO4NUi77Uj+cYvrOU4Ks5G49fj0fQYIq6rpI=; b=Op3J0dwniqFz2DKZUvUWpFCezm8JVHQVul4q564jB/x7OkGJqH6eVzlJ5cObrSUxDJ PGRN47Ew+X59oqZ8CIGQCeDG6twzIvpFlxuiYA3i8iuxZor+znHGgkpdqIjukKW/xnpH 8ucFzc0k2bhrHjSYYpAd/d3lelI+/kVaa9X6WnUFvTOEsBxqM5z1joI962QXGZCOEq7L s01Qi3VgcqeCcAMVJ/E/gVzHHx2su5bGzsgxx2ROHBYtPk15Y1ZVxJ1qby8UGTXJfcNV 1wb1irj/elqAYffkd8xGa5eAgd8hSvalX40TVOaTybRplKm2TNmJMZ5lAN0wsbQRRDVm pWYg== X-Gm-Message-State: AOAM532P+EsBbHsQu81ufsIFpfIdAPmrSFNnDQ14GiUyEh042AIbow8l zca6qlF4L6LiUO+krzByZCIBBIKJHmKD X-Google-Smtp-Source: ABdhPJyAyrN45gMOAlaStcGcdibOpYIOibhoQ9RmoK/yB/n96peiRQ6P/R0p8ef0yM8amQWGs/batNr68Iti X-Received: from luke.lon.corp.google.com ([2a00:79e0:d:210:1a96:a43f:6c2e:bb5a]) (user=qperret job=sendgmr) by 2002:a17:906:d8da:: with SMTP id re26mr26655364ejb.205.1626691681533; Mon, 19 Jul 2021 03:48:01 -0700 (PDT) Date: Mon, 19 Jul 2021 11:47:30 +0100 In-Reply-To: <20210719104735.3681732-1-qperret@google.com> Message-Id: <20210719104735.3681732-10-qperret@google.com> Mime-Version: 1.0 References: <20210719104735.3681732-1-qperret@google.com> X-Mailer: git-send-email 2.32.0.402.g57bb445576-goog Subject: [PATCH 09/14] KVM: arm64: Mark host bss and rodata section as shared From: Quentin Perret To: maz@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, ardb@kernel.org, qwandor@google.com, tabba@google.com, dbrazdil@google.com, kernel-team@android.com, Quentin Perret X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210719_034804_093002_3362D260 X-CRM114-Status: GOOD ( 20.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As the hypervisor maps the host's .bss and .rodata sections in its stage-1, make sure to tag them as shared in hyp and host page-tables. But since the hypervisor relies on the presence of these mappings, we cannot let the host in complete control of the memory regions -- it must not unshare or donate them to another entity for example. To prevent this, let's transfer the ownership of those ranges to the hypervisor itself, and share the page back with the host. Signed-off-by: Quentin Perret --- arch/arm64/kvm/hyp/include/nvhe/mem_protect.h | 1 + arch/arm64/kvm/hyp/nvhe/mem_protect.c | 7 ++- arch/arm64/kvm/hyp/nvhe/setup.c | 52 ++++++++++++++++--- 3 files changed, 51 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h index 9c227d87c36d..b39047463075 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mem_protect.h @@ -23,6 +23,7 @@ extern struct host_kvm host_kvm; int __pkvm_prot_finalize(void); int __pkvm_mark_hyp(phys_addr_t start, phys_addr_t end); +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot); int kvm_host_prepare_stage2(void *pgt_pool_base); void handle_host_mem_abort(struct kvm_cpu_context *host_ctxt); diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvhe/mem_protect.c index cdace80d3e28..6f28edf58407 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -235,6 +235,11 @@ static bool host_stage2_want_pte_cb(u64 addr, u64 end, enum kvm_pgtable_prot pro return prot != KVM_PGTABLE_PROT_RW; } +int host_stage2_idmap_locked(u64 start, u64 end, enum kvm_pgtable_prot prot) +{ + return host_stage2_try(__host_stage2_idmap, start, end, prot); +} + static int host_stage2_idmap(u64 addr) { enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RW; @@ -250,7 +255,7 @@ static int host_stage2_idmap(u64 addr) if (ret) goto unlock; - ret = host_stage2_try(__host_stage2_idmap, range.start, range.end, prot); + ret = host_stage2_idmap_locked(range.start, range.end, prot); unlock: hyp_spin_unlock(&host_kvm.lock); diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 0b574d106519..74dce83a6fad 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -83,10 +83,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__start_rodata, __end_rodata, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(__hyp_rodata_start, __hyp_rodata_end, PAGE_HYP_RO); if (ret) return ret; @@ -95,10 +91,6 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, if (ret) return ret; - ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, PAGE_HYP_RO); - if (ret) - return ret; - ret = pkvm_create_mappings(virt, virt + size, PAGE_HYP); if (ret) return ret; @@ -117,6 +109,25 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, return ret; } + /* + * Map the host's .bss and .rodata sections RO in the hypervisor, but + * transfer the ownerhsip from the host to the hypervisor itself to + * make sure it can't be donated or shared with another entity. + * + * The ownership transtion requires matching changes in the host + * stage-2. This will done later (see finalize_mappings()) once the + * hyp_vmemmap is addressable. + */ + ret = pkvm_create_mappings(__start_rodata, __end_rodata, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + + ret = pkvm_create_mappings(__hyp_bss_end, __bss_stop, + PAGE_HYP_RO | KVM_PGTABLE_STATE_SHARED); + if (ret) + return ret; + return 0; } @@ -148,6 +159,27 @@ static void hpool_put_page(void *addr) hyp_put_page(&hpool, addr); } +static int finalize_mappings(void) +{ + enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_RWX; + int ret; + + /* + * The host's .bss and .rodata sections are now conceptually owned by + * the hypervisor, so mark them as 'borrowed' in the host stage-2. We + * can safely use host_stage2_idmap_locked() at this point since the + * host stage-2 has not been enabled yet. + */ + prot |= KVM_PGTABLE_STATE_SHARED | KVM_PGTABLE_STATE_BORROWED; + ret = host_stage2_idmap_locked(__hyp_pa(__start_rodata), + __hyp_pa(__end_rodata), prot); + if (ret) + return ret; + + return host_stage2_idmap_locked(__hyp_pa(__hyp_bss_end), + __hyp_pa(__bss_stop), prot); +} + void __noreturn __pkvm_init_finalise(void) { struct kvm_host_data *host_data = this_cpu_ptr(&kvm_host_data); @@ -167,6 +199,10 @@ void __noreturn __pkvm_init_finalise(void) if (ret) goto out; + ret = finalize_mappings(); + if (ret) + goto out; + pkvm_pgtable_mm_ops = (struct kvm_pgtable_mm_ops) { .zalloc_page = hyp_zalloc_hyp_page, .phys_to_virt = hyp_phys_to_virt, -- 2.32.0.402.g57bb445576-goog _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel