From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6E4B1548D; Fri, 14 Jul 2023 16:28:30 +0000 (UTC) Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.201]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4R2cKr749Qz6J6f6; Sat, 15 Jul 2023 00:26:08 +0800 (CST) Received: from localhost (10.202.227.76) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Fri, 14 Jul 2023 17:28:26 +0100 Date: Fri, 14 Jul 2023 17:28:25 +0100 From: Jonathan Cameron To: Suzuki K Poulose CC: , , , , , Alexandru Elisei , Andrew Jones , "Catalin Marinas" , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , "Paolo Bonzini" , Quentin Perret , "Sean Christopherson" , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , Subject: Re: [RFC] Support for Arm CCA VMs on Linux Message-ID: <20230714172825.00003e81@Huawei.com> In-Reply-To: <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> References: <20230127112248.136810-1-suzuki.poulose@arm.com> <20230714144657.000064ef@Huawei.com> <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.202.227.76] X-ClientProxiedBy: lhrpeml500005.china.huawei.com (7.191.163.240) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected On Fri, 14 Jul 2023 16:03:37 +0100 Suzuki K Poulose wrote: > Hi Jonathan > > On 14/07/2023 14:46, Jonathan Cameron wrote: > > On Fri, 27 Jan 2023 11:22:48 +0000 > > Suzuki K Poulose wrote: > > > > > > Hi Suzuki, > > > > Looking at this has been on the backlog for a while from our side and we are finally > > getting to it. So before we dive in and given it's been 6 months, I wanted to check > > if you expect to post a new version shortly or if there is a rebased tree available? > > Thanks for your interest. We have been updating our trees to the latest > RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of > v6.5-rc1. We will post this as soon as we have all the components ready > (and the TF-RMM). At the earliest, this would be around early September. > > That said, the revised version will have the following changes : > - Changes to the Stage2 management > - Changes to RMM memory management for Realm > - PMU/SVE support > > Otherwise, most of the changes remain the same (e.g., UABI). Happy to > hear feedback on those areas. Hi Suzuki, Thanks for the update. If there is any chance of visibility of changes via a git tree etc that would be great in the meantime. If not, such is life and I'll try to wait patiently :) + we'll review the existing code. Jonathan > > > Kind regards > Suzuki > > > > > Jonathan > > > >> We are happy to announce the early RFC version of the Arm > >> Confidential Compute Architecture (CCA) support for the Linux > >> stack. The intention is to seek early feedback in the following areas: > >> * KVM integration of the Arm CCA > >> * KVM UABI for managing the Realms, seeking to generalise the operations > >> wherever possible with other Confidential Compute solutions. > >> Note: This version doesn't support Guest Private memory, which will be added > >> later (see below). > >> * Linux Guest support for Realms > >> > >> Arm CCA Introduction > >> ===================== > >> > >> The Arm CCA is a reference software architecture and implementation that builds > >> on the Realm Management Extension (RME), enabling the execution of Virtual > >> machines, while preventing access by more privileged software, such as hypervisor. > >> The Arm CCA allows the hypervisor to control the VM, but removes the right for > >> access to the code, register state or data that is used by VM. > >> More information on the architecture is available here[0]. > >> > >> Arm CCA Reference Software Architecture > >> > >> Realm World || Normal World || Secure World || > >> || | || || > >> EL0 x-------x || x----x | x------x || || > >> | Realm | || | | | | | || || > >> | | || | VM | | | | || || > >> ----| VM* |---------||-| |---| |-||----------------|| > >> | | || | | | | H | || || > >> EL1 x-------x || x----x | | | || || > >> ^ || | | o | || || > >> | || | | | || || > >> ------- R*------------------------| s -|--------------------- > >> S || | | || || > >> I || | t | || || > >> | || | | || || > >> v || x------x || || > >> EL2 RMM* || ^ || || > >> ^ || | || || > >> ========|=============================|======================== > >> | | SMC > >> x--------- *RMI* -------------x > >> > >> EL3 Root World > >> EL3 Firmware > >> =============================================================== > >> Where : > >> RMM - Realm Management Monitor > >> RMI - Realm Management Interface > >> RSI - Realm Service Interface > >> SMC - Secure Monitor Call > >> > >> RME introduces a new security state "Realm world", in addition to the > >> traditional Secure and Non-Secure states. The Arm CCA defines a new component, > >> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of > >> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at > >> system boot. > >> > >> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the > >> Normal world hypervisor to manage the VMs running in the Realm world (also called > >> Realms in short). These are exposed via SMC and are routed through the EL3 > >> firmwre. > >> The RMI interface includes: > >> - Move a physical page from the Normal world to the Realm world > >> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD) > >> - Creating VCPUs aka Realm Execution Context (REC), with initial register state. > >> - Create stage2 translation table at any level. > >> - Load initial images into Realm Memory from normal world memory > >> - Schedule RECs (vCPUs) and handle exits > >> - Inject virtual interrupts into the Realm > >> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM). > >> - Create "shared" mappings that can be accessed by VMM/Hyp. > >> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables) > >> > >> However v1.0 of RMM specifications doesn't support: > >> - Paging protected memory of a Realm VM. Thus the pages backing the protected > >> memory region must be pinned. > >> - Live migration of Realms. > >> - Trusted Device assignment. > >> - Physical interrupt backed Virtual interrupts for Realms > >> > >> RMM also provides certain services to the Realms via SMC, called Realm Service > >> Interface (RSI). These include: > >> - Realm Guest Configuration. > >> - Attestation & Measurement services > >> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page. > >> - Host Call service (Communication with the Normal world Hypervisor) > >> > >> The specifications for the RMM software is currently at *v1.0-Beta2* and the > >> latest version is available here [1]. > >> > >> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM - > >> available here [3]. > >> > >> Implementation > >> ================= > >> > >> This version of the stack is based on the RMM specification v1.0-Beta0[2], with > >> following exceptions : > >> - TF-RMM/KVM currently doesn't support the optional features of PMU, > >> SVE and Self-hosted debug (coming soon). > >> - The RSI_HOST_CALL structure alignment requirement is reduced to match > >> RMM v1.0 Beta1 > >> - RMI/RSI version numbers do not match the RMM spec. This will be > >> resolved once the spec/implementation is complete, across TF-RMM+Linux stack. > >> > >> We plan to update the stack to support the latest version of the RMMv1.0 spec > >> in the coming revisions. > >> > >> This release includes the following components : > >> > >> a) Linux Kernel > >> i) Host / KVM support - Support for driving the Realms via RMI. This is > >> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides > >> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page > >> size, matching the Stage2 granule supported by RMM. The VMM is responsible > >> for making sure the guest memory is locked. > >> > >> TODO: Guest Private memory[10] integration - We have been following the > >> series and support will be added once it is merged upstream. > >> > >> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at > >> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0 > >> only). All I/O are treated as non-secure/shared. > >> > >> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory > >> as mentioned above. > >> d) kvm-unit-tests - Support for running in Realms along with additional tests > >> for RSI ABI. > >> > >> Running the stack > >> ==================== > >> > >> To run/test the stack, you would need the following components : > >> > >> 1) FVP Base AEM RevC model with FEAT_RME support [4] > >> 2) TF-A firmware for EL3 [5] > >> 3) TF-A RMM for R-EL2 [3] > >> 4) Linux Kernel [6] > >> 5) kvmtool [7] > >> 6) kvm-unit-tests [8] > >> > >> Instructions for building the firmware components and running the model are > >> available here [9]. Once, the host kernel is booted, a Realm can be launched by > >> invoking the `lkvm` commad as follows: > >> > >> $ lkvm run --realm \ > >> --measurement-algo=["sha256", "sha512"] \ > >> --disable-sve \ > >> > >> > >> Where: > >> * --measurement-algo (Optional) specifies the algorithm selected for creating the > >> initial measurements by the RMM for this Realm (defaults to sha256). > >> * GICv3 is mandatory for the Realms. > >> * SVE is not yet supported in the TF-RMM, and thus must be disabled using > >> --disable-sve > >> > >> You may also run the kvm-unit-tests inside the Realm world, using the similar > >> options as above. > >> > >> > >> Links > >> ============ > >> > >> [0] Arm CCA Landing page (See Key Resources section for various documentations) > >> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture > >> > >> [1] RMM Specification Latest > >> https://developer.arm.com/documentation/den0137/latest > >> > >> [2] RMM v1.0-Beta0 specification > >> https://developer.arm.com/documentation/den0137/1-0bet0/ > >> > >> [3] Trusted Firmware RMM - TF-RMM > >> https://www.trustedfirmware.org/projects/tf-rmm/ > >> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git > >> > >> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux) > >> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms > >> > >> [5] Trusted Firmware for A class > >> https://www.trustedfirmware.org/projects/tf-a/ > >> > >> [6] Linux kernel support for Arm-CCA > >> https://gitlab.arm.com/linux-arm/linux-cca > >> Host Support branch: cca-host/rfc-v1 > >> Guest Support branch: cca-guest/rfc-v1 > >> > >> [7] kvmtool support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1 > >> > >> [8] kvm-unit-tests support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1 > >> > >> [9] Instructions for Building Firmware components and running the model, see > >> section 4.19.2 "Building and running TF-A with RME" > >> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme > >> > >> [10] fd based Guest Private memory for KVM > >> https://lkml.kernel.org/r/20221202061347.1070246-1-chao.p.peng@linux.intel.com > >> > >> Cc: Alexandru Elisei > >> Cc: Andrew Jones > >> Cc: Catalin Marinas > >> Cc: Chao Peng > >> Cc: Christoffer Dall > >> Cc: Fuad Tabba > >> Cc: James Morse > >> Cc: Jean-Philippe Brucker > >> Cc: Joey Gouly > >> Cc: Marc Zyngier > >> Cc: Mark Rutland > >> Cc: Oliver Upton > >> Cc: Paolo Bonzini > >> Cc: Quentin Perret > >> Cc: Sean Christopherson > >> Cc: Steven Price > >> Cc: Thomas Huth > >> Cc: Will Deacon > >> Cc: Zenghui Yu > >> To: linux-coco@lists.linux.dev > >> To: kvmarm@lists.linux.dev > >> Cc: kvmarm@lists.cs.columbia.edu > >> Cc: linux-arm-kernel@lists.infradead.org > >> To: linux-kernel@vger.kernel.org > >> To: kvm@vger.kernel.org > >> > >> _______________________________________________ > >> linux-arm-kernel mailing list > >> linux-arm-kernel@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 54603EB64DC for ; Fri, 14 Jul 2023 16:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Subject:CC:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9fN1am2lkretRSuFs8fmMlcGTgpBaxYVvVuVPFQfguE=; b=2KP3yZvbYkx0N/ jtdmmR9frYR5/34rFKIDhxJ7jz/EVhMEig8OtDvONLHQyH4IfUA3vg1ak3GvEfEqCLoNjAJhYMDbR YDsJoiJEwvmgygAZRJak3A4XMmghQ6LFaLAU4S6lmkn55dqWrpU7qHPwclTkms/rIotI1Cbvf4Qwi +KQcWUIMOmIEWevzQ4FG4FFkrcp5JCbKMF+bthm8ajCd4Jhvhif2Gad/9YWGxgTbFeGWqMx1fpqLA 9pjClkBXD02wDv5SNh2lZ8DUk3QZ4Pdfq5WqeGIUP+Ig+eu5yOU4W+3lgyG/BM5FPPWlGpNv+zYiW CgYNe76skQMfbeh5wJTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qKLes-006e8E-1v; Fri, 14 Jul 2023 16:28:42 +0000 Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qKLen-006e6q-0f for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 16:28:40 +0000 Received: from lhrpeml500005.china.huawei.com (unknown [172.18.147.201]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4R2cKr749Qz6J6f6; Sat, 15 Jul 2023 00:26:08 +0800 (CST) Received: from localhost (10.202.227.76) by lhrpeml500005.china.huawei.com (7.191.163.240) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Fri, 14 Jul 2023 17:28:26 +0100 Date: Fri, 14 Jul 2023 17:28:25 +0100 From: Jonathan Cameron To: Suzuki K Poulose CC: , , , , , Alexandru Elisei , Andrew Jones , "Catalin Marinas" , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , "Paolo Bonzini" , Quentin Perret , "Sean Christopherson" , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , Subject: Re: [RFC] Support for Arm CCA VMs on Linux Message-ID: <20230714172825.00003e81@Huawei.com> In-Reply-To: <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> References: <20230127112248.136810-1-suzuki.poulose@arm.com> <20230714144657.000064ef@Huawei.com> <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> Organization: Huawei Technologies Research and Development (UK) Ltd. X-Mailer: Claws Mail 4.1.0 (GTK 3.24.33; x86_64-w64-mingw32) MIME-Version: 1.0 X-Originating-IP: [10.202.227.76] X-ClientProxiedBy: lhrpeml500005.china.huawei.com (7.191.163.240) To lhrpeml500005.china.huawei.com (7.191.163.240) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230714_092837_660866_E658C26B X-CRM114-Status: GOOD ( 51.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 14 Jul 2023 16:03:37 +0100 Suzuki K Poulose wrote: > Hi Jonathan > > On 14/07/2023 14:46, Jonathan Cameron wrote: > > On Fri, 27 Jan 2023 11:22:48 +0000 > > Suzuki K Poulose wrote: > > > > > > Hi Suzuki, > > > > Looking at this has been on the backlog for a while from our side and we are finally > > getting to it. So before we dive in and given it's been 6 months, I wanted to check > > if you expect to post a new version shortly or if there is a rebased tree available? > > Thanks for your interest. We have been updating our trees to the latest > RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of > v6.5-rc1. We will post this as soon as we have all the components ready > (and the TF-RMM). At the earliest, this would be around early September. > > That said, the revised version will have the following changes : > - Changes to the Stage2 management > - Changes to RMM memory management for Realm > - PMU/SVE support > > Otherwise, most of the changes remain the same (e.g., UABI). Happy to > hear feedback on those areas. Hi Suzuki, Thanks for the update. If there is any chance of visibility of changes via a git tree etc that would be great in the meantime. If not, such is life and I'll try to wait patiently :) + we'll review the existing code. Jonathan > > > Kind regards > Suzuki > > > > > Jonathan > > > >> We are happy to announce the early RFC version of the Arm > >> Confidential Compute Architecture (CCA) support for the Linux > >> stack. The intention is to seek early feedback in the following areas: > >> * KVM integration of the Arm CCA > >> * KVM UABI for managing the Realms, seeking to generalise the operations > >> wherever possible with other Confidential Compute solutions. > >> Note: This version doesn't support Guest Private memory, which will be added > >> later (see below). > >> * Linux Guest support for Realms > >> > >> Arm CCA Introduction > >> ===================== > >> > >> The Arm CCA is a reference software architecture and implementation that builds > >> on the Realm Management Extension (RME), enabling the execution of Virtual > >> machines, while preventing access by more privileged software, such as hypervisor. > >> The Arm CCA allows the hypervisor to control the VM, but removes the right for > >> access to the code, register state or data that is used by VM. > >> More information on the architecture is available here[0]. > >> > >> Arm CCA Reference Software Architecture > >> > >> Realm World || Normal World || Secure World || > >> || | || || > >> EL0 x-------x || x----x | x------x || || > >> | Realm | || | | | | | || || > >> | | || | VM | | | | || || > >> ----| VM* |---------||-| |---| |-||----------------|| > >> | | || | | | | H | || || > >> EL1 x-------x || x----x | | | || || > >> ^ || | | o | || || > >> | || | | | || || > >> ------- R*------------------------| s -|--------------------- > >> S || | | || || > >> I || | t | || || > >> | || | | || || > >> v || x------x || || > >> EL2 RMM* || ^ || || > >> ^ || | || || > >> ========|=============================|======================== > >> | | SMC > >> x--------- *RMI* -------------x > >> > >> EL3 Root World > >> EL3 Firmware > >> =============================================================== > >> Where : > >> RMM - Realm Management Monitor > >> RMI - Realm Management Interface > >> RSI - Realm Service Interface > >> SMC - Secure Monitor Call > >> > >> RME introduces a new security state "Realm world", in addition to the > >> traditional Secure and Non-Secure states. The Arm CCA defines a new component, > >> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of > >> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at > >> system boot. > >> > >> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the > >> Normal world hypervisor to manage the VMs running in the Realm world (also called > >> Realms in short). These are exposed via SMC and are routed through the EL3 > >> firmwre. > >> The RMI interface includes: > >> - Move a physical page from the Normal world to the Realm world > >> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD) > >> - Creating VCPUs aka Realm Execution Context (REC), with initial register state. > >> - Create stage2 translation table at any level. > >> - Load initial images into Realm Memory from normal world memory > >> - Schedule RECs (vCPUs) and handle exits > >> - Inject virtual interrupts into the Realm > >> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM). > >> - Create "shared" mappings that can be accessed by VMM/Hyp. > >> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables) > >> > >> However v1.0 of RMM specifications doesn't support: > >> - Paging protected memory of a Realm VM. Thus the pages backing the protected > >> memory region must be pinned. > >> - Live migration of Realms. > >> - Trusted Device assignment. > >> - Physical interrupt backed Virtual interrupts for Realms > >> > >> RMM also provides certain services to the Realms via SMC, called Realm Service > >> Interface (RSI). These include: > >> - Realm Guest Configuration. > >> - Attestation & Measurement services > >> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page. > >> - Host Call service (Communication with the Normal world Hypervisor) > >> > >> The specifications for the RMM software is currently at *v1.0-Beta2* and the > >> latest version is available here [1]. > >> > >> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM - > >> available here [3]. > >> > >> Implementation > >> ================= > >> > >> This version of the stack is based on the RMM specification v1.0-Beta0[2], with > >> following exceptions : > >> - TF-RMM/KVM currently doesn't support the optional features of PMU, > >> SVE and Self-hosted debug (coming soon). > >> - The RSI_HOST_CALL structure alignment requirement is reduced to match > >> RMM v1.0 Beta1 > >> - RMI/RSI version numbers do not match the RMM spec. This will be > >> resolved once the spec/implementation is complete, across TF-RMM+Linux stack. > >> > >> We plan to update the stack to support the latest version of the RMMv1.0 spec > >> in the coming revisions. > >> > >> This release includes the following components : > >> > >> a) Linux Kernel > >> i) Host / KVM support - Support for driving the Realms via RMI. This is > >> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides > >> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page > >> size, matching the Stage2 granule supported by RMM. The VMM is responsible > >> for making sure the guest memory is locked. > >> > >> TODO: Guest Private memory[10] integration - We have been following the > >> series and support will be added once it is merged upstream. > >> > >> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at > >> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0 > >> only). All I/O are treated as non-secure/shared. > >> > >> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory > >> as mentioned above. > >> d) kvm-unit-tests - Support for running in Realms along with additional tests > >> for RSI ABI. > >> > >> Running the stack > >> ==================== > >> > >> To run/test the stack, you would need the following components : > >> > >> 1) FVP Base AEM RevC model with FEAT_RME support [4] > >> 2) TF-A firmware for EL3 [5] > >> 3) TF-A RMM for R-EL2 [3] > >> 4) Linux Kernel [6] > >> 5) kvmtool [7] > >> 6) kvm-unit-tests [8] > >> > >> Instructions for building the firmware components and running the model are > >> available here [9]. Once, the host kernel is booted, a Realm can be launched by > >> invoking the `lkvm` commad as follows: > >> > >> $ lkvm run --realm \ > >> --measurement-algo=["sha256", "sha512"] \ > >> --disable-sve \ > >> > >> > >> Where: > >> * --measurement-algo (Optional) specifies the algorithm selected for creating the > >> initial measurements by the RMM for this Realm (defaults to sha256). > >> * GICv3 is mandatory for the Realms. > >> * SVE is not yet supported in the TF-RMM, and thus must be disabled using > >> --disable-sve > >> > >> You may also run the kvm-unit-tests inside the Realm world, using the similar > >> options as above. > >> > >> > >> Links > >> ============ > >> > >> [0] Arm CCA Landing page (See Key Resources section for various documentations) > >> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture > >> > >> [1] RMM Specification Latest > >> https://developer.arm.com/documentation/den0137/latest > >> > >> [2] RMM v1.0-Beta0 specification > >> https://developer.arm.com/documentation/den0137/1-0bet0/ > >> > >> [3] Trusted Firmware RMM - TF-RMM > >> https://www.trustedfirmware.org/projects/tf-rmm/ > >> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git > >> > >> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux) > >> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms > >> > >> [5] Trusted Firmware for A class > >> https://www.trustedfirmware.org/projects/tf-a/ > >> > >> [6] Linux kernel support for Arm-CCA > >> https://gitlab.arm.com/linux-arm/linux-cca > >> Host Support branch: cca-host/rfc-v1 > >> Guest Support branch: cca-guest/rfc-v1 > >> > >> [7] kvmtool support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1 > >> > >> [8] kvm-unit-tests support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1 > >> > >> [9] Instructions for Building Firmware components and running the model, see > >> section 4.19.2 "Building and running TF-A with RME" > >> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme > >> > >> [10] fd based Guest Private memory for KVM > >> https://lkml.kernel.org/r/20221202061347.1070246-1-chao.p.peng@linux.intel.com > >> > >> Cc: Alexandru Elisei > >> Cc: Andrew Jones > >> Cc: Catalin Marinas > >> Cc: Chao Peng > >> Cc: Christoffer Dall > >> Cc: Fuad Tabba > >> Cc: James Morse > >> Cc: Jean-Philippe Brucker > >> Cc: Joey Gouly > >> Cc: Marc Zyngier > >> Cc: Mark Rutland > >> Cc: Oliver Upton > >> Cc: Paolo Bonzini > >> Cc: Quentin Perret > >> Cc: Sean Christopherson > >> Cc: Steven Price > >> Cc: Thomas Huth > >> Cc: Will Deacon > >> Cc: Zenghui Yu > >> To: linux-coco@lists.linux.dev > >> To: kvmarm@lists.linux.dev > >> Cc: kvmarm@lists.cs.columbia.edu > >> Cc: linux-arm-kernel@lists.infradead.org > >> To: linux-kernel@vger.kernel.org > >> To: kvm@vger.kernel.org > >> > >> _______________________________________________ > >> linux-arm-kernel mailing list > >> linux-arm-kernel@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qKLen-006e6q-0f for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 16:28:40 +0000 Date: Fri, 14 Jul 2023 17:28:25 +0100 From: Jonathan Cameron Subject: Re: [RFC] Support for Arm CCA VMs on Linux Message-ID: <20230714172825.00003e81@Huawei.com> In-Reply-To: <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> References: <20230127112248.136810-1-suzuki.poulose@arm.com> <20230714144657.000064ef@Huawei.com> <42cbffac-05a8-a279-9bdb-f76354c1a1b1@arm.com> MIME-Version: 1.0 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+lwn-linux-arm-kernel=archive.lwn.net@lists.infradead.org List-Archive: To: Suzuki K Poulose Cc: linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andrew Jones , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu Message-ID: <20230714162825.CWzyKp6Zs8JGtPwePeA81RsVifYMi9xk61lvwkBiv8w@z> On Fri, 14 Jul 2023 16:03:37 +0100 Suzuki K Poulose wrote: > Hi Jonathan > > On 14/07/2023 14:46, Jonathan Cameron wrote: > > On Fri, 27 Jan 2023 11:22:48 +0000 > > Suzuki K Poulose wrote: > > > > > > Hi Suzuki, > > > > Looking at this has been on the backlog for a while from our side and we are finally > > getting to it. So before we dive in and given it's been 6 months, I wanted to check > > if you expect to post a new version shortly or if there is a rebased tree available? > > Thanks for your interest. We have been updating our trees to the latest > RMM specification (v1.0-eac2 now) and also rebasing Linux/KVM on top of > v6.5-rc1. We will post this as soon as we have all the components ready > (and the TF-RMM). At the earliest, this would be around early September. > > That said, the revised version will have the following changes : > - Changes to the Stage2 management > - Changes to RMM memory management for Realm > - PMU/SVE support > > Otherwise, most of the changes remain the same (e.g., UABI). Happy to > hear feedback on those areas. Hi Suzuki, Thanks for the update. If there is any chance of visibility of changes via a git tree etc that would be great in the meantime. If not, such is life and I'll try to wait patiently :) + we'll review the existing code. Jonathan > > > Kind regards > Suzuki > > > > > Jonathan > > > >> We are happy to announce the early RFC version of the Arm > >> Confidential Compute Architecture (CCA) support for the Linux > >> stack. The intention is to seek early feedback in the following areas: > >> * KVM integration of the Arm CCA > >> * KVM UABI for managing the Realms, seeking to generalise the operations > >> wherever possible with other Confidential Compute solutions. > >> Note: This version doesn't support Guest Private memory, which will be added > >> later (see below). > >> * Linux Guest support for Realms > >> > >> Arm CCA Introduction > >> ===================== > >> > >> The Arm CCA is a reference software architecture and implementation that builds > >> on the Realm Management Extension (RME), enabling the execution of Virtual > >> machines, while preventing access by more privileged software, such as hypervisor. > >> The Arm CCA allows the hypervisor to control the VM, but removes the right for > >> access to the code, register state or data that is used by VM. > >> More information on the architecture is available here[0]. > >> > >> Arm CCA Reference Software Architecture > >> > >> Realm World || Normal World || Secure World || > >> || | || || > >> EL0 x-------x || x----x | x------x || || > >> | Realm | || | | | | | || || > >> | | || | VM | | | | || || > >> ----| VM* |---------||-| |---| |-||----------------|| > >> | | || | | | | H | || || > >> EL1 x-------x || x----x | | | || || > >> ^ || | | o | || || > >> | || | | | || || > >> ------- R*------------------------| s -|--------------------- > >> S || | | || || > >> I || | t | || || > >> | || | | || || > >> v || x------x || || > >> EL2 RMM* || ^ || || > >> ^ || | || || > >> ========|=============================|======================== > >> | | SMC > >> x--------- *RMI* -------------x > >> > >> EL3 Root World > >> EL3 Firmware > >> =============================================================== > >> Where : > >> RMM - Realm Management Monitor > >> RMI - Realm Management Interface > >> RSI - Realm Service Interface > >> SMC - Secure Monitor Call > >> > >> RME introduces a new security state "Realm world", in addition to the > >> traditional Secure and Non-Secure states. The Arm CCA defines a new component, > >> Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of > >> firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at > >> system boot. > >> > >> The RMM provides standard interfaces - Realm Management Interface (RMI) - to the > >> Normal world hypervisor to manage the VMs running in the Realm world (also called > >> Realms in short). These are exposed via SMC and are routed through the EL3 > >> firmwre. > >> The RMI interface includes: > >> - Move a physical page from the Normal world to the Realm world > >> - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD) > >> - Creating VCPUs aka Realm Execution Context (REC), with initial register state. > >> - Create stage2 translation table at any level. > >> - Load initial images into Realm Memory from normal world memory > >> - Schedule RECs (vCPUs) and handle exits > >> - Inject virtual interrupts into the Realm > >> - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM). > >> - Create "shared" mappings that can be accessed by VMM/Hyp. > >> - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables) > >> > >> However v1.0 of RMM specifications doesn't support: > >> - Paging protected memory of a Realm VM. Thus the pages backing the protected > >> memory region must be pinned. > >> - Live migration of Realms. > >> - Trusted Device assignment. > >> - Physical interrupt backed Virtual interrupts for Realms > >> > >> RMM also provides certain services to the Realms via SMC, called Realm Service > >> Interface (RSI). These include: > >> - Realm Guest Configuration. > >> - Attestation & Measurement services > >> - Managing the state of an Intermediate Physical Address (IPA aka GPA) page. > >> - Host Call service (Communication with the Normal world Hypervisor) > >> > >> The specifications for the RMM software is currently at *v1.0-Beta2* and the > >> latest version is available here [1]. > >> > >> The Trusted Firmware foundation has an implementation of the RMM - TF-RMM - > >> available here [3]. > >> > >> Implementation > >> ================= > >> > >> This version of the stack is based on the RMM specification v1.0-Beta0[2], with > >> following exceptions : > >> - TF-RMM/KVM currently doesn't support the optional features of PMU, > >> SVE and Self-hosted debug (coming soon). > >> - The RSI_HOST_CALL structure alignment requirement is reduced to match > >> RMM v1.0 Beta1 > >> - RMI/RSI version numbers do not match the RMM spec. This will be > >> resolved once the spec/implementation is complete, across TF-RMM+Linux stack. > >> > >> We plan to update the stack to support the latest version of the RMMv1.0 spec > >> in the coming revisions. > >> > >> This release includes the following components : > >> > >> a) Linux Kernel > >> i) Host / KVM support - Support for driving the Realms via RMI. This is > >> dependent on running in the Kernel at EL2 (aka VHE mode). Also provides > >> UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page > >> size, matching the Stage2 granule supported by RMM. The VMM is responsible > >> for making sure the guest memory is locked. > >> > >> TODO: Guest Private memory[10] integration - We have been following the > >> series and support will be added once it is merged upstream. > >> > >> ii) Guest support - Support for a Linux Kernel to run in the Realm VM at > >> Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0 > >> only). All I/O are treated as non-secure/shared. > >> > >> c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory > >> as mentioned above. > >> d) kvm-unit-tests - Support for running in Realms along with additional tests > >> for RSI ABI. > >> > >> Running the stack > >> ==================== > >> > >> To run/test the stack, you would need the following components : > >> > >> 1) FVP Base AEM RevC model with FEAT_RME support [4] > >> 2) TF-A firmware for EL3 [5] > >> 3) TF-A RMM for R-EL2 [3] > >> 4) Linux Kernel [6] > >> 5) kvmtool [7] > >> 6) kvm-unit-tests [8] > >> > >> Instructions for building the firmware components and running the model are > >> available here [9]. Once, the host kernel is booted, a Realm can be launched by > >> invoking the `lkvm` commad as follows: > >> > >> $ lkvm run --realm \ > >> --measurement-algo=["sha256", "sha512"] \ > >> --disable-sve \ > >> > >> > >> Where: > >> * --measurement-algo (Optional) specifies the algorithm selected for creating the > >> initial measurements by the RMM for this Realm (defaults to sha256). > >> * GICv3 is mandatory for the Realms. > >> * SVE is not yet supported in the TF-RMM, and thus must be disabled using > >> --disable-sve > >> > >> You may also run the kvm-unit-tests inside the Realm world, using the similar > >> options as above. > >> > >> > >> Links > >> ============ > >> > >> [0] Arm CCA Landing page (See Key Resources section for various documentations) > >> https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture > >> > >> [1] RMM Specification Latest > >> https://developer.arm.com/documentation/den0137/latest > >> > >> [2] RMM v1.0-Beta0 specification > >> https://developer.arm.com/documentation/den0137/1-0bet0/ > >> > >> [3] Trusted Firmware RMM - TF-RMM > >> https://www.trustedfirmware.org/projects/tf-rmm/ > >> GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git > >> > >> [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux) > >> https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms > >> > >> [5] Trusted Firmware for A class > >> https://www.trustedfirmware.org/projects/tf-a/ > >> > >> [6] Linux kernel support for Arm-CCA > >> https://gitlab.arm.com/linux-arm/linux-cca > >> Host Support branch: cca-host/rfc-v1 > >> Guest Support branch: cca-guest/rfc-v1 > >> > >> [7] kvmtool support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1 > >> > >> [8] kvm-unit-tests support for Arm CCA > >> https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca cca/rfc-v1 > >> > >> [9] Instructions for Building Firmware components and running the model, see > >> section 4.19.2 "Building and running TF-A with RME" > >> https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme > >> > >> [10] fd based Guest Private memory for KVM > >> https://lkml.kernel.org/r/20221202061347.1070246-1-chao.p.peng@linux.intel.com > >> > >> Cc: Alexandru Elisei > >> Cc: Andrew Jones > >> Cc: Catalin Marinas > >> Cc: Chao Peng > >> Cc: Christoffer Dall > >> Cc: Fuad Tabba > >> Cc: James Morse > >> Cc: Jean-Philippe Brucker > >> Cc: Joey Gouly > >> Cc: Marc Zyngier > >> Cc: Mark Rutland > >> Cc: Oliver Upton > >> Cc: Paolo Bonzini > >> Cc: Quentin Perret > >> Cc: Sean Christopherson > >> Cc: Steven Price > >> Cc: Thomas Huth > >> Cc: Will Deacon > >> Cc: Zenghui Yu > >> To: linux-coco@lists.linux.dev > >> To: kvmarm@lists.linux.dev > >> Cc: kvmarm@lists.cs.columbia.edu > >> Cc: linux-arm-kernel@lists.infradead.org > >> To: linux-kernel@vger.kernel.org > >> To: kvm@vger.kernel.org > >> > >> _______________________________________________ > >> linux-arm-kernel mailing list > >> linux-arm-kernel@lists.infradead.org > >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel