From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C196F2F2B; Mon, 20 Feb 2023 10:52:04 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C879915BF; Mon, 20 Feb 2023 02:52:40 -0800 (PST) Received: from [10.57.75.211] (unknown [10.57.75.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4EDEA3F703; Mon, 20 Feb 2023 02:51:53 -0800 (PST) Message-ID: <09cc1a81-d4a2-7db2-2add-d56121bbd7d7@arm.com> Date: Mon, 20 Feb 2023 10:51:51 +0000 Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Subject: Re: [RFC] Support for Arm CCA VMs on Linux Content-Language: en-US To: Itaru Kitayama Cc: Suzuki K Poulose , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andrew Jones , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu References: <20230127112248.136810-1-suzuki.poulose@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 17/02/2023 08:02, Itaru Kitayama wrote: > On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama wrote: >> >> On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts wrote: >>> >>> On 27/01/2023 11:22, Suzuki K Poulose wrote: >>>> [...] >>> >>>> Running the stack >>>> ==================== >>>> >>>> To run/test the stack, you would need the following components : >>>> >>>> 1) FVP Base AEM RevC model with FEAT_RME support [4] >>>> 2) TF-A firmware for EL3 [5] >>>> 3) TF-A RMM for R-EL2 [3] >>>> 4) Linux Kernel [6] >>>> 5) kvmtool [7] >>>> 6) kvm-unit-tests [8] >>>> >>>> Instructions for building the firmware components and running the model are >>>> available here [9]. Once, the host kernel is booted, a Realm can be launched by >>>> invoking the `lkvm` commad as follows: >>>> >>>> $ lkvm run --realm \ >>>> --measurement-algo=["sha256", "sha512"] \ >>>> --disable-sve \ >>>> >>>> >>>> Where: >>>> * --measurement-algo (Optional) specifies the algorithm selected for creating the >>>> initial measurements by the RMM for this Realm (defaults to sha256). >>>> * GICv3 is mandatory for the Realms. >>>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using >>>> --disable-sve >>>> >>>> You may also run the kvm-unit-tests inside the Realm world, using the similar >>>> options as above. >>> >>> Building all of these components and configuring the FVP correctly can be quite >>> tricky, so I thought I would plug a tool we have called Shrinkwrap, which can >>> simplify all of this. >>> >>> The tool accepts a yaml input configuration that describes how a set of >>> components should be built and packaged, and how the FVP should be configured >>> and booted. And by default, it uses a Docker container on its backend, which >>> contains all the required tools, including the FVP. You can optionally use >>> Podman or have it run on your native system if you prefer. It supports both >>> x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of >>> shell commands that would have been executed. >>> >>> It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM, >>> Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml >>> adds Hafnium and some demo SPs for the secure world (although since Hafnium >>> requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64 >>> build host). >>> >>> See the documentation [1] and repository [2] for more info. >>> >>> Brief instructions to get you up and running: >>> >>> # Install shrinkwrap. (I assume you have Docker installed): >>> sudo pip3 install pyyaml termcolor tuxmake >>> git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git >>> export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH >>> >>> # If running Python < 3.9: >>> sudo pip3 install graphlib-backport >>> >>> # Build all the CCA components: >>> shrinkwrap build cca-3world.yaml [--dry-run] >> >> This has been working on my Multipass instance on M1, thanks for the tool. >> >> Thanks, >> Itaru. > > It took a while though I've just booted an Ubuntu 22.10 disk image > with the cca-3world.yaml config on M1. That's good to hear - If you have any feedback (or patches ;-)) for Shrinkwrap that would improve the experience, do let me know! > > Thanks, > Itaru. > >> >>> >>> # Run the stack in the FVP: >>> shrinkwrap run cca-3world.yaml -r ROOTFS= [--dry-run] >>> >>> By default, building is done at ~/.shrinkwrap/build/cca-3world and the package >>> is created at ~/.shrinkwrap/package/cca-3world (this can be changed with >>> envvars). >>> >>> The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and >>> mount the provided rootfs. You will likely want to have copied the userspace >>> pieces into the rootfs before running, so you can create realms: >>> >>> - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support) >>> - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms) >>> - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests) >>> >>> Once the FVP is booted to a shell, you can do something like this to launch a >>> Linux guest in a realm: >>> >>> lkvm run --realm --disable-sve -c 1 -m 256 -k Image >>> >>> [1] https://shrinkwrap.docs.arm.com >>> [2] https://gitlab.arm.com/tooling/shrinkwrap >>> >>> >>> _______________________________________________ >>> linux-arm-kernel mailing list >>> linux-arm-kernel@lists.infradead.org >>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F4B9C636CC for ; Mon, 20 Feb 2023 10:53:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BBbXIxQNPBVhmPfmfeJaTRGkHVdOUN/x8IXtkI2S8B4=; b=nNMSDDJtnxT0YH tP58TiLGx7YTenRFG4RIdWU7CNfVu9T6e9ohBdSaGOba47LFVITL8XqZxYTe2rzqnbSsWzsoCnFgb 77+xAXiod0f6l039XYEpbQjsZoW6+GVUTEhvvleW/oTKkgflSyZRiWFASgqjxUOSxWnAnwYDYW4h+ 75rBEC1nCcaPp/p9EIyuCpd4+hC7Eyf3F6kxyFzZDYYO/YnScwkTby5RNAwNCgoBRhZp+RLVe1oHk Abc3Z3n6/wWzlMlb03qJqhTKEYHJObaOagySvXDNN5cvPFUpqEeOvO+hGnWK5PgFF7k5Rtsgmpwa3 RLHHA3NnC/0EHqTybeZA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pU3mZ-003jbf-7u; Mon, 20 Feb 2023 10:52:32 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pU3m7-003jQf-R4 for linux-arm-kernel@lists.infradead.org; Mon, 20 Feb 2023 10:52:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C879915BF; Mon, 20 Feb 2023 02:52:40 -0800 (PST) Received: from [10.57.75.211] (unknown [10.57.75.211]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4EDEA3F703; Mon, 20 Feb 2023 02:51:53 -0800 (PST) Message-ID: <09cc1a81-d4a2-7db2-2add-d56121bbd7d7@arm.com> Date: Mon, 20 Feb 2023 10:51:51 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.7.2 Subject: Re: [RFC] Support for Arm CCA VMs on Linux Content-Language: en-US To: Itaru Kitayama Cc: Suzuki K Poulose , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andrew Jones , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu References: <20230127112248.136810-1-suzuki.poulose@arm.com> From: Ryan Roberts In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230220_025204_912644_DCCCD4E1 X-CRM114-Status: GOOD ( 22.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 17/02/2023 08:02, Itaru Kitayama wrote: > On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama wrote: >> >> On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts wrote: >>> >>> On 27/01/2023 11:22, Suzuki K Poulose wrote: >>>> [...] >>> >>>> Running the stack >>>> ==================== >>>> >>>> To run/test the stack, you would need the following components : >>>> >>>> 1) FVP Base AEM RevC model with FEAT_RME support [4] >>>> 2) TF-A firmware for EL3 [5] >>>> 3) TF-A RMM for R-EL2 [3] >>>> 4) Linux Kernel [6] >>>> 5) kvmtool [7] >>>> 6) kvm-unit-tests [8] >>>> >>>> Instructions for building the firmware components and running the model are >>>> available here [9]. Once, the host kernel is booted, a Realm can be launched by >>>> invoking the `lkvm` commad as follows: >>>> >>>> $ lkvm run --realm \ >>>> --measurement-algo=["sha256", "sha512"] \ >>>> --disable-sve \ >>>> >>>> >>>> Where: >>>> * --measurement-algo (Optional) specifies the algorithm selected for creating the >>>> initial measurements by the RMM for this Realm (defaults to sha256). >>>> * GICv3 is mandatory for the Realms. >>>> * SVE is not yet supported in the TF-RMM, and thus must be disabled using >>>> --disable-sve >>>> >>>> You may also run the kvm-unit-tests inside the Realm world, using the similar >>>> options as above. >>> >>> Building all of these components and configuring the FVP correctly can be quite >>> tricky, so I thought I would plug a tool we have called Shrinkwrap, which can >>> simplify all of this. >>> >>> The tool accepts a yaml input configuration that describes how a set of >>> components should be built and packaged, and how the FVP should be configured >>> and booted. And by default, it uses a Docker container on its backend, which >>> contains all the required tools, including the FVP. You can optionally use >>> Podman or have it run on your native system if you prefer. It supports both >>> x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of >>> shell commands that would have been executed. >>> >>> It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM, >>> Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml >>> adds Hafnium and some demo SPs for the secure world (although since Hafnium >>> requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64 >>> build host). >>> >>> See the documentation [1] and repository [2] for more info. >>> >>> Brief instructions to get you up and running: >>> >>> # Install shrinkwrap. (I assume you have Docker installed): >>> sudo pip3 install pyyaml termcolor tuxmake >>> git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git >>> export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH >>> >>> # If running Python < 3.9: >>> sudo pip3 install graphlib-backport >>> >>> # Build all the CCA components: >>> shrinkwrap build cca-3world.yaml [--dry-run] >> >> This has been working on my Multipass instance on M1, thanks for the tool. >> >> Thanks, >> Itaru. > > It took a while though I've just booted an Ubuntu 22.10 disk image > with the cca-3world.yaml config on M1. That's good to hear - If you have any feedback (or patches ;-)) for Shrinkwrap that would improve the experience, do let me know! > > Thanks, > Itaru. > >> >>> >>> # Run the stack in the FVP: >>> shrinkwrap run cca-3world.yaml -r ROOTFS= [--dry-run] >>> >>> By default, building is done at ~/.shrinkwrap/build/cca-3world and the package >>> is created at ~/.shrinkwrap/package/cca-3world (this can be changed with >>> envvars). >>> >>> The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and >>> mount the provided rootfs. You will likely want to have copied the userspace >>> pieces into the rootfs before running, so you can create realms: >>> >>> - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support) >>> - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms) >>> - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests) >>> >>> Once the FVP is booted to a shell, you can do something like this to launch a >>> Linux guest in a realm: >>> >>> lkvm run --realm --disable-sve -c 1 -m 256 -k Image >>> >>> [1] https://shrinkwrap.docs.arm.com >>> [2] https://gitlab.arm.com/tooling/shrinkwrap >>> >>> >>> _______________________________________________ >>> linux-arm-kernel mailing list >>> linux-arm-kernel@lists.infradead.org >>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel