From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C8337F; Fri, 17 Feb 2023 08:02:34 +0000 (UTC) Received: by mail-pg1-f178.google.com with SMTP id o3so71885pga.1; Fri, 17 Feb 2023 00:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=5phApT5KIAC+uU4xT31TKIBplRF0IkqRKlCgfLJol30=; b=Xby5Ak4MGCvbfJQDH9RgIBpJ7ud8v/bYNBMmTNvPlgRfw4ceXutUEndOOBZ6FvKmne /m4Mmf5w0gBLFFk1FBOs2ho4xYFfIaI1zMG8mqRPXG3uj6K7N/ZPwX28SYTy3ekopzbs hq0/55FgbSOALCjOmBUQw7u+1wMO8jXxfkehHfXHOsMB1NiAaPEkidC282DKS+9Q6sw0 nXXwPcC7+YfDcYOplELpS7swqFr9W3r/o+Ct4Zte+dC4tcH6eRuY8CYnFevaCMBJ7I+z R4hPz97jW56z52zUlMAneBAmi49J72is2YmRZS/xSDbXP/FryfwHQg88N3TsX8Io9roQ sD0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=5phApT5KIAC+uU4xT31TKIBplRF0IkqRKlCgfLJol30=; b=07MdeAQJVWkruYBgPsq4VP29+jpe3Da6K2VMBkZL+HW8MnpXvB/Apfuksd/CwbcSvZ m50UJiEbFJpf7qfuvaIYD4Fow0ehbSkO3qNVSC5Muyc3xAfWaL4fhsHse3ZX30u2UryT 34zAcQQCUo0kmRSlyaKmS6fNZw1Jx0qBn2TNLMAhy/mNZ5WUxLAt/Sm+dZrn6xk8MQTU GnRHwqgrGFww+fxCRPb/DSLGTMxXcWQIsQh5STe5HEQv8mMz+QPTs6cuKJK1Pa47/AU7 zaAvAXq37FO6HJDbH8jmKC0VS0AMa1XiI59V6si6dBM2x1d07vuq+FdRc2KuMLyTX/ZQ 8vKA== X-Gm-Message-State: AO0yUKV4B3kGP3+Rp74Fi9UIzGRVSD27kI7/2q53Vvh4arzjqdTisBtv WLqbFsnpifHhbRnVQsr1LGTF901qiI1u3NwCUAk= X-Google-Smtp-Source: AK7set9c2bqfEa/WvsjgUSoNZf3yG4q6yN5HhL1cKAlR3naz5zYCmz1a+lbbOE7KDjDQKA7s66o9VdJMNMaoYdA/jQE= X-Received: by 2002:a63:3ec8:0:b0:4fb:8fb6:37e3 with SMTP id l191-20020a633ec8000000b004fb8fb637e3mr48435pga.6.1676620953779; Fri, 17 Feb 2023 00:02:33 -0800 (PST) Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20230127112248.136810-1-suzuki.poulose@arm.com> In-Reply-To: From: Itaru Kitayama Date: Fri, 17 Feb 2023 17:02:22 +0900 Message-ID: Subject: Re: [RFC] Support for Arm CCA VMs on Linux To: Ryan Roberts Cc: Suzuki K Poulose , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andrew Jones , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu Content-Type: text/plain; charset="UTF-8" On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama wrote: > > On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts wrote: > > > > On 27/01/2023 11:22, Suzuki K Poulose wrote: > > > [...] > > > > > Running the stack > > > ==================== > > > > > > To run/test the stack, you would need the following components : > > > > > > 1) FVP Base AEM RevC model with FEAT_RME support [4] > > > 2) TF-A firmware for EL3 [5] > > > 3) TF-A RMM for R-EL2 [3] > > > 4) Linux Kernel [6] > > > 5) kvmtool [7] > > > 6) kvm-unit-tests [8] > > > > > > Instructions for building the firmware components and running the model are > > > available here [9]. Once, the host kernel is booted, a Realm can be launched by > > > invoking the `lkvm` commad as follows: > > > > > > $ lkvm run --realm \ > > > --measurement-algo=["sha256", "sha512"] \ > > > --disable-sve \ > > > > > > > > > Where: > > > * --measurement-algo (Optional) specifies the algorithm selected for creating the > > > initial measurements by the RMM for this Realm (defaults to sha256). > > > * GICv3 is mandatory for the Realms. > > > * SVE is not yet supported in the TF-RMM, and thus must be disabled using > > > --disable-sve > > > > > > You may also run the kvm-unit-tests inside the Realm world, using the similar > > > options as above. > > > > Building all of these components and configuring the FVP correctly can be quite > > tricky, so I thought I would plug a tool we have called Shrinkwrap, which can > > simplify all of this. > > > > The tool accepts a yaml input configuration that describes how a set of > > components should be built and packaged, and how the FVP should be configured > > and booted. And by default, it uses a Docker container on its backend, which > > contains all the required tools, including the FVP. You can optionally use > > Podman or have it run on your native system if you prefer. It supports both > > x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of > > shell commands that would have been executed. > > > > It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM, > > Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml > > adds Hafnium and some demo SPs for the secure world (although since Hafnium > > requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64 > > build host). > > > > See the documentation [1] and repository [2] for more info. > > > > Brief instructions to get you up and running: > > > > # Install shrinkwrap. (I assume you have Docker installed): > > sudo pip3 install pyyaml termcolor tuxmake > > git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git > > export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH > > > > # If running Python < 3.9: > > sudo pip3 install graphlib-backport > > > > # Build all the CCA components: > > shrinkwrap build cca-3world.yaml [--dry-run] > > This has been working on my Multipass instance on M1, thanks for the tool. > > Thanks, > Itaru. It took a while though I've just booted an Ubuntu 22.10 disk image with the cca-3world.yaml config on M1. Thanks, Itaru. > > > > > # Run the stack in the FVP: > > shrinkwrap run cca-3world.yaml -r ROOTFS= [--dry-run] > > > > By default, building is done at ~/.shrinkwrap/build/cca-3world and the package > > is created at ~/.shrinkwrap/package/cca-3world (this can be changed with > > envvars). > > > > The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and > > mount the provided rootfs. You will likely want to have copied the userspace > > pieces into the rootfs before running, so you can create realms: > > > > - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support) > > - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms) > > - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests) > > > > Once the FVP is booted to a shell, you can do something like this to launch a > > Linux guest in a realm: > > > > lkvm run --realm --disable-sve -c 1 -m 256 -k Image > > > > [1] https://shrinkwrap.docs.arm.com > > [2] https://gitlab.arm.com/tooling/shrinkwrap > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 05E1AC05027 for ; Fri, 17 Feb 2023 08:04:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=s1b9nKWsnCcI6DPnibzmHgzKV5rtLOXOmkrymvt/btQ=; b=pIhX7rwilzMezR w7fcKfgJKt9dCRMMZ1wkzpu21zkpoQ0LrT5izHMIx+EX+8hKhYnsnkmXCPD3viDa5yikk2S8237YG V54oq+3SOh7B329d/WS8jYssorrGfjjqEMZFLN/LbM+TnFDOUdk4jLon8IjWltfs0v3f5/8exsojY qc9XsWW3lDEbibow20I/KypZFfZmAmwLZdhJx18JvajcWtHEhGRaOjWWZZ+yCPCTZOFk22l96D92Q +54b0mNzIJEGC9eRM3D5kuuZTCrc5UPBOUMfP1J/7vV0Wu1TOh6V2x9YM7X22PD2RGm8e7dsGbQX7 7nR2OgUxhyhV0IVil+1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSvhg-00DC9K-83; Fri, 17 Feb 2023 08:02:48 +0000 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pSvhW-00DC4f-3X for linux-arm-kernel@lists.infradead.org; Fri, 17 Feb 2023 08:02:40 +0000 Received: by mail-pg1-x533.google.com with SMTP id r11so74925pgu.9 for ; Fri, 17 Feb 2023 00:02:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=5phApT5KIAC+uU4xT31TKIBplRF0IkqRKlCgfLJol30=; b=Xby5Ak4MGCvbfJQDH9RgIBpJ7ud8v/bYNBMmTNvPlgRfw4ceXutUEndOOBZ6FvKmne /m4Mmf5w0gBLFFk1FBOs2ho4xYFfIaI1zMG8mqRPXG3uj6K7N/ZPwX28SYTy3ekopzbs hq0/55FgbSOALCjOmBUQw7u+1wMO8jXxfkehHfXHOsMB1NiAaPEkidC282DKS+9Q6sw0 nXXwPcC7+YfDcYOplELpS7swqFr9W3r/o+Ct4Zte+dC4tcH6eRuY8CYnFevaCMBJ7I+z R4hPz97jW56z52zUlMAneBAmi49J72is2YmRZS/xSDbXP/FryfwHQg88N3TsX8Io9roQ sD0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=5phApT5KIAC+uU4xT31TKIBplRF0IkqRKlCgfLJol30=; b=aWLLhiydcR6yA/RGpLBunW68vwzSh9q3cBnB8ImN3PTejzjg+mAZ/kBYhs7ZoT2kNq xyb/eVyZWZM7xfAxyj+VFqnDlggGEmOZbuzgJ2jUFXm+cs1R8WHAAfrzMUI6Mdr1lODi WraVovrveralxxW5aJ4c2d10yB/nNtstq5b2axonfI1oVARPT4MGNh/UVmFueAqcqgRL i3rpvPYzAAmxj9aEde/TRLVrbtRxXUmPxek0hrMSoTDGJvmefb1Eb8H4dmvF10b2sVnu +8KTwE70IhEvr6KAAhzy50vfIqwjRAQPUFQIsvAMZuIjt3GBecKxnmJSqbn1yL5WyS2/ hcAQ== X-Gm-Message-State: AO0yUKWuwkUPBSdHOWXNHpvGAPCo9uAMuerk/MgNN2ChG+rs4M/2oz6g 4a+r9c7aCTyT5PB0KjS7bnAKen3O1G8X51lAN7o= X-Google-Smtp-Source: AK7set9c2bqfEa/WvsjgUSoNZf3yG4q6yN5HhL1cKAlR3naz5zYCmz1a+lbbOE7KDjDQKA7s66o9VdJMNMaoYdA/jQE= X-Received: by 2002:a63:3ec8:0:b0:4fb:8fb6:37e3 with SMTP id l191-20020a633ec8000000b004fb8fb637e3mr48435pga.6.1676620953779; Fri, 17 Feb 2023 00:02:33 -0800 (PST) MIME-Version: 1.0 References: <20230127112248.136810-1-suzuki.poulose@arm.com> In-Reply-To: From: Itaru Kitayama Date: Fri, 17 Feb 2023 17:02:22 +0900 Message-ID: Subject: Re: [RFC] Support for Arm CCA VMs on Linux To: Ryan Roberts Cc: Suzuki K Poulose , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Andrew Jones , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Jean-Philippe Brucker , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230217_000238_219642_8EFFA219 X-CRM114-Status: GOOD ( 39.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sat, Feb 11, 2023 at 7:53 AM Itaru Kitayama wrote: > > On Sat, Feb 11, 2023 at 1:56 AM Ryan Roberts wrote: > > > > On 27/01/2023 11:22, Suzuki K Poulose wrote: > > > [...] > > > > > Running the stack > > > ==================== > > > > > > To run/test the stack, you would need the following components : > > > > > > 1) FVP Base AEM RevC model with FEAT_RME support [4] > > > 2) TF-A firmware for EL3 [5] > > > 3) TF-A RMM for R-EL2 [3] > > > 4) Linux Kernel [6] > > > 5) kvmtool [7] > > > 6) kvm-unit-tests [8] > > > > > > Instructions for building the firmware components and running the model are > > > available here [9]. Once, the host kernel is booted, a Realm can be launched by > > > invoking the `lkvm` commad as follows: > > > > > > $ lkvm run --realm \ > > > --measurement-algo=["sha256", "sha512"] \ > > > --disable-sve \ > > > > > > > > > Where: > > > * --measurement-algo (Optional) specifies the algorithm selected for creating the > > > initial measurements by the RMM for this Realm (defaults to sha256). > > > * GICv3 is mandatory for the Realms. > > > * SVE is not yet supported in the TF-RMM, and thus must be disabled using > > > --disable-sve > > > > > > You may also run the kvm-unit-tests inside the Realm world, using the similar > > > options as above. > > > > Building all of these components and configuring the FVP correctly can be quite > > tricky, so I thought I would plug a tool we have called Shrinkwrap, which can > > simplify all of this. > > > > The tool accepts a yaml input configuration that describes how a set of > > components should be built and packaged, and how the FVP should be configured > > and booted. And by default, it uses a Docker container on its backend, which > > contains all the required tools, including the FVP. You can optionally use > > Podman or have it run on your native system if you prefer. It supports both > > x86_64 and aarch64. And you can even run it in --dry-run mode to see the set of > > shell commands that would have been executed. > > > > It comes with two CCA configs out-of-the-box; cca-3world.yaml builds TF-A, RMM, > > Linux (for both host and guest), kvmtool and kvm-unit-tests. cca-4world.yaml > > adds Hafnium and some demo SPs for the secure world (although since Hafnium > > requires x86_64 to build, cca-4world.yaml doesn't currently work on an aarch64 > > build host). > > > > See the documentation [1] and repository [2] for more info. > > > > Brief instructions to get you up and running: > > > > # Install shrinkwrap. (I assume you have Docker installed): > > sudo pip3 install pyyaml termcolor tuxmake > > git clone https://git.gitlab.arm.com/tooling/shrinkwrap.git > > export PATH=$PWD/shrinkwrap/shrinkwrap:$PATH > > > > # If running Python < 3.9: > > sudo pip3 install graphlib-backport > > > > # Build all the CCA components: > > shrinkwrap build cca-3world.yaml [--dry-run] > > This has been working on my Multipass instance on M1, thanks for the tool. > > Thanks, > Itaru. It took a while though I've just booted an Ubuntu 22.10 disk image with the cca-3world.yaml config on M1. Thanks, Itaru. > > > > > # Run the stack in the FVP: > > shrinkwrap run cca-3world.yaml -r ROOTFS= [--dry-run] > > > > By default, building is done at ~/.shrinkwrap/build/cca-3world and the package > > is created at ~/.shrinkwrap/package/cca-3world (this can be changed with > > envvars). > > > > The 'run' command will boot TF-A, RMM and host Linux kernel in the FVP, and > > mount the provided rootfs. You will likely want to have copied the userspace > > pieces into the rootfs before running, so you can create realms: > > > > - ~/.shrinkwrap/package/cca-3world/Image (kernel with RMI and RSI support) > > - ~/.shrinkwrap/package/cca-3world/lkvm (kvmtool able to launch realms) > > - ~/.shrinkwrap/package/cca-3world/kvm-unit-tests.tgz (built kvm-unit-tests) > > > > Once the FVP is booted to a shell, you can do something like this to launch a > > Linux guest in a realm: > > > > lkvm run --realm --disable-sve -c 1 -m 256 -k Image > > > > [1] https://shrinkwrap.docs.arm.com > > [2] https://gitlab.arm.com/tooling/shrinkwrap > > > > > > _______________________________________________ > > linux-arm-kernel mailing list > > linux-arm-kernel@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel