From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 488AD6FC0 for ; Fri, 3 Mar 2023 13:57:29 +0000 (UTC) Received: by mail-wr1-f49.google.com with SMTP id q16so2369701wrw.2 for ; Fri, 03 Mar 2023 05:57:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1677851847; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hrHDSk+aKQWG+5fv3AwNWB+cSl2suAJ8jLnBdEelixg=; b=wA0JLwiXTgP78aOg9Fy3S212QN3lFzf2xs0OkU4uR31gq+5+VXY70j7NEr2MI0kFNV aw799C8JRza89IaPD0o+Fs2abJDPkOBqyXasvRBRxd0f6rMlI8Ny5gNw1NpOpezNqLzA B0l375Z2imh4lIC0h9fvMZ8LoX3n6XaSES2/wj2PeCobmWfQDNYS7Y4kUyvxPWNYMgjW eWayThJDFs4mFcKfGpAJIXj/g+NjvilR1orOgShnTKWoprXtrzv95Jnv9y+vBN5qqvq8 Hx4zUOADzcq/Z9tx/fvFnlSCgniS4bsZmRdSEGN3H+bIIgSPkYfifD4O5vUULy29Mjzm gtQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677851847; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hrHDSk+aKQWG+5fv3AwNWB+cSl2suAJ8jLnBdEelixg=; b=1V7bgUc3wkATLk2Jc0gFn1ocLgyeXfjCuIBPVa+NK4lHZ21jT2bA0zkR19OeR2r0Hm ObA2mEwdvnIwKXCMWnqtKywfKX6DX7Zth7F7mZ0GFdrYiNAoEifwJ0gQB/mD5UZXxu7F AYJM205Vj1pfQlowAp4V3gSbYmmAwc6TuxHmWj7GUdm5NEIBqoN0k3FocPCYQY3R+Jz5 nSMh4AVtd9QMAFYA4x50axbFh4nimVtG74RRPtLWHLE9bJy+7idMuS6z+dve+PaQiNkD EMNxKiC40MKWpfHxGrc70bJ2SvrcxY4NQ8RVWStX7komNZuSOzWpWwYpx99KzF/SWKmm iXyg== X-Gm-Message-State: AO0yUKU9sp7a9abEvT2sbD9Lc72V1m6xTaL0TlvuUp/iCBjHk1mZ9Cbq s93HcbCkrxav8nbj/De+OatEzQ== X-Google-Smtp-Source: AK7set8CZLbAGLTlPG82iW3Ddrry4HX/pR8GcNB6glVvipnsuFTiB+l0Sif0R9izgtavLnk1COykyA== X-Received: by 2002:a5d:6545:0:b0:2c7:cc8:782c with SMTP id z5-20020a5d6545000000b002c70cc8782cmr4300861wrv.1.1677851847356; Fri, 03 Mar 2023 05:57:27 -0800 (PST) Received: from myrica (054592b0.skybroadband.com. [5.69.146.176]) by smtp.gmail.com with ESMTPSA id j6-20020a05600c42c600b003eb192787bfsm2445486wme.25.2023.03.03.05.57.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 05:57:26 -0800 (PST) Date: Fri, 3 Mar 2023 13:57:27 +0000 From: Jean-Philippe Brucker To: Cornelia Huck Cc: Suzuki K Poulose , Andrew Jones , Jean-Philippe Brucker , Itaru Kitayama , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu Subject: Re: [RFC] Support for Arm CCA VMs on Linux Message-ID: <20230303135727.GA473581@myrica> References: <20230303094618.GC361458@myrica> <1c91b777-982e-e71a-4829-51744e9555c5@arm.com> <20230303113905.GD361458@myrica> <20230303120800.ahtyc6et77ig4s27@orel> <2418536c-2658-18d6-f70c-c1af5adaa816@arm.com> <875ybi0ytc.fsf@redhat.com> Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <875ybi0ytc.fsf@redhat.com> On Fri, Mar 03, 2023 at 02:06:07PM +0100, Cornelia Huck wrote: > On Fri, Mar 03 2023, Suzuki K Poulose wrote: > > > On 03/03/2023 12:08, Andrew Jones wrote: > >> On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote: > >>> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote: > >>>> On 03/03/2023 09:46, Jean-Philippe Brucker wrote: > >>>>> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote: > >>>>>>>> I've tried your series in Real on CCA Host, but the KVM arch init > >>>>>>>> emits an Invalid argument error and terminates. > >>>>> > >>>>> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I > >>>>> didn't enable SVE in the host but shrinkwrap enables more options. > >>>> > >>>> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we > >>>> changed to using the VM instance and that would prevent using SVE, > >>>> until the RMM supports it. > >>> > >>> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or > >>> complementing it with a VM check in my next version, it seems to work > >>> (though I need to double-check the VM fd lifetime). Same goes for > >>> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment. > >> > >> I'm probably missing something since I haven't looked at this, but I'm > >> wondering what the "VM instance" check is and why it should be necessary. > > > > Userspace can check for a KVM_CAP_ on KVM fd (/dev/kvm) or a VM fd > > (returned via KVM_CREATE_VM). > > > >> Shouldn't KVM only expose capabilities which it can provide? I.e. the > > > > Correct, given now that we have different "types" of VMs possible on > > Arm64, (Normal vs Realm vs pVM), the capabilities of each of these > > could be different and thus we should use the KVM_CAP_ on the VM fd ( > > referred to VM instance above) and not the generic KVM fd. > > Using the vm ioctl is even encouraged in the doc for > KVM_CHECK_EXTENSION: > > "Based on their initialization different VMs may have different capabilities. > It is thus encouraged to use the vm ioctl to query for capabilities" > > It would probably make sense to convert QEMU to use the vm ioctl > everywhere (the wrapper falls back to the global version on failure > anyway.) Indeed, I'll see if I can come up with something generic, thanks for the pointer Thanks, Jean From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2D5B9C678D4 for ; Fri, 3 Mar 2023 13:58:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NA6C0hUC/K1cjNHSVTiWVpY98bzW9iVQD4JSW2Gx2FQ=; b=M0ciPvrwtfUioO 3rubWvOQXXokZ7YSOn28F1stPKyQ3/jVH13QS1arTgFQlXkEYREC15O4j1KJL0cLrmzDJdmYihcbO uboKTJo4+dCS5KlQ5cFlbyLa7k1Fsr2mRqDQ0Lc0SCTGoiIXaC93OOV1AtzTObD69nIl71cEWBirP 1dvYO7J+LJETLJhVLUxGaz0SJtKXM3x7WN3lbu8SPOtSEQOMjnZvt5LLYnYhWYtbNAUbxLV3ky2Ms XRIS7q9xeqhj8CWwe36kuVldaIMIROSNaELBvCoI/PtZiML1e5K0MbEpBi+3wY7E7IZ1TLjYIm5ae gFRAZ8rQXgRCR/y5sZAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pY5uf-006XW5-JQ; Fri, 03 Mar 2023 13:57:33 +0000 Received: from mail-wr1-x42f.google.com ([2a00:1450:4864:20::42f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pY5uc-006XVd-Fw for linux-arm-kernel@lists.infradead.org; Fri, 03 Mar 2023 13:57:32 +0000 Received: by mail-wr1-x42f.google.com with SMTP id t15so2348513wrz.7 for ; Fri, 03 Mar 2023 05:57:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1677851847; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hrHDSk+aKQWG+5fv3AwNWB+cSl2suAJ8jLnBdEelixg=; b=wA0JLwiXTgP78aOg9Fy3S212QN3lFzf2xs0OkU4uR31gq+5+VXY70j7NEr2MI0kFNV aw799C8JRza89IaPD0o+Fs2abJDPkOBqyXasvRBRxd0f6rMlI8Ny5gNw1NpOpezNqLzA B0l375Z2imh4lIC0h9fvMZ8LoX3n6XaSES2/wj2PeCobmWfQDNYS7Y4kUyvxPWNYMgjW eWayThJDFs4mFcKfGpAJIXj/g+NjvilR1orOgShnTKWoprXtrzv95Jnv9y+vBN5qqvq8 Hx4zUOADzcq/Z9tx/fvFnlSCgniS4bsZmRdSEGN3H+bIIgSPkYfifD4O5vUULy29Mjzm gtQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677851847; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hrHDSk+aKQWG+5fv3AwNWB+cSl2suAJ8jLnBdEelixg=; b=1TwUBqAr/23zu+2hhESTtaYciAgqzT5oMglckptsxx0eueK2W41m7na/verZnCxJFz ZLbbq1ielAfnFS9Dd0cjyQDlS2Qt4tISeW+X3hwJe+EHc4l7mUIFnfxezf7U/6qA+RV8 gGNWnDQkUsuu7oDSIdnRAA8ZqzrBHT+6qrvk0nSnTYW+K4bn6oEDmgr9EO0/LXxGnQ/p VYkYKFvenpTqk3ytTgiZJphxZ7/I0rY6eIYke3PnMu0b98hWqAVkQRaJ760MbdeoBJaH pC0EFReMwJJg3EUH61OuQrBb+ubBJKL+Bo5Y4r244IYUZUckCy4V4WP/StPINSNyS3rk MMPw== X-Gm-Message-State: AO0yUKXFD6Gku7RHtVntkZgcRDpfrM6EiyRh3QQS/bXbrdR9amUg6A64 ow9pWTUIr4IWobz2EFxMJHuxCQ== X-Google-Smtp-Source: AK7set8CZLbAGLTlPG82iW3Ddrry4HX/pR8GcNB6glVvipnsuFTiB+l0Sif0R9izgtavLnk1COykyA== X-Received: by 2002:a5d:6545:0:b0:2c7:cc8:782c with SMTP id z5-20020a5d6545000000b002c70cc8782cmr4300861wrv.1.1677851847356; Fri, 03 Mar 2023 05:57:27 -0800 (PST) Received: from myrica (054592b0.skybroadband.com. [5.69.146.176]) by smtp.gmail.com with ESMTPSA id j6-20020a05600c42c600b003eb192787bfsm2445486wme.25.2023.03.03.05.57.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 05:57:26 -0800 (PST) Date: Fri, 3 Mar 2023 13:57:27 +0000 From: Jean-Philippe Brucker To: Cornelia Huck Cc: Suzuki K Poulose , Andrew Jones , Jean-Philippe Brucker , Itaru Kitayama , linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, Alexandru Elisei , Catalin Marinas , Chao Peng , Christoffer Dall , Fuad Tabba , James Morse , Joey Gouly , Marc Zyngier , Mark Rutland , Oliver Upton , Paolo Bonzini , Quentin Perret , Sean Christopherson , Steven Price , Thomas Huth , Will Deacon , Zenghui Yu , kvmarm@lists.cs.columbia.edu Subject: Re: [RFC] Support for Arm CCA VMs on Linux Message-ID: <20230303135727.GA473581@myrica> References: <20230303094618.GC361458@myrica> <1c91b777-982e-e71a-4829-51744e9555c5@arm.com> <20230303113905.GD361458@myrica> <20230303120800.ahtyc6et77ig4s27@orel> <2418536c-2658-18d6-f70c-c1af5adaa816@arm.com> <875ybi0ytc.fsf@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <875ybi0ytc.fsf@redhat.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230303_055730_623609_8E97F428 X-CRM114-Status: GOOD ( 27.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Mar 03, 2023 at 02:06:07PM +0100, Cornelia Huck wrote: > On Fri, Mar 03 2023, Suzuki K Poulose wrote: > > > On 03/03/2023 12:08, Andrew Jones wrote: > >> On Fri, Mar 03, 2023 at 11:39:05AM +0000, Jean-Philippe Brucker wrote: > >>> On Fri, Mar 03, 2023 at 09:54:47AM +0000, Suzuki K Poulose wrote: > >>>> On 03/03/2023 09:46, Jean-Philippe Brucker wrote: > >>>>> On Thu, Mar 02, 2023 at 07:12:24AM +0900, Itaru Kitayama wrote: > >>>>>>>> I've tried your series in Real on CCA Host, but the KVM arch init > >>>>>>>> emits an Invalid argument error and terminates. > >>>>> > >>>>> This was the KVM_SET_ONE_REG for the SVE vector size. During my tests I > >>>>> didn't enable SVE in the host but shrinkwrap enables more options. > >>>> > >>>> Does the Qemu check for SVE capability on /dev/kvm ? For kvmtool, we > >>>> changed to using the VM instance and that would prevent using SVE, > >>>> until the RMM supports it. > >>> > >>> Yes, QEMU does check the SVE cap on /dev/kvm. I can propose changing it or > >>> complementing it with a VM check in my next version, it seems to work > >>> (though I need to double-check the VM fd lifetime). Same goes for > >>> KVM_CAP_STEAL_TIME, which I need to disable explicitly at the moment. > >> > >> I'm probably missing something since I haven't looked at this, but I'm > >> wondering what the "VM instance" check is and why it should be necessary. > > > > Userspace can check for a KVM_CAP_ on KVM fd (/dev/kvm) or a VM fd > > (returned via KVM_CREATE_VM). > > > >> Shouldn't KVM only expose capabilities which it can provide? I.e. the > > > > Correct, given now that we have different "types" of VMs possible on > > Arm64, (Normal vs Realm vs pVM), the capabilities of each of these > > could be different and thus we should use the KVM_CAP_ on the VM fd ( > > referred to VM instance above) and not the generic KVM fd. > > Using the vm ioctl is even encouraged in the doc for > KVM_CHECK_EXTENSION: > > "Based on their initialization different VMs may have different capabilities. > It is thus encouraged to use the vm ioctl to query for capabilities" > > It would probably make sense to convert QEMU to use the vm ioctl > everywhere (the wrapper falls back to the global version on failure > anyway.) Indeed, I'll see if I can come up with something generic, thanks for the pointer Thanks, Jean _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel