From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marc Zyngier Subject: Re: [PATCH v5 22/26] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths Date: Thu, 7 Mar 2019 13:47:09 +0000 Message-ID: <5a3fda6d-5da8-8aa9-9b0e-36d6bcd7441b@arm.com> References: <1550519559-15915-1-git-send-email-Dave.Martin@arm.com> <1550519559-15915-23-git-send-email-Dave.Martin@arm.com> <34b7691f-7acb-c9aa-6e53-2857e429f148@arm.com> <20190226121347.GT3567@e103592.cambridge.arm.com> <20190301145503.GI3567@e103592.cambridge.arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6D8574A4A4 for ; Thu, 7 Mar 2019 08:47:15 -0500 (EST) Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZAlR7K4HV8N1 for ; Thu, 7 Mar 2019 08:47:14 -0500 (EST) Received: from foss.arm.com (foss.arm.com [217.140.101.70]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E63A74A455 for ; Thu, 7 Mar 2019 08:47:13 -0500 (EST) In-Reply-To: <20190301145503.GI3567@e103592.cambridge.arm.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu To: Dave Martin , Julien Thierry Cc: Okamoto Takayuki , Christoffer Dall , Ard Biesheuvel , Catalin Marinas , Will Deacon , Zhang Lei , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org List-Id: kvmarm@lists.cs.columbia.edu Hi Dave, On 01/03/2019 14:55, Dave Martin wrote: > [FYI Peter, your views on the proposal outlined torward the end of the > mail would be appreciated.] > > On Fri, Mar 01, 2019 at 01:28:19PM +0000, Julien Thierry wrote: [...] >> I might be over-thinking it, but if there is a way to move that >> finalization call I'd prefer that. > > I know what you mean, but there's not really another clear place to put > it either, right now. Making finalization a side-effect of only KVM_RUN > and KVM_GET_REG_LIST would mean that if userspace is squirting in the > state of a migrated vcpu, a dummy KVM_GET_REG_LIST call would need to be > inserted between setting KVM_REG_ARM64_SVE_VLS and setting the other SVE > regs. > > One thing we could to is get rid of the implicit finalization behaviour > completely, and require an explicit ioctl KVM_ARM_VCPU_FINALIZE to do +1. In the past, implicit finalization has been a major pain, and the "implicit GICv2 configuration" has been an absolute disaster. > this. This makes a clear barrier between reg writes that manipulate the > "physical" configuration of the vcpu, and reg reads/writes that merely > manipulate the vcpu's runtime state. Say: > > KVM_ARM_VCPU_INIT(features[] = KVM_ARM_VCPU_SVE) -> ok > KVM_RUN -> -EPERM (too early) > KVM_GET_REG_LIST -> -EPERM (too early) > ... > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_VLS) -> ok > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_ZREG(0, 0)) -> -EPERM (too early) > ... > KVM_RUN -> -EPERM (too early) > ... > KVM_ARM_VCPU_FINALIZE -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > ... > KVM_REG_REG_LIST -> ok > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_VLS) -> -EPERM (too late) > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_ZREG(0, 0)) -> ok > KVM_RUN -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > > This would change all existing kvm_arm_vcpu_finalize() calls to > > if (!kvm_arm_vcpu_finalized()) > return -EPERM; > > (or similar). I find this rather compelling, assuming we can easily map features that are associated to finalization. > > Without an affected vcpu feature enabled, for backwards compatibility > we would not require the KVM_ARM_VCPU_FINALIZE call (or perhaps even > forbid it, since userspace that wants to backwards compatible cannot > use the new ioctl anyway. I'm in two minds about this. Either way, > calling _FINALIZE on an old kvm is harmless, so long as userspace knows > to ignore this failure case.) > > The backwards-compatible flow would be: > > KVM_ARM_VCPU_INIT(features[] = 0) -> ok > ... > KVM_ARM_VCPU_FINALIZE -> ok (or -EPERM) > ... > KVM_RUN -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > > > Thoughts? This goes back to the above question: how do we ensure that userspace knows which features are subject to being finalized. As it is currently described, KVM_ARM_VCPU_FINALIZE isn't qualified by a feature set, nor can the kernel report what feature flags requires being finalized. It is also unclear to me whether all "finalizable" features would have the same init cycle. So either we take the simplest approach and make KVM_ARM_VCPU_FINALIZE strictly SVE specific (renaming it in the process), or it takes some argument that allow specifying the feature set it applies to. Maybe we can get away with the kernel not reporting which features require to be finalized as long that it is clearly documented. > I may not have time to rework this for -rc1, and being a late ABI > change I'd like to get others' views on it before heading too far into > the tunnel. Please do your best to have something as close as possible to the final version for -rc1. From that point on, I'd expect changes to be mostly cosmetic. Thanks, M. -- Jazz is not dead. It just smells funny... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B769C43381 for ; Thu, 7 Mar 2019 13:47:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 12E0220675 for ; Thu, 7 Mar 2019 13:47:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XwcL+LAw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 12E0220675 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Bs4eRNCSOZ9OjYyXh2pRX+oPHb3W6CVDzEd7q+0xDTs=; b=XwcL+LAw2Xc3dQ h+g4A26yCWcVtPMD1QjcjAX1hnAs1kyDfD/zg5D69YXS5klVdUJjy8BuxkO/9pXzeriDuB6BBouxE wSuvGSmbEkkvpd/tfN32mjMHdrApvuduHBE34omlL0spH5kJxl9MdtXXsfQkJfdPhYT6eqNG4aOqs AfLqj8nu/XhYaFje8lyMQOY6cGLE3RhvVln3r9xMn9/u39p7M5LIA38MRjLjfW0rfBXEOmTq+AOSf FiZsxmQh4UwZSBQcTIE82KwYPS/FBjO8EbAb48YpWhV7HYQHzeszLiGzIYroR8YgJTA3azeFwO4vR q76sCmGcBmXEeCAW1iog==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h1tMk-0007U8-5s; Thu, 07 Mar 2019 13:47:18 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h1tMg-0007TQ-Dp for linux-arm-kernel@lists.infradead.org; Thu, 07 Mar 2019 13:47:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A776AEBD; Thu, 7 Mar 2019 05:47:12 -0800 (PST) Received: from [10.1.196.92] (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D06FE3F703; Thu, 7 Mar 2019 05:47:10 -0800 (PST) Subject: Re: [PATCH v5 22/26] KVM: arm64/sve: Add pseudo-register for the guest's vector lengths To: Dave Martin , Julien Thierry References: <1550519559-15915-1-git-send-email-Dave.Martin@arm.com> <1550519559-15915-23-git-send-email-Dave.Martin@arm.com> <34b7691f-7acb-c9aa-6e53-2857e429f148@arm.com> <20190226121347.GT3567@e103592.cambridge.arm.com> <20190301145503.GI3567@e103592.cambridge.arm.com> From: Marc Zyngier Openpgp: preference=signencrypt Autocrypt: addr=marc.zyngier@arm.com; prefer-encrypt=mutual; keydata= mQINBE6Jf0UBEADLCxpix34Ch3kQKA9SNlVQroj9aHAEzzl0+V8jrvT9a9GkK+FjBOIQz4KE g+3p+lqgJH4NfwPm9H5I5e3wa+Scz9wAqWLTT772Rqb6hf6kx0kKd0P2jGv79qXSmwru28vJ t9NNsmIhEYwS5eTfCbsZZDCnR31J6qxozsDHpCGLHlYym/VbC199Uq/pN5gH+5JHZyhyZiNW ozUCjMqC4eNW42nYVKZQfbj/k4W9xFfudFaFEhAf/Vb1r6F05eBP1uopuzNkAN7vqS8XcgQH qXI357YC4ToCbmqLue4HK9+2mtf7MTdHZYGZ939OfTlOGuxFW+bhtPQzsHiW7eNe0ew0+LaL 3wdNzT5abPBscqXWVGsZWCAzBmrZato+Pd2bSCDPLInZV0j+rjt7MWiSxEAEowue3IcZA++7 ifTDIscQdpeKT8hcL+9eHLgoSDH62SlubO/y8bB1hV8JjLW/jQpLnae0oz25h39ij4ijcp8N t5slf5DNRi1NLz5+iaaLg4gaM3ywVK2VEKdBTg+JTg3dfrb3DH7ctTQquyKun9IVY8AsxMc6 lxl4HxrpLX7HgF10685GG5fFla7R1RUnW5svgQhz6YVU33yJjk5lIIrrxKI/wLlhn066mtu1 DoD9TEAjwOmpa6ofV6rHeBPehUwMZEsLqlKfLsl0PpsJwov8TQARAQABtCNNYXJjIFp5bmdp ZXIgPG1hcmMuenluZ2llckBhcm0uY29tPokCOwQTAQIAJQIbAwYLCQgHAwIGFQgCCQoLBBYC AwECHgECF4AFAk6NvYYCGQEACgkQI9DQutE9ekObww/+NcUATWXOcnoPflpYG43GZ0XjQLng LQFjBZL+CJV5+1XMDfz4ATH37cR+8gMO1UwmWPv5tOMKLHhw6uLxGG4upPAm0qxjRA/SE3LC 22kBjWiSMrkQgv5FDcwdhAcj8A+gKgcXBeyXsGBXLjo5UQOGvPTQXcqNXB9A3ZZN9vS6QUYN TXFjnUnzCJd+PVI/4jORz9EUVw1q/+kZgmA8/GhfPH3xNetTGLyJCJcQ86acom2liLZZX4+1 6Hda2x3hxpoQo7pTu+XA2YC4XyUstNDYIsE4F4NVHGi88a3N8yWE+Z7cBI2HjGvpfNxZnmKX 6bws6RQ4LHDPhy0yzWFowJXGTqM/e79c1UeqOVxKGFF3VhJJu1nMlh+5hnW4glXOoy/WmDEM UMbl9KbJUfo+GgIQGMp8mwgW0vK4HrSmevlDeMcrLdfbbFbcZLNeFFBn6KqxFZaTd+LpylIH bOPN6fy1Dxf7UZscogYw5Pt0JscgpciuO3DAZo3eXz6ffj2NrWchnbj+SpPBiH4srfFmHY+Y LBemIIOmSqIsjoSRjNEZeEObkshDVG5NncJzbAQY+V3Q3yo9og/8ZiaulVWDbcpKyUpzt7pv cdnY3baDE8ate/cymFP5jGJK++QCeA6u6JzBp7HnKbngqWa6g8qDSjPXBPCLmmRWbc5j0lvA 6ilrF8m5Ag0ETol/RQEQAM/2pdLYCWmf3rtIiP8Wj5NwyjSL6/UrChXtoX9wlY8a4h3EX6E3 64snIJVMLbyr4bwdmPKULlny7T/R8dx/mCOWu/DztrVNQiXWOTKJnd/2iQblBT+W5W8ep/nS w3qUIckKwKdplQtzSKeE+PJ+GMS+DoNDDkcrVjUnsoCEr0aK3cO6g5hLGu8IBbC1CJYSpple VVb/sADnWF3SfUvJ/l4K8Uk4B4+X90KpA7U9MhvDTCy5mJGaTsFqDLpnqp/yqaT2P7kyMG2E w+eqtVIqwwweZA0S+tuqput5xdNAcsj2PugVx9tlw/LJo39nh8NrMxAhv5aQ+JJ2I8UTiHLX QvoC0Yc/jZX/JRB5r4x4IhK34Mv5TiH/gFfZbwxd287Y1jOaD9lhnke1SX5MXF7eCT3cgyB+ hgSu42w+2xYl3+rzIhQqxXhaP232t/b3ilJO00ZZ19d4KICGcakeiL6ZBtD8TrtkRiewI3v0 o8rUBWtjcDRgg3tWx/PcJvZnw1twbmRdaNvsvnlapD2Y9Js3woRLIjSAGOijwzFXSJyC2HU1 AAuR9uo4/QkeIrQVHIxP7TJZdJ9sGEWdeGPzzPlKLHwIX2HzfbdtPejPSXm5LJ026qdtJHgz BAb3NygZG6BH6EC1NPDQ6O53EXorXS1tsSAgp5ZDSFEBklpRVT3E0NrDABEBAAGJAh8EGAEC AAkFAk6Jf0UCGwwACgkQI9DQutE9ekMLBQ//U+Mt9DtFpzMCIHFPE9nNlsCm75j22lNiw6mX mx3cUA3pl+uRGQr/zQC5inQNtjFUmwGkHqrAw+SmG5gsgnM4pSdYvraWaCWOZCQCx1lpaCOl MotrNcwMJTJLQGc4BjJyOeSH59HQDitKfKMu/yjRhzT8CXhys6R0kYMrEN0tbe1cFOJkxSbV 0GgRTDF4PKyLT+RncoKxQe8lGxuk5614aRpBQa0LPafkirwqkUtxsPnarkPUEfkBlnIhAR8L kmneYLu0AvbWjfJCUH7qfpyS/FRrQCoBq9QIEcf2v1f0AIpA27f9KCEv5MZSHXGCdNcbjKw1 39YxYZhmXaHFKDSZIC29YhQJeXWlfDEDq6nIhvurZy3mSh2OMQgaIoFexPCsBBOclH8QUtMk a3jW/qYyrV+qUq9Wf3SKPrXf7B3xB332jFCETbyZQXqmowV+2b3rJFRWn5hK5B+xwvuxKyGq qDOGjof2dKl2zBIxbFgOclV7wqCVkhxSJi/QaOj2zBqSNPXga5DWtX3ekRnJLa1+ijXxmdjz hApihi08gwvP5G9fNGKQyRETePEtEAWt0b7dOqMzYBYGRVr7uS4uT6WP7fzOwAJC4lU7ZYWZ yVshCa0IvTtp1085RtT3qhh9mobkcZ+7cQOY+Tx2RGXS9WeOh2jZjdoWUv6CevXNQyOUXMM= Organization: ARM Ltd Message-ID: <5a3fda6d-5da8-8aa9-9b0e-36d6bcd7441b@arm.com> Date: Thu, 7 Mar 2019 13:47:09 +0000 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <20190301145503.GI3567@e103592.cambridge.arm.com> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190307_054714_478051_CD4140C1 X-CRM114-Status: GOOD ( 24.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Okamoto Takayuki , Christoffer Dall , Ard Biesheuvel , Catalin Marinas , Will Deacon , Zhang Lei , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Dave, On 01/03/2019 14:55, Dave Martin wrote: > [FYI Peter, your views on the proposal outlined torward the end of the > mail would be appreciated.] > > On Fri, Mar 01, 2019 at 01:28:19PM +0000, Julien Thierry wrote: [...] >> I might be over-thinking it, but if there is a way to move that >> finalization call I'd prefer that. > > I know what you mean, but there's not really another clear place to put > it either, right now. Making finalization a side-effect of only KVM_RUN > and KVM_GET_REG_LIST would mean that if userspace is squirting in the > state of a migrated vcpu, a dummy KVM_GET_REG_LIST call would need to be > inserted between setting KVM_REG_ARM64_SVE_VLS and setting the other SVE > regs. > > One thing we could to is get rid of the implicit finalization behaviour > completely, and require an explicit ioctl KVM_ARM_VCPU_FINALIZE to do +1. In the past, implicit finalization has been a major pain, and the "implicit GICv2 configuration" has been an absolute disaster. > this. This makes a clear barrier between reg writes that manipulate the > "physical" configuration of the vcpu, and reg reads/writes that merely > manipulate the vcpu's runtime state. Say: > > KVM_ARM_VCPU_INIT(features[] = KVM_ARM_VCPU_SVE) -> ok > KVM_RUN -> -EPERM (too early) > KVM_GET_REG_LIST -> -EPERM (too early) > ... > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_VLS) -> ok > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_ZREG(0, 0)) -> -EPERM (too early) > ... > KVM_RUN -> -EPERM (too early) > ... > KVM_ARM_VCPU_FINALIZE -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > ... > KVM_REG_REG_LIST -> ok > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_VLS) -> -EPERM (too late) > KVM_SET_ONE_REG(KVM_REG_ARM64_SVE_ZREG(0, 0)) -> ok > KVM_RUN -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > > This would change all existing kvm_arm_vcpu_finalize() calls to > > if (!kvm_arm_vcpu_finalized()) > return -EPERM; > > (or similar). I find this rather compelling, assuming we can easily map features that are associated to finalization. > > Without an affected vcpu feature enabled, for backwards compatibility > we would not require the KVM_ARM_VCPU_FINALIZE call (or perhaps even > forbid it, since userspace that wants to backwards compatible cannot > use the new ioctl anyway. I'm in two minds about this. Either way, > calling _FINALIZE on an old kvm is harmless, so long as userspace knows > to ignore this failure case.) > > The backwards-compatible flow would be: > > KVM_ARM_VCPU_INIT(features[] = 0) -> ok > ... > KVM_ARM_VCPU_FINALIZE -> ok (or -EPERM) > ... > KVM_RUN -> ok > KVM_ARM_VCPU_FINALIZE -> -EPERM (too late) > > > Thoughts? This goes back to the above question: how do we ensure that userspace knows which features are subject to being finalized. As it is currently described, KVM_ARM_VCPU_FINALIZE isn't qualified by a feature set, nor can the kernel report what feature flags requires being finalized. It is also unclear to me whether all "finalizable" features would have the same init cycle. So either we take the simplest approach and make KVM_ARM_VCPU_FINALIZE strictly SVE specific (renaming it in the process), or it takes some argument that allow specifying the feature set it applies to. Maybe we can get away with the kernel not reporting which features require to be finalized as long that it is clearly documented. > I may not have time to rework this for -rc1, and being a late ABI > change I'd like to get others' views on it before heading too far into > the tunnel. Please do your best to have something as close as possible to the final version for -rc1. From that point on, I'd expect changes to be mostly cosmetic. Thanks, M. -- Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel