[RFC PATCH v3 24/24] KVM: arm64/sve: Document KVM API extensions for SVE

[Dave Martin <Dave.Martin@arm.com>]

This patch adds sections to the KVM API documentation describing
the extensions for supporting the Scalable Vector Extension (SVE)
in guests.

Signed-off-by: Dave Martin <Dave.Martin@arm.com>

---

Changes since RFC v2:

 * Fix documentation regarding which SVE Zn register bits must be
   accessed in order to get at Vn on an SVE-enabled vcpu.

 * Update documentation to describe KVM_VM_TYPE_ARM_SVE and related
   API changes.

 * Move comment about max_vq==0 semantics for
   KVM_ARM_SVE_CONFIG_GET to the correct place.

 * Miscellaneous wording updates to describe the new initialisation
   semantics.

The documentation remains inaccurate / misleading in places.

Since the current API design is still needs discussion I expect another
respin with API changes.  I didn't want to waste effort on a
time-consuming documentation rewrite in the meantime.
---
 Documentation/virtual/kvm/api.txt | 176 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 173 insertions(+), 3 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 5f3c525..2a0605d 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -154,6 +154,13 @@ size of the address translated by the stage2 level (guest physical to
 host physical address translations).
 
 
+Also on arm64, if capability KVM_CAP_ARM_SVE is present then the
+KVM_VM_TYPE_ARM_SVE flag may be set in the machine type identifier to
+enable the KVM API extensions for the Arm Scalable Vector Extension
+(SVE) for the created VM.  This is required in order to create vcpus
+that support SVE.  See section 4.117 (KVM_ARM_SVE_CONFIG) for details.
+
+
 4.3 KVM_GET_MSR_INDEX_LIST, KVM_GET_MSR_FEATURE_INDEX_LIST
 
 Capability: basic, KVM_CAP_GET_MSR_FEATURES for KVM_GET_MSR_FEATURE_INDEX_LIST
@@ -2099,13 +2106,21 @@ Specifically:
   0x6030 0000 0010 004c SPSR_UND    64  spsr[KVM_SPSR_UND]
   0x6030 0000 0010 004e SPSR_IRQ    64  spsr[KVM_SPSR_IRQ]
   0x6060 0000 0010 0050 SPSR_FIQ    64  spsr[KVM_SPSR_FIQ]
-  0x6040 0000 0010 0054 V0         128  fp_regs.vregs[0]
-  0x6040 0000 0010 0058 V1         128  fp_regs.vregs[1]
+  0x6040 0000 0010 0054 V0         128  fp_regs.vregs[0]    (*)
+  0x6040 0000 0010 0058 V1         128  fp_regs.vregs[1]    (*)
     ...
-  0x6040 0000 0010 00d0 V31        128  fp_regs.vregs[31]
+  0x6040 0000 0010 00d0 V31        128  fp_regs.vregs[31]   (*)
   0x6020 0000 0010 00d4 FPSR        32  fp_regs.fpsr
   0x6020 0000 0010 00d5 FPCR        32  fp_regs.fpcr
 
+(*) These encodings are not accepted for SVE-enabled vcpus.  See
+    KVM_ARM_SVE_CONFIG for details of how SVE support is configured for
+    a vcpu.
+
+    The equivalent register content can be accessed via bits [127:0] of
+    the corresponding SVE Zn registers instead for vcpus that have SVE
+    enabled (see below).
+
 arm64 CCSIDR registers are demultiplexed by CSSELR value:
   0x6020 0000 0011 00 <csselr:8>
 
@@ -2115,6 +2130,14 @@ arm64 system registers have the following id bit patterns:
 arm64 firmware pseudo-registers have the following bit pattern:
   0x6030 0000 0014 <regno:16>
 
+arm64 SVE registers have the following bit patterns:
+  0x6080 0000 0015 00 <n:5> <slice:5>   Zn bits[2048*slice + 2047 : 2048*slice]
+  0x6050 0000 0015 04 <n:4> <slice:5>   Pn bits[256*slice + 255 : 256*slice]
+  0x6050 0000 0015 060 <slice:5>        FFR bits[256*slice + 255 : 256*slice]
+
+  These registers are only accessible on SVE-enabled vcpus.  See
+  KVM_ARM_SVE_CONFIG for details.
+
 
 MIPS registers are mapped using the lower 32 bits.  The upper 16 of that is
 the register group type:
@@ -2632,6 +2655,8 @@ Parameters: struct kvm_vcpu_init (in)
 Returns: 0 on success; -1 on error
 Errors:
   EINVAL:    the target is unknown, or the combination of features is invalid.
+  EBADFD:    further configuration required before this ioctl (see sections
+             4.2 KVM_CREATE_VM, 4.117 KVM_ARM_SVE_CONFIG)
   ENOENT:    a features bit specified is unknown.
 
 This tells KVM what type of CPU to present to the guest, and what
@@ -2694,6 +2719,8 @@ Returns: 0 on success; -1 on error
 Errors:
   E2BIG:     the reg index list is too big to fit in the array specified by
              the user (the number required will be written into n).
+  EBADFD:    (arm64) further configuration required before this ioctl (see
+             sections 4.2 KVM_CREATE_VM, 4.117 KVM_ARM_SVE_CONFIG)
 
 struct kvm_reg_list {
 	__u64 n; /* number of registers in reg[] */
@@ -3777,6 +3804,149 @@ Coalesced pio is based on coalesced mmio. There is little difference
 between coalesced mmio and pio except that coalesced pio records accesses
 to I/O ports.
 
+4.117 KVM_ARM_SVE_CONFIG
+
+Capability: KVM_CAP_ARM_SVE
+Architectures: arm64
+Type: vm and vcpu ioctl
+Parameters: struct kvm_sve_vls (in/out)
+Returns: 0 on success
+Errors:
+  EINVAL:    Unrecognised subcommand or bad arguments, or SVE API not enabled
+             (see section 2.4, KVM_CREATE_VM)
+  EBADFD:    vcpu in wrong state for request
+             (KVM_ARM_SVE_CONFIG_SET, KVM_ARM_SVE_CONFIG_SET)
+  ENOMEM:    Out of memory
+  EFAULT:    Bad user address
+
+struct kvm_sve_vls {
+	__u16 cmd;
+	__u16 max_vq;
+	__u16 _reserved[2];
+	__u64 required_vqs[8];
+};
+
+General:
+
+In addition to requiring KVM_CAP_ARM_SVE, this ioctl is only available
+when the SVE API extensions have been enabled by creating the
+corresponding VM with the KVM_VM_TYPE_ARM_SVE flag.  See section 4.2
+(KVM_CREATE_VM) for details.
+
+This should be the first vcpu ioctl issued after creating the vcpu via
+KVM_CREATE_VCPU: until SVE configuration for the vcpu is completed via a
+successful KVM_ARM_SVE_CONFIG_SET subcommand (see below), non-trivial
+vcpu ioctls will be rejected with EBADFD or another appropriate error.
+
+Parameters:
+
+cmd: This ioctl supports a few different subcommands, selected by the
+value of cmd (described in detail in the following sections).
+
+_reserved[]: these fields may be meaningful to later kernels.  For
+forward compatibility, they must be zeroed before invoking this ioctl
+for the first time on a given struct kvm_sve_vls object.  (So, memset()
+it to zero before first use, or allocate with calloc() for example.)
+
+max_vq, required_vqs[]:
+
+If max_vq == 0, SVE is disabled for this vcpu.
+
+Otherwise, max_vq and required_vqs[] encode a set of SVE vector
+lengths to attempt to configure for this vcpu.  The set is encoded as
+follows:
+
+If (a * 64 + b + 1) <= max_vq, then the bit represented by
+
+    required_vqs[a] & ((__u64)1 << b)
+
+(where a is in the range 0..7 and b is in the range 0..63)
+indicates that the vector length (a * 64 + b + 1) * 128 bits is
+supported (KVM_ARM_SVE_CONFIG_QUERY, KVM_ARM_SVE_CONFIG_GET) or required
+(KVM_ARM_SVE_CONFIG_SET).
+
+If (a * 64 + b + 1) > max_vq, then the vector length
+(a * 64 + b + 1) * 128 bits is unsupported or prohibited respectively.
+In other words, only the first max_vq bits in required_vqs[] are
+significant; remaining bits are implicitly treated as if they were zero.
+
+max_vq must be in the range SVE_VQ_MIN (1) to SVE_VQ_MAX (512).
+
+See Documentation/arm64/sve.txt for an explanation of vector lengths and
+the meaning associated with "VQ".
+
+Subcommands:
+
+/* values for cmd: */
+#define KVM_ARM_SVE_CONFIG_QUERY	0 /* query what the host can support */
+#define KVM_ARM_SVE_CONFIG_SET		1 /* enable SVE for vcpu and set VLs */
+#define KVM_ARM_SVE_CONFIG_GET		2 /* read the set of VLs for a vcpu */
+
+Subcommand details:
+
+4.117.1 KVM_ARM_SVE_CONFIG_QUERY
+Type: vm and vcpu
+
+Retrieve the full set of SVE vector lengths available for use by KVM
+guests on this host.  The result is independent of which vcpu this
+command is invoked on.  As a convenience, it may also be invoked on a
+vm file descriptor, eliminating the need to create a vcpu first.
+
+4.117.2 KVM_ARM_SVE_CONFIG_SET
+Type: vcpu only
+
+Sets whether SVE is enabled for the vcpu, and if so sets the set of
+SVE vector lengths that will be visible to the guest.
+
+This is the only way to enable SVE for a vcpu: if this command is not
+invoked for a vcpu then SVE will not be available to the guest on this
+vcpu.
+
+This subcommand is only permitted once per vcpu, before KVM_RUN has been
+invoked for the vcpu for the first time.  Otherwise, the command fails
+with -EBADFD and the state of the vcpu is not modified.
+
+In typical use, the user should call KVM_ARM_SVE_CONFIG_QUERY first to
+populate a struct kvm_sve_vls with the full set of vector lengths
+available on the host, then set cmd = KVM_ARM_SVE_CONFIG_SET and
+re-issue the KVM_ARM_SVE_CONFIG ioctl on the desired vcpu.  This will
+configure the best set of vector lengths available.  When following this
+approach, the maximum available vector length can also be restricted by
+reducing the value of max_vq before invoking KVM_ARM_SVE_CONFIG_SET.
+
+Every requested vector length in the struct kvm_sve_vls argument must be
+supported by the hardware.  In addition, except for vector lengths
+greater than the maximum requested vector length, every vector length
+not requested must *not* be supported by the hardware.  (The latter
+restriction may be relaxed in the future.)  If the requested set of
+vector lengths is not supportable, the command fails with -EINVAL and
+the state of the vcpu is not modified.
+
+Different vcpus of a vm may be configured with different sets of vector
+lengths.  Equally, some vcpus may have SVE enabled and some not.
+However, such configurations are not recommended except for testing and
+experimentation purposes.  Architecturally compliant guest OSes will
+work, but may or may not make effective use of the resulting
+configuration.
+
+After a successful KVM_ARM_SVE_CONFIG_SET, KVM_ARM_SVE_CONFIG_GET can be
+used to retrieve the configured set of vector lengths.
+
+4.117.3 KVM_ARM_SVE_CONFIG_GET
+Type: vcpu only
+
+This subcommand returns the set of vector lengths enabled for the vcpu.
+SVE must have been disabled or enabled and configured for this vcpu by a
+successful prior KVM_ARM_SVE_CONFIG_SET call.  Otherwise, -EBADFD is
+returned.
+
+If SVE is disabled for this vcpu, this subcommand will yield
+max_vq == 0; otherwise max_vq and required_vqs[] indicate the
+(non-empty) set of configured vector lengths.
+
+The state of the vcpu is unchanged.
+
+
 5. The kvm_run structure
 ------------------------
 
-- 
2.1.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel