* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
This patchset adds API for saving and restoring
of VGICv3 registers to support live migration with new vgic feature.
This API definition is as per version of VGICv3 specification
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
The patch 3 & 4 are picked from the Pavel's previous implementation.
http://www.spinics.net/lists/kvm/msg122040.html
v7 => v8:
- Rebased to 4.9-rc3
- Fixed wrong parameter to VGIC_TO_MPIDR
v6 => v7:
- Rename all patches heading from vgic-new to vgic
- Moved caching of priority and ID bits from vgic global struct
to vgic_cpu struct.
v5 => v6:
- Collated all register definitions to single patch (4)
- Introduce macro to convert userspace MPIDR format to MPIDR reg format
- Check on ICC_CTLR_EL1.CBPR value is made while accessing ICC_BPR1_EL1
- Cached ich_vtr_el2 and guests priority and ID bits
- Check on number of priority and ID bits when ICC_CTRL_EL1 write is made
- Check is made on SRE bit for ICC_SRE_EL1 write
v4 => v5:
- ICC_CTLR_EL1 access is updated to reflect HW values
- Updated ICC reg access mask and shift macros
- Introduced patch 4 for VMCR changes
- Other minor fixes.
v3 => v4:
- Rebased to latest code base
- Moved vgic_uaccess() from vgic-mmio-v2.c to vgic-mmio.c
- Dropped macro REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED_UACCESS
- Dropped LE conversion for userspace access
- Introduced vgic_uaccess_write_pending() for ISPENDR write
- Change macro KVM_DEV_ARM_VGIC_V3_CPUID_MASK to KVM_DEV_ARM_VGIC_V3_MIDR_MASK
- Refactored some code as common code.
- Changed handing of ICC_* registers
- Allowed ICC_SRE_EL1 read by userspace
- Fixed KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_* macros
v2 => v3:
- Implemented separate API for ISPENDR and ICPENDR to
read soft_pending instead of pending for level triggerred interrupts
- Implemented ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO to access line level
- Rebased on top of Christoffer's patch set
http://www.spinics.net/lists/kvm/msg136840.html
NOTE: GICD_STATUSR and GICR_STATUSR are implemented as RAZ/WI.
v1 => v2:
- The init sequence change patch is no more required.
Fixed in patch 2 by using static vgic_io_dev regions structure instead
of using dynamic allocation pointer.
- Updated commit message of patch 4.
- Dropped usage of union to manage 32-bit and 64-bit access in patch 1.
Used local variable for 32-bit access.
- Updated macro __ARM64_SYS_REG and ARM64_SYS_REG in
arch/arm64/include/uapi/asm/kvm.h as per qemu requirements.
*** BLURB HERE ***
Vijaya Kumar K (7):
arm/arm64: vgic: Implement support for userspace access
arm/arm64: vgic: Add distributor and redistributor access
arm/arm64: vgic: Introduce find_reg_by_id()
irqchip/gic-v3: Add missing system register definitions
arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
arm/arm64: vgic: Implement VGICv3 CPU interface access
arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
arch/arm64/include/uapi/asm/kvm.h | 13 ++
arch/arm64/kvm/Makefile | 1 +
arch/arm64/kvm/sys_regs.c | 22 ++-
arch/arm64/kvm/sys_regs.h | 4 +
include/kvm/arm_vgic.h | 9 +
include/linux/irqchip/arm-gic-v3.h | 45 ++++-
virt/kvm/arm/vgic/vgic-kvm-device.c | 224 +++++++++++++++++++++++--
virt/kvm/arm/vgic/vgic-mmio-v2.c | 57 +------
virt/kvm/arm/vgic/vgic-mmio-v3.c | 200 ++++++++++++++++++++--
virt/kvm/arm/vgic/vgic-mmio.c | 149 ++++++++++++++++-
virt/kvm/arm/vgic/vgic-mmio.h | 28 ++++
virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 18 +-
virt/kvm/arm/vgic/vgic.h | 45 +++++
14 files changed, 1035 insertions(+), 104 deletions(-)
create mode 100644 virt/kvm/arm/vgic/vgic-sys-reg-v3.c
--
1.9.1
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
This patchset adds API for saving and restoring
of VGICv3 registers to support live migration with new vgic feature.
This API definition is as per version of VGICv3 specification
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
The patch 3 & 4 are picked from the Pavel's previous implementation.
http://www.spinics.net/lists/kvm/msg122040.html
v7 => v8:
- Rebased to 4.9-rc3
- Fixed wrong parameter to VGIC_TO_MPIDR
v6 => v7:
- Rename all patches heading from vgic-new to vgic
- Moved caching of priority and ID bits from vgic global struct
to vgic_cpu struct.
v5 => v6:
- Collated all register definitions to single patch (4)
- Introduce macro to convert userspace MPIDR format to MPIDR reg format
- Check on ICC_CTLR_EL1.CBPR value is made while accessing ICC_BPR1_EL1
- Cached ich_vtr_el2 and guests priority and ID bits
- Check on number of priority and ID bits when ICC_CTRL_EL1 write is made
- Check is made on SRE bit for ICC_SRE_EL1 write
v4 => v5:
- ICC_CTLR_EL1 access is updated to reflect HW values
- Updated ICC reg access mask and shift macros
- Introduced patch 4 for VMCR changes
- Other minor fixes.
v3 => v4:
- Rebased to latest code base
- Moved vgic_uaccess() from vgic-mmio-v2.c to vgic-mmio.c
- Dropped macro REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED_UACCESS
- Dropped LE conversion for userspace access
- Introduced vgic_uaccess_write_pending() for ISPENDR write
- Change macro KVM_DEV_ARM_VGIC_V3_CPUID_MASK to KVM_DEV_ARM_VGIC_V3_MIDR_MASK
- Refactored some code as common code.
- Changed handing of ICC_* registers
- Allowed ICC_SRE_EL1 read by userspace
- Fixed KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_* macros
v2 => v3:
- Implemented separate API for ISPENDR and ICPENDR to
read soft_pending instead of pending for level triggerred interrupts
- Implemented ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO to access line level
- Rebased on top of Christoffer's patch set
http://www.spinics.net/lists/kvm/msg136840.html
NOTE: GICD_STATUSR and GICR_STATUSR are implemented as RAZ/WI.
v1 => v2:
- The init sequence change patch is no more required.
Fixed in patch 2 by using static vgic_io_dev regions structure instead
of using dynamic allocation pointer.
- Updated commit message of patch 4.
- Dropped usage of union to manage 32-bit and 64-bit access in patch 1.
Used local variable for 32-bit access.
- Updated macro __ARM64_SYS_REG and ARM64_SYS_REG in
arch/arm64/include/uapi/asm/kvm.h as per qemu requirements.
*** BLURB HERE ***
Vijaya Kumar K (7):
arm/arm64: vgic: Implement support for userspace access
arm/arm64: vgic: Add distributor and redistributor access
arm/arm64: vgic: Introduce find_reg_by_id()
irqchip/gic-v3: Add missing system register definitions
arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
arm/arm64: vgic: Implement VGICv3 CPU interface access
arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
arch/arm64/include/uapi/asm/kvm.h | 13 ++
arch/arm64/kvm/Makefile | 1 +
arch/arm64/kvm/sys_regs.c | 22 ++-
arch/arm64/kvm/sys_regs.h | 4 +
include/kvm/arm_vgic.h | 9 +
include/linux/irqchip/arm-gic-v3.h | 45 ++++-
virt/kvm/arm/vgic/vgic-kvm-device.c | 224 +++++++++++++++++++++++--
virt/kvm/arm/vgic/vgic-mmio-v2.c | 57 +------
virt/kvm/arm/vgic/vgic-mmio-v3.c | 200 ++++++++++++++++++++--
virt/kvm/arm/vgic/vgic-mmio.c | 149 ++++++++++++++++-
virt/kvm/arm/vgic/vgic-mmio.h | 28 ++++
virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 18 +-
virt/kvm/arm/vgic/vgic.h | 45 +++++
14 files changed, 1035 insertions(+), 104 deletions(-)
create mode 100644 virt/kvm/arm/vgic/vgic-sys-reg-v3.c
--
1.9.1
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Read and write of some registers like ISPENDR and ICPENDR
from userspace requires special handling when compared to
guest access for these registers.
Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
for handling of ISPENDR, ICPENDR registers handling.
Add infrastructure to support guest and userspace read
and write for the required registers
Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
4 files changed, 169 insertions(+), 51 deletions(-)
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index b44b359..0b32f40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
return -ENXIO;
}
-/*
- * When userland tries to access the VGIC register handlers, we need to
- * create a usable struct vgic_io_device to be passed to the handlers and we
- * have to set up a buffer similar to what would have happened if a guest MMIO
- * access occurred, including doing endian conversions on BE systems.
- */
-static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
- bool is_write, int offset, u32 *val)
-{
- unsigned int len = 4;
- u8 buf[4];
- int ret;
-
- if (is_write) {
- vgic_data_host_to_mmio_bus(buf, len, *val);
- ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
- } else {
- ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
- if (!ret)
- *val = vgic_data_mmio_bus_to_host(buf, len);
- }
-
- return ret;
-}
-
int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val)
{
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 0d3c76a..ce2708d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
return 0;
}
+static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+ u32 value = 0;
+ int i;
+
+ /*
+ * A level triggerred interrupt pending state is latched in both
+ * "soft_pending" and "line_level" variables. Userspace will save
+ * and restore soft_pending and line_level separately.
+ * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
+ * handling of ISPENDR and ICPENDR.
+ */
+ for (i = 0; i < len * 8; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
+ value |= (1U << i);
+ if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
+ value |= (1U << i);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return value;
+}
+
+static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len,
+ unsigned long val)
+{
+ u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+ int i;
+
+ for (i = 0; i < len * 8; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ spin_lock(&irq->irq_lock);
+ if (test_bit(i, &val)) {
+ irq->pending = true;
+ irq->soft_pending = true;
+ vgic_queue_irq_unlock(vcpu->kvm, irq);
+ } else {
+ irq->soft_pending = false;
+ if (irq->config == VGIC_CONFIG_EDGE ||
+ (irq->config == VGIC_CONFIG_LEVEL &&
+ !irq->line_level))
+ irq->pending = false;
+ spin_unlock(&irq->irq_lock);
+ }
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
/* We want to avoid outer shareable. */
u64 vgic_sanitise_shareability(u64 field)
{
@@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
* We take some special care here to fix the calculation of the register
* offset.
*/
-#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc) \
+#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
{ \
.reg_offset = off, \
.bits_per_irq = bpi, \
@@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
.access_flags = acc, \
.read = rd, \
.write = wr, \
+ .uaccess_read = ur, \
+ .uaccess_write = uw, \
}
static const struct vgic_register_region vgic_v3_dist_registers[] = {
@@ -380,40 +438,42 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
- vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
+ vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
- vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
+ vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
- vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
+ vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
- vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
+ vgic_mmio_read_pending, vgic_mmio_write_spending,
+ vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
- vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
+ vgic_mmio_read_pending, vgic_mmio_write_cpending,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
- vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
+ vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
- vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
+ vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
- vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
- VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+ vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
+ 8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
- vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
- vgic_mmio_read_config, vgic_mmio_write_config, 2,
+ vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
- vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
- vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
+ vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
@@ -451,11 +511,13 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
VGIC_ACCESS_32bit),
- REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
- vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
+ vgic_mmio_read_pending, vgic_mmio_write_spending,
+ vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 4,
VGIC_ACCESS_32bit),
- REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
- vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
+ vgic_mmio_read_pending, vgic_mmio_write_cpending,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index e18b30d..31f85df 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
return false;
}
+static const struct vgic_register_region *
+vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
+{
+ const struct vgic_register_region *region;
+
+ region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
+ addr - iodev->base_addr);
+ if (!region || !check_region(region, addr, len))
+ return NULL;
+
+ return region;
+}
+
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+ gpa_t addr, u32 *val)
+{
+ struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+ const struct vgic_register_region *region;
+ struct kvm_vcpu *r_vcpu;
+
+ region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+ if (!region) {
+ *val = 0;
+ return 0;
+ }
+
+ r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+ if (region->uaccess_read)
+ *val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
+ else
+ *val = region->read(r_vcpu, addr, sizeof(u32));
+
+ return 0;
+}
+
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+ gpa_t addr, const u32 *val)
+{
+ struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+ const struct vgic_register_region *region;
+ struct kvm_vcpu *r_vcpu;
+
+ region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+ if (!region)
+ return 0;
+
+ r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+ if (region->uaccess_write)
+ region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
+ else
+ region->write(r_vcpu, addr, sizeof(u32), *val);
+
+ return 0;
+}
+
+/*
+ * Userland access to VGIC registers.
+ */
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+ bool is_write, int offset, u32 *val)
+{
+ if (is_write)
+ return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+ else
+ return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+}
+
static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
@@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
const struct vgic_register_region *region;
unsigned long data = 0;
- region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
- addr - iodev->base_addr);
- if (!region || !check_region(region, addr, len)) {
+ region = vgic_get_mmio_region(iodev, addr, len);
+ if (!region) {
memset(val, 0, len);
return 0;
}
@@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
const struct vgic_register_region *region;
unsigned long data = vgic_data_mmio_bus_to_host(val, len);
- region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
- addr - iodev->base_addr);
+ region = vgic_get_mmio_region(iodev, addr, len);
if (!region)
return 0;
- if (!check_region(region, addr, len))
- return 0;
-
switch (iodev->iodev_type) {
case IODEV_CPUIF:
region->write(vcpu, addr, len, data);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 4c34d39..97e6df7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -34,6 +34,10 @@ struct vgic_register_region {
gpa_t addr, unsigned int len,
unsigned long val);
};
+ unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len);
+ void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val);
};
extern struct kvm_io_device_ops kvm_io_gic_ops;
@@ -86,6 +90,18 @@ struct vgic_register_region {
.write = wr, \
}
+#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
+ { \
+ .reg_offset = off, \
+ .bits_per_irq = 0, \
+ .len = length, \
+ .access_flags = acc, \
+ .read = rd, \
+ .write = wr, \
+ .uaccess_read = urd, \
+ .uaccess_write = uwr, \
+ }
+
int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
struct vgic_register_region *reg_desc,
struct vgic_io_device *region,
@@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
gpa_t addr, unsigned int len,
unsigned long val);
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+ bool is_write, int offset, u32 *val);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Read and write of some registers like ISPENDR and ICPENDR
from userspace requires special handling when compared to
guest access for these registers.
Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
for handling of ISPENDR, ICPENDR registers handling.
Add infrastructure to support guest and userspace read
and write for the required registers
Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
4 files changed, 169 insertions(+), 51 deletions(-)
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index b44b359..0b32f40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
return -ENXIO;
}
-/*
- * When userland tries to access the VGIC register handlers, we need to
- * create a usable struct vgic_io_device to be passed to the handlers and we
- * have to set up a buffer similar to what would have happened if a guest MMIO
- * access occurred, including doing endian conversions on BE systems.
- */
-static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
- bool is_write, int offset, u32 *val)
-{
- unsigned int len = 4;
- u8 buf[4];
- int ret;
-
- if (is_write) {
- vgic_data_host_to_mmio_bus(buf, len, *val);
- ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
- } else {
- ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
- if (!ret)
- *val = vgic_data_mmio_bus_to_host(buf, len);
- }
-
- return ret;
-}
-
int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val)
{
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 0d3c76a..ce2708d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
return 0;
}
+static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len)
+{
+ u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+ u32 value = 0;
+ int i;
+
+ /*
+ * A level triggerred interrupt pending state is latched in both
+ * "soft_pending" and "line_level" variables. Userspace will save
+ * and restore soft_pending and line_level separately.
+ * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
+ * handling of ISPENDR and ICPENDR.
+ */
+ for (i = 0; i < len * 8; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
+ value |= (1U << i);
+ if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
+ value |= (1U << i);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return value;
+}
+
+static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
+ gpa_t addr, unsigned int len,
+ unsigned long val)
+{
+ u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+ int i;
+
+ for (i = 0; i < len * 8; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ spin_lock(&irq->irq_lock);
+ if (test_bit(i, &val)) {
+ irq->pending = true;
+ irq->soft_pending = true;
+ vgic_queue_irq_unlock(vcpu->kvm, irq);
+ } else {
+ irq->soft_pending = false;
+ if (irq->config == VGIC_CONFIG_EDGE ||
+ (irq->config == VGIC_CONFIG_LEVEL &&
+ !irq->line_level))
+ irq->pending = false;
+ spin_unlock(&irq->irq_lock);
+ }
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
/* We want to avoid outer shareable. */
u64 vgic_sanitise_shareability(u64 field)
{
@@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
* We take some special care here to fix the calculation of the register
* offset.
*/
-#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc) \
+#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
{ \
.reg_offset = off, \
.bits_per_irq = bpi, \
@@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
.access_flags = acc, \
.read = rd, \
.write = wr, \
+ .uaccess_read = ur, \
+ .uaccess_write = uw, \
}
static const struct vgic_register_region vgic_v3_dist_registers[] = {
@@ -380,40 +438,42 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
- vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
+ vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
- vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
+ vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
- vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
+ vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
- vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
+ vgic_mmio_read_pending, vgic_mmio_write_spending,
+ vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
- vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
+ vgic_mmio_read_pending, vgic_mmio_write_cpending,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
- vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
+ vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
- vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
+ vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
- vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
- VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+ vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
+ 8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
- vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
- vgic_mmio_read_config, vgic_mmio_write_config, 2,
+ vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
- vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
- vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
+ vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
@@ -451,11 +511,13 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
VGIC_ACCESS_32bit),
- REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
- vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
+ vgic_mmio_read_pending, vgic_mmio_write_spending,
+ vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 4,
VGIC_ACCESS_32bit),
- REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
- vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
+ REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
+ vgic_mmio_read_pending, vgic_mmio_write_cpending,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index e18b30d..31f85df 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
return false;
}
+static const struct vgic_register_region *
+vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
+{
+ const struct vgic_register_region *region;
+
+ region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
+ addr - iodev->base_addr);
+ if (!region || !check_region(region, addr, len))
+ return NULL;
+
+ return region;
+}
+
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+ gpa_t addr, u32 *val)
+{
+ struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+ const struct vgic_register_region *region;
+ struct kvm_vcpu *r_vcpu;
+
+ region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+ if (!region) {
+ *val = 0;
+ return 0;
+ }
+
+ r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+ if (region->uaccess_read)
+ *val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
+ else
+ *val = region->read(r_vcpu, addr, sizeof(u32));
+
+ return 0;
+}
+
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+ gpa_t addr, const u32 *val)
+{
+ struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+ const struct vgic_register_region *region;
+ struct kvm_vcpu *r_vcpu;
+
+ region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+ if (!region)
+ return 0;
+
+ r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+ if (region->uaccess_write)
+ region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
+ else
+ region->write(r_vcpu, addr, sizeof(u32), *val);
+
+ return 0;
+}
+
+/*
+ * Userland access to VGIC registers.
+ */
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+ bool is_write, int offset, u32 *val)
+{
+ if (is_write)
+ return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+ else
+ return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+}
+
static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
@@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
const struct vgic_register_region *region;
unsigned long data = 0;
- region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
- addr - iodev->base_addr);
- if (!region || !check_region(region, addr, len)) {
+ region = vgic_get_mmio_region(iodev, addr, len);
+ if (!region) {
memset(val, 0, len);
return 0;
}
@@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
const struct vgic_register_region *region;
unsigned long data = vgic_data_mmio_bus_to_host(val, len);
- region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
- addr - iodev->base_addr);
+ region = vgic_get_mmio_region(iodev, addr, len);
if (!region)
return 0;
- if (!check_region(region, addr, len))
- return 0;
-
switch (iodev->iodev_type) {
case IODEV_CPUIF:
region->write(vcpu, addr, len, data);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 4c34d39..97e6df7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -34,6 +34,10 @@ struct vgic_register_region {
gpa_t addr, unsigned int len,
unsigned long val);
};
+ unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len);
+ void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
+ unsigned int len, unsigned long val);
};
extern struct kvm_io_device_ops kvm_io_gic_ops;
@@ -86,6 +90,18 @@ struct vgic_register_region {
.write = wr, \
}
+#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
+ { \
+ .reg_offset = off, \
+ .bits_per_irq = 0, \
+ .len = length, \
+ .access_flags = acc, \
+ .read = rd, \
+ .write = wr, \
+ .uaccess_read = urd, \
+ .uaccess_write = uwr, \
+ }
+
int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
struct vgic_register_region *reg_desc,
struct vgic_io_device *region,
@@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
gpa_t addr, unsigned int len,
unsigned long val);
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+ bool is_write, int offset, u32 *val);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 2/7] arm/arm64: vgic: Add distributor and redistributor access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
VGICv3 Distributor and Redistributor registers are accessed using
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
These registers are accessed as 32-bit and cpu mpidr
value passed along with register offset is used to identify the
cpu for redistributor registers access.
The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 4 +
virt/kvm/arm/vgic/vgic-kvm-device.c | 149 +++++++++++++++++++++++++++++++++---
virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 +---
virt/kvm/arm/vgic/vgic-mmio-v3.c | 72 +++++++++++++++++
virt/kvm/arm/vgic/vgic-mmio.c | 22 ++++++
virt/kvm/arm/vgic/vgic-mmio.h | 4 +
virt/kvm/arm/vgic/vgic.h | 33 ++++++++
7 files changed, 276 insertions(+), 24 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 3051f86..56dc08d 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
+#define KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
+ (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
#define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
+#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
/* Device Control API on vcpu fd */
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index ce1f4ed..6c7d30c 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -237,7 +237,7 @@ struct vgic_reg_attr {
gpa_t addr;
};
-static int parse_vgic_v2_attr(struct kvm_device *dev,
+static int vgic_v2_parse_attr(struct kvm_device *dev,
struct kvm_device_attr *attr,
struct vgic_reg_attr *reg_attr)
{
@@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
}
/**
- * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
+ * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
*
* @dev: kvm device handle
* @attr: kvm device attribute
* @reg: address the value is read or written
* @is_write: true if userspace is writing a register
*/
-static int vgic_attr_regs_access_v2(struct kvm_device *dev,
+static int vgic_v2_attr_regs_access(struct kvm_device *dev,
struct kvm_device_attr *attr,
u32 *reg, bool is_write)
{
@@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
struct kvm_vcpu *vcpu;
int ret;
- ret = parse_vgic_v2_attr(dev, attr, ®_attr);
+ ret = vgic_v2_parse_attr(dev, attr, ®_attr);
if (ret)
return ret;
@@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
mutex_lock(&dev->kvm->lock);
- ret = vgic_init(dev->kvm);
- if (ret)
+ if (unlikely(!vgic_initialized(dev->kvm))) {
+ ret = -EBUSY;
goto out;
+ }
if (!lock_all_vcpus(dev->kvm)) {
ret = -EBUSY;
@@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
if (get_user(reg, uaddr))
return -EFAULT;
- return vgic_attr_regs_access_v2(dev, attr, ®, true);
+ return vgic_v2_attr_regs_access(dev, attr, ®, true);
}
}
@@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
u32 __user *uaddr = (u32 __user *)(long)attr->addr;
u32 reg = 0;
- ret = vgic_attr_regs_access_v2(dev, attr, ®, false);
+ ret = vgic_v2_attr_regs_access(dev, attr, ®, false);
if (ret)
return ret;
return put_user(reg, uaddr);
@@ -430,16 +431,141 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
.has_attr = vgic_v2_has_attr,
};
+static int vgic_v3_parse_attr(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ struct vgic_reg_attr *reg_attr)
+{
+ unsigned long vgic_mpidr, mpidr_reg;
+
+ vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+ KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+ mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
+ reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
+ if (!reg_attr->vcpu)
+ return -EINVAL;
+
+ reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+ return 0;
+}
+
+/*
+ * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
+ *
+ * @dev: kvm device handle
+ * @attr: kvm device attribute
+ * @reg: address the value is read or written
+ * @is_write: true if userspace is writing a register
+ */
+static int vgic_v3_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u64 *reg, bool is_write)
+{
+ struct vgic_reg_attr reg_attr;
+ gpa_t addr;
+ struct kvm_vcpu *vcpu;
+ int ret;
+ u32 tmp32;
+
+ ret = vgic_v3_parse_attr(dev, attr, ®_attr);
+ if (ret)
+ return ret;
+
+ vcpu = reg_attr.vcpu;
+ addr = reg_attr.addr;
+
+ mutex_lock(&dev->kvm->lock);
+
+ if (unlikely(!vgic_initialized(dev->kvm))) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (!lock_all_vcpus(dev->kvm)) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ if (is_write)
+ tmp32 = *reg;
+
+ ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ if (is_write)
+ tmp32 = *reg;
+
+ ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ unlock_all_vcpus(dev->kvm);
+out:
+ mutex_unlock(&dev->kvm->lock);
+ return ret;
+}
+
static int vgic_v3_set_attr(struct kvm_device *dev,
struct kvm_device_attr *attr)
{
- return vgic_set_common_attr(dev, attr);
+ int ret;
+
+ ret = vgic_set_common_attr(dev, attr);
+ if (ret != -ENXIO)
+ return ret;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 tmp32;
+ u64 reg;
+
+ if (get_user(tmp32, uaddr))
+ return -EFAULT;
+
+ reg = tmp32;
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
+ }
+ return -ENXIO;
}
static int vgic_v3_get_attr(struct kvm_device *dev,
struct kvm_device_attr *attr)
{
- return vgic_get_common_attr(dev, attr);
+ int ret;
+
+ ret = vgic_get_common_attr(dev, attr);
+ if (ret != -ENXIO)
+ return ret;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ tmp32 = reg;
+ return put_user(tmp32, uaddr);
+ }
+ }
+
+ return -ENXIO;
}
static int vgic_v3_has_attr(struct kvm_device *dev,
@@ -453,6 +579,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
return 0;
}
break;
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
case KVM_DEV_ARM_VGIC_GRP_CTRL:
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 0b32f40..2cb04b7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
{
- int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
const struct vgic_register_region *regions;
gpa_t addr;
- int nr_regions, i, len;
+ int nr_regions;
addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
@@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
if (addr & 3)
return -ENXIO;
- for (i = 0; i < nr_regions; i++) {
- if (regions[i].bits_per_irq)
- len = (regions[i].bits_per_irq * nr_irqs) / 8;
- else
- len = regions[i].len;
-
- if (regions[i].reg_offset <= addr &&
- regions[i].reg_offset + len > addr)
- return 0;
- }
-
- return -ENXIO;
+ return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
}
int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index ce2708d..b35fb83 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -18,6 +18,8 @@
#include <kvm/arm_vgic.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
#include "vgic.h"
#include "vgic-mmio.h"
@@ -437,6 +439,9 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
+ vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
@@ -484,12 +489,18 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
@@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
return ret;
}
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ const struct vgic_register_region *regions;
+ gpa_t addr;
+ int nr_regions;
+
+ addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ regions = vgic_v3_dist_registers;
+ nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
+ regions = vgic_v3_rdbase_registers;
+ nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
+ break;
+ }
+ default:
+ return -ENXIO;
+ }
+
+ /* We only support aligned 32-bit accesses. */
+ if (addr & 3)
+ return -ENXIO;
+
+ return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
+}
/*
* Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
* generation register ICC_SGI1R_EL1) with a given VCPU.
@@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
vgic_put_irq(vcpu->kvm, irq);
}
}
+
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val)
+{
+ struct vgic_io_device dev = {
+ .regions = vgic_v3_dist_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
+ };
+
+ return vgic_uaccess(vcpu, &dev, is_write, offset, val);
+}
+
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val)
+{
+ struct vgic_io_device rd_dev = {
+ .regions = vgic_v3_rdbase_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
+ };
+
+ struct vgic_io_device sgi_dev = {
+ .regions = vgic_v3_sgibase_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
+ };
+
+ /* SGI_base is the next 64K frame after RD_base */
+ if (offset >= SZ_64K)
+ return vgic_uaccess(vcpu, &sgi_dev, is_write,
+ offset - SZ_64K, val);
+ else
+ return vgic_uaccess(vcpu, &rd_dev, is_write,
+ offset, val);
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 31f85df..9939d1d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -394,6 +394,28 @@ static int match_region(const void *key, const void *elt)
sizeof(region[0]), match_region);
}
+/* Check if address falls within the region */
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+ const struct vgic_register_region *regions,
+ int nr_regions, gpa_t addr)
+{
+ int i, len;
+ int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+ for (i = 0; i < nr_regions; i++) {
+ if (regions[i].bits_per_irq)
+ len = (regions[i].bits_per_irq * nr_irqs) / 8;
+ else
+ len = regions[i].len;
+
+ if (regions[i].reg_offset <= addr &&
+ regions[i].reg_offset + len > addr)
+ return 0;
+ }
+
+ return -ENXIO;
+}
+
/*
* kvm_mmio_read_buf() returns a value in a format where it can be converted
* to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 97e6df7..acbf99e 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
bool is_write, int offset, u32 *val);
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+ const struct vgic_register_region *regions,
+ int nr_regions, gpa_t addr);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9d9e014..d901b0c 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -30,6 +30,34 @@
#define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
+#define VGIC_AFFINITY_0_SHIFT 0
+#define VGIC_AFFINITY_0_MASK (0xffUL << VGIC_AFFINITY_0_SHIFT)
+#define VGIC_AFFINITY_1_SHIFT 8
+#define VGIC_AFFINITY_1_MASK (0xffUL << VGIC_AFFINITY_1_SHIFT)
+#define VGIC_AFFINITY_2_SHIFT 16
+#define VGIC_AFFINITY_2_MASK (0xffUL << VGIC_AFFINITY_2_SHIFT)
+#define VGIC_AFFINITY_3_SHIFT 24
+#define VGIC_AFFINITY_3_MASK (0xffUL << VGIC_AFFINITY_3_SHIFT)
+
+#define VGIC_AFFINITY_LEVEL(reg, level) \
+ ((((reg) & VGIC_AFFINITY_## level ##_MASK) \
+ >> VGIC_AFFINITY_## level ##_SHIFT) << MPIDR_LEVEL_SHIFT(level))
+
+/*
+ * The userspace encode the affinity differently from the MPIDR,
+ * Below macro converts vgic userspace format to MPIDR reg format.
+ */
+#define VGIC_TO_MPIDR(val) (VGIC_AFFINITY_LEVEL(val, 0) | \
+ VGIC_AFFINITY_LEVEL(val, 1) | \
+ VGIC_AFFINITY_LEVEL(val, 2) | \
+ VGIC_AFFINITY_LEVEL(val, 3))
+
+#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
+ KVM_REG_ARM64_SYSREG_OP1_MASK | \
+ KVM_REG_ARM64_SYSREG_CRN_MASK | \
+ KVM_REG_ARM64_SYSREG_CRM_MASK | \
+ KVM_REG_ARM64_SYSREG_OP2_MASK)
+
struct vgic_vmcr {
u32 ctlr;
u32 abpr;
@@ -90,6 +118,11 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
int kvm_vgic_register_its_device(void);
void vgic_enable_lpis(struct kvm_vcpu *vcpu);
int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val);
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val);
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 2/7] arm/arm64: vgic: Add distributor and redistributor access
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
VGICv3 Distributor and Redistributor registers are accessed using
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
These registers are accessed as 32-bit and cpu mpidr
value passed along with register offset is used to identify the
cpu for redistributor registers access.
The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 4 +
virt/kvm/arm/vgic/vgic-kvm-device.c | 149 +++++++++++++++++++++++++++++++++---
virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 +---
virt/kvm/arm/vgic/vgic-mmio-v3.c | 72 +++++++++++++++++
virt/kvm/arm/vgic/vgic-mmio.c | 22 ++++++
virt/kvm/arm/vgic/vgic-mmio.h | 4 +
virt/kvm/arm/vgic/vgic.h | 33 ++++++++
7 files changed, 276 insertions(+), 24 deletions(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 3051f86..56dc08d 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
+#define KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
+ (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
#define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
+#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
/* Device Control API on vcpu fd */
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index ce1f4ed..6c7d30c 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -237,7 +237,7 @@ struct vgic_reg_attr {
gpa_t addr;
};
-static int parse_vgic_v2_attr(struct kvm_device *dev,
+static int vgic_v2_parse_attr(struct kvm_device *dev,
struct kvm_device_attr *attr,
struct vgic_reg_attr *reg_attr)
{
@@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
}
/**
- * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
+ * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
*
* @dev: kvm device handle
* @attr: kvm device attribute
* @reg: address the value is read or written
* @is_write: true if userspace is writing a register
*/
-static int vgic_attr_regs_access_v2(struct kvm_device *dev,
+static int vgic_v2_attr_regs_access(struct kvm_device *dev,
struct kvm_device_attr *attr,
u32 *reg, bool is_write)
{
@@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
struct kvm_vcpu *vcpu;
int ret;
- ret = parse_vgic_v2_attr(dev, attr, ®_attr);
+ ret = vgic_v2_parse_attr(dev, attr, ®_attr);
if (ret)
return ret;
@@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
mutex_lock(&dev->kvm->lock);
- ret = vgic_init(dev->kvm);
- if (ret)
+ if (unlikely(!vgic_initialized(dev->kvm))) {
+ ret = -EBUSY;
goto out;
+ }
if (!lock_all_vcpus(dev->kvm)) {
ret = -EBUSY;
@@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
if (get_user(reg, uaddr))
return -EFAULT;
- return vgic_attr_regs_access_v2(dev, attr, ®, true);
+ return vgic_v2_attr_regs_access(dev, attr, ®, true);
}
}
@@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
u32 __user *uaddr = (u32 __user *)(long)attr->addr;
u32 reg = 0;
- ret = vgic_attr_regs_access_v2(dev, attr, ®, false);
+ ret = vgic_v2_attr_regs_access(dev, attr, ®, false);
if (ret)
return ret;
return put_user(reg, uaddr);
@@ -430,16 +431,141 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
.has_attr = vgic_v2_has_attr,
};
+static int vgic_v3_parse_attr(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ struct vgic_reg_attr *reg_attr)
+{
+ unsigned long vgic_mpidr, mpidr_reg;
+
+ vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+ KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+ mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
+ reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
+ if (!reg_attr->vcpu)
+ return -EINVAL;
+
+ reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+ return 0;
+}
+
+/*
+ * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
+ *
+ * @dev: kvm device handle
+ * @attr: kvm device attribute
+ * @reg: address the value is read or written
+ * @is_write: true if userspace is writing a register
+ */
+static int vgic_v3_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u64 *reg, bool is_write)
+{
+ struct vgic_reg_attr reg_attr;
+ gpa_t addr;
+ struct kvm_vcpu *vcpu;
+ int ret;
+ u32 tmp32;
+
+ ret = vgic_v3_parse_attr(dev, attr, ®_attr);
+ if (ret)
+ return ret;
+
+ vcpu = reg_attr.vcpu;
+ addr = reg_attr.addr;
+
+ mutex_lock(&dev->kvm->lock);
+
+ if (unlikely(!vgic_initialized(dev->kvm))) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ if (!lock_all_vcpus(dev->kvm)) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ if (is_write)
+ tmp32 = *reg;
+
+ ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ if (is_write)
+ tmp32 = *reg;
+
+ ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ unlock_all_vcpus(dev->kvm);
+out:
+ mutex_unlock(&dev->kvm->lock);
+ return ret;
+}
+
static int vgic_v3_set_attr(struct kvm_device *dev,
struct kvm_device_attr *attr)
{
- return vgic_set_common_attr(dev, attr);
+ int ret;
+
+ ret = vgic_set_common_attr(dev, attr);
+ if (ret != -ENXIO)
+ return ret;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 tmp32;
+ u64 reg;
+
+ if (get_user(tmp32, uaddr))
+ return -EFAULT;
+
+ reg = tmp32;
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
+ }
+ return -ENXIO;
}
static int vgic_v3_get_attr(struct kvm_device *dev,
struct kvm_device_attr *attr)
{
- return vgic_get_common_attr(dev, attr);
+ int ret;
+
+ ret = vgic_get_common_attr(dev, attr);
+ if (ret != -ENXIO)
+ return ret;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ tmp32 = reg;
+ return put_user(tmp32, uaddr);
+ }
+ }
+
+ return -ENXIO;
}
static int vgic_v3_has_attr(struct kvm_device *dev,
@@ -453,6 +579,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
return 0;
}
break;
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
case KVM_DEV_ARM_VGIC_GRP_CTRL:
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 0b32f40..2cb04b7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
{
- int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
const struct vgic_register_region *regions;
gpa_t addr;
- int nr_regions, i, len;
+ int nr_regions;
addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
@@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
if (addr & 3)
return -ENXIO;
- for (i = 0; i < nr_regions; i++) {
- if (regions[i].bits_per_irq)
- len = (regions[i].bits_per_irq * nr_irqs) / 8;
- else
- len = regions[i].len;
-
- if (regions[i].reg_offset <= addr &&
- regions[i].reg_offset + len > addr)
- return 0;
- }
-
- return -ENXIO;
+ return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
}
int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index ce2708d..b35fb83 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -18,6 +18,8 @@
#include <kvm/arm_vgic.h>
#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
#include "vgic.h"
#include "vgic-mmio.h"
@@ -437,6 +439,9 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
+ vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
VGIC_ACCESS_32bit),
@@ -484,12 +489,18 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+ REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
+ vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+ VGIC_ACCESS_32bit),
REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
@@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
return ret;
}
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ const struct vgic_register_region *regions;
+ gpa_t addr;
+ int nr_regions;
+
+ addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ regions = vgic_v3_dist_registers;
+ nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
+ regions = vgic_v3_rdbase_registers;
+ nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
+ break;
+ }
+ default:
+ return -ENXIO;
+ }
+
+ /* We only support aligned 32-bit accesses. */
+ if (addr & 3)
+ return -ENXIO;
+
+ return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
+}
/*
* Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
* generation register ICC_SGI1R_EL1) with a given VCPU.
@@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
vgic_put_irq(vcpu->kvm, irq);
}
}
+
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val)
+{
+ struct vgic_io_device dev = {
+ .regions = vgic_v3_dist_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
+ };
+
+ return vgic_uaccess(vcpu, &dev, is_write, offset, val);
+}
+
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val)
+{
+ struct vgic_io_device rd_dev = {
+ .regions = vgic_v3_rdbase_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
+ };
+
+ struct vgic_io_device sgi_dev = {
+ .regions = vgic_v3_sgibase_registers,
+ .nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
+ };
+
+ /* SGI_base is the next 64K frame after RD_base */
+ if (offset >= SZ_64K)
+ return vgic_uaccess(vcpu, &sgi_dev, is_write,
+ offset - SZ_64K, val);
+ else
+ return vgic_uaccess(vcpu, &rd_dev, is_write,
+ offset, val);
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 31f85df..9939d1d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -394,6 +394,28 @@ static int match_region(const void *key, const void *elt)
sizeof(region[0]), match_region);
}
+/* Check if address falls within the region */
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+ const struct vgic_register_region *regions,
+ int nr_regions, gpa_t addr)
+{
+ int i, len;
+ int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+ for (i = 0; i < nr_regions; i++) {
+ if (regions[i].bits_per_irq)
+ len = (regions[i].bits_per_irq * nr_irqs) / 8;
+ else
+ len = regions[i].len;
+
+ if (regions[i].reg_offset <= addr &&
+ regions[i].reg_offset + len > addr)
+ return 0;
+ }
+
+ return -ENXIO;
+}
+
/*
* kvm_mmio_read_buf() returns a value in a format where it can be converted
* to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 97e6df7..acbf99e 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
bool is_write, int offset, u32 *val);
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+ const struct vgic_register_region *regions,
+ int nr_regions, gpa_t addr);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9d9e014..d901b0c 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -30,6 +30,34 @@
#define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
+#define VGIC_AFFINITY_0_SHIFT 0
+#define VGIC_AFFINITY_0_MASK (0xffUL << VGIC_AFFINITY_0_SHIFT)
+#define VGIC_AFFINITY_1_SHIFT 8
+#define VGIC_AFFINITY_1_MASK (0xffUL << VGIC_AFFINITY_1_SHIFT)
+#define VGIC_AFFINITY_2_SHIFT 16
+#define VGIC_AFFINITY_2_MASK (0xffUL << VGIC_AFFINITY_2_SHIFT)
+#define VGIC_AFFINITY_3_SHIFT 24
+#define VGIC_AFFINITY_3_MASK (0xffUL << VGIC_AFFINITY_3_SHIFT)
+
+#define VGIC_AFFINITY_LEVEL(reg, level) \
+ ((((reg) & VGIC_AFFINITY_## level ##_MASK) \
+ >> VGIC_AFFINITY_## level ##_SHIFT) << MPIDR_LEVEL_SHIFT(level))
+
+/*
+ * The userspace encode the affinity differently from the MPIDR,
+ * Below macro converts vgic userspace format to MPIDR reg format.
+ */
+#define VGIC_TO_MPIDR(val) (VGIC_AFFINITY_LEVEL(val, 0) | \
+ VGIC_AFFINITY_LEVEL(val, 1) | \
+ VGIC_AFFINITY_LEVEL(val, 2) | \
+ VGIC_AFFINITY_LEVEL(val, 3))
+
+#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
+ KVM_REG_ARM64_SYSREG_OP1_MASK | \
+ KVM_REG_ARM64_SYSREG_CRN_MASK | \
+ KVM_REG_ARM64_SYSREG_CRM_MASK | \
+ KVM_REG_ARM64_SYSREG_OP2_MASK)
+
struct vgic_vmcr {
u32 ctlr;
u32 abpr;
@@ -90,6 +118,11 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
int kvm_vgic_register_its_device(void);
void vgic_enable_lpis(struct kvm_vcpu *vcpu);
int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val);
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ int offset, u32 *val);
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 3/7] arm/arm64: vgic: Introduce find_reg_by_id()
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
In order to implement vGICv3 CPU interface access, we will need to perform
table lookup of system registers. We would need both index_to_params() and
find_reg() exported for that purpose, but instead we export a single
function which combines them both.
Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/sys_regs.c | 22 +++++++++++++++-------
arch/arm64/kvm/sys_regs.h | 4 ++++
2 files changed, 19 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index f302fdb..1330d4c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1789,6 +1789,17 @@ static bool index_to_params(u64 id, struct sys_reg_params *params)
}
}
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+ struct sys_reg_params *params,
+ const struct sys_reg_desc table[],
+ unsigned int num)
+{
+ if (!index_to_params(id, params))
+ return NULL;
+
+ return find_reg(params, table, num);
+}
+
/* Decode an index value, and find the sys_reg_desc entry. */
static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
u64 id)
@@ -1912,10 +1923,8 @@ static int get_invariant_sys_reg(u64 id, void __user *uaddr)
struct sys_reg_params params;
const struct sys_reg_desc *r;
- if (!index_to_params(id, ¶ms))
- return -ENOENT;
-
- r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+ r = find_reg_by_id(id, ¶ms, invariant_sys_regs,
+ ARRAY_SIZE(invariant_sys_regs));
if (!r)
return -ENOENT;
@@ -1929,9 +1938,8 @@ static int set_invariant_sys_reg(u64 id, void __user *uaddr)
int err;
u64 val = 0; /* Make sure high bits are 0 for 32-bit regs */
- if (!index_to_params(id, ¶ms))
- return -ENOENT;
- r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+ r = find_reg_by_id(id, ¶ms, invariant_sys_regs,
+ ARRAY_SIZE(invariant_sys_regs));
if (!r)
return -ENOENT;
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index dbbb01c..9c6ffd0 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -136,6 +136,10 @@ static inline int cmp_sys_reg(const struct sys_reg_desc *i1,
return i1->Op2 - i2->Op2;
}
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+ struct sys_reg_params *params,
+ const struct sys_reg_desc table[],
+ unsigned int num);
#define Op0(_x) .Op0 = _x
#define Op1(_x) .Op1 = _x
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 3/7] arm/arm64: vgic: Introduce find_reg_by_id()
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
In order to implement vGICv3 CPU interface access, we will need to perform
table lookup of system registers. We would need both index_to_params() and
find_reg() exported for that purpose, but instead we export a single
function which combines them both.
Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm64/kvm/sys_regs.c | 22 +++++++++++++++-------
arch/arm64/kvm/sys_regs.h | 4 ++++
2 files changed, 19 insertions(+), 7 deletions(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index f302fdb..1330d4c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1789,6 +1789,17 @@ static bool index_to_params(u64 id, struct sys_reg_params *params)
}
}
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+ struct sys_reg_params *params,
+ const struct sys_reg_desc table[],
+ unsigned int num)
+{
+ if (!index_to_params(id, params))
+ return NULL;
+
+ return find_reg(params, table, num);
+}
+
/* Decode an index value, and find the sys_reg_desc entry. */
static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
u64 id)
@@ -1912,10 +1923,8 @@ static int get_invariant_sys_reg(u64 id, void __user *uaddr)
struct sys_reg_params params;
const struct sys_reg_desc *r;
- if (!index_to_params(id, ¶ms))
- return -ENOENT;
-
- r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+ r = find_reg_by_id(id, ¶ms, invariant_sys_regs,
+ ARRAY_SIZE(invariant_sys_regs));
if (!r)
return -ENOENT;
@@ -1929,9 +1938,8 @@ static int set_invariant_sys_reg(u64 id, void __user *uaddr)
int err;
u64 val = 0; /* Make sure high bits are 0 for 32-bit regs */
- if (!index_to_params(id, ¶ms))
- return -ENOENT;
- r = find_reg(¶ms, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+ r = find_reg_by_id(id, ¶ms, invariant_sys_regs,
+ ARRAY_SIZE(invariant_sys_regs));
if (!r)
return -ENOENT;
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index dbbb01c..9c6ffd0 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -136,6 +136,10 @@ static inline int cmp_sys_reg(const struct sys_reg_desc *i1,
return i1->Op2 - i2->Op2;
}
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+ struct sys_reg_params *params,
+ const struct sys_reg_desc table[],
+ unsigned int num);
#define Op0(_x) .Op0 = _x
#define Op1(_x) .Op1 = _x
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 4/7] irqchip/gic-v3: Add missing system register definitions
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Define register definitions for ICH_VMCR_EL2, ICC_CTLR_EL1 and
ICH_VTR_EL2, ICC_BPR0_EL1, ICC_BPR1_EL1 registers.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
include/linux/irqchip/arm-gic-v3.h | 43 ++++++++++++++++++++++++++++++++++++--
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index b7e3431..d48d886 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -352,8 +352,30 @@
/*
* CPU interface registers
*/
-#define ICC_CTLR_EL1_EOImode_drop_dir (0U << 1)
-#define ICC_CTLR_EL1_EOImode_drop (1U << 1)
+#define ICC_CTLR_EL1_EOImode_SHIFT (1)
+#define ICC_CTLR_EL1_EOImode_drop_dir (0U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_drop (1U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_MASK (1 << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_CBPR_SHIFT 0
+#define ICC_CTLR_EL1_CBPR_MASK (1 << ICC_CTLR_EL1_CBPR_SHIFT)
+#define ICC_CTLR_EL1_PRI_BITS_SHIFT 8
+#define ICC_CTLR_EL1_PRI_BITS_MASK (0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
+#define ICC_CTLR_EL1_ID_BITS_SHIFT 11
+#define ICC_CTLR_EL1_ID_BITS_MASK (0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
+#define ICC_CTLR_EL1_SEIS_SHIFT 14
+#define ICC_CTLR_EL1_SEIS_MASK (0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
+#define ICC_CTLR_EL1_A3V_SHIFT 15
+#define ICC_CTLR_EL1_A3V_MASK (0x1 << ICC_CTLR_EL1_A3V_SHIFT)
+#define ICC_PMR_EL1_SHIFT 0
+#define ICC_PMR_EL1_MASK (0xff << ICC_PMR_EL1_SHIFT)
+#define ICC_BPR0_EL1_SHIFT 0
+#define ICC_BPR0_EL1_MASK (0x7 << ICC_BPR0_EL1_SHIFT)
+#define ICC_BPR1_EL1_SHIFT 0
+#define ICC_BPR1_EL1_MASK (0x7 << ICC_BPR1_EL1_SHIFT)
+#define ICC_IGRPEN0_EL1_SHIFT 0
+#define ICC_IGRPEN0_EL1_MASK (1 << ICC_IGRPEN0_EL1_SHIFT)
+#define ICC_IGRPEN1_EL1_SHIFT 0
+#define ICC_IGRPEN1_EL1_MASK (1 << ICC_IGRPEN1_EL1_SHIFT)
#define ICC_SRE_EL1_SRE (1U << 0)
/*
@@ -384,12 +406,29 @@
#define ICH_VMCR_CTLR_SHIFT 0
#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
+#define ICH_VMCR_CBPR_SHIFT 4
+#define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
+#define ICH_VMCR_EOIM_SHIFT 9
+#define ICH_VMCR_EOIM_MASK (1 << ICH_VMCR_EOIM_SHIFT)
#define ICH_VMCR_BPR1_SHIFT 18
#define ICH_VMCR_BPR1_MASK (7 << ICH_VMCR_BPR1_SHIFT)
#define ICH_VMCR_BPR0_SHIFT 21
#define ICH_VMCR_BPR0_MASK (7 << ICH_VMCR_BPR0_SHIFT)
#define ICH_VMCR_PMR_SHIFT 24
#define ICH_VMCR_PMR_MASK (0xffUL << ICH_VMCR_PMR_SHIFT)
+#define ICH_VMCR_ENG0_SHIFT 0
+#define ICH_VMCR_ENG0_MASK (1 << ICH_VMCR_ENG0_SHIFT)
+#define ICH_VMCR_ENG1_SHIFT 1
+#define ICH_VMCR_ENG1_MASK (1 << ICH_VMCR_ENG1_SHIFT)
+
+#define ICH_VTR_PRI_BITS_SHIFT 29
+#define ICH_VTR_PRI_BITS_MASK (7 << ICH_VTR_PRI_BITS_SHIFT)
+#define ICH_VTR_ID_BITS_SHIFT 23
+#define ICH_VTR_ID_BITS_MASK (7 << ICH_VTR_ID_BITS_SHIFT)
+#define ICH_VTR_SEIS_SHIFT 22
+#define ICH_VTR_SEIS_MASK (1 << ICH_VTR_SEIS_SHIFT)
+#define ICH_VTR_A3V_SHIFT 21
+#define ICH_VTR_A3V_MASK (1 << ICH_VTR_A3V_SHIFT)
#define ICC_IAR1_EL1_SPURIOUS 0x3ff
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 4/7] irqchip/gic-v3: Add missing system register definitions
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Define register definitions for ICH_VMCR_EL2, ICC_CTLR_EL1 and
ICH_VTR_EL2, ICC_BPR0_EL1, ICC_BPR1_EL1 registers.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
include/linux/irqchip/arm-gic-v3.h | 43 ++++++++++++++++++++++++++++++++++++--
1 file changed, 41 insertions(+), 2 deletions(-)
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index b7e3431..d48d886 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -352,8 +352,30 @@
/*
* CPU interface registers
*/
-#define ICC_CTLR_EL1_EOImode_drop_dir (0U << 1)
-#define ICC_CTLR_EL1_EOImode_drop (1U << 1)
+#define ICC_CTLR_EL1_EOImode_SHIFT (1)
+#define ICC_CTLR_EL1_EOImode_drop_dir (0U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_drop (1U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_MASK (1 << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_CBPR_SHIFT 0
+#define ICC_CTLR_EL1_CBPR_MASK (1 << ICC_CTLR_EL1_CBPR_SHIFT)
+#define ICC_CTLR_EL1_PRI_BITS_SHIFT 8
+#define ICC_CTLR_EL1_PRI_BITS_MASK (0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
+#define ICC_CTLR_EL1_ID_BITS_SHIFT 11
+#define ICC_CTLR_EL1_ID_BITS_MASK (0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
+#define ICC_CTLR_EL1_SEIS_SHIFT 14
+#define ICC_CTLR_EL1_SEIS_MASK (0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
+#define ICC_CTLR_EL1_A3V_SHIFT 15
+#define ICC_CTLR_EL1_A3V_MASK (0x1 << ICC_CTLR_EL1_A3V_SHIFT)
+#define ICC_PMR_EL1_SHIFT 0
+#define ICC_PMR_EL1_MASK (0xff << ICC_PMR_EL1_SHIFT)
+#define ICC_BPR0_EL1_SHIFT 0
+#define ICC_BPR0_EL1_MASK (0x7 << ICC_BPR0_EL1_SHIFT)
+#define ICC_BPR1_EL1_SHIFT 0
+#define ICC_BPR1_EL1_MASK (0x7 << ICC_BPR1_EL1_SHIFT)
+#define ICC_IGRPEN0_EL1_SHIFT 0
+#define ICC_IGRPEN0_EL1_MASK (1 << ICC_IGRPEN0_EL1_SHIFT)
+#define ICC_IGRPEN1_EL1_SHIFT 0
+#define ICC_IGRPEN1_EL1_MASK (1 << ICC_IGRPEN1_EL1_SHIFT)
#define ICC_SRE_EL1_SRE (1U << 0)
/*
@@ -384,12 +406,29 @@
#define ICH_VMCR_CTLR_SHIFT 0
#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
+#define ICH_VMCR_CBPR_SHIFT 4
+#define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
+#define ICH_VMCR_EOIM_SHIFT 9
+#define ICH_VMCR_EOIM_MASK (1 << ICH_VMCR_EOIM_SHIFT)
#define ICH_VMCR_BPR1_SHIFT 18
#define ICH_VMCR_BPR1_MASK (7 << ICH_VMCR_BPR1_SHIFT)
#define ICH_VMCR_BPR0_SHIFT 21
#define ICH_VMCR_BPR0_MASK (7 << ICH_VMCR_BPR0_SHIFT)
#define ICH_VMCR_PMR_SHIFT 24
#define ICH_VMCR_PMR_MASK (0xffUL << ICH_VMCR_PMR_SHIFT)
+#define ICH_VMCR_ENG0_SHIFT 0
+#define ICH_VMCR_ENG0_MASK (1 << ICH_VMCR_ENG0_SHIFT)
+#define ICH_VMCR_ENG1_SHIFT 1
+#define ICH_VMCR_ENG1_MASK (1 << ICH_VMCR_ENG1_SHIFT)
+
+#define ICH_VTR_PRI_BITS_SHIFT 29
+#define ICH_VTR_PRI_BITS_MASK (7 << ICH_VTR_PRI_BITS_SHIFT)
+#define ICH_VTR_ID_BITS_SHIFT 23
+#define ICH_VTR_ID_BITS_MASK (7 << ICH_VTR_ID_BITS_SHIFT)
+#define ICH_VTR_SEIS_SHIFT 22
+#define ICH_VTR_SEIS_MASK (1 << ICH_VTR_SEIS_SHIFT)
+#define ICH_VTR_A3V_SHIFT 21
+#define ICH_VTR_A3V_MASK (1 << ICH_VTR_A3V_SHIFT)
#define ICC_IAR1_EL1_SPURIOUS 0x3ff
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
variables to struct vmcr to support read and write of these fields.
Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
include/linux/irqchip/arm-gic-v3.h | 2 --
virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
virt/kvm/arm/vgic/vgic.h | 5 +++++
5 files changed, 29 insertions(+), 20 deletions(-)
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index d48d886..61646aa 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -404,8 +404,6 @@
#define ICH_HCR_EN (1 << 0)
#define ICH_HCR_UIE (1 << 1)
-#define ICH_VMCR_CTLR_SHIFT 0
-#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
#define ICH_VMCR_CBPR_SHIFT 4
#define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
#define ICH_VMCR_EOIM_SHIFT 9
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 2cb04b7..ad353b5 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
}
}
-static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
- if (kvm_vgic_global_state.type == VGIC_V2)
- vgic_v2_set_vmcr(vcpu, vmcr);
- else
- vgic_v3_set_vmcr(vcpu, vmcr);
-}
-
-static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
- if (kvm_vgic_global_state.type == VGIC_V2)
- vgic_v2_get_vmcr(vcpu, vmcr);
- else
- vgic_v3_get_vmcr(vcpu, vmcr);
-}
-
#define GICC_ARCH_VERSION_V2 0x2
/* These are for userland accesses only, there is no guest-facing emulation. */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 9939d1d..173d6f0 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
return -ENXIO;
}
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+ if (kvm_vgic_global_state.type == VGIC_V2)
+ vgic_v2_set_vmcr(vcpu, vmcr);
+ else
+ vgic_v3_set_vmcr(vcpu, vmcr);
+}
+
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+ if (kvm_vgic_global_state.type == VGIC_V2)
+ vgic_v2_get_vmcr(vcpu, vmcr);
+ else
+ vgic_v3_get_vmcr(vcpu, vmcr);
+}
+
/*
* kvm_mmio_read_buf() returns a value in a format where it can be converted
* to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 9f0dae3..967c295 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
{
u32 vmcr;
- vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
+ vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
+ vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
+ vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
+ vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
}
@@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
{
u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
- vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
+ vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
+ vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
vmcrp->bpr = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
vmcrp->pmr = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
+ vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
+ vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
}
#define INITIAL_PENDBASER_VALUE \
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index d901b0c..c461f6b 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -63,6 +63,9 @@ struct vgic_vmcr {
u32 abpr;
u32 bpr;
u32 pmr;
+ /* Below member variable are valid only for GICv3 */
+ u32 grpen0;
+ u32 grpen1;
};
struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
@@ -150,6 +153,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
#endif
int kvm_register_vgic_device(unsigned long type);
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
int vgic_lazy_init(struct kvm *kvm);
int vgic_init(struct kvm *kvm);
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
variables to struct vmcr to support read and write of these fields.
Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
include/linux/irqchip/arm-gic-v3.h | 2 --
virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
virt/kvm/arm/vgic/vgic.h | 5 +++++
5 files changed, 29 insertions(+), 20 deletions(-)
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index d48d886..61646aa 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -404,8 +404,6 @@
#define ICH_HCR_EN (1 << 0)
#define ICH_HCR_UIE (1 << 1)
-#define ICH_VMCR_CTLR_SHIFT 0
-#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
#define ICH_VMCR_CBPR_SHIFT 4
#define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
#define ICH_VMCR_EOIM_SHIFT 9
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 2cb04b7..ad353b5 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
}
}
-static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
- if (kvm_vgic_global_state.type == VGIC_V2)
- vgic_v2_set_vmcr(vcpu, vmcr);
- else
- vgic_v3_set_vmcr(vcpu, vmcr);
-}
-
-static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
- if (kvm_vgic_global_state.type == VGIC_V2)
- vgic_v2_get_vmcr(vcpu, vmcr);
- else
- vgic_v3_get_vmcr(vcpu, vmcr);
-}
-
#define GICC_ARCH_VERSION_V2 0x2
/* These are for userland accesses only, there is no guest-facing emulation. */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 9939d1d..173d6f0 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
return -ENXIO;
}
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+ if (kvm_vgic_global_state.type == VGIC_V2)
+ vgic_v2_set_vmcr(vcpu, vmcr);
+ else
+ vgic_v3_set_vmcr(vcpu, vmcr);
+}
+
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+ if (kvm_vgic_global_state.type == VGIC_V2)
+ vgic_v2_get_vmcr(vcpu, vmcr);
+ else
+ vgic_v3_get_vmcr(vcpu, vmcr);
+}
+
/*
* kvm_mmio_read_buf() returns a value in a format where it can be converted
* to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 9f0dae3..967c295 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
{
u32 vmcr;
- vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
+ vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
+ vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
+ vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
+ vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
}
@@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
{
u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
- vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
+ vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
+ vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
vmcrp->bpr = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
vmcrp->pmr = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
+ vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
+ vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
}
#define INITIAL_PENDBASER_VALUE \
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index d901b0c..c461f6b 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -63,6 +63,9 @@ struct vgic_vmcr {
u32 abpr;
u32 bpr;
u32 pmr;
+ /* Below member variable are valid only for GICv3 */
+ u32 grpen0;
+ u32 grpen1;
};
struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
@@ -150,6 +153,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
#endif
int kvm_register_vgic_device(unsigned long type);
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
int vgic_lazy_init(struct kvm *kvm);
int vgic_init(struct kvm *kvm);
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
VGICv3 CPU interface registers are accessed using
KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
as 64-bit. The cpu MPIDR value is passed along with register id.
is used to identify the cpu for registers access.
The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 3 +
arch/arm64/kvm/Makefile | 1 +
include/kvm/arm_vgic.h | 9 +
virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 8 +
virt/kvm/arm/vgic/vgic.h | 4 +
8 files changed, 395 insertions(+)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 56dc08d..91c7137 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
+#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
#define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
+#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
+
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
/* Device Control API on vcpu fd */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d50a82a..1a14e29 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 002f092..730a18a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -71,6 +71,9 @@ struct vgic_global {
/* GIC system register CPU interface */
struct static_key_false gicv3_cpuif;
+
+ /* Cache ICH_VTR_EL2 reg value */
+ u32 ich_vtr_el2;
};
extern struct vgic_global kvm_vgic_global_state;
@@ -269,6 +272,12 @@ struct vgic_cpu {
u64 pendbaser;
bool lpis_enabled;
+
+ /* Cache guest priority bits */
+ u32 num_pri_bits;
+
+ /* Cache guest interrupt ID bits */
+ u32 num_id_bits;
};
extern struct static_key_false vgic_v2_cpuif_trap;
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 6c7d30c..da532d1 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
if (!is_write)
*reg = tmp32;
break;
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 regid;
+
+ regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+ ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
+ regid, reg);
+ break;
+ }
default:
ret = -EINVAL;
break;
@@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
reg = tmp32;
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 reg;
+
+ if (get_user(reg, uaddr))
+ return -EFAULT;
+
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
}
return -ENXIO;
}
@@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
tmp32 = reg;
return put_user(tmp32, uaddr);
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 reg;
+
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ return put_user(reg, uaddr);
+ }
}
return -ENXIO;
@@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
break;
case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index b35fb83..519b919 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,6 +23,7 @@
#include "vgic.h"
#include "vgic-mmio.h"
+#include "sys_regs.h"
/* extract @num bytes at @offset bytes offset in data */
unsigned long extract_bytes(u64 data, unsigned int offset,
@@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
break;
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 reg, id;
+ unsigned long vgic_mpidr, mpidr_reg;
+ struct kvm_vcpu *vcpu;
+
+ vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+ KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+ /* Convert plain mpidr value to MPIDR reg format */
+ mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
+
+ vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
+ if (!vcpu)
+ return -EINVAL;
+
+ id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+ return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
+ }
default:
return -ENXIO;
}
diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
new file mode 100644
index 0000000..69d8597
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
@@ -0,0 +1,324 @@
+#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <kvm/iodev.h>
+#include <kvm/arm_vgic.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+#include "sys_regs.h"
+
+static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_vmcr vmcr;
+ u64 val;
+ u32 num_pri_bits, num_id_bits;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ val = p->regval;
+
+ /*
+ * Does not allow update of ICC_CTLR_EL1 if HW does not support
+ * guest programmed ID and PRI bits
+ */
+ num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
+ ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
+ if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
+ return false;
+
+ vgic_v3_cpu->num_pri_bits = num_pri_bits;
+
+ num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
+ ICC_CTLR_EL1_ID_BITS_SHIFT;
+ if (num_id_bits > vgic_v3_cpu->num_id_bits)
+ return false;
+
+ vgic_v3_cpu->num_id_bits = num_id_bits;
+
+ vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
+ vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
+ ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
+ vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
+ ICC_CTLR_EL1_EOImode_SHIFT) <<
+ ICH_VMCR_EOIM_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ val = 0;
+ val |= (vgic_v3_cpu->num_pri_bits - 1) <<
+ ICC_CTLR_EL1_PRI_BITS_SHIFT;
+ val |= vgic_v3_cpu->num_id_bits <<
+ ICC_CTLR_EL1_ID_BITS_SHIFT;
+ val |= ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
+ ICC_CTLR_EL1_SEIS_SHIFT;
+ val |= ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
+ ICC_CTLR_EL1_A3V_SHIFT;
+ val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
+ ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
+ val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
+ ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
+
+ p->regval = val;
+ }
+
+ return true;
+}
+
+static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
+ ICC_BPR0_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
+ ICC_BPR0_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ if (!p->is_write)
+ p->regval = 0;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
+ if (p->is_write) {
+ vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
+ ICC_BPR1_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
+ ICC_BPR1_EL1_MASK;
+ }
+ } else {
+ if (!p->is_write)
+ p->regval = min((vmcr.bpr + 1), 7U);
+ }
+
+ return true;
+}
+
+static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
+ ICC_IGRPEN0_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
+ ICC_IGRPEN0_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
+ ICC_IGRPEN1_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
+ ICC_IGRPEN1_EL1_MASK;
+ }
+
+ return true;
+}
+
+static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p, u8 apr, u8 idx)
+{
+ struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+ uint32_t *ap_reg;
+
+ if (apr)
+ ap_reg = &vgicv3->vgic_ap1r[idx];
+ else
+ ap_reg = &vgicv3->vgic_ap0r[idx];
+
+ if (p->is_write)
+ *ap_reg = p->regval;
+ else
+ p->regval = *ap_reg;
+}
+
+static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r, u8 apr)
+{
+ struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
+ u8 idx = r->Op2 & 3;
+
+ switch (vgic_v3_cpu->num_pri_bits) {
+ case 7:
+ if (idx > 3)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ break;
+ case 6:
+ if (idx > 1)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ break;
+ default:
+ if (idx > 0)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ }
+
+ return;
+err:
+ if (!p->is_write)
+ p->regval = 0;
+}
+
+static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ access_gic_aprn(vcpu, p, r, 0);
+
+ return true;
+}
+
+static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ access_gic_aprn(vcpu, p, r, 1);
+
+ return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ /* Validate SRE bit */
+ if (p->is_write) {
+ if (!(p->regval & ICC_SRE_EL1_SRE))
+ return false;
+ } else {
+ p->regval = vgicv3->vgic_sre;
+ }
+
+ return true;
+}
+
+static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
+ /* ICC_PMR_EL1 */
+ { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
+ /* ICC_BPR0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
+ /* ICC_AP0R0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
+ /* ICC_AP0R1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
+ /* ICC_AP0R2_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
+ /* ICC_AP0R3_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
+ /* ICC_AP1R0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
+ /* ICC_AP1R1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
+ /* ICC_AP1R2_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
+ /* ICC_AP1R3_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
+ /* ICC_BPR1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
+ /* ICC_CTLR_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
+ /* ICC_SRE_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
+ /* ICC_IGRPEN0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
+ /* ICC_GRPEN1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
+};
+
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg)
+{
+ struct sys_reg_params params;
+ u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+ params.regval = *reg;
+ params.is_write = is_write;
+ params.is_aarch32 = false;
+ params.is_32bit = false;
+
+ if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
+ ARRAY_SIZE(gic_v3_icc_reg_descs)))
+ return 0;
+
+ return -ENXIO;
+}
+
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg)
+{
+ struct sys_reg_params params;
+ const struct sys_reg_desc *r;
+ u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+ if (is_write)
+ params.regval = *reg;
+ params.is_write = is_write;
+ params.is_aarch32 = false;
+ params.is_32bit = false;
+
+ r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
+ ARRAY_SIZE(gic_v3_icc_reg_descs));
+ if (!r)
+ return -ENXIO;
+
+ if (!r->access(vcpu, ¶ms, r))
+ return -EINVAL;
+
+ if (!is_write)
+ *reg = params.regval;
+
+ return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 967c295..1139971 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
vgic_v3->vgic_sre = 0;
}
+ vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_ID_BITS_MASK) >>
+ ICH_VTR_ID_BITS_SHIFT;
+ vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_PRI_BITS_MASK) >>
+ ICH_VTR_PRI_BITS_SHIFT) + 1;
+
/* Get the show on the road... */
vgic_v3->vgic_hcr = ICH_HCR_EN;
}
@@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
*/
kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
kvm_vgic_global_state.can_emulate_gicv2 = false;
+ kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
if (!info->vcpu.start) {
kvm_info("GICv3: no GICV resource entry\n");
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index c461f6b..0e632d0 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val);
int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val);
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u64 id, u64 *val);
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg);
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
VGICv3 CPU interface registers are accessed using
KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
as 64-bit. The cpu MPIDR value is passed along with register id.
is used to identify the cpu for registers access.
The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 3 +
arch/arm64/kvm/Makefile | 1 +
include/kvm/arm_vgic.h | 9 +
virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-v3.c | 8 +
virt/kvm/arm/vgic/vgic.h | 4 +
8 files changed, 395 insertions(+)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 56dc08d..91c7137 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
+#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
#define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
+#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
+
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
/* Device Control API on vcpu fd */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d50a82a..1a14e29 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 002f092..730a18a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -71,6 +71,9 @@ struct vgic_global {
/* GIC system register CPU interface */
struct static_key_false gicv3_cpuif;
+
+ /* Cache ICH_VTR_EL2 reg value */
+ u32 ich_vtr_el2;
};
extern struct vgic_global kvm_vgic_global_state;
@@ -269,6 +272,12 @@ struct vgic_cpu {
u64 pendbaser;
bool lpis_enabled;
+
+ /* Cache guest priority bits */
+ u32 num_pri_bits;
+
+ /* Cache guest interrupt ID bits */
+ u32 num_id_bits;
};
extern struct static_key_false vgic_v2_cpuif_trap;
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index 6c7d30c..da532d1 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
if (!is_write)
*reg = tmp32;
break;
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 regid;
+
+ regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+ ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
+ regid, reg);
+ break;
+ }
default:
ret = -EINVAL;
break;
@@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
reg = tmp32;
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 reg;
+
+ if (get_user(reg, uaddr))
+ return -EFAULT;
+
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
}
return -ENXIO;
}
@@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
tmp32 = reg;
return put_user(tmp32, uaddr);
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 reg;
+
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ return put_user(reg, uaddr);
+ }
}
return -ENXIO;
@@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
break;
case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index b35fb83..519b919 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,6 +23,7 @@
#include "vgic.h"
#include "vgic-mmio.h"
+#include "sys_regs.h"
/* extract @num bytes at @offset bytes offset in data */
unsigned long extract_bytes(u64 data, unsigned int offset,
@@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
break;
}
+ case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+ u64 reg, id;
+ unsigned long vgic_mpidr, mpidr_reg;
+ struct kvm_vcpu *vcpu;
+
+ vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+ KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+ /* Convert plain mpidr value to MPIDR reg format */
+ mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
+
+ vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
+ if (!vcpu)
+ return -EINVAL;
+
+ id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+ return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
+ }
default:
return -ENXIO;
}
diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
new file mode 100644
index 0000000..69d8597
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
@@ -0,0 +1,324 @@
+#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <kvm/iodev.h>
+#include <kvm/arm_vgic.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+#include "sys_regs.h"
+
+static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
+ struct vgic_vmcr vmcr;
+ u64 val;
+ u32 num_pri_bits, num_id_bits;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ val = p->regval;
+
+ /*
+ * Does not allow update of ICC_CTLR_EL1 if HW does not support
+ * guest programmed ID and PRI bits
+ */
+ num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
+ ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
+ if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
+ return false;
+
+ vgic_v3_cpu->num_pri_bits = num_pri_bits;
+
+ num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
+ ICC_CTLR_EL1_ID_BITS_SHIFT;
+ if (num_id_bits > vgic_v3_cpu->num_id_bits)
+ return false;
+
+ vgic_v3_cpu->num_id_bits = num_id_bits;
+
+ vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
+ vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
+ ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
+ vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
+ ICC_CTLR_EL1_EOImode_SHIFT) <<
+ ICH_VMCR_EOIM_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ val = 0;
+ val |= (vgic_v3_cpu->num_pri_bits - 1) <<
+ ICC_CTLR_EL1_PRI_BITS_SHIFT;
+ val |= vgic_v3_cpu->num_id_bits <<
+ ICC_CTLR_EL1_ID_BITS_SHIFT;
+ val |= ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
+ ICC_CTLR_EL1_SEIS_SHIFT;
+ val |= ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
+ ICC_CTLR_EL1_A3V_SHIFT;
+ val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
+ ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
+ val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
+ ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
+
+ p->regval = val;
+ }
+
+ return true;
+}
+
+static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
+ ICC_BPR0_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
+ ICC_BPR0_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ if (!p->is_write)
+ p->regval = 0;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
+ if (p->is_write) {
+ vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
+ ICC_BPR1_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
+ ICC_BPR1_EL1_MASK;
+ }
+ } else {
+ if (!p->is_write)
+ p->regval = min((vmcr.bpr + 1), 7U);
+ }
+
+ return true;
+}
+
+static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
+ ICC_IGRPEN0_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
+ ICC_IGRPEN0_EL1_MASK;
+ }
+
+ return true;
+}
+
+static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_vmcr vmcr;
+
+ vgic_get_vmcr(vcpu, &vmcr);
+ if (p->is_write) {
+ vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
+ ICC_IGRPEN1_EL1_SHIFT;
+ vgic_set_vmcr(vcpu, &vmcr);
+ } else {
+ p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
+ ICC_IGRPEN1_EL1_MASK;
+ }
+
+ return true;
+}
+
+static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p, u8 apr, u8 idx)
+{
+ struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+ uint32_t *ap_reg;
+
+ if (apr)
+ ap_reg = &vgicv3->vgic_ap1r[idx];
+ else
+ ap_reg = &vgicv3->vgic_ap0r[idx];
+
+ if (p->is_write)
+ *ap_reg = p->regval;
+ else
+ p->regval = *ap_reg;
+}
+
+static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r, u8 apr)
+{
+ struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
+ u8 idx = r->Op2 & 3;
+
+ switch (vgic_v3_cpu->num_pri_bits) {
+ case 7:
+ if (idx > 3)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ break;
+ case 6:
+ if (idx > 1)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ break;
+ default:
+ if (idx > 0)
+ goto err;
+ vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+ }
+
+ return;
+err:
+ if (!p->is_write)
+ p->regval = 0;
+}
+
+static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ access_gic_aprn(vcpu, p, r, 0);
+
+ return true;
+}
+
+static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ access_gic_aprn(vcpu, p, r, 1);
+
+ return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+
+ /* Validate SRE bit */
+ if (p->is_write) {
+ if (!(p->regval & ICC_SRE_EL1_SRE))
+ return false;
+ } else {
+ p->regval = vgicv3->vgic_sre;
+ }
+
+ return true;
+}
+
+static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
+ /* ICC_PMR_EL1 */
+ { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
+ /* ICC_BPR0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
+ /* ICC_AP0R0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
+ /* ICC_AP0R1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
+ /* ICC_AP0R2_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
+ /* ICC_AP0R3_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
+ /* ICC_AP1R0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
+ /* ICC_AP1R1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
+ /* ICC_AP1R2_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
+ /* ICC_AP1R3_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
+ /* ICC_BPR1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
+ /* ICC_CTLR_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
+ /* ICC_SRE_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
+ /* ICC_IGRPEN0_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
+ /* ICC_GRPEN1_EL1 */
+ { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
+};
+
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg)
+{
+ struct sys_reg_params params;
+ u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+ params.regval = *reg;
+ params.is_write = is_write;
+ params.is_aarch32 = false;
+ params.is_32bit = false;
+
+ if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
+ ARRAY_SIZE(gic_v3_icc_reg_descs)))
+ return 0;
+
+ return -ENXIO;
+}
+
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg)
+{
+ struct sys_reg_params params;
+ const struct sys_reg_desc *r;
+ u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+ if (is_write)
+ params.regval = *reg;
+ params.is_write = is_write;
+ params.is_aarch32 = false;
+ params.is_32bit = false;
+
+ r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
+ ARRAY_SIZE(gic_v3_icc_reg_descs));
+ if (!r)
+ return -ENXIO;
+
+ if (!r->access(vcpu, ¶ms, r))
+ return -EINVAL;
+
+ if (!is_write)
+ *reg = params.regval;
+
+ return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 967c295..1139971 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
vgic_v3->vgic_sre = 0;
}
+ vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_ID_BITS_MASK) >>
+ ICH_VTR_ID_BITS_SHIFT;
+ vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
+ ICH_VTR_PRI_BITS_MASK) >>
+ ICH_VTR_PRI_BITS_SHIFT) + 1;
+
/* Get the show on the road... */
vgic_v3->vgic_hcr = ICH_HCR_EN;
}
@@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
*/
kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
kvm_vgic_global_state.can_emulate_gicv2 = false;
+ kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
if (!info->vcpu.start) {
kvm_info("GICv3: no GICV resource entry\n");
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index c461f6b..0e632d0 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val);
int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
int offset, u32 *val);
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u64 id, u64 *val);
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+ u64 *reg);
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 7/7] arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
-1 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari @ 2016-11-04 11:13 UTC (permalink / raw)
To: marc.zyngier, christoffer.dall, peter.maydell
Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Userspace requires to store and restore of line_level for
level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 6 +++++
virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
virt/kvm/arm/vgic/vgic-mmio-v3.c | 11 ++++++++
virt/kvm/arm/vgic/vgic-mmio.c | 33 ++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-mmio.h | 5 ++++
virt/kvm/arm/vgic/vgic.h | 3 +++
6 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 91c7137..4100f8c 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
+#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
+ (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK 0x3ff
+#define VGIC_LEVEL_INFO_LINE_LEVEL 0
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index da532d1..0f82a91 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -512,6 +512,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
regid, reg);
break;
}
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ unsigned int info, intid;
+
+ info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
+ if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
+ if (is_write)
+ tmp32 = *reg;
+ intid = attr->attr &
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
+ ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
+ intid, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ } else {
+ ret = -EINVAL;
+ }
+ break;
+ }
default:
ret = -EINVAL;
break;
@@ -554,6 +573,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+
+ if (get_user(tmp32, uaddr))
+ return -EFAULT;
+
+ reg = tmp32;
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
}
return -ENXIO;
}
@@ -589,8 +619,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
return ret;
return put_user(reg, uaddr);
}
- }
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ tmp32 = reg;
+ return put_user(tmp32, uaddr);
+ }
+ }
return -ENXIO;
}
@@ -611,11 +651,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
+ VGIC_LEVEL_INFO_LINE_LEVEL)
+ return 0;
+ break;
+ }
case KVM_DEV_ARM_VGIC_GRP_CTRL:
switch (attr->attr) {
case KVM_DEV_ARM_VGIC_CTRL_INIT:
return 0;
}
+ break;
}
return -ENXIO;
}
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 519b919..38b481c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -807,3 +807,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
return vgic_uaccess(vcpu, &rd_dev, is_write,
offset, val);
}
+
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u32 intid, u32 *val)
+{
+ if (is_write)
+ vgic_write_irq_line_level_info(vcpu, intid, *val);
+ else
+ *val = vgic_read_irq_line_level_info(vcpu, intid);
+
+ return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 173d6f0..fb018eb 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
}
}
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
+{
+ int i;
+ unsigned long val = 0;
+
+ for (i = 0; i < 32; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ if (irq->line_level)
+ val |= (1U << i);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return val;
+}
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+ const unsigned long val)
+{
+ int i;
+
+ for_each_set_bit(i, &val, 32) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ spin_lock(&irq->irq_lock);
+ irq->line_level = true;
+ spin_unlock(&irq->irq_lock);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
static int match_region(const void *key, const void *elt)
{
const unsigned int offset = (unsigned long)key;
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index acbf99e..938702c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
const struct vgic_register_region *regions,
int nr_regions, gpa_t addr);
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+ const unsigned long val);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 0e632d0..77d3d84 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -130,6 +130,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
u64 id, u64 *val);
int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
u64 *reg);
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u32 intid, u32 *val);
+
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v8 7/7] arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
@ 2016-11-04 11:13 ` vijay.kilari at gmail.com
0 siblings, 0 replies; 62+ messages in thread
From: vijay.kilari at gmail.com @ 2016-11-04 11:13 UTC (permalink / raw)
To: linux-arm-kernel
From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Userspace requires to store and restore of line_level for
level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
arch/arm64/include/uapi/asm/kvm.h | 6 +++++
virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
virt/kvm/arm/vgic/vgic-mmio-v3.c | 11 ++++++++
virt/kvm/arm/vgic/vgic-mmio.c | 33 ++++++++++++++++++++++++
virt/kvm/arm/vgic/vgic-mmio.h | 5 ++++
virt/kvm/arm/vgic/vgic.h | 3 +++
6 files changed, 107 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 91c7137..4100f8c 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
#define KVM_DEV_ARM_VGIC_GRP_CTRL 4
#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
+#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
+ (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK 0x3ff
+#define VGIC_LEVEL_INFO_LINE_LEVEL 0
#define KVM_DEV_ARM_VGIC_CTRL_INIT 0
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index da532d1..0f82a91 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -512,6 +512,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
regid, reg);
break;
}
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ unsigned int info, intid;
+
+ info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
+ if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
+ if (is_write)
+ tmp32 = *reg;
+ intid = attr->attr &
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
+ ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
+ intid, &tmp32);
+ if (!is_write)
+ *reg = tmp32;
+ } else {
+ ret = -EINVAL;
+ }
+ break;
+ }
default:
ret = -EINVAL;
break;
@@ -554,6 +573,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
return vgic_v3_attr_regs_access(dev, attr, ®, true);
}
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+
+ if (get_user(tmp32, uaddr))
+ return -EFAULT;
+
+ reg = tmp32;
+ return vgic_v3_attr_regs_access(dev, attr, ®, true);
+ }
}
return -ENXIO;
}
@@ -589,8 +619,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
return ret;
return put_user(reg, uaddr);
}
- }
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u64 reg;
+ u32 tmp32;
+ ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
+ if (ret)
+ return ret;
+ tmp32 = reg;
+ return put_user(tmp32, uaddr);
+ }
+ }
return -ENXIO;
}
@@ -611,11 +651,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
return vgic_v3_has_attr_regs(dev, attr);
case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
return 0;
+ case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+ if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+ KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
+ VGIC_LEVEL_INFO_LINE_LEVEL)
+ return 0;
+ break;
+ }
case KVM_DEV_ARM_VGIC_GRP_CTRL:
switch (attr->attr) {
case KVM_DEV_ARM_VGIC_CTRL_INIT:
return 0;
}
+ break;
}
return -ENXIO;
}
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 519b919..38b481c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -807,3 +807,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
return vgic_uaccess(vcpu, &rd_dev, is_write,
offset, val);
}
+
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u32 intid, u32 *val)
+{
+ if (is_write)
+ vgic_write_irq_line_level_info(vcpu, intid, *val);
+ else
+ *val = vgic_read_irq_line_level_info(vcpu, intid);
+
+ return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 173d6f0..fb018eb 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
}
}
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
+{
+ int i;
+ unsigned long val = 0;
+
+ for (i = 0; i < 32; i++) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ if (irq->line_level)
+ val |= (1U << i);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+
+ return val;
+}
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+ const unsigned long val)
+{
+ int i;
+
+ for_each_set_bit(i, &val, 32) {
+ struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+ spin_lock(&irq->irq_lock);
+ irq->line_level = true;
+ spin_unlock(&irq->irq_lock);
+
+ vgic_put_irq(vcpu->kvm, irq);
+ }
+}
+
static int match_region(const void *key, const void *elt)
{
const unsigned int offset = (unsigned long)key;
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index acbf99e..938702c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
const struct vgic_register_region *regions,
int nr_regions, gpa_t addr);
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+ const unsigned long val);
+
unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 0e632d0..77d3d84 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -130,6 +130,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
u64 id, u64 *val);
int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
u64 *reg);
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+ u32 intid, u32 *val);
+
#else
static inline int vgic_register_its_iodevs(struct kvm *kvm)
{
--
1.9.1
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 11:47 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 11:47 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> This patchset adds API for saving and restoring
> of VGICv3 registers to support live migration with new vgic feature.
> This API definition is as per version of VGICv3 specification
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> The patch 3 & 4 are picked from the Pavel's previous implementation.
> http://www.spinics.net/lists/kvm/msg122040.html
Do we have QEMU/kvmtool patches somewhere at this point so that I can
test this?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-16 11:47 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 11:47 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> This patchset adds API for saving and restoring
> of VGICv3 registers to support live migration with new vgic feature.
> This API definition is as per version of VGICv3 specification
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> The patch 3 & 4 are picked from the Pavel's previous implementation.
> http://www.spinics.net/lists/kvm/msg122040.html
Do we have QEMU/kvmtool patches somewhere at this point so that I can
test this?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
2016-11-16 11:47 ` Christoffer Dall
@ 2016-11-16 14:54 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-16 14:54 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Wed, Nov 16, 2016 at 5:17 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> This patchset adds API for saving and restoring
>> of VGICv3 registers to support live migration with new vgic feature.
>> This API definition is as per version of VGICv3 specification
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> The patch 3 & 4 are picked from the Pavel's previous implementation.
>> http://www.spinics.net/lists/kvm/msg122040.html
>
> Do we have QEMU/kvmtool patches somewhere at this point so that I can
> test this?
I will send you next revision of QEMU patches tomorrow.
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-16 14:54 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-16 14:54 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 16, 2016 at 5:17 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari at gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> This patchset adds API for saving and restoring
>> of VGICv3 registers to support live migration with new vgic feature.
>> This API definition is as per version of VGICv3 specification
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> The patch 3 & 4 are picked from the Pavel's previous implementation.
>> http://www.spinics.net/lists/kvm/msg122040.html
>
> Do we have QEMU/kvmtool patches somewhere at this point so that I can
> test this?
I will send you next revision of QEMU patches tomorrow.
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
2016-11-16 14:54 ` Vijay Kilari
@ 2016-11-16 15:11 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 15:11 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Wed, Nov 16, 2016 at 08:24:16PM +0530, Vijay Kilari wrote:
> On Wed, Nov 16, 2016 at 5:17 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari@gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> This patchset adds API for saving and restoring
> >> of VGICv3 registers to support live migration with new vgic feature.
> >> This API definition is as per version of VGICv3 specification
> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>
> >> The patch 3 & 4 are picked from the Pavel's previous implementation.
> >> http://www.spinics.net/lists/kvm/msg122040.html
> >
> > Do we have QEMU/kvmtool patches somewhere at this point so that I can
> > test this?
>
> I will send you next revision of QEMU patches tomorrow.
>
Sounds good, thanks.
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-16 15:11 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 15:11 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Nov 16, 2016 at 08:24:16PM +0530, Vijay Kilari wrote:
> On Wed, Nov 16, 2016 at 5:17 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari at gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> This patchset adds API for saving and restoring
> >> of VGICv3 registers to support live migration with new vgic feature.
> >> This API definition is as per version of VGICv3 specification
> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>
> >> The patch 3 & 4 are picked from the Pavel's previous implementation.
> >> http://www.spinics.net/lists/kvm/msg122040.html
> >
> > Do we have QEMU/kvmtool patches somewhere at this point so that I can
> > test this?
>
> I will send you next revision of QEMU patches tomorrow.
>
Sounds good, thanks.
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 18:52 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> Read and write of some registers like ISPENDR and ICPENDR
> from userspace requires special handling when compared to
> guest access for these registers.
>
> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> for handling of ISPENDR, ICPENDR registers handling.
>
> Add infrastructure to support guest and userspace read
> and write for the required registers
> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
> 4 files changed, 169 insertions(+), 51 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index b44b359..0b32f40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> return -ENXIO;
> }
>
> -/*
> - * When userland tries to access the VGIC register handlers, we need to
> - * create a usable struct vgic_io_device to be passed to the handlers and we
> - * have to set up a buffer similar to what would have happened if a guest MMIO
> - * access occurred, including doing endian conversions on BE systems.
> - */
> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> - bool is_write, int offset, u32 *val)
> -{
> - unsigned int len = 4;
> - u8 buf[4];
> - int ret;
> -
> - if (is_write) {
> - vgic_data_host_to_mmio_bus(buf, len, *val);
> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> - } else {
> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> - if (!ret)
> - *val = vgic_data_mmio_bus_to_host(buf, len);
> - }
> -
> - return ret;
> -}
> -
> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val)
> {
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 0d3c76a..ce2708d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
> return 0;
> }
>
> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
> + gpa_t addr, unsigned int len)
> +{
> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> + u32 value = 0;
> + int i;
> +
> + /*
> + * A level triggerred interrupt pending state is latched in both
> + * "soft_pending" and "line_level" variables. Userspace will save
> + * and restore soft_pending and line_level separately.
> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> + * handling of ISPENDR and ICPENDR.
> + */
> + for (i = 0; i < len * 8; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> + value |= (1U << i);
> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> + value |= (1U << i);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +
> + return value;
> +}
> +
> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
> + gpa_t addr, unsigned int len,
> + unsigned long val)
> +{
> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> + int i;
> +
> + for (i = 0; i < len * 8; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + spin_lock(&irq->irq_lock);
> + if (test_bit(i, &val)) {
> + irq->pending = true;
> + irq->soft_pending = true;
In the vgic_mmio_write_spending function we only set the soft_pending
state to true if the interrupt is a level-triggered interrupt.
Should we check if that's the case here as well before setting the
soft_pending state?
Otherwise, this patch looks good.
Thanks,
-Christoffer
> + vgic_queue_irq_unlock(vcpu->kvm, irq);
> + } else {
> + irq->soft_pending = false;
> + if (irq->config == VGIC_CONFIG_EDGE ||
> + (irq->config == VGIC_CONFIG_LEVEL &&
> + !irq->line_level))
> + irq->pending = false;
> + spin_unlock(&irq->irq_lock);
> + }
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +}
> +
> /* We want to avoid outer shareable. */
> u64 vgic_sanitise_shareability(u64 field)
> {
> @@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> * We take some special care here to fix the calculation of the register
> * offset.
> */
> -#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc) \
> +#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
> { \
> .reg_offset = off, \
> .bits_per_irq = bpi, \
> @@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> .access_flags = acc, \
> .read = rd, \
> .write = wr, \
> + .uaccess_read = ur, \
> + .uaccess_write = uw, \
> }
>
> static const struct vgic_register_region vgic_v3_dist_registers[] = {
> @@ -380,40 +438,42 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> - vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
> + vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
> - vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
> + vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
> - vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
> + vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
> - vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
> + vgic_mmio_read_pending, vgic_mmio_write_spending,
> + vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
> - vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
> + vgic_mmio_read_pending, vgic_mmio_write_cpending,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
> - vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
> + vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
> - vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
> + vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
> - vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
> - VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> + vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
> + 8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
> - vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
> VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
> - vgic_mmio_read_config, vgic_mmio_write_config, 2,
> + vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
> - vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
> - vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
> + vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
> vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
> @@ -451,11 +511,13 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
> vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
> VGIC_ACCESS_32bit),
> - REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
> - vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
> + REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
> + vgic_mmio_read_pending, vgic_mmio_write_spending,
> + vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 4,
> VGIC_ACCESS_32bit),
> - REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
> - vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
> + REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
> + vgic_mmio_read_pending, vgic_mmio_write_cpending,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
> vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index e18b30d..31f85df 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
> return false;
> }
>
> +static const struct vgic_register_region *
> +vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
> +{
> + const struct vgic_register_region *region;
> +
> + region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> + addr - iodev->base_addr);
> + if (!region || !check_region(region, addr, len))
> + return NULL;
> +
> + return region;
> +}
> +
> +static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> + gpa_t addr, u32 *val)
> +{
> + struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> + const struct vgic_register_region *region;
> + struct kvm_vcpu *r_vcpu;
> +
> + region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> + if (!region) {
> + *val = 0;
> + return 0;
> + }
> +
> + r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> + if (region->uaccess_read)
> + *val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
> + else
> + *val = region->read(r_vcpu, addr, sizeof(u32));
> +
> + return 0;
> +}
> +
> +static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> + gpa_t addr, const u32 *val)
> +{
> + struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> + const struct vgic_register_region *region;
> + struct kvm_vcpu *r_vcpu;
> +
> + region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> + if (!region)
> + return 0;
> +
> + r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> + if (region->uaccess_write)
> + region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
> + else
> + region->write(r_vcpu, addr, sizeof(u32), *val);
> +
> + return 0;
> +}
> +
> +/*
> + * Userland access to VGIC registers.
> + */
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> + bool is_write, int offset, u32 *val)
> +{
> + if (is_write)
> + return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
> + else
> + return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
> +}
> +
> static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> gpa_t addr, int len, void *val)
> {
> @@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> const struct vgic_register_region *region;
> unsigned long data = 0;
>
> - region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> - addr - iodev->base_addr);
> - if (!region || !check_region(region, addr, len)) {
> + region = vgic_get_mmio_region(iodev, addr, len);
> + if (!region) {
> memset(val, 0, len);
> return 0;
> }
> @@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> const struct vgic_register_region *region;
> unsigned long data = vgic_data_mmio_bus_to_host(val, len);
>
> - region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> - addr - iodev->base_addr);
> + region = vgic_get_mmio_region(iodev, addr, len);
> if (!region)
> return 0;
>
> - if (!check_region(region, addr, len))
> - return 0;
> -
> switch (iodev->iodev_type) {
> case IODEV_CPUIF:
> region->write(vcpu, addr, len, data);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 4c34d39..97e6df7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -34,6 +34,10 @@ struct vgic_register_region {
> gpa_t addr, unsigned int len,
> unsigned long val);
> };
> + unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
> + unsigned int len);
> + void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
> + unsigned int len, unsigned long val);
> };
>
> extern struct kvm_io_device_ops kvm_io_gic_ops;
> @@ -86,6 +90,18 @@ struct vgic_register_region {
> .write = wr, \
> }
>
> +#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
> + { \
> + .reg_offset = off, \
> + .bits_per_irq = 0, \
> + .len = length, \
> + .access_flags = acc, \
> + .read = rd, \
> + .write = wr, \
> + .uaccess_read = urd, \
> + .uaccess_write = uwr, \
> + }
> +
> int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
> struct vgic_register_region *reg_desc,
> struct vgic_io_device *region,
> @@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> gpa_t addr, unsigned int len,
> unsigned long val);
>
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> + bool is_write, int offset, u32 *val);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> --
> 1.9.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
@ 2016-11-16 18:52 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> Read and write of some registers like ISPENDR and ICPENDR
> from userspace requires special handling when compared to
> guest access for these registers.
>
> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> for handling of ISPENDR, ICPENDR registers handling.
>
> Add infrastructure to support guest and userspace read
> and write for the required registers
> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
> 4 files changed, 169 insertions(+), 51 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index b44b359..0b32f40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> return -ENXIO;
> }
>
> -/*
> - * When userland tries to access the VGIC register handlers, we need to
> - * create a usable struct vgic_io_device to be passed to the handlers and we
> - * have to set up a buffer similar to what would have happened if a guest MMIO
> - * access occurred, including doing endian conversions on BE systems.
> - */
> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> - bool is_write, int offset, u32 *val)
> -{
> - unsigned int len = 4;
> - u8 buf[4];
> - int ret;
> -
> - if (is_write) {
> - vgic_data_host_to_mmio_bus(buf, len, *val);
> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> - } else {
> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> - if (!ret)
> - *val = vgic_data_mmio_bus_to_host(buf, len);
> - }
> -
> - return ret;
> -}
> -
> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val)
> {
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 0d3c76a..ce2708d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
> return 0;
> }
>
> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
> + gpa_t addr, unsigned int len)
> +{
> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> + u32 value = 0;
> + int i;
> +
> + /*
> + * A level triggerred interrupt pending state is latched in both
> + * "soft_pending" and "line_level" variables. Userspace will save
> + * and restore soft_pending and line_level separately.
> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> + * handling of ISPENDR and ICPENDR.
> + */
> + for (i = 0; i < len * 8; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> + value |= (1U << i);
> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> + value |= (1U << i);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +
> + return value;
> +}
> +
> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
> + gpa_t addr, unsigned int len,
> + unsigned long val)
> +{
> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> + int i;
> +
> + for (i = 0; i < len * 8; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + spin_lock(&irq->irq_lock);
> + if (test_bit(i, &val)) {
> + irq->pending = true;
> + irq->soft_pending = true;
In the vgic_mmio_write_spending function we only set the soft_pending
state to true if the interrupt is a level-triggered interrupt.
Should we check if that's the case here as well before setting the
soft_pending state?
Otherwise, this patch looks good.
Thanks,
-Christoffer
> + vgic_queue_irq_unlock(vcpu->kvm, irq);
> + } else {
> + irq->soft_pending = false;
> + if (irq->config == VGIC_CONFIG_EDGE ||
> + (irq->config == VGIC_CONFIG_LEVEL &&
> + !irq->line_level))
> + irq->pending = false;
> + spin_unlock(&irq->irq_lock);
> + }
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +}
> +
> /* We want to avoid outer shareable. */
> u64 vgic_sanitise_shareability(u64 field)
> {
> @@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> * We take some special care here to fix the calculation of the register
> * offset.
> */
> -#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc) \
> +#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
> { \
> .reg_offset = off, \
> .bits_per_irq = bpi, \
> @@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> .access_flags = acc, \
> .read = rd, \
> .write = wr, \
> + .uaccess_read = ur, \
> + .uaccess_write = uw, \
> }
>
> static const struct vgic_register_region vgic_v3_dist_registers[] = {
> @@ -380,40 +438,42 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> - vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
> + vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
> - vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
> + vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
> - vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
> + vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
> - vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
> + vgic_mmio_read_pending, vgic_mmio_write_spending,
> + vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
> - vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
> + vgic_mmio_read_pending, vgic_mmio_write_cpending,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
> - vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
> + vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
> - vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
> + vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
> - vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
> - VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> + vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
> + 8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
> - vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
> VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
> - vgic_mmio_read_config, vgic_mmio_write_config, 2,
> + vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
> - vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
> - vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
> + vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
> vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
> @@ -451,11 +511,13 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
> vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
> VGIC_ACCESS_32bit),
> - REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
> - vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
> + REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
> + vgic_mmio_read_pending, vgic_mmio_write_spending,
> + vgic_v3_uaccess_read_pending, vgic_v3_uaccess_write_pending, 4,
> VGIC_ACCESS_32bit),
> - REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
> - vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
> + REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
> + vgic_mmio_read_pending, vgic_mmio_write_cpending,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
> vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index e18b30d..31f85df 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
> return false;
> }
>
> +static const struct vgic_register_region *
> +vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
> +{
> + const struct vgic_register_region *region;
> +
> + region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> + addr - iodev->base_addr);
> + if (!region || !check_region(region, addr, len))
> + return NULL;
> +
> + return region;
> +}
> +
> +static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> + gpa_t addr, u32 *val)
> +{
> + struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> + const struct vgic_register_region *region;
> + struct kvm_vcpu *r_vcpu;
> +
> + region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> + if (!region) {
> + *val = 0;
> + return 0;
> + }
> +
> + r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> + if (region->uaccess_read)
> + *val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
> + else
> + *val = region->read(r_vcpu, addr, sizeof(u32));
> +
> + return 0;
> +}
> +
> +static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> + gpa_t addr, const u32 *val)
> +{
> + struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> + const struct vgic_register_region *region;
> + struct kvm_vcpu *r_vcpu;
> +
> + region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> + if (!region)
> + return 0;
> +
> + r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> + if (region->uaccess_write)
> + region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
> + else
> + region->write(r_vcpu, addr, sizeof(u32), *val);
> +
> + return 0;
> +}
> +
> +/*
> + * Userland access to VGIC registers.
> + */
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> + bool is_write, int offset, u32 *val)
> +{
> + if (is_write)
> + return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
> + else
> + return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
> +}
> +
> static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> gpa_t addr, int len, void *val)
> {
> @@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> const struct vgic_register_region *region;
> unsigned long data = 0;
>
> - region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> - addr - iodev->base_addr);
> - if (!region || !check_region(region, addr, len)) {
> + region = vgic_get_mmio_region(iodev, addr, len);
> + if (!region) {
> memset(val, 0, len);
> return 0;
> }
> @@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> const struct vgic_register_region *region;
> unsigned long data = vgic_data_mmio_bus_to_host(val, len);
>
> - region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> - addr - iodev->base_addr);
> + region = vgic_get_mmio_region(iodev, addr, len);
> if (!region)
> return 0;
>
> - if (!check_region(region, addr, len))
> - return 0;
> -
> switch (iodev->iodev_type) {
> case IODEV_CPUIF:
> region->write(vcpu, addr, len, data);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 4c34d39..97e6df7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -34,6 +34,10 @@ struct vgic_register_region {
> gpa_t addr, unsigned int len,
> unsigned long val);
> };
> + unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
> + unsigned int len);
> + void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
> + unsigned int len, unsigned long val);
> };
>
> extern struct kvm_io_device_ops kvm_io_gic_ops;
> @@ -86,6 +90,18 @@ struct vgic_register_region {
> .write = wr, \
> }
>
> +#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
> + { \
> + .reg_offset = off, \
> + .bits_per_irq = 0, \
> + .len = length, \
> + .access_flags = acc, \
> + .read = rd, \
> + .write = wr, \
> + .uaccess_read = urd, \
> + .uaccess_write = uwr, \
> + }
> +
> int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
> struct vgic_register_region *reg_desc,
> struct vgic_io_device *region,
> @@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> gpa_t addr, unsigned int len,
> unsigned long val);
>
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> + bool is_write, int offset, u32 *val);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> --
> 1.9.1
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 2/7] arm/arm64: vgic: Add distributor and redistributor access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 18:52 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:28PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> VGICv3 Distributor and Redistributor registers are accessed using
> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
DIST_REGS and REDIST_REGS ?
> with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
> These registers are accessed as 32-bit and cpu mpidr
> value passed along with register offset is used to identify the
> cpu for redistributor registers access.
>
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
I think you should just point to the Documentation/... path in the
kernel now when it's merged.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 4 +
> virt/kvm/arm/vgic/vgic-kvm-device.c | 149 +++++++++++++++++++++++++++++++++---
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 +---
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 72 +++++++++++++++++
> virt/kvm/arm/vgic/vgic-mmio.c | 22 ++++++
> virt/kvm/arm/vgic/vgic-mmio.h | 4 +
> virt/kvm/arm/vgic/vgic.h | 33 ++++++++
> 7 files changed, 276 insertions(+), 24 deletions(-)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 3051f86..56dc08d 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
> #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> +#define KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
> +#define KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
> + (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> /* Device Control API on vcpu fd */
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index ce1f4ed..6c7d30c 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -237,7 +237,7 @@ struct vgic_reg_attr {
> gpa_t addr;
> };
>
> -static int parse_vgic_v2_attr(struct kvm_device *dev,
> +static int vgic_v2_parse_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr,
> struct vgic_reg_attr *reg_attr)
> {
> @@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
> }
>
> /**
> - * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
> + * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
> *
> * @dev: kvm device handle
> * @attr: kvm device attribute
> * @reg: address the value is read or written
> * @is_write: true if userspace is writing a register
> */
> -static int vgic_attr_regs_access_v2(struct kvm_device *dev,
> +static int vgic_v2_attr_regs_access(struct kvm_device *dev,
> struct kvm_device_attr *attr,
> u32 *reg, bool is_write)
> {
> @@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
> struct kvm_vcpu *vcpu;
> int ret;
>
> - ret = parse_vgic_v2_attr(dev, attr, ®_attr);
> + ret = vgic_v2_parse_attr(dev, attr, ®_attr);
> if (ret)
> return ret;
>
> @@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
>
> mutex_lock(&dev->kvm->lock);
>
> - ret = vgic_init(dev->kvm);
> - if (ret)
> + if (unlikely(!vgic_initialized(dev->kvm))) {
> + ret = -EBUSY;
> goto out;
> + }
eh, GICv2 should still support lazy init, shouldn't it? Am I
misunderstanding this change?
>
> if (!lock_all_vcpus(dev->kvm)) {
> ret = -EBUSY;
> @@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
> if (get_user(reg, uaddr))
> return -EFAULT;
>
> - return vgic_attr_regs_access_v2(dev, attr, ®, true);
> + return vgic_v2_attr_regs_access(dev, attr, ®, true);
> }
> }
>
> @@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
> u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> u32 reg = 0;
>
> - ret = vgic_attr_regs_access_v2(dev, attr, ®, false);
> + ret = vgic_v2_attr_regs_access(dev, attr, ®, false);
> if (ret)
> return ret;
> return put_user(reg, uaddr);
> @@ -430,16 +431,141 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
> .has_attr = vgic_v2_has_attr,
> };
>
> +static int vgic_v3_parse_attr(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + struct vgic_reg_attr *reg_attr)
> +{
> + unsigned long vgic_mpidr, mpidr_reg;
> +
> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> + reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> + if (!reg_attr->vcpu)
> + return -EINVAL;
> +
> + reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> +
> + return 0;
> +}
> +
> +/*
> + * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
> + *
> + * @dev: kvm device handle
> + * @attr: kvm device attribute
> + * @reg: address the value is read or written
> + * @is_write: true if userspace is writing a register
> + */
> +static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + u64 *reg, bool is_write)
> +{
> + struct vgic_reg_attr reg_attr;
> + gpa_t addr;
> + struct kvm_vcpu *vcpu;
> + int ret;
> + u32 tmp32;
> +
> + ret = vgic_v3_parse_attr(dev, attr, ®_attr);
> + if (ret)
> + return ret;
> +
> + vcpu = reg_attr.vcpu;
> + addr = reg_attr.addr;
> +
> + mutex_lock(&dev->kvm->lock);
> +
> + if (unlikely(!vgic_initialized(dev->kvm))) {
> + ret = -EBUSY;
> + goto out;
> + }
> +
> + if (!lock_all_vcpus(dev->kvm)) {
> + ret = -EBUSY;
> + goto out;
> + }
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + if (is_write)
> + tmp32 = *reg;
> +
> + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + if (is_write)
> + tmp32 = *reg;
> +
> + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + unlock_all_vcpus(dev->kvm);
> +out:
> + mutex_unlock(&dev->kvm->lock);
> + return ret;
> +}
> +
> static int vgic_v3_set_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr)
> {
> - return vgic_set_common_attr(dev, attr);
> + int ret;
> +
> + ret = vgic_set_common_attr(dev, attr);
> + if (ret != -ENXIO)
> + return ret;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 tmp32;
> + u64 reg;
> +
> + if (get_user(tmp32, uaddr))
> + return -EFAULT;
> +
> + reg = tmp32;
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> + }
> + return -ENXIO;
> }
>
> static int vgic_v3_get_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr)
> {
> - return vgic_get_common_attr(dev, attr);
> + int ret;
> +
> + ret = vgic_get_common_attr(dev, attr);
> + if (ret != -ENXIO)
> + return ret;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
> +
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + tmp32 = reg;
> + return put_user(tmp32, uaddr);
> + }
> + }
> +
> + return -ENXIO;
> }
>
> static int vgic_v3_has_attr(struct kvm_device *dev,
> @@ -453,6 +579,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> return 0;
> }
> break;
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> case KVM_DEV_ARM_VGIC_GRP_CTRL:
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index 0b32f40..2cb04b7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
>
> int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> {
> - int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
> const struct vgic_register_region *regions;
> gpa_t addr;
> - int nr_regions, i, len;
> + int nr_regions;
>
> addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
>
> @@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> if (addr & 3)
> return -ENXIO;
>
> - for (i = 0; i < nr_regions; i++) {
> - if (regions[i].bits_per_irq)
> - len = (regions[i].bits_per_irq * nr_irqs) / 8;
> - else
> - len = regions[i].len;
> -
> - if (regions[i].reg_offset <= addr &&
> - regions[i].reg_offset + len > addr)
> - return 0;
> - }
> -
> - return -ENXIO;
> + return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
> }
>
> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index ce2708d..b35fb83 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -18,6 +18,8 @@
> #include <kvm/arm_vgic.h>
>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
>
> #include "vgic.h"
> #include "vgic-mmio.h"
> @@ -437,6 +439,9 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
> vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
> VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
> + vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> @@ -484,12 +489,18 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
> vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
> VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
> vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
> vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
> vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> @@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
> return ret;
> }
>
> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> +{
> + const struct vgic_register_region *regions;
> + gpa_t addr;
> + int nr_regions;
> +
> + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + regions = vgic_v3_dist_registers;
> + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
> + regions = vgic_v3_rdbase_registers;
> + nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> + break;
> + }
> + default:
> + return -ENXIO;
> + }
> +
> + /* We only support aligned 32-bit accesses. */
> + if (addr & 3)
> + return -ENXIO;
> +
> + return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
> +}
> /*
> * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
> * generation register ICC_SGI1R_EL1) with a given VCPU.
> @@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> vgic_put_irq(vcpu->kvm, irq);
> }
> }
> +
> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val)
> +{
> + struct vgic_io_device dev = {
> + .regions = vgic_v3_dist_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
> + };
> +
> + return vgic_uaccess(vcpu, &dev, is_write, offset, val);
> +}
> +
> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val)
> +{
> + struct vgic_io_device rd_dev = {
> + .regions = vgic_v3_rdbase_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
> + };
> +
> + struct vgic_io_device sgi_dev = {
> + .regions = vgic_v3_sgibase_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
> + };
> +
> + /* SGI_base is the next 64K frame after RD_base */
> + if (offset >= SZ_64K)
> + return vgic_uaccess(vcpu, &sgi_dev, is_write,
> + offset - SZ_64K, val);
> + else
> + return vgic_uaccess(vcpu, &rd_dev, is_write,
> + offset, val);
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 31f85df..9939d1d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -394,6 +394,28 @@ static int match_region(const void *key, const void *elt)
> sizeof(region[0]), match_region);
> }
>
> +/* Check if address falls within the region */
> +int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> + const struct vgic_register_region *regions,
> + int nr_regions, gpa_t addr)
> +{
> + int i, len;
> + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
> +
> + for (i = 0; i < nr_regions; i++) {
> + if (regions[i].bits_per_irq)
> + len = (regions[i].bits_per_irq * nr_irqs) / 8;
> + else
> + len = regions[i].len;
> +
> + if (regions[i].reg_offset <= addr &&
> + regions[i].reg_offset + len > addr)
> + return 0;
> + }
> +
> + return -ENXIO;
> +}
> +
> /*
> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> * to a byte array and be directly observed as the guest wanted it to appear
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 97e6df7..acbf99e 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> bool is_write, int offset, u32 *val);
>
> +int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> + const struct vgic_register_region *regions,
> + int nr_regions, gpa_t addr);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 9d9e014..d901b0c 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -30,6 +30,34 @@
>
> #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
>
> +#define VGIC_AFFINITY_0_SHIFT 0
> +#define VGIC_AFFINITY_0_MASK (0xffUL << VGIC_AFFINITY_0_SHIFT)
> +#define VGIC_AFFINITY_1_SHIFT 8
> +#define VGIC_AFFINITY_1_MASK (0xffUL << VGIC_AFFINITY_1_SHIFT)
> +#define VGIC_AFFINITY_2_SHIFT 16
> +#define VGIC_AFFINITY_2_MASK (0xffUL << VGIC_AFFINITY_2_SHIFT)
> +#define VGIC_AFFINITY_3_SHIFT 24
> +#define VGIC_AFFINITY_3_MASK (0xffUL << VGIC_AFFINITY_3_SHIFT)
> +
> +#define VGIC_AFFINITY_LEVEL(reg, level) \
> + ((((reg) & VGIC_AFFINITY_## level ##_MASK) \
> + >> VGIC_AFFINITY_## level ##_SHIFT) << MPIDR_LEVEL_SHIFT(level))
> +
> +/*
> + * The userspace encode the affinity differently from the MPIDR,
Userspace encodes
> + * Below macro converts vgic userspace format to MPIDR reg format.
> + */
> +#define VGIC_TO_MPIDR(val) (VGIC_AFFINITY_LEVEL(val, 0) | \
> + VGIC_AFFINITY_LEVEL(val, 1) | \
> + VGIC_AFFINITY_LEVEL(val, 2) | \
> + VGIC_AFFINITY_LEVEL(val, 3))
> +
> +#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
> + KVM_REG_ARM64_SYSREG_OP1_MASK | \
> + KVM_REG_ARM64_SYSREG_CRN_MASK | \
> + KVM_REG_ARM64_SYSREG_CRM_MASK | \
> + KVM_REG_ARM64_SYSREG_OP2_MASK)
> +
> struct vgic_vmcr {
> u32 ctlr;
> u32 abpr;
> @@ -90,6 +118,11 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
> int kvm_vgic_register_its_device(void);
> void vgic_enable_lpis(struct kvm_vcpu *vcpu);
> int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val);
> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val);
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
> 1.9.1
>
Otherwise looks ok.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 2/7] arm/arm64: vgic: Add distributor and redistributor access
@ 2016-11-16 18:52 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:28PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> VGICv3 Distributor and Redistributor registers are accessed using
> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
DIST_REGS and REDIST_REGS ?
> with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
> These registers are accessed as 32-bit and cpu mpidr
> value passed along with register offset is used to identify the
> cpu for redistributor registers access.
>
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
I think you should just point to the Documentation/... path in the
kernel now when it's merged.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 4 +
> virt/kvm/arm/vgic/vgic-kvm-device.c | 149 +++++++++++++++++++++++++++++++++---
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 +---
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 72 +++++++++++++++++
> virt/kvm/arm/vgic/vgic-mmio.c | 22 ++++++
> virt/kvm/arm/vgic/vgic-mmio.h | 4 +
> virt/kvm/arm/vgic/vgic.h | 33 ++++++++
> 7 files changed, 276 insertions(+), 24 deletions(-)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 3051f86..56dc08d 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
> #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> #define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> #define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> +#define KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
> +#define KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
> + (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> +#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> /* Device Control API on vcpu fd */
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index ce1f4ed..6c7d30c 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -237,7 +237,7 @@ struct vgic_reg_attr {
> gpa_t addr;
> };
>
> -static int parse_vgic_v2_attr(struct kvm_device *dev,
> +static int vgic_v2_parse_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr,
> struct vgic_reg_attr *reg_attr)
> {
> @@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
> }
>
> /**
> - * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
> + * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
> *
> * @dev: kvm device handle
> * @attr: kvm device attribute
> * @reg: address the value is read or written
> * @is_write: true if userspace is writing a register
> */
> -static int vgic_attr_regs_access_v2(struct kvm_device *dev,
> +static int vgic_v2_attr_regs_access(struct kvm_device *dev,
> struct kvm_device_attr *attr,
> u32 *reg, bool is_write)
> {
> @@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
> struct kvm_vcpu *vcpu;
> int ret;
>
> - ret = parse_vgic_v2_attr(dev, attr, ®_attr);
> + ret = vgic_v2_parse_attr(dev, attr, ®_attr);
> if (ret)
> return ret;
>
> @@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
>
> mutex_lock(&dev->kvm->lock);
>
> - ret = vgic_init(dev->kvm);
> - if (ret)
> + if (unlikely(!vgic_initialized(dev->kvm))) {
> + ret = -EBUSY;
> goto out;
> + }
eh, GICv2 should still support lazy init, shouldn't it? Am I
misunderstanding this change?
>
> if (!lock_all_vcpus(dev->kvm)) {
> ret = -EBUSY;
> @@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
> if (get_user(reg, uaddr))
> return -EFAULT;
>
> - return vgic_attr_regs_access_v2(dev, attr, ®, true);
> + return vgic_v2_attr_regs_access(dev, attr, ®, true);
> }
> }
>
> @@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
> u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> u32 reg = 0;
>
> - ret = vgic_attr_regs_access_v2(dev, attr, ®, false);
> + ret = vgic_v2_attr_regs_access(dev, attr, ®, false);
> if (ret)
> return ret;
> return put_user(reg, uaddr);
> @@ -430,16 +431,141 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
> .has_attr = vgic_v2_has_attr,
> };
>
> +static int vgic_v3_parse_attr(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + struct vgic_reg_attr *reg_attr)
> +{
> + unsigned long vgic_mpidr, mpidr_reg;
> +
> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> + reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> + if (!reg_attr->vcpu)
> + return -EINVAL;
> +
> + reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> +
> + return 0;
> +}
> +
> +/*
> + * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
> + *
> + * @dev: kvm device handle
> + * @attr: kvm device attribute
> + * @reg: address the value is read or written
> + * @is_write: true if userspace is writing a register
> + */
> +static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + u64 *reg, bool is_write)
> +{
> + struct vgic_reg_attr reg_attr;
> + gpa_t addr;
> + struct kvm_vcpu *vcpu;
> + int ret;
> + u32 tmp32;
> +
> + ret = vgic_v3_parse_attr(dev, attr, ®_attr);
> + if (ret)
> + return ret;
> +
> + vcpu = reg_attr.vcpu;
> + addr = reg_attr.addr;
> +
> + mutex_lock(&dev->kvm->lock);
> +
> + if (unlikely(!vgic_initialized(dev->kvm))) {
> + ret = -EBUSY;
> + goto out;
> + }
> +
> + if (!lock_all_vcpus(dev->kvm)) {
> + ret = -EBUSY;
> + goto out;
> + }
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + if (is_write)
> + tmp32 = *reg;
> +
> + ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + if (is_write)
> + tmp32 = *reg;
> +
> + ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + unlock_all_vcpus(dev->kvm);
> +out:
> + mutex_unlock(&dev->kvm->lock);
> + return ret;
> +}
> +
> static int vgic_v3_set_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr)
> {
> - return vgic_set_common_attr(dev, attr);
> + int ret;
> +
> + ret = vgic_set_common_attr(dev, attr);
> + if (ret != -ENXIO)
> + return ret;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 tmp32;
> + u64 reg;
> +
> + if (get_user(tmp32, uaddr))
> + return -EFAULT;
> +
> + reg = tmp32;
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> + }
> + return -ENXIO;
> }
>
> static int vgic_v3_get_attr(struct kvm_device *dev,
> struct kvm_device_attr *attr)
> {
> - return vgic_get_common_attr(dev, attr);
> + int ret;
> +
> + ret = vgic_get_common_attr(dev, attr);
> + if (ret != -ENXIO)
> + return ret;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
> +
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + tmp32 = reg;
> + return put_user(tmp32, uaddr);
> + }
> + }
> +
> + return -ENXIO;
> }
>
> static int vgic_v3_has_attr(struct kvm_device *dev,
> @@ -453,6 +579,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> return 0;
> }
> break;
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> case KVM_DEV_ARM_VGIC_GRP_CTRL:
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index 0b32f40..2cb04b7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
>
> int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> {
> - int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
> const struct vgic_register_region *regions;
> gpa_t addr;
> - int nr_regions, i, len;
> + int nr_regions;
>
> addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
>
> @@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> if (addr & 3)
> return -ENXIO;
>
> - for (i = 0; i < nr_regions; i++) {
> - if (regions[i].bits_per_irq)
> - len = (regions[i].bits_per_irq * nr_irqs) / 8;
> - else
> - len = regions[i].len;
> -
> - if (regions[i].reg_offset <= addr &&
> - regions[i].reg_offset + len > addr)
> - return 0;
> - }
> -
> - return -ENXIO;
> + return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
> }
>
> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index ce2708d..b35fb83 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -18,6 +18,8 @@
> #include <kvm/arm_vgic.h>
>
> #include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
>
> #include "vgic.h"
> #include "vgic-mmio.h"
> @@ -437,6 +439,9 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
> vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
> VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
> + vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
> VGIC_ACCESS_32bit),
> @@ -484,12 +489,18 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
> REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
> vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
> VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
> vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
> VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
> vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> + REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
> + vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> + VGIC_ACCESS_32bit),
> REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
> vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
> VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
> @@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
> return ret;
> }
>
> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> +{
> + const struct vgic_register_region *regions;
> + gpa_t addr;
> + int nr_regions;
> +
> + addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> +
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + regions = vgic_v3_dist_registers;
> + nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
> + regions = vgic_v3_rdbase_registers;
> + nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> + break;
> + }
> + default:
> + return -ENXIO;
> + }
> +
> + /* We only support aligned 32-bit accesses. */
> + if (addr & 3)
> + return -ENXIO;
> +
> + return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
> +}
> /*
> * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
> * generation register ICC_SGI1R_EL1) with a given VCPU.
> @@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
> vgic_put_irq(vcpu->kvm, irq);
> }
> }
> +
> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val)
> +{
> + struct vgic_io_device dev = {
> + .regions = vgic_v3_dist_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
> + };
> +
> + return vgic_uaccess(vcpu, &dev, is_write, offset, val);
> +}
> +
> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val)
> +{
> + struct vgic_io_device rd_dev = {
> + .regions = vgic_v3_rdbase_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
> + };
> +
> + struct vgic_io_device sgi_dev = {
> + .regions = vgic_v3_sgibase_registers,
> + .nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
> + };
> +
> + /* SGI_base is the next 64K frame after RD_base */
> + if (offset >= SZ_64K)
> + return vgic_uaccess(vcpu, &sgi_dev, is_write,
> + offset - SZ_64K, val);
> + else
> + return vgic_uaccess(vcpu, &rd_dev, is_write,
> + offset, val);
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 31f85df..9939d1d 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -394,6 +394,28 @@ static int match_region(const void *key, const void *elt)
> sizeof(region[0]), match_region);
> }
>
> +/* Check if address falls within the region */
> +int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> + const struct vgic_register_region *regions,
> + int nr_regions, gpa_t addr)
> +{
> + int i, len;
> + int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
> +
> + for (i = 0; i < nr_regions; i++) {
> + if (regions[i].bits_per_irq)
> + len = (regions[i].bits_per_irq * nr_irqs) / 8;
> + else
> + len = regions[i].len;
> +
> + if (regions[i].reg_offset <= addr &&
> + regions[i].reg_offset + len > addr)
> + return 0;
> + }
> +
> + return -ENXIO;
> +}
> +
> /*
> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> * to a byte array and be directly observed as the guest wanted it to appear
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 97e6df7..acbf99e 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> bool is_write, int offset, u32 *val);
>
> +int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> + const struct vgic_register_region *regions,
> + int nr_regions, gpa_t addr);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 9d9e014..d901b0c 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -30,6 +30,34 @@
>
> #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
>
> +#define VGIC_AFFINITY_0_SHIFT 0
> +#define VGIC_AFFINITY_0_MASK (0xffUL << VGIC_AFFINITY_0_SHIFT)
> +#define VGIC_AFFINITY_1_SHIFT 8
> +#define VGIC_AFFINITY_1_MASK (0xffUL << VGIC_AFFINITY_1_SHIFT)
> +#define VGIC_AFFINITY_2_SHIFT 16
> +#define VGIC_AFFINITY_2_MASK (0xffUL << VGIC_AFFINITY_2_SHIFT)
> +#define VGIC_AFFINITY_3_SHIFT 24
> +#define VGIC_AFFINITY_3_MASK (0xffUL << VGIC_AFFINITY_3_SHIFT)
> +
> +#define VGIC_AFFINITY_LEVEL(reg, level) \
> + ((((reg) & VGIC_AFFINITY_## level ##_MASK) \
> + >> VGIC_AFFINITY_## level ##_SHIFT) << MPIDR_LEVEL_SHIFT(level))
> +
> +/*
> + * The userspace encode the affinity differently from the MPIDR,
Userspace encodes
> + * Below macro converts vgic userspace format to MPIDR reg format.
> + */
> +#define VGIC_TO_MPIDR(val) (VGIC_AFFINITY_LEVEL(val, 0) | \
> + VGIC_AFFINITY_LEVEL(val, 1) | \
> + VGIC_AFFINITY_LEVEL(val, 2) | \
> + VGIC_AFFINITY_LEVEL(val, 3))
> +
> +#define KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
> + KVM_REG_ARM64_SYSREG_OP1_MASK | \
> + KVM_REG_ARM64_SYSREG_CRN_MASK | \
> + KVM_REG_ARM64_SYSREG_CRM_MASK | \
> + KVM_REG_ARM64_SYSREG_OP2_MASK)
> +
> struct vgic_vmcr {
> u32 ctlr;
> u32 abpr;
> @@ -90,6 +118,11 @@ static inline void vgic_get_irq_kref(struct vgic_irq *irq)
> int kvm_vgic_register_its_device(void);
> void vgic_enable_lpis(struct kvm_vcpu *vcpu);
> int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
> +int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
> +int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val);
> +int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + int offset, u32 *val);
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
> 1.9.1
>
Otherwise looks ok.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 18:52 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> variables to struct vmcr to support read and write of these fields.
>
> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
> .
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> include/linux/irqchip/arm-gic-v3.h | 2 --
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
> virt/kvm/arm/vgic/vgic.h | 5 +++++
> 5 files changed, 29 insertions(+), 20 deletions(-)
>
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index d48d886..61646aa 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -404,8 +404,6 @@
> #define ICH_HCR_EN (1 << 0)
> #define ICH_HCR_UIE (1 << 1)
>
> -#define ICH_VMCR_CTLR_SHIFT 0
> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
> #define ICH_VMCR_CBPR_SHIFT 4
> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
> #define ICH_VMCR_EOIM_SHIFT 9
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index 2cb04b7..ad353b5 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
> }
> }
>
> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> - if (kvm_vgic_global_state.type == VGIC_V2)
> - vgic_v2_set_vmcr(vcpu, vmcr);
> - else
> - vgic_v3_set_vmcr(vcpu, vmcr);
> -}
> -
> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> - if (kvm_vgic_global_state.type == VGIC_V2)
> - vgic_v2_get_vmcr(vcpu, vmcr);
> - else
> - vgic_v3_get_vmcr(vcpu, vmcr);
> -}
> -
> #define GICC_ARCH_VERSION_V2 0x2
>
> /* These are for userland accesses only, there is no guest-facing emulation. */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 9939d1d..173d6f0 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> return -ENXIO;
> }
>
> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> +{
> + if (kvm_vgic_global_state.type == VGIC_V2)
> + vgic_v2_set_vmcr(vcpu, vmcr);
> + else
> + vgic_v3_set_vmcr(vcpu, vmcr);
> +}
> +
> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> +{
> + if (kvm_vgic_global_state.type == VGIC_V2)
> + vgic_v2_get_vmcr(vcpu, vmcr);
> + else
> + vgic_v3_get_vmcr(vcpu, vmcr);
> +}
> +
> /*
> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> * to a byte array and be directly observed as the guest wanted it to appear
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 9f0dae3..967c295 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> {
> u32 vmcr;
>
> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
9 and then masking off everything by bit 9, so you'll end with never
being able to set VEOIM I think...
Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
understand because it's deprecated, but why the first? This particular
piece of information would be very nice to have in the commit message.
> vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
> vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
> vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
> + vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
> + vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
>
> vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
> }
> @@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> {
> u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
>
> - vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
> + vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
> + vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
> vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
> vmcrp->bpr = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
> vmcrp->pmr = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
> + vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
> + vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
> }
>
> #define INITIAL_PENDBASER_VALUE \
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index d901b0c..c461f6b 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -63,6 +63,9 @@ struct vgic_vmcr {
> u32 abpr;
> u32 bpr;
> u32 pmr;
> + /* Below member variable are valid only for GICv3 */
> + u32 grpen0;
> + u32 grpen1;
> };
>
> struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
> @@ -150,6 +153,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
> #endif
>
> int kvm_register_vgic_device(unsigned long type);
> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
> int vgic_lazy_init(struct kvm *kvm);
> int vgic_init(struct kvm *kvm);
>
> --
> 1.9.1
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-11-16 18:52 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> variables to struct vmcr to support read and write of these fields.
>
> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
> .
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> include/linux/irqchip/arm-gic-v3.h | 2 --
> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
> virt/kvm/arm/vgic/vgic.h | 5 +++++
> 5 files changed, 29 insertions(+), 20 deletions(-)
>
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index d48d886..61646aa 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -404,8 +404,6 @@
> #define ICH_HCR_EN (1 << 0)
> #define ICH_HCR_UIE (1 << 1)
>
> -#define ICH_VMCR_CTLR_SHIFT 0
> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
> #define ICH_VMCR_CBPR_SHIFT 4
> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
> #define ICH_VMCR_EOIM_SHIFT 9
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index 2cb04b7..ad353b5 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
> }
> }
>
> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> - if (kvm_vgic_global_state.type == VGIC_V2)
> - vgic_v2_set_vmcr(vcpu, vmcr);
> - else
> - vgic_v3_set_vmcr(vcpu, vmcr);
> -}
> -
> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> -{
> - if (kvm_vgic_global_state.type == VGIC_V2)
> - vgic_v2_get_vmcr(vcpu, vmcr);
> - else
> - vgic_v3_get_vmcr(vcpu, vmcr);
> -}
> -
> #define GICC_ARCH_VERSION_V2 0x2
>
> /* These are for userland accesses only, there is no guest-facing emulation. */
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 9939d1d..173d6f0 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> return -ENXIO;
> }
>
> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> +{
> + if (kvm_vgic_global_state.type == VGIC_V2)
> + vgic_v2_set_vmcr(vcpu, vmcr);
> + else
> + vgic_v3_set_vmcr(vcpu, vmcr);
> +}
> +
> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> +{
> + if (kvm_vgic_global_state.type == VGIC_V2)
> + vgic_v2_get_vmcr(vcpu, vmcr);
> + else
> + vgic_v3_get_vmcr(vcpu, vmcr);
> +}
> +
> /*
> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> * to a byte array and be directly observed as the guest wanted it to appear
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 9f0dae3..967c295 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> {
> u32 vmcr;
>
> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
9 and then masking off everything by bit 9, so you'll end with never
being able to set VEOIM I think...
Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
understand because it's deprecated, but why the first? This particular
piece of information would be very nice to have in the commit message.
> vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
> vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
> vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
> + vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
> + vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
>
> vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
> }
> @@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> {
> u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
>
> - vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
> + vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
> + vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
> vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
> vmcrp->bpr = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
> vmcrp->pmr = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
> + vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
> + vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
> }
>
> #define INITIAL_PENDBASER_VALUE \
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index d901b0c..c461f6b 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -63,6 +63,9 @@ struct vgic_vmcr {
> u32 abpr;
> u32 bpr;
> u32 pmr;
> + /* Below member variable are valid only for GICv3 */
> + u32 grpen0;
> + u32 grpen1;
> };
>
> struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
> @@ -150,6 +153,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
> #endif
>
> int kvm_register_vgic_device(unsigned long type);
> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
> int vgic_lazy_init(struct kvm *kvm);
> int vgic_init(struct kvm *kvm);
>
> --
> 1.9.1
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 18:52 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> VGICv3 CPU interface registers are accessed using
> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> as 64-bit. The cpu MPIDR value is passed along with register id.
> is used to identify the cpu for registers access.
>
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 3 +
> arch/arm64/kvm/Makefile | 1 +
> include/kvm/arm_vgic.h | 9 +
> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> virt/kvm/arm/vgic/vgic.h | 4 +
> 8 files changed, 395 insertions(+)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 56dc08d..91c7137 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> +
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> /* Device Control API on vcpu fd */
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index d50a82a..1a14e29 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
Thi is making me wonder: Are we properly handling GICv3 save/restore
for AArch32 now that we have GICv3 support for AArch32? By properly I
mean that either it is clearly only supported on AArch64 systems or it's
supported on both AArch64 and AArch32, but it shouldn't break randomly
on AArch32.
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 002f092..730a18a 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -71,6 +71,9 @@ struct vgic_global {
>
> /* GIC system register CPU interface */
> struct static_key_false gicv3_cpuif;
> +
> + /* Cache ICH_VTR_EL2 reg value */
> + u32 ich_vtr_el2;
> };
>
> extern struct vgic_global kvm_vgic_global_state;
> @@ -269,6 +272,12 @@ struct vgic_cpu {
> u64 pendbaser;
>
> bool lpis_enabled;
> +
> + /* Cache guest priority bits */
> + u32 num_pri_bits;
> +
> + /* Cache guest interrupt ID bits */
> + u32 num_id_bits;
> };
>
> extern struct static_key_false vgic_v2_cpuif_trap;
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index 6c7d30c..da532d1 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> if (!is_write)
> *reg = tmp32;
> break;
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 regid;
> +
> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> + regid, reg);
> + break;
> + }
> default:
> ret = -EINVAL;
> break;
> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> reg = tmp32;
> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> + u64 reg;
> +
> + if (get_user(reg, uaddr))
> + return -EFAULT;
> +
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> }
> return -ENXIO;
> }
> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> tmp32 = reg;
> return put_user(tmp32, uaddr);
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> + u64 reg;
> +
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + return put_user(reg, uaddr);
> + }
> }
>
> return -ENXIO;
> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> break;
> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index b35fb83..519b919 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,6 +23,7 @@
>
> #include "vgic.h"
> #include "vgic-mmio.h"
> +#include "sys_regs.h"
>
> /* extract @num bytes at @offset bytes offset in data */
> unsigned long extract_bytes(u64 data, unsigned int offset,
> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> break;
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 reg, id;
> + unsigned long vgic_mpidr, mpidr_reg;
> + struct kvm_vcpu *vcpu;
> +
> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> + /* Convert plain mpidr value to MPIDR reg format */
> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> +
> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> + if (!vcpu)
> + return -EINVAL;
> +
> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> + }
> default:
> return -ENXIO;
> }
> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> new file mode 100644
> index 0000000..69d8597
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
Shouldn't we have a GPL header here?
> @@ -0,0 +1,324 @@
> +#include <linux/irqchip/arm-gic-v3.h>
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +#include <kvm/iodev.h>
> +#include <kvm/arm_vgic.h>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
> +
> +#include "vgic.h"
> +#include "vgic-mmio.h"
> +#include "sys_regs.h"
> +
> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> + struct vgic_vmcr vmcr;
> + u64 val;
> + u32 num_pri_bits, num_id_bits;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + val = p->regval;
> +
> + /*
> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> + * guest programmed ID and PRI bits
> + */
I would suggest rewording this comment:
Disallow restoring VM state not supported by this hardware.
> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> + return false;
> +
> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
understand which effect this is intended to have?
Sure, it may limit what you do with other registers later, but since
there's no ordering requirement that the ctlr be restored first, I'm not
sure it makes sense.
Also, since this field is RO in the ICH_VTR, we'll have a strange
situation during runtime after a GICv3 restore where the
vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
which is never the case if you didn't do a save/restore.
Finally, should we somehow ensure that this field is set to the same
value across VCPUs or is that not an architectural requirement?
> +
> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> + return false;
> +
> + vgic_v3_cpu->num_id_bits = num_id_bits;
same questions
> +
> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> + ICH_VMCR_EOIM_SHIFT;
I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
format or in the VMCR format? I would assume the former, since
otherwise I don't get the point with this indirection, and for GICv2
vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
into VMCR values.
Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
ring.
> + vgic_set_vmcr(vcpu, &vmcr);
Should we check compatibility between the source and destination for the
SEIS and A3V support here?
> + } else {
> + val = 0;
> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> + val |= vgic_v3_cpu->num_id_bits <<
> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> + ICC_CTLR_EL1_SEIS_SHIFT;
> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> + ICC_CTLR_EL1_A3V_SHIFT;
> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
again, these last two look weird to me.
> +
> + p->regval = val;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> + ICC_BPR0_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> + ICC_BPR0_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + if (!p->is_write)
> + p->regval = 0;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> + if (p->is_write) {
> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> + ICC_BPR1_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> + ICC_BPR1_EL1_MASK;
> + }
> + } else {
> + if (!p->is_write)
> + p->regval = min((vmcr.bpr + 1), 7U);
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> + ICC_IGRPEN0_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> + ICC_IGRPEN0_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> + ICC_IGRPEN1_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> + ICC_IGRPEN1_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p, u8 apr, u8 idx)
> +{
> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> + uint32_t *ap_reg;
> +
> + if (apr)
> + ap_reg = &vgicv3->vgic_ap1r[idx];
> + else
> + ap_reg = &vgicv3->vgic_ap0r[idx];
> +
> + if (p->is_write)
> + *ap_reg = p->regval;
> + else
> + p->regval = *ap_reg;
> +}
> +
> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r, u8 apr)
> +{
> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> + u8 idx = r->Op2 & 3;
> +
> + switch (vgic_v3_cpu->num_pri_bits) {
> + case 7:
> + if (idx > 3)
> + goto err;
idx cannot be higher than three given the mask above, right?
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + break;
> + case 6:
> + if (idx > 1)
> + goto err;
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + break;
> + default:
> + if (idx > 0)
> + goto err;
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + }
what's the rationale behind ignoring the case where userspace is using
unsupported priorities? Is it that this will be checked during
save/restore of the ctlr?
This sort of thing just looks like the case that's impossible to debug,
because userspace could be scratching its head trying to understand why
the value it wrote isn't recorded anywhere...
If there's a good rationale for doing it this way, then could we have a
comment to that effect?
> +
> + return;
> +err:
> + if (!p->is_write)
> + p->regval = 0;
> +}
> +
> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + access_gic_aprn(vcpu, p, r, 0);
> +
> + return true;
> +}
> +
> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + access_gic_aprn(vcpu, p, r, 1);
> +
> + return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + /* Validate SRE bit */
> + if (p->is_write) {
> + if (!(p->regval & ICC_SRE_EL1_SRE))
> + return false;
> + } else {
> + p->regval = vgicv3->vgic_sre;
> + }
> +
> + return true;
> +}
> +
> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> + /* ICC_PMR_EL1 */
> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> + /* ICC_BPR0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> + /* ICC_AP0R0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> + /* ICC_AP0R1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> + /* ICC_AP0R2_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> + /* ICC_AP0R3_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> + /* ICC_AP1R0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> + /* ICC_AP1R1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> + /* ICC_AP1R2_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> + /* ICC_AP1R3_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> + /* ICC_BPR1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> + /* ICC_CTLR_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> + /* ICC_SRE_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> + /* ICC_IGRPEN0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> + /* ICC_GRPEN1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> +};
> +
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg)
> +{
> + struct sys_reg_params params;
> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> + params.regval = *reg;
> + params.is_write = is_write;
> + params.is_aarch32 = false;
> + params.is_32bit = false;
> +
> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> + return 0;
> +
> + return -ENXIO;
> +}
> +
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg)
> +{
> + struct sys_reg_params params;
> + const struct sys_reg_desc *r;
> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> + if (is_write)
> + params.regval = *reg;
> + params.is_write = is_write;
> + params.is_aarch32 = false;
> + params.is_32bit = false;
> +
> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> + if (!r)
> + return -ENXIO;
> +
> + if (!r->access(vcpu, ¶ms, r))
> + return -EINVAL;
According to the API, EINVAL meansinvalid mpidr. Should we expand on
how it can be used or allocate a new error code?
> +
> + if (!is_write)
> + *reg = params.regval;
> +
> + return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 967c295..1139971 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> vgic_v3->vgic_sre = 0;
> }
>
> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_ID_BITS_MASK) >>
> + ICH_VTR_ID_BITS_SHIFT;
> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_PRI_BITS_MASK) >>
> + ICH_VTR_PRI_BITS_SHIFT) + 1;
> +
> /* Get the show on the road... */
> vgic_v3->vgic_hcr = ICH_HCR_EN;
> }
> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
> */
> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
> kvm_vgic_global_state.can_emulate_gicv2 = false;
> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>
> if (!info->vcpu.start) {
> kvm_info("GICv3: no GICV resource entry\n");
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index c461f6b..0e632d0 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val);
> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val);
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u64 id, u64 *val);
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg);
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-16 18:52 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> VGICv3 CPU interface registers are accessed using
> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> as 64-bit. The cpu MPIDR value is passed along with register id.
> is used to identify the cpu for registers access.
>
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 3 +
> arch/arm64/kvm/Makefile | 1 +
> include/kvm/arm_vgic.h | 9 +
> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> virt/kvm/arm/vgic/vgic.h | 4 +
> 8 files changed, 395 insertions(+)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 56dc08d..91c7137 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> +
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> /* Device Control API on vcpu fd */
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index d50a82a..1a14e29 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
Thi is making me wonder: Are we properly handling GICv3 save/restore
for AArch32 now that we have GICv3 support for AArch32? By properly I
mean that either it is clearly only supported on AArch64 systems or it's
supported on both AArch64 and AArch32, but it shouldn't break randomly
on AArch32.
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 002f092..730a18a 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -71,6 +71,9 @@ struct vgic_global {
>
> /* GIC system register CPU interface */
> struct static_key_false gicv3_cpuif;
> +
> + /* Cache ICH_VTR_EL2 reg value */
> + u32 ich_vtr_el2;
> };
>
> extern struct vgic_global kvm_vgic_global_state;
> @@ -269,6 +272,12 @@ struct vgic_cpu {
> u64 pendbaser;
>
> bool lpis_enabled;
> +
> + /* Cache guest priority bits */
> + u32 num_pri_bits;
> +
> + /* Cache guest interrupt ID bits */
> + u32 num_id_bits;
> };
>
> extern struct static_key_false vgic_v2_cpuif_trap;
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index 6c7d30c..da532d1 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> if (!is_write)
> *reg = tmp32;
> break;
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 regid;
> +
> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> + regid, reg);
> + break;
> + }
> default:
> ret = -EINVAL;
> break;
> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> reg = tmp32;
> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> + u64 reg;
> +
> + if (get_user(reg, uaddr))
> + return -EFAULT;
> +
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> }
> return -ENXIO;
> }
> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> tmp32 = reg;
> return put_user(tmp32, uaddr);
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> + u64 reg;
> +
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + return put_user(reg, uaddr);
> + }
> }
>
> return -ENXIO;
> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> break;
> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index b35fb83..519b919 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,6 +23,7 @@
>
> #include "vgic.h"
> #include "vgic-mmio.h"
> +#include "sys_regs.h"
>
> /* extract @num bytes at @offset bytes offset in data */
> unsigned long extract_bytes(u64 data, unsigned int offset,
> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> break;
> }
> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> + u64 reg, id;
> + unsigned long vgic_mpidr, mpidr_reg;
> + struct kvm_vcpu *vcpu;
> +
> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> + /* Convert plain mpidr value to MPIDR reg format */
> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> +
> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> + if (!vcpu)
> + return -EINVAL;
> +
> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> + }
> default:
> return -ENXIO;
> }
> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> new file mode 100644
> index 0000000..69d8597
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
Shouldn't we have a GPL header here?
> @@ -0,0 +1,324 @@
> +#include <linux/irqchip/arm-gic-v3.h>
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +#include <kvm/iodev.h>
> +#include <kvm/arm_vgic.h>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
> +
> +#include "vgic.h"
> +#include "vgic-mmio.h"
> +#include "sys_regs.h"
> +
> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> + struct vgic_vmcr vmcr;
> + u64 val;
> + u32 num_pri_bits, num_id_bits;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + val = p->regval;
> +
> + /*
> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> + * guest programmed ID and PRI bits
> + */
I would suggest rewording this comment:
Disallow restoring VM state not supported by this hardware.
> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> + return false;
> +
> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
understand which effect this is intended to have?
Sure, it may limit what you do with other registers later, but since
there's no ordering requirement that the ctlr be restored first, I'm not
sure it makes sense.
Also, since this field is RO in the ICH_VTR, we'll have a strange
situation during runtime after a GICv3 restore where the
vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
which is never the case if you didn't do a save/restore.
Finally, should we somehow ensure that this field is set to the same
value across VCPUs or is that not an architectural requirement?
> +
> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> + return false;
> +
> + vgic_v3_cpu->num_id_bits = num_id_bits;
same questions
> +
> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> + ICH_VMCR_EOIM_SHIFT;
I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
format or in the VMCR format? I would assume the former, since
otherwise I don't get the point with this indirection, and for GICv2
vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
into VMCR values.
Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
ring.
> + vgic_set_vmcr(vcpu, &vmcr);
Should we check compatibility between the source and destination for the
SEIS and A3V support here?
> + } else {
> + val = 0;
> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> + val |= vgic_v3_cpu->num_id_bits <<
> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> + ICC_CTLR_EL1_SEIS_SHIFT;
> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> + ICC_CTLR_EL1_A3V_SHIFT;
> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
again, these last two look weird to me.
> +
> + p->regval = val;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> + ICC_BPR0_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> + ICC_BPR0_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + if (!p->is_write)
> + p->regval = 0;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> + if (p->is_write) {
> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> + ICC_BPR1_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> + ICC_BPR1_EL1_MASK;
> + }
> + } else {
> + if (!p->is_write)
> + p->regval = min((vmcr.bpr + 1), 7U);
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> + ICC_IGRPEN0_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> + ICC_IGRPEN0_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_vmcr vmcr;
> +
> + vgic_get_vmcr(vcpu, &vmcr);
> + if (p->is_write) {
> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> + ICC_IGRPEN1_EL1_SHIFT;
> + vgic_set_vmcr(vcpu, &vmcr);
> + } else {
> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> + ICC_IGRPEN1_EL1_MASK;
> + }
> +
> + return true;
> +}
> +
> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p, u8 apr, u8 idx)
> +{
> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> + uint32_t *ap_reg;
> +
> + if (apr)
> + ap_reg = &vgicv3->vgic_ap1r[idx];
> + else
> + ap_reg = &vgicv3->vgic_ap0r[idx];
> +
> + if (p->is_write)
> + *ap_reg = p->regval;
> + else
> + p->regval = *ap_reg;
> +}
> +
> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r, u8 apr)
> +{
> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> + u8 idx = r->Op2 & 3;
> +
> + switch (vgic_v3_cpu->num_pri_bits) {
> + case 7:
> + if (idx > 3)
> + goto err;
idx cannot be higher than three given the mask above, right?
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + break;
> + case 6:
> + if (idx > 1)
> + goto err;
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + break;
> + default:
> + if (idx > 0)
> + goto err;
> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> + }
what's the rationale behind ignoring the case where userspace is using
unsupported priorities? Is it that this will be checked during
save/restore of the ctlr?
This sort of thing just looks like the case that's impossible to debug,
because userspace could be scratching its head trying to understand why
the value it wrote isn't recorded anywhere...
If there's a good rationale for doing it this way, then could we have a
comment to that effect?
> +
> + return;
> +err:
> + if (!p->is_write)
> + p->regval = 0;
> +}
> +
> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + access_gic_aprn(vcpu, p, r, 0);
> +
> + return true;
> +}
> +
> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + access_gic_aprn(vcpu, p, r, 1);
> +
> + return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> + /* Validate SRE bit */
> + if (p->is_write) {
> + if (!(p->regval & ICC_SRE_EL1_SRE))
> + return false;
> + } else {
> + p->regval = vgicv3->vgic_sre;
> + }
> +
> + return true;
> +}
> +
> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> + /* ICC_PMR_EL1 */
> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> + /* ICC_BPR0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> + /* ICC_AP0R0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> + /* ICC_AP0R1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> + /* ICC_AP0R2_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> + /* ICC_AP0R3_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> + /* ICC_AP1R0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> + /* ICC_AP1R1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> + /* ICC_AP1R2_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> + /* ICC_AP1R3_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> + /* ICC_BPR1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> + /* ICC_CTLR_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> + /* ICC_SRE_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> + /* ICC_IGRPEN0_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> + /* ICC_GRPEN1_EL1 */
> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> +};
> +
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg)
> +{
> + struct sys_reg_params params;
> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> + params.regval = *reg;
> + params.is_write = is_write;
> + params.is_aarch32 = false;
> + params.is_32bit = false;
> +
> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> + return 0;
> +
> + return -ENXIO;
> +}
> +
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg)
> +{
> + struct sys_reg_params params;
> + const struct sys_reg_desc *r;
> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> + if (is_write)
> + params.regval = *reg;
> + params.is_write = is_write;
> + params.is_aarch32 = false;
> + params.is_32bit = false;
> +
> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> + if (!r)
> + return -ENXIO;
> +
> + if (!r->access(vcpu, ¶ms, r))
> + return -EINVAL;
According to the API, EINVAL meansinvalid mpidr. Should we expand on
how it can be used or allocate a new error code?
> +
> + if (!is_write)
> + *reg = params.regval;
> +
> + return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> index 967c295..1139971 100644
> --- a/virt/kvm/arm/vgic/vgic-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> vgic_v3->vgic_sre = 0;
> }
>
> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_ID_BITS_MASK) >>
> + ICH_VTR_ID_BITS_SHIFT;
> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
> + ICH_VTR_PRI_BITS_MASK) >>
> + ICH_VTR_PRI_BITS_SHIFT) + 1;
> +
> /* Get the show on the road... */
> vgic_v3->vgic_hcr = ICH_HCR_EN;
> }
> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
> */
> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
> kvm_vgic_global_state.can_emulate_gicv2 = false;
> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>
> if (!info->vcpu.start) {
> kvm_info("GICv3: no GICV resource entry\n");
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index c461f6b..0e632d0 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val);
> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> int offset, u32 *val);
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u64 id, u64 *val);
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> + u64 *reg);
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 7/7] arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-16 18:52 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:33PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> Userspace requires to store and restore of line_level for
> level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 6 +++++
> virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 11 ++++++++
> virt/kvm/arm/vgic/vgic-mmio.c | 33 ++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic-mmio.h | 5 ++++
> virt/kvm/arm/vgic/vgic.h | 3 +++
> 6 files changed, 107 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 91c7137..4100f8c 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> #define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> +#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
> + (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK 0x3ff
> +#define VGIC_LEVEL_INFO_LINE_LEVEL 0
>
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index da532d1..0f82a91 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -512,6 +512,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> regid, reg);
> break;
> }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + unsigned int info, intid;
> +
> + info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
> + if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
> + if (is_write)
> + tmp32 = *reg;
> + intid = attr->attr &
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
> + ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
> + intid, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
I think you can avoid the indirection with tmp32 here by just making the
line level interface use an unsigned long.
> + } else {
> + ret = -EINVAL;
> + }
> + break;
> + }
> default:
> ret = -EINVAL;
> break;
> @@ -554,6 +573,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>
> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
> +
> + if (get_user(tmp32, uaddr))
> + return -EFAULT;
> +
> + reg = tmp32;
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> }
> return -ENXIO;
> }
> @@ -589,8 +619,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> return ret;
> return put_user(reg, uaddr);
> }
> - }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
>
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + tmp32 = reg;
> + return put_user(tmp32, uaddr);
> + }
> + }
> return -ENXIO;
> }
>
> @@ -611,11 +651,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
> + VGIC_LEVEL_INFO_LINE_LEVEL)
> + return 0;
> + break;
> + }
> case KVM_DEV_ARM_VGIC_GRP_CTRL:
> switch (attr->attr) {
> case KVM_DEV_ARM_VGIC_CTRL_INIT:
> return 0;
> }
> + break;
spurious change?
> }
> return -ENXIO;
> }
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 519b919..38b481c 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -807,3 +807,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> return vgic_uaccess(vcpu, &rd_dev, is_write,
> offset, val);
> }
> +
> +int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u32 intid, u32 *val)
> +{
> + if (is_write)
> + vgic_write_irq_line_level_info(vcpu, intid, *val);
> + else
> + *val = vgic_read_irq_line_level_info(vcpu, intid);
> +
> + return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 173d6f0..fb018eb 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> }
> }
>
> +unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
> +{
> + int i;
> + unsigned long val = 0;
> +
> + for (i = 0; i < 32; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + if (irq->line_level)
> + val |= (1U << i);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +
> + return val;
> +}
> +
> +void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
> + const unsigned long val)
> +{
> + int i;
> +
> + for_each_set_bit(i, &val, 32) {
I think you misunderstood this part of the API. Userspace should be
able to both set an asserted and deasserted line level, regardless of
what the value was before. So you need to loop through all of them and
set the level as nneded.
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + spin_lock(&irq->irq_lock);
> + irq->line_level = true;
why don't you have to set pending as well and potentially queue the interrupt?
> + spin_unlock(&irq->irq_lock);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +}
> +
> static int match_region(const void *key, const void *elt)
> {
> const unsigned int offset = (unsigned long)key;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index acbf99e..938702c 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> const struct vgic_register_region *regions,
> int nr_regions, gpa_t addr);
>
> +unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
> +
> +void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
> + const unsigned long val);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 0e632d0..77d3d84 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -130,6 +130,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> u64 id, u64 *val);
> int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> u64 *reg);
> +int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u32 intid, u32 *val);
> +
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
> 1.9.1
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 7/7] arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
@ 2016-11-16 18:52 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-16 18:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 04, 2016 at 04:43:33PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> Userspace requires to store and restore of line_level for
> level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.
>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
> arch/arm64/include/uapi/asm/kvm.h | 6 +++++
> virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 11 ++++++++
> virt/kvm/arm/vgic/vgic-mmio.c | 33 ++++++++++++++++++++++++
> virt/kvm/arm/vgic/vgic-mmio.h | 5 ++++
> virt/kvm/arm/vgic/vgic.h | 3 +++
> 6 files changed, 107 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 91c7137..4100f8c 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> #define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> +#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT 10
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
> + (0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
> +#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK 0x3ff
> +#define VGIC_LEVEL_INFO_LINE_LEVEL 0
>
> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index da532d1..0f82a91 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -512,6 +512,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> regid, reg);
> break;
> }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + unsigned int info, intid;
> +
> + info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
> + if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
> + if (is_write)
> + tmp32 = *reg;
> + intid = attr->attr &
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
> + ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
> + intid, &tmp32);
> + if (!is_write)
> + *reg = tmp32;
I think you can avoid the indirection with tmp32 here by just making the
line level interface use an unsigned long.
> + } else {
> + ret = -EINVAL;
> + }
> + break;
> + }
> default:
> ret = -EINVAL;
> break;
> @@ -554,6 +573,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>
> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
> +
> + if (get_user(tmp32, uaddr))
> + return -EFAULT;
> +
> + reg = tmp32;
> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> + }
> }
> return -ENXIO;
> }
> @@ -589,8 +619,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> return ret;
> return put_user(reg, uaddr);
> }
> - }
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u64 reg;
> + u32 tmp32;
>
> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> + if (ret)
> + return ret;
> + tmp32 = reg;
> + return put_user(tmp32, uaddr);
> + }
> + }
> return -ENXIO;
> }
>
> @@ -611,11 +651,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> return vgic_v3_has_attr_regs(dev, attr);
> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> return 0;
> + case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> + if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> + KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
> + VGIC_LEVEL_INFO_LINE_LEVEL)
> + return 0;
> + break;
> + }
> case KVM_DEV_ARM_VGIC_GRP_CTRL:
> switch (attr->attr) {
> case KVM_DEV_ARM_VGIC_CTRL_INIT:
> return 0;
> }
> + break;
spurious change?
> }
> return -ENXIO;
> }
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 519b919..38b481c 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -807,3 +807,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> return vgic_uaccess(vcpu, &rd_dev, is_write,
> offset, val);
> }
> +
> +int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u32 intid, u32 *val)
> +{
> + if (is_write)
> + vgic_write_irq_line_level_info(vcpu, intid, *val);
> + else
> + *val = vgic_read_irq_line_level_info(vcpu, intid);
> +
> + return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index 173d6f0..fb018eb 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
> }
> }
>
> +unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
> +{
> + int i;
> + unsigned long val = 0;
> +
> + for (i = 0; i < 32; i++) {
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + if (irq->line_level)
> + val |= (1U << i);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +
> + return val;
> +}
> +
> +void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
> + const unsigned long val)
> +{
> + int i;
> +
> + for_each_set_bit(i, &val, 32) {
I think you misunderstood this part of the API. Userspace should be
able to both set an asserted and deasserted line level, regardless of
what the value was before. So you need to loop through all of them and
set the level as nneded.
> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> + spin_lock(&irq->irq_lock);
> + irq->line_level = true;
why don't you have to set pending as well and potentially queue the interrupt?
> + spin_unlock(&irq->irq_lock);
> +
> + vgic_put_irq(vcpu->kvm, irq);
> + }
> +}
> +
> static int match_region(const void *key, const void *elt)
> {
> const unsigned int offset = (unsigned long)key;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index acbf99e..938702c 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> const struct vgic_register_region *regions,
> int nr_regions, gpa_t addr);
>
> +unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
> +
> +void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
> + const unsigned long val);
> +
> unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>
> unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 0e632d0..77d3d84 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -130,6 +130,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> u64 id, u64 *val);
> int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> u64 *reg);
> +int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> + u32 intid, u32 *val);
> +
> #else
> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> {
> --
> 1.9.1
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
2016-11-16 18:52 ` Christoffer Dall
@ 2016-11-17 11:26 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 11:26 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> Read and write of some registers like ISPENDR and ICPENDR
>> from userspace requires special handling when compared to
>> guest access for these registers.
>>
>> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
>> for handling of ISPENDR, ICPENDR registers handling.
>>
>> Add infrastructure to support guest and userspace read
>> and write for the required registers
>> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
>> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
>> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
>> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
>> 4 files changed, 169 insertions(+), 51 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> index b44b359..0b32f40 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> return -ENXIO;
>> }
>>
>> -/*
>> - * When userland tries to access the VGIC register handlers, we need to
>> - * create a usable struct vgic_io_device to be passed to the handlers and we
>> - * have to set up a buffer similar to what would have happened if a guest MMIO
>> - * access occurred, including doing endian conversions on BE systems.
>> - */
>> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
>> - bool is_write, int offset, u32 *val)
>> -{
>> - unsigned int len = 4;
>> - u8 buf[4];
>> - int ret;
>> -
>> - if (is_write) {
>> - vgic_data_host_to_mmio_bus(buf, len, *val);
>> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
>> - } else {
>> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
>> - if (!ret)
>> - *val = vgic_data_mmio_bus_to_host(buf, len);
>> - }
>> -
>> - return ret;
>> -}
>> -
>> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val)
>> {
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index 0d3c76a..ce2708d 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
>> return 0;
>> }
>>
>> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
>> + gpa_t addr, unsigned int len)
>> +{
>> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
>> + u32 value = 0;
>> + int i;
>> +
>> + /*
>> + * A level triggerred interrupt pending state is latched in both
>> + * "soft_pending" and "line_level" variables. Userspace will save
>> + * and restore soft_pending and line_level separately.
>> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
>> + * handling of ISPENDR and ICPENDR.
>> + */
>> + for (i = 0; i < len * 8; i++) {
>> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
>> +
>> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
>> + value |= (1U << i);
>> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
>> + value |= (1U << i);
>> +
>> + vgic_put_irq(vcpu->kvm, irq);
>> + }
>> +
>> + return value;
>> +}
>> +
>> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
>> + gpa_t addr, unsigned int len,
>> + unsigned long val)
>> +{
>> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
>> + int i;
>> +
>> + for (i = 0; i < len * 8; i++) {
>> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
>> +
>> + spin_lock(&irq->irq_lock);
>> + if (test_bit(i, &val)) {
>> + irq->pending = true;
>> + irq->soft_pending = true;
>
> In the vgic_mmio_write_spending function we only set the soft_pending
> state to true if the interrupt is a level-triggered interrupt.
>
> Should we check if that's the case here as well before setting the
> soft_pending state?
Yes, can be done. But it puts hard requirement that irq config should
be restored
before updating pending state.
In any case, the soft_pending is used only if interrupt is level-triggered.
>
> Otherwise, this patch looks good.
>
> Thanks,
> -Christoffer
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
@ 2016-11-17 11:26 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 11:26 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari at gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> Read and write of some registers like ISPENDR and ICPENDR
>> from userspace requires special handling when compared to
>> guest access for these registers.
>>
>> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
>> for handling of ISPENDR, ICPENDR registers handling.
>>
>> Add infrastructure to support guest and userspace read
>> and write for the required registers
>> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
>>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
>> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
>> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
>> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
>> 4 files changed, 169 insertions(+), 51 deletions(-)
>>
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> index b44b359..0b32f40 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> return -ENXIO;
>> }
>>
>> -/*
>> - * When userland tries to access the VGIC register handlers, we need to
>> - * create a usable struct vgic_io_device to be passed to the handlers and we
>> - * have to set up a buffer similar to what would have happened if a guest MMIO
>> - * access occurred, including doing endian conversions on BE systems.
>> - */
>> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
>> - bool is_write, int offset, u32 *val)
>> -{
>> - unsigned int len = 4;
>> - u8 buf[4];
>> - int ret;
>> -
>> - if (is_write) {
>> - vgic_data_host_to_mmio_bus(buf, len, *val);
>> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
>> - } else {
>> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
>> - if (!ret)
>> - *val = vgic_data_mmio_bus_to_host(buf, len);
>> - }
>> -
>> - return ret;
>> -}
>> -
>> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val)
>> {
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index 0d3c76a..ce2708d 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
>> return 0;
>> }
>>
>> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
>> + gpa_t addr, unsigned int len)
>> +{
>> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
>> + u32 value = 0;
>> + int i;
>> +
>> + /*
>> + * A level triggerred interrupt pending state is latched in both
>> + * "soft_pending" and "line_level" variables. Userspace will save
>> + * and restore soft_pending and line_level separately.
>> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
>> + * handling of ISPENDR and ICPENDR.
>> + */
>> + for (i = 0; i < len * 8; i++) {
>> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
>> +
>> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
>> + value |= (1U << i);
>> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
>> + value |= (1U << i);
>> +
>> + vgic_put_irq(vcpu->kvm, irq);
>> + }
>> +
>> + return value;
>> +}
>> +
>> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
>> + gpa_t addr, unsigned int len,
>> + unsigned long val)
>> +{
>> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
>> + int i;
>> +
>> + for (i = 0; i < len * 8; i++) {
>> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
>> +
>> + spin_lock(&irq->irq_lock);
>> + if (test_bit(i, &val)) {
>> + irq->pending = true;
>> + irq->soft_pending = true;
>
> In the vgic_mmio_write_spending function we only set the soft_pending
> state to true if the interrupt is a level-triggered interrupt.
>
> Should we check if that's the case here as well before setting the
> soft_pending state?
Yes, can be done. But it puts hard requirement that irq config should
be restored
before updating pending state.
In any case, the soft_pending is used only if interrupt is level-triggered.
>
> Otherwise, this patch looks good.
>
> Thanks,
> -Christoffer
>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
2016-11-17 11:26 ` Vijay Kilari
@ 2016-11-17 11:40 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 11:40 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 04:56:53PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari@gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> Read and write of some registers like ISPENDR and ICPENDR
> >> from userspace requires special handling when compared to
> >> guest access for these registers.
> >>
> >> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> >> for handling of ISPENDR, ICPENDR registers handling.
> >>
> >> Add infrastructure to support guest and userspace read
> >> and write for the required registers
> >> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
> >>
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
> >> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
> >> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
> >> 4 files changed, 169 insertions(+), 51 deletions(-)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> index b44b359..0b32f40 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> return -ENXIO;
> >> }
> >>
> >> -/*
> >> - * When userland tries to access the VGIC register handlers, we need to
> >> - * create a usable struct vgic_io_device to be passed to the handlers and we
> >> - * have to set up a buffer similar to what would have happened if a guest MMIO
> >> - * access occurred, including doing endian conversions on BE systems.
> >> - */
> >> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> >> - bool is_write, int offset, u32 *val)
> >> -{
> >> - unsigned int len = 4;
> >> - u8 buf[4];
> >> - int ret;
> >> -
> >> - if (is_write) {
> >> - vgic_data_host_to_mmio_bus(buf, len, *val);
> >> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> >> - } else {
> >> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> >> - if (!ret)
> >> - *val = vgic_data_mmio_bus_to_host(buf, len);
> >> - }
> >> -
> >> - return ret;
> >> -}
> >> -
> >> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val)
> >> {
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index 0d3c76a..ce2708d 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
> >> return 0;
> >> }
> >>
> >> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
> >> + gpa_t addr, unsigned int len)
> >> +{
> >> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> >> + u32 value = 0;
> >> + int i;
> >> +
> >> + /*
> >> + * A level triggerred interrupt pending state is latched in both
> >> + * "soft_pending" and "line_level" variables. Userspace will save
> >> + * and restore soft_pending and line_level separately.
> >> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> >> + * handling of ISPENDR and ICPENDR.
> >> + */
> >> + for (i = 0; i < len * 8; i++) {
> >> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> >> +
> >> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> >> + value |= (1U << i);
> >> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> >> + value |= (1U << i);
> >> +
> >> + vgic_put_irq(vcpu->kvm, irq);
> >> + }
> >> +
> >> + return value;
> >> +}
> >> +
> >> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
> >> + gpa_t addr, unsigned int len,
> >> + unsigned long val)
> >> +{
> >> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> >> + int i;
> >> +
> >> + for (i = 0; i < len * 8; i++) {
> >> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> >> +
> >> + spin_lock(&irq->irq_lock);
> >> + if (test_bit(i, &val)) {
> >> + irq->pending = true;
> >> + irq->soft_pending = true;
> >
> > In the vgic_mmio_write_spending function we only set the soft_pending
> > state to true if the interrupt is a level-triggered interrupt.
> >
> > Should we check if that's the case here as well before setting the
> > soft_pending state?
>
> Yes, can be done. But it puts hard requirement that irq config should
> be restored
> before updating pending state.
Ah, I see.
ok, I think you should keep it the way it is then, but please add a
comment to that effect.
> In any case, the soft_pending is used only if interrupt is level-triggered.
>
Yes, indeed.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access
@ 2016-11-17 11:40 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 11:40 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 04:56:53PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:27PM +0530, vijay.kilari at gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> Read and write of some registers like ISPENDR and ICPENDR
> >> from userspace requires special handling when compared to
> >> guest access for these registers.
> >>
> >> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> >> for handling of ISPENDR, ICPENDR registers handling.
> >>
> >> Add infrastructure to support guest and userspace read
> >> and write for the required registers
> >> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
> >>
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
> >> virt/kvm/arm/vgic/vgic-mmio.c | 78 ++++++++++++++++++++++++++++----
> >> virt/kvm/arm/vgic/vgic-mmio.h | 19 ++++++++
> >> 4 files changed, 169 insertions(+), 51 deletions(-)
> >>
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> index b44b359..0b32f40 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> return -ENXIO;
> >> }
> >>
> >> -/*
> >> - * When userland tries to access the VGIC register handlers, we need to
> >> - * create a usable struct vgic_io_device to be passed to the handlers and we
> >> - * have to set up a buffer similar to what would have happened if a guest MMIO
> >> - * access occurred, including doing endian conversions on BE systems.
> >> - */
> >> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> >> - bool is_write, int offset, u32 *val)
> >> -{
> >> - unsigned int len = 4;
> >> - u8 buf[4];
> >> - int ret;
> >> -
> >> - if (is_write) {
> >> - vgic_data_host_to_mmio_bus(buf, len, *val);
> >> - ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> >> - } else {
> >> - ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> >> - if (!ret)
> >> - *val = vgic_data_mmio_bus_to_host(buf, len);
> >> - }
> >> -
> >> - return ret;
> >> -}
> >> -
> >> int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val)
> >> {
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index 0d3c76a..ce2708d 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
> >> return 0;
> >> }
> >>
> >> +static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
> >> + gpa_t addr, unsigned int len)
> >> +{
> >> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> >> + u32 value = 0;
> >> + int i;
> >> +
> >> + /*
> >> + * A level triggerred interrupt pending state is latched in both
> >> + * "soft_pending" and "line_level" variables. Userspace will save
> >> + * and restore soft_pending and line_level separately.
> >> + * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> >> + * handling of ISPENDR and ICPENDR.
> >> + */
> >> + for (i = 0; i < len * 8; i++) {
> >> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> >> +
> >> + if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> >> + value |= (1U << i);
> >> + if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> >> + value |= (1U << i);
> >> +
> >> + vgic_put_irq(vcpu->kvm, irq);
> >> + }
> >> +
> >> + return value;
> >> +}
> >> +
> >> +static void vgic_v3_uaccess_write_pending(struct kvm_vcpu *vcpu,
> >> + gpa_t addr, unsigned int len,
> >> + unsigned long val)
> >> +{
> >> + u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> >> + int i;
> >> +
> >> + for (i = 0; i < len * 8; i++) {
> >> + struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> >> +
> >> + spin_lock(&irq->irq_lock);
> >> + if (test_bit(i, &val)) {
> >> + irq->pending = true;
> >> + irq->soft_pending = true;
> >
> > In the vgic_mmio_write_spending function we only set the soft_pending
> > state to true if the interrupt is a level-triggered interrupt.
> >
> > Should we check if that's the case here as well before setting the
> > soft_pending state?
>
> Yes, can be done. But it puts hard requirement that irq config should
> be restored
> before updating pending state.
Ah, I see.
ok, I think you should keep it the way it is then, but please add a
comment to that effect.
> In any case, the soft_pending is used only if interrupt is level-triggered.
>
Yes, indeed.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
2016-11-04 11:13 ` vijay.kilari at gmail.com
@ 2016-11-17 11:41 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 11:41 UTC (permalink / raw)
To: vijay.kilari; +Cc: marc.zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
Hi Vijaya,
On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> This patchset adds API for saving and restoring
> of VGICv3 registers to support live migration with new vgic feature.
> This API definition is as per version of VGICv3 specification
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> The patch 3 & 4 are picked from the Pavel's previous implementation.
> http://www.spinics.net/lists/kvm/msg122040.html
>
> v7 => v8:
> - Rebased to 4.9-rc3
I noticed when looking at this series that there are quite a few
conflicts when applying to kvmarm/queue, so if you could rebase the next
verson on kvmarm/queue, then you'll make Marc's life a bit easier.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration
@ 2016-11-17 11:41 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 11:41 UTC (permalink / raw)
To: linux-arm-kernel
Hi Vijaya,
On Fri, Nov 04, 2016 at 04:43:26PM +0530, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>
> This patchset adds API for saving and restoring
> of VGICv3 registers to support live migration with new vgic feature.
> This API definition is as per version of VGICv3 specification
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>
> The patch 3 & 4 are picked from the Pavel's previous implementation.
> http://www.spinics.net/lists/kvm/msg122040.html
>
> v7 => v8:
> - Rebased to 4.9-rc3
I noticed when looking at this series that there are quite a few
conflicts when applying to kvmarm/queue, so if you could rebase the next
verson on kvmarm/queue, then you'll make Marc's life a bit easier.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
2016-11-16 18:52 ` Christoffer Dall
@ 2016-11-17 12:42 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 12:42 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
>> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
>> variables to struct vmcr to support read and write of these fields.
>>
>> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
>> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
>> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
>> .
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> include/linux/irqchip/arm-gic-v3.h | 2 --
>> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
>> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
>> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
>> virt/kvm/arm/vgic/vgic.h | 5 +++++
>> 5 files changed, 29 insertions(+), 20 deletions(-)
>>
>> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
>> index d48d886..61646aa 100644
>> --- a/include/linux/irqchip/arm-gic-v3.h
>> +++ b/include/linux/irqchip/arm-gic-v3.h
>> @@ -404,8 +404,6 @@
>> #define ICH_HCR_EN (1 << 0)
>> #define ICH_HCR_UIE (1 << 1)
>>
>> -#define ICH_VMCR_CTLR_SHIFT 0
>> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
>> #define ICH_VMCR_CBPR_SHIFT 4
>> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
>> #define ICH_VMCR_EOIM_SHIFT 9
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> index 2cb04b7..ad353b5 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
>> }
>> }
>>
>> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> -{
>> - if (kvm_vgic_global_state.type == VGIC_V2)
>> - vgic_v2_set_vmcr(vcpu, vmcr);
>> - else
>> - vgic_v3_set_vmcr(vcpu, vmcr);
>> -}
>> -
>> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> -{
>> - if (kvm_vgic_global_state.type == VGIC_V2)
>> - vgic_v2_get_vmcr(vcpu, vmcr);
>> - else
>> - vgic_v3_get_vmcr(vcpu, vmcr);
>> -}
>> -
>> #define GICC_ARCH_VERSION_V2 0x2
>>
>> /* These are for userland accesses only, there is no guest-facing emulation. */
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
>> index 9939d1d..173d6f0 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
>> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
>> return -ENXIO;
>> }
>>
>> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> +{
>> + if (kvm_vgic_global_state.type == VGIC_V2)
>> + vgic_v2_set_vmcr(vcpu, vmcr);
>> + else
>> + vgic_v3_set_vmcr(vcpu, vmcr);
>> +}
>> +
>> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> +{
>> + if (kvm_vgic_global_state.type == VGIC_V2)
>> + vgic_v2_get_vmcr(vcpu, vmcr);
>> + else
>> + vgic_v3_get_vmcr(vcpu, vmcr);
>> +}
>> +
>> /*
>> * kvm_mmio_read_buf() returns a value in a format where it can be converted
>> * to a byte array and be directly observed as the guest wanted it to appear
>> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> index 9f0dae3..967c295 100644
>> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
>> {
>> u32 vmcr;
>>
>> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
>> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
>> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
>
> This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
> bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
> 9 and then masking off everything by bit 9, so you'll end with never
> being able to set VEOIM I think...
>
OK
> Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
> understand because it's deprecated, but why the first? This particular
> piece of information would be very nice to have in the commit message.
I understand that group 0 interrupts are not handled. So vFIQEn can be ignored.
Spec says, if SRE=1 (non-secure) this bit is RES1 also it is alias to
ICC_CTLR_EL1
if SRE is 1. However there is no bit in ICC_CTLR_EL1 for FIQen. It is defined
only in GICV_CTLR which is used when SRE=0.
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-11-17 12:42 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 12:42 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari at gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
>> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
>> variables to struct vmcr to support read and write of these fields.
>>
>> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
>> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
>> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
>> .
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> include/linux/irqchip/arm-gic-v3.h | 2 --
>> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
>> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
>> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
>> virt/kvm/arm/vgic/vgic.h | 5 +++++
>> 5 files changed, 29 insertions(+), 20 deletions(-)
>>
>> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
>> index d48d886..61646aa 100644
>> --- a/include/linux/irqchip/arm-gic-v3.h
>> +++ b/include/linux/irqchip/arm-gic-v3.h
>> @@ -404,8 +404,6 @@
>> #define ICH_HCR_EN (1 << 0)
>> #define ICH_HCR_UIE (1 << 1)
>>
>> -#define ICH_VMCR_CTLR_SHIFT 0
>> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
>> #define ICH_VMCR_CBPR_SHIFT 4
>> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
>> #define ICH_VMCR_EOIM_SHIFT 9
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> index 2cb04b7..ad353b5 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
>> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
>> }
>> }
>>
>> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> -{
>> - if (kvm_vgic_global_state.type == VGIC_V2)
>> - vgic_v2_set_vmcr(vcpu, vmcr);
>> - else
>> - vgic_v3_set_vmcr(vcpu, vmcr);
>> -}
>> -
>> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> -{
>> - if (kvm_vgic_global_state.type == VGIC_V2)
>> - vgic_v2_get_vmcr(vcpu, vmcr);
>> - else
>> - vgic_v3_get_vmcr(vcpu, vmcr);
>> -}
>> -
>> #define GICC_ARCH_VERSION_V2 0x2
>>
>> /* These are for userland accesses only, there is no guest-facing emulation. */
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
>> index 9939d1d..173d6f0 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
>> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
>> return -ENXIO;
>> }
>>
>> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> +{
>> + if (kvm_vgic_global_state.type == VGIC_V2)
>> + vgic_v2_set_vmcr(vcpu, vmcr);
>> + else
>> + vgic_v3_set_vmcr(vcpu, vmcr);
>> +}
>> +
>> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
>> +{
>> + if (kvm_vgic_global_state.type == VGIC_V2)
>> + vgic_v2_get_vmcr(vcpu, vmcr);
>> + else
>> + vgic_v3_get_vmcr(vcpu, vmcr);
>> +}
>> +
>> /*
>> * kvm_mmio_read_buf() returns a value in a format where it can be converted
>> * to a byte array and be directly observed as the guest wanted it to appear
>> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> index 9f0dae3..967c295 100644
>> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
>> {
>> u32 vmcr;
>>
>> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
>> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
>> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
>
> This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
> bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
> 9 and then masking off everything by bit 9, so you'll end with never
> being able to set VEOIM I think...
>
OK
> Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
> understand because it's deprecated, but why the first? This particular
> piece of information would be very nice to have in the commit message.
I understand that group 0 interrupts are not handled. So vFIQEn can be ignored.
Spec says, if SRE=1 (non-secure) this bit is RES1 also it is alias to
ICC_CTLR_EL1
if SRE is 1. However there is no bit in ICC_CTLR_EL1 for FIQen. It is defined
only in GICV_CTLR which is used when SRE=0.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-16 18:52 ` Christoffer Dall
@ 2016-11-17 15:55 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 15:55 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> VGICv3 CPU interface registers are accessed using
>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> is used to identify the cpu for registers access.
>>
>> The version of VGIC v3 specification is define here
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> arch/arm64/kvm/Makefile | 1 +
>> include/kvm/arm_vgic.h | 9 +
>> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> virt/kvm/arm/vgic/vgic.h | 4 +
>> 8 files changed, 395 insertions(+)
>>
>> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> index 56dc08d..91c7137 100644
>> --- a/arch/arm64/include/uapi/asm/kvm.h
>> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> +
>> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>>
>> /* Device Control API on vcpu fd */
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index d50a82a..1a14e29 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>
> Thi is making me wonder: Are we properly handling GICv3 save/restore
> for AArch32 now that we have GICv3 support for AArch32? By properly I
> mean that either it is clearly only supported on AArch64 systems or it's
> supported on both AArch64 and AArch32, but it shouldn't break randomly
> on AArch32.
It supports both AArch64 and AArch64 in handling of system registers
save/restore.
All system registers that we save/restore are 32-bit for both aarch64
and aarch32.
Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
are same. However the codes sent by qemu is matched and register
are handled properly irrespective of AArch32 or AArch64.
I don't have platform which support AArch32 guests to verify.
>
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 002f092..730a18a 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -71,6 +71,9 @@ struct vgic_global {
>>
>> /* GIC system register CPU interface */
>> struct static_key_false gicv3_cpuif;
>> +
>> + /* Cache ICH_VTR_EL2 reg value */
>> + u32 ich_vtr_el2;
>> };
>>
>> extern struct vgic_global kvm_vgic_global_state;
>> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> u64 pendbaser;
>>
>> bool lpis_enabled;
>> +
>> + /* Cache guest priority bits */
>> + u32 num_pri_bits;
>> +
>> + /* Cache guest interrupt ID bits */
>> + u32 num_id_bits;
>> };
>>
>> extern struct static_key_false vgic_v2_cpuif_trap;
>> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> index 6c7d30c..da532d1 100644
>> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> if (!is_write)
>> *reg = tmp32;
>> break;
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 regid;
>> +
>> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> + regid, reg);
>> + break;
>> + }
>> default:
>> ret = -EINVAL;
>> break;
>> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> reg = tmp32;
>> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> + u64 reg;
>> +
>> + if (get_user(reg, uaddr))
>> + return -EFAULT;
>> +
>> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> + }
>> }
>> return -ENXIO;
>> }
>> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> tmp32 = reg;
>> return put_user(tmp32, uaddr);
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> + u64 reg;
>> +
>> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> + if (ret)
>> + return ret;
>> + return put_user(reg, uaddr);
>> + }
>> }
>>
>> return -ENXIO;
>> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> break;
>> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> return vgic_v3_has_attr_regs(dev, attr);
>> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> return 0;
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index b35fb83..519b919 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,6 +23,7 @@
>>
>> #include "vgic.h"
>> #include "vgic-mmio.h"
>> +#include "sys_regs.h"
>>
>> /* extract @num bytes at @offset bytes offset in data */
>> unsigned long extract_bytes(u64 data, unsigned int offset,
>> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> break;
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 reg, id;
>> + unsigned long vgic_mpidr, mpidr_reg;
>> + struct kvm_vcpu *vcpu;
>> +
>> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> +
>> + /* Convert plain mpidr value to MPIDR reg format */
>> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> +
>> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> + if (!vcpu)
>> + return -EINVAL;
>> +
>> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> + }
>> default:
>> return -ENXIO;
>> }
>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> new file mode 100644
>> index 0000000..69d8597
>> --- /dev/null
>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>
> Shouldn't we have a GPL header here?
>
>> @@ -0,0 +1,324 @@
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +#include <linux/kvm.h>
>> +#include <linux/kvm_host.h>
>> +#include <kvm/iodev.h>
>> +#include <kvm/arm_vgic.h>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_arm.h>
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "vgic.h"
>> +#include "vgic-mmio.h"
>> +#include "sys_regs.h"
>> +
>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> + struct vgic_vmcr vmcr;
>> + u64 val;
>> + u32 num_pri_bits, num_id_bits;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + val = p->regval;
>> +
>> + /*
>> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> + * guest programmed ID and PRI bits
>> + */
>
> I would suggest rewording this comment:
> Disallow restoring VM state not supported by this hardware.
>
>> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> + return false;
>> +
>> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>
> hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> understand which effect this is intended to have?
>
> Sure, it may limit what you do with other registers later, but since
> there's no ordering requirement that the ctlr be restored first, I'm not
> sure it makes sense.
>
> Also, since this field is RO in the ICH_VTR, we'll have a strange
> situation during runtime after a GICv3 restore where the
> vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> which is never the case if you didn't do a save/restore.
Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
than HW supported
value.
>
> Finally, should we somehow ensure that this field is set to the same
> value across VCPUs or is that not an architectural requirement?
>
Yes it is nice to have it same across VCPUs. But should be ok as
we are ensuring value is not greater than HW supported value.
There is no single point of place where we can make such a check
>> +
>> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
>> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
>> + return false;
>> +
>> + vgic_v3_cpu->num_id_bits = num_id_bits;
>
> same questions
>
>> +
>> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> + ICC_CTLR_EL1_EOImode_SHIFT) <<
>> + ICH_VMCR_EOIM_SHIFT;
>
> I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> format or in the VMCR format? I would assume the former, since
> otherwise I don't get the point with this indirection, and for GICv2
> vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> into VMCR values.
>
> Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> ring.
I will check and fix it.
>
>> + vgic_set_vmcr(vcpu, &vmcr);
>
> Should we check compatibility between the source and destination for the
> SEIS and A3V support here?
Can be checked. But I feel A3V check makes more sense than checking for
SEIS.
>
>> + } else {
>> + val = 0;
>> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
>> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> + val |= vgic_v3_cpu->num_id_bits <<
>> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
>> + ICC_CTLR_EL1_SEIS_SHIFT;
>> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
>> + ICC_CTLR_EL1_A3V_SHIFT;
>> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>
> again, these last two look weird to me.
>
>> +
>> + p->regval = val;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> + ICC_BPR0_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> + ICC_BPR0_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + if (!p->is_write)
>> + p->regval = 0;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
>> + if (p->is_write) {
>> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> + ICC_BPR1_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> + ICC_BPR1_EL1_MASK;
>> + }
>> + } else {
>> + if (!p->is_write)
>> + p->regval = min((vmcr.bpr + 1), 7U);
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> + ICC_IGRPEN0_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> + ICC_IGRPEN0_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> + ICC_IGRPEN1_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> + ICC_IGRPEN1_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> + struct sys_reg_params *p, u8 apr, u8 idx)
>> +{
>> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> + uint32_t *ap_reg;
>> +
>> + if (apr)
>> + ap_reg = &vgicv3->vgic_ap1r[idx];
>> + else
>> + ap_reg = &vgicv3->vgic_ap0r[idx];
>> +
>> + if (p->is_write)
>> + *ap_reg = p->regval;
>> + else
>> + p->regval = *ap_reg;
>> +}
>> +
>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r, u8 apr)
>> +{
>> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> + u8 idx = r->Op2 & 3;
>> +
>> + switch (vgic_v3_cpu->num_pri_bits) {
>> + case 7:
>> + if (idx > 3)
>> + goto err;
>
> idx cannot be higher than three given the mask above, right?
>
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + break;
>> + case 6:
>> + if (idx > 1)
>> + goto err;
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + break;
>> + default:
>> + if (idx > 0)
>> + goto err;
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + }
>
> what's the rationale behind ignoring the case where userspace is using
> unsupported priorities? Is it that this will be checked during
> save/restore of the ctlr?
>
> This sort of thing just looks like the case that's impossible to debug,
> because userspace could be scratching its head trying to understand why
> the value it wrote isn't recorded anywhere...
>
> If there's a good rationale for doing it this way, then could we have a
> comment to that effect?
Accessing umplemented priority registers raised UNDEF exception.
So userspace accesing should be ignored instead of recording unsupported
values.
>
>> +
>> + return;
>> +err:
>> + if (!p->is_write)
>> + p->regval = 0;
>> +}
>> +
>> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + access_gic_aprn(vcpu, p, r, 0);
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + access_gic_aprn(vcpu, p, r, 1);
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> +
>> + /* Validate SRE bit */
>> + if (p->is_write) {
>> + if (!(p->regval & ICC_SRE_EL1_SRE))
>> + return false;
>> + } else {
>> + p->regval = vgicv3->vgic_sre;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
>> + /* ICC_PMR_EL1 */
>> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
>> + /* ICC_BPR0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
>> + /* ICC_AP0R0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
>> + /* ICC_AP0R1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
>> + /* ICC_AP0R2_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
>> + /* ICC_AP0R3_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
>> + /* ICC_AP1R0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
>> + /* ICC_AP1R1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
>> + /* ICC_AP1R2_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
>> + /* ICC_AP1R3_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
>> + /* ICC_BPR1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
>> + /* ICC_CTLR_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
>> + /* ICC_SRE_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
>> + /* ICC_IGRPEN0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
>> + /* ICC_GRPEN1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
>> +};
>> +
>> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg)
>> +{
>> + struct sys_reg_params params;
>> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> +
>> + params.regval = *reg;
>> + params.is_write = is_write;
>> + params.is_aarch32 = false;
>> + params.is_32bit = false;
>> +
>> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
>> + return 0;
>> +
>> + return -ENXIO;
>> +}
>> +
>> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg)
>> +{
>> + struct sys_reg_params params;
>> + const struct sys_reg_desc *r;
>> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> +
>> + if (is_write)
>> + params.regval = *reg;
>> + params.is_write = is_write;
>> + params.is_aarch32 = false;
>> + params.is_32bit = false;
>> +
>> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> + ARRAY_SIZE(gic_v3_icc_reg_descs));
>> + if (!r)
>> + return -ENXIO;
>> +
>> + if (!r->access(vcpu, ¶ms, r))
>> + return -EINVAL;
>
> According to the API, EINVAL meansinvalid mpidr. Should we expand on
> how it can be used or allocate a new error code?
How abt EACCES error code?
>
>> +
>> + if (!is_write)
>> + *reg = params.regval;
>> +
>> + return 0;
>> +}
>> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> index 967c295..1139971 100644
>> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
>> vgic_v3->vgic_sre = 0;
>> }
>>
>> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_ID_BITS_MASK) >>
>> + ICH_VTR_ID_BITS_SHIFT;
>> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_PRI_BITS_MASK) >>
>> + ICH_VTR_PRI_BITS_SHIFT) + 1;
>> +
>> /* Get the show on the road... */
>> vgic_v3->vgic_hcr = ICH_HCR_EN;
>> }
>> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>> */
>> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
>> kvm_vgic_global_state.can_emulate_gicv2 = false;
>> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>>
>> if (!info->vcpu.start) {
>> kvm_info("GICv3: no GICV resource entry\n");
>> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> index c461f6b..0e632d0 100644
>> --- a/virt/kvm/arm/vgic/vgic.h
>> +++ b/virt/kvm/arm/vgic/vgic.h
>> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val);
>> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val);
>> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> + u64 id, u64 *val);
>> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg);
>> #else
>> static inline int vgic_register_its_iodevs(struct kvm *kvm)
>> {
>> --
>
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-17 15:55 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-17 15:55 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> VGICv3 CPU interface registers are accessed using
>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> is used to identify the cpu for registers access.
>>
>> The version of VGIC v3 specification is define here
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> arch/arm64/kvm/Makefile | 1 +
>> include/kvm/arm_vgic.h | 9 +
>> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> virt/kvm/arm/vgic/vgic.h | 4 +
>> 8 files changed, 395 insertions(+)
>>
>> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> index 56dc08d..91c7137 100644
>> --- a/arch/arm64/include/uapi/asm/kvm.h
>> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> +
>> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>>
>> /* Device Control API on vcpu fd */
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index d50a82a..1a14e29 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>
> Thi is making me wonder: Are we properly handling GICv3 save/restore
> for AArch32 now that we have GICv3 support for AArch32? By properly I
> mean that either it is clearly only supported on AArch64 systems or it's
> supported on both AArch64 and AArch32, but it shouldn't break randomly
> on AArch32.
It supports both AArch64 and AArch64 in handling of system registers
save/restore.
All system registers that we save/restore are 32-bit for both aarch64
and aarch32.
Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
are same. However the codes sent by qemu is matched and register
are handled properly irrespective of AArch32 or AArch64.
I don't have platform which support AArch32 guests to verify.
>
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 002f092..730a18a 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -71,6 +71,9 @@ struct vgic_global {
>>
>> /* GIC system register CPU interface */
>> struct static_key_false gicv3_cpuif;
>> +
>> + /* Cache ICH_VTR_EL2 reg value */
>> + u32 ich_vtr_el2;
>> };
>>
>> extern struct vgic_global kvm_vgic_global_state;
>> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> u64 pendbaser;
>>
>> bool lpis_enabled;
>> +
>> + /* Cache guest priority bits */
>> + u32 num_pri_bits;
>> +
>> + /* Cache guest interrupt ID bits */
>> + u32 num_id_bits;
>> };
>>
>> extern struct static_key_false vgic_v2_cpuif_trap;
>> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> index 6c7d30c..da532d1 100644
>> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> if (!is_write)
>> *reg = tmp32;
>> break;
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 regid;
>> +
>> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> + regid, reg);
>> + break;
>> + }
>> default:
>> ret = -EINVAL;
>> break;
>> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> reg = tmp32;
>> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> + u64 reg;
>> +
>> + if (get_user(reg, uaddr))
>> + return -EFAULT;
>> +
>> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> + }
>> }
>> return -ENXIO;
>> }
>> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> tmp32 = reg;
>> return put_user(tmp32, uaddr);
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> + u64 reg;
>> +
>> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> + if (ret)
>> + return ret;
>> + return put_user(reg, uaddr);
>> + }
>> }
>>
>> return -ENXIO;
>> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> break;
>> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> return vgic_v3_has_attr_regs(dev, attr);
>> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> return 0;
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index b35fb83..519b919 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,6 +23,7 @@
>>
>> #include "vgic.h"
>> #include "vgic-mmio.h"
>> +#include "sys_regs.h"
>>
>> /* extract @num bytes at @offset bytes offset in data */
>> unsigned long extract_bytes(u64 data, unsigned int offset,
>> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> break;
>> }
>> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> + u64 reg, id;
>> + unsigned long vgic_mpidr, mpidr_reg;
>> + struct kvm_vcpu *vcpu;
>> +
>> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> +
>> + /* Convert plain mpidr value to MPIDR reg format */
>> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> +
>> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> + if (!vcpu)
>> + return -EINVAL;
>> +
>> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> + }
>> default:
>> return -ENXIO;
>> }
>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> new file mode 100644
>> index 0000000..69d8597
>> --- /dev/null
>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>
> Shouldn't we have a GPL header here?
>
>> @@ -0,0 +1,324 @@
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +#include <linux/kvm.h>
>> +#include <linux/kvm_host.h>
>> +#include <kvm/iodev.h>
>> +#include <kvm/arm_vgic.h>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_arm.h>
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "vgic.h"
>> +#include "vgic-mmio.h"
>> +#include "sys_regs.h"
>> +
>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> + struct vgic_vmcr vmcr;
>> + u64 val;
>> + u32 num_pri_bits, num_id_bits;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + val = p->regval;
>> +
>> + /*
>> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> + * guest programmed ID and PRI bits
>> + */
>
> I would suggest rewording this comment:
> Disallow restoring VM state not supported by this hardware.
>
>> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> + return false;
>> +
>> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>
> hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> understand which effect this is intended to have?
>
> Sure, it may limit what you do with other registers later, but since
> there's no ordering requirement that the ctlr be restored first, I'm not
> sure it makes sense.
>
> Also, since this field is RO in the ICH_VTR, we'll have a strange
> situation during runtime after a GICv3 restore where the
> vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> which is never the case if you didn't do a save/restore.
Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
than HW supported
value.
>
> Finally, should we somehow ensure that this field is set to the same
> value across VCPUs or is that not an architectural requirement?
>
Yes it is nice to have it same across VCPUs. But should be ok as
we are ensuring value is not greater than HW supported value.
There is no single point of place where we can make such a check
>> +
>> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
>> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
>> + return false;
>> +
>> + vgic_v3_cpu->num_id_bits = num_id_bits;
>
> same questions
>
>> +
>> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> + ICC_CTLR_EL1_EOImode_SHIFT) <<
>> + ICH_VMCR_EOIM_SHIFT;
>
> I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> format or in the VMCR format? I would assume the former, since
> otherwise I don't get the point with this indirection, and for GICv2
> vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> into VMCR values.
>
> Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> ring.
I will check and fix it.
>
>> + vgic_set_vmcr(vcpu, &vmcr);
>
> Should we check compatibility between the source and destination for the
> SEIS and A3V support here?
Can be checked. But I feel A3V check makes more sense than checking for
SEIS.
>
>> + } else {
>> + val = 0;
>> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
>> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> + val |= vgic_v3_cpu->num_id_bits <<
>> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
>> + ICC_CTLR_EL1_SEIS_SHIFT;
>> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
>> + ICC_CTLR_EL1_A3V_SHIFT;
>> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>
> again, these last two look weird to me.
>
>> +
>> + p->regval = val;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> + ICC_BPR0_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> + ICC_BPR0_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + if (!p->is_write)
>> + p->regval = 0;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
>> + if (p->is_write) {
>> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> + ICC_BPR1_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> + ICC_BPR1_EL1_MASK;
>> + }
>> + } else {
>> + if (!p->is_write)
>> + p->regval = min((vmcr.bpr + 1), 7U);
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> + ICC_IGRPEN0_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> + ICC_IGRPEN0_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_vmcr vmcr;
>> +
>> + vgic_get_vmcr(vcpu, &vmcr);
>> + if (p->is_write) {
>> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> + ICC_IGRPEN1_EL1_SHIFT;
>> + vgic_set_vmcr(vcpu, &vmcr);
>> + } else {
>> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> + ICC_IGRPEN1_EL1_MASK;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> + struct sys_reg_params *p, u8 apr, u8 idx)
>> +{
>> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> + uint32_t *ap_reg;
>> +
>> + if (apr)
>> + ap_reg = &vgicv3->vgic_ap1r[idx];
>> + else
>> + ap_reg = &vgicv3->vgic_ap0r[idx];
>> +
>> + if (p->is_write)
>> + *ap_reg = p->regval;
>> + else
>> + p->regval = *ap_reg;
>> +}
>> +
>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r, u8 apr)
>> +{
>> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> + u8 idx = r->Op2 & 3;
>> +
>> + switch (vgic_v3_cpu->num_pri_bits) {
>> + case 7:
>> + if (idx > 3)
>> + goto err;
>
> idx cannot be higher than three given the mask above, right?
>
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + break;
>> + case 6:
>> + if (idx > 1)
>> + goto err;
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + break;
>> + default:
>> + if (idx > 0)
>> + goto err;
>> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> + }
>
> what's the rationale behind ignoring the case where userspace is using
> unsupported priorities? Is it that this will be checked during
> save/restore of the ctlr?
>
> This sort of thing just looks like the case that's impossible to debug,
> because userspace could be scratching its head trying to understand why
> the value it wrote isn't recorded anywhere...
>
> If there's a good rationale for doing it this way, then could we have a
> comment to that effect?
Accessing umplemented priority registers raised UNDEF exception.
So userspace accesing should be ignored instead of recording unsupported
values.
>
>> +
>> + return;
>> +err:
>> + if (!p->is_write)
>> + p->regval = 0;
>> +}
>> +
>> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + access_gic_aprn(vcpu, p, r, 0);
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + access_gic_aprn(vcpu, p, r, 1);
>> +
>> + return true;
>> +}
>> +
>> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> + const struct sys_reg_desc *r)
>> +{
>> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> +
>> + /* Validate SRE bit */
>> + if (p->is_write) {
>> + if (!(p->regval & ICC_SRE_EL1_SRE))
>> + return false;
>> + } else {
>> + p->regval = vgicv3->vgic_sre;
>> + }
>> +
>> + return true;
>> +}
>> +
>> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
>> + /* ICC_PMR_EL1 */
>> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
>> + /* ICC_BPR0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
>> + /* ICC_AP0R0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
>> + /* ICC_AP0R1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
>> + /* ICC_AP0R2_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
>> + /* ICC_AP0R3_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
>> + /* ICC_AP1R0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
>> + /* ICC_AP1R1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
>> + /* ICC_AP1R2_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
>> + /* ICC_AP1R3_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
>> + /* ICC_BPR1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
>> + /* ICC_CTLR_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
>> + /* ICC_SRE_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
>> + /* ICC_IGRPEN0_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
>> + /* ICC_GRPEN1_EL1 */
>> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
>> +};
>> +
>> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg)
>> +{
>> + struct sys_reg_params params;
>> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> +
>> + params.regval = *reg;
>> + params.is_write = is_write;
>> + params.is_aarch32 = false;
>> + params.is_32bit = false;
>> +
>> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
>> + return 0;
>> +
>> + return -ENXIO;
>> +}
>> +
>> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg)
>> +{
>> + struct sys_reg_params params;
>> + const struct sys_reg_desc *r;
>> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> +
>> + if (is_write)
>> + params.regval = *reg;
>> + params.is_write = is_write;
>> + params.is_aarch32 = false;
>> + params.is_32bit = false;
>> +
>> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> + ARRAY_SIZE(gic_v3_icc_reg_descs));
>> + if (!r)
>> + return -ENXIO;
>> +
>> + if (!r->access(vcpu, ¶ms, r))
>> + return -EINVAL;
>
> According to the API, EINVAL meansinvalid mpidr. Should we expand on
> how it can be used or allocate a new error code?
How abt EACCES error code?
>
>> +
>> + if (!is_write)
>> + *reg = params.regval;
>> +
>> + return 0;
>> +}
>> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> index 967c295..1139971 100644
>> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
>> vgic_v3->vgic_sre = 0;
>> }
>>
>> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_ID_BITS_MASK) >>
>> + ICH_VTR_ID_BITS_SHIFT;
>> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
>> + ICH_VTR_PRI_BITS_MASK) >>
>> + ICH_VTR_PRI_BITS_SHIFT) + 1;
>> +
>> /* Get the show on the road... */
>> vgic_v3->vgic_hcr = ICH_HCR_EN;
>> }
>> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>> */
>> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
>> kvm_vgic_global_state.can_emulate_gicv2 = false;
>> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>>
>> if (!info->vcpu.start) {
>> kvm_info("GICv3: no GICV resource entry\n");
>> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> index c461f6b..0e632d0 100644
>> --- a/virt/kvm/arm/vgic/vgic.h
>> +++ b/virt/kvm/arm/vgic/vgic.h
>> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val);
>> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> int offset, u32 *val);
>> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> + u64 id, u64 *val);
>> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> + u64 *reg);
>> #else
>> static inline int vgic_register_its_iodevs(struct kvm *kvm)
>> {
>> --
>
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
2016-11-17 12:42 ` Vijay Kilari
@ 2016-11-17 16:01 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 16:01 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 06:12:39PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari@gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> >> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> >> variables to struct vmcr to support read and write of these fields.
> >>
> >> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> >> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
> >> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
> >> .
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> include/linux/irqchip/arm-gic-v3.h | 2 --
> >> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
> >> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
> >> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
> >> virt/kvm/arm/vgic/vgic.h | 5 +++++
> >> 5 files changed, 29 insertions(+), 20 deletions(-)
> >>
> >> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> >> index d48d886..61646aa 100644
> >> --- a/include/linux/irqchip/arm-gic-v3.h
> >> +++ b/include/linux/irqchip/arm-gic-v3.h
> >> @@ -404,8 +404,6 @@
> >> #define ICH_HCR_EN (1 << 0)
> >> #define ICH_HCR_UIE (1 << 1)
> >>
> >> -#define ICH_VMCR_CTLR_SHIFT 0
> >> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
> >> #define ICH_VMCR_CBPR_SHIFT 4
> >> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
> >> #define ICH_VMCR_EOIM_SHIFT 9
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> index 2cb04b7..ad353b5 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
> >> }
> >> }
> >>
> >> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> -{
> >> - if (kvm_vgic_global_state.type == VGIC_V2)
> >> - vgic_v2_set_vmcr(vcpu, vmcr);
> >> - else
> >> - vgic_v3_set_vmcr(vcpu, vmcr);
> >> -}
> >> -
> >> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> -{
> >> - if (kvm_vgic_global_state.type == VGIC_V2)
> >> - vgic_v2_get_vmcr(vcpu, vmcr);
> >> - else
> >> - vgic_v3_get_vmcr(vcpu, vmcr);
> >> -}
> >> -
> >> #define GICC_ARCH_VERSION_V2 0x2
> >>
> >> /* These are for userland accesses only, there is no guest-facing emulation. */
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> >> index 9939d1d..173d6f0 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> >> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> >> return -ENXIO;
> >> }
> >>
> >> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> +{
> >> + if (kvm_vgic_global_state.type == VGIC_V2)
> >> + vgic_v2_set_vmcr(vcpu, vmcr);
> >> + else
> >> + vgic_v3_set_vmcr(vcpu, vmcr);
> >> +}
> >> +
> >> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> +{
> >> + if (kvm_vgic_global_state.type == VGIC_V2)
> >> + vgic_v2_get_vmcr(vcpu, vmcr);
> >> + else
> >> + vgic_v3_get_vmcr(vcpu, vmcr);
> >> +}
> >> +
> >> /*
> >> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> >> * to a byte array and be directly observed as the guest wanted it to appear
> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> >> index 9f0dae3..967c295 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> >> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> >> {
> >> u32 vmcr;
> >>
> >> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
> >> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
> >> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
> >
> > This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
> > bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
> > 9 and then masking off everything by bit 9, so you'll end with never
> > being able to set VEOIM I think...
> >
> OK
> > Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
> > understand because it's deprecated, but why the first? This particular
> > piece of information would be very nice to have in the commit message.
>
> I understand that group 0 interrupts are not handled. So vFIQEn can be ignored.
> Spec says, if SRE=1 (non-secure) this bit is RES1 also it is alias to
> ICC_CTLR_EL1
> if SRE is 1. However there is no bit in ICC_CTLR_EL1 for FIQen. It is defined
> only in GICV_CTLR which is used when SRE=0.
So you should add a comment or the very least add to the commit message
that we ignore the FIQen bit, because our GIC emulation always implies
SRE=1 which means the vFIQEn bit is also RES1 (if I got this right).
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-11-17 16:01 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 16:01 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 06:12:39PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:31PM +0530, vijay.kilari at gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> >> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> >> variables to struct vmcr to support read and write of these fields.
> >>
> >> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> >> Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
> >> use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros
> >> .
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> include/linux/irqchip/arm-gic-v3.h | 2 --
> >> virt/kvm/arm/vgic/vgic-mmio-v2.c | 16 ----------------
> >> virt/kvm/arm/vgic/vgic-mmio.c | 16 ++++++++++++++++
> >> virt/kvm/arm/vgic/vgic-v3.c | 10 ++++++++--
> >> virt/kvm/arm/vgic/vgic.h | 5 +++++
> >> 5 files changed, 29 insertions(+), 20 deletions(-)
> >>
> >> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> >> index d48d886..61646aa 100644
> >> --- a/include/linux/irqchip/arm-gic-v3.h
> >> +++ b/include/linux/irqchip/arm-gic-v3.h
> >> @@ -404,8 +404,6 @@
> >> #define ICH_HCR_EN (1 << 0)
> >> #define ICH_HCR_UIE (1 << 1)
> >>
> >> -#define ICH_VMCR_CTLR_SHIFT 0
> >> -#define ICH_VMCR_CTLR_MASK (0x21f << ICH_VMCR_CTLR_SHIFT)
> >> #define ICH_VMCR_CBPR_SHIFT 4
> >> #define ICH_VMCR_CBPR_MASK (1 << ICH_VMCR_CBPR_SHIFT)
> >> #define ICH_VMCR_EOIM_SHIFT 9
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> index 2cb04b7..ad353b5 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> >> @@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
> >> }
> >> }
> >>
> >> -static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> -{
> >> - if (kvm_vgic_global_state.type == VGIC_V2)
> >> - vgic_v2_set_vmcr(vcpu, vmcr);
> >> - else
> >> - vgic_v3_set_vmcr(vcpu, vmcr);
> >> -}
> >> -
> >> -static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> -{
> >> - if (kvm_vgic_global_state.type == VGIC_V2)
> >> - vgic_v2_get_vmcr(vcpu, vmcr);
> >> - else
> >> - vgic_v3_get_vmcr(vcpu, vmcr);
> >> -}
> >> -
> >> #define GICC_ARCH_VERSION_V2 0x2
> >>
> >> /* These are for userland accesses only, there is no guest-facing emulation. */
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> >> index 9939d1d..173d6f0 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> >> @@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
> >> return -ENXIO;
> >> }
> >>
> >> +void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> +{
> >> + if (kvm_vgic_global_state.type == VGIC_V2)
> >> + vgic_v2_set_vmcr(vcpu, vmcr);
> >> + else
> >> + vgic_v3_set_vmcr(vcpu, vmcr);
> >> +}
> >> +
> >> +void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
> >> +{
> >> + if (kvm_vgic_global_state.type == VGIC_V2)
> >> + vgic_v2_get_vmcr(vcpu, vmcr);
> >> + else
> >> + vgic_v3_get_vmcr(vcpu, vmcr);
> >> +}
> >> +
> >> /*
> >> * kvm_mmio_read_buf() returns a value in a format where it can be converted
> >> * to a byte array and be directly observed as the guest wanted it to appear
> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> >> index 9f0dae3..967c295 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> >> @@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
> >> {
> >> u32 vmcr;
> >>
> >> - vmcr = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
> >> + vmcr = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
> >> + vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
> >
> > This looks weird: The EOImode field is bit[2] in the CTLR, and VEOIM is
> > bit[9] in the ICH_VMCR, but you're just shifting the ctlr field left by
> > 9 and then masking off everything by bit 9, so you'll end with never
> > being able to set VEOIM I think...
> >
> OK
> > Also, we do we now forget about VFIQEn and VAckCtl? The latter I can
> > understand because it's deprecated, but why the first? This particular
> > piece of information would be very nice to have in the commit message.
>
> I understand that group 0 interrupts are not handled. So vFIQEn can be ignored.
> Spec says, if SRE=1 (non-secure) this bit is RES1 also it is alias to
> ICC_CTLR_EL1
> if SRE is 1. However there is no bit in ICC_CTLR_EL1 for FIQen. It is defined
> only in GICV_CTLR which is used when SRE=0.
So you should add a comment or the very least add to the commit message
that we ignore the FIQen bit, because our GIC emulation always implies
SRE=1 which means the vFIQEn bit is also RES1 (if I got this right).
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-17 15:55 ` Vijay Kilari
@ 2016-11-17 16:09 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 16:09 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> VGICv3 CPU interface registers are accessed using
> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> is used to identify the cpu for registers access.
> >>
> >> The version of VGIC v3 specification is define here
> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>
> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> arch/arm64/kvm/Makefile | 1 +
> >> include/kvm/arm_vgic.h | 9 +
> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> 8 files changed, 395 insertions(+)
> >>
> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> index 56dc08d..91c7137 100644
> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> +
> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >>
> >> /* Device Control API on vcpu fd */
> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> index d50a82a..1a14e29 100644
> >> --- a/arch/arm64/kvm/Makefile
> >> +++ b/arch/arm64/kvm/Makefile
> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >
> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> > mean that either it is clearly only supported on AArch64 systems or it's
> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> > on AArch32.
>
> It supports both AArch64 and AArch64 in handling of system registers
> save/restore.
> All system registers that we save/restore are 32-bit for both aarch64
> and aarch32.
> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> are same. However the codes sent by qemu is matched and register
> are handled properly irrespective of AArch32 or AArch64.
>
> I don't have platform which support AArch32 guests to verify.
Actually this is not about the guest, it's about an ARMv8 AArch32 host
that has a GICv3.
I just tried to do a v7 compile with your patches, and it results in an
epic failure, so there's something for you to look at.
> >
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 002f092..730a18a 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >>
> >> /* GIC system register CPU interface */
> >> struct static_key_false gicv3_cpuif;
> >> +
> >> + /* Cache ICH_VTR_EL2 reg value */
> >> + u32 ich_vtr_el2;
> >> };
> >>
> >> extern struct vgic_global kvm_vgic_global_state;
> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> u64 pendbaser;
> >>
> >> bool lpis_enabled;
> >> +
> >> + /* Cache guest priority bits */
> >> + u32 num_pri_bits;
> >> +
> >> + /* Cache guest interrupt ID bits */
> >> + u32 num_id_bits;
> >> };
> >>
> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> index 6c7d30c..da532d1 100644
> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> if (!is_write)
> >> *reg = tmp32;
> >> break;
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 regid;
> >> +
> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> + regid, reg);
> >> + break;
> >> + }
> >> default:
> >> ret = -EINVAL;
> >> break;
> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> reg = tmp32;
> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> + u64 reg;
> >> +
> >> + if (get_user(reg, uaddr))
> >> + return -EFAULT;
> >> +
> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> + }
> >> }
> >> return -ENXIO;
> >> }
> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> tmp32 = reg;
> >> return put_user(tmp32, uaddr);
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> + u64 reg;
> >> +
> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> + if (ret)
> >> + return ret;
> >> + return put_user(reg, uaddr);
> >> + }
> >> }
> >>
> >> return -ENXIO;
> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> break;
> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> return vgic_v3_has_attr_regs(dev, attr);
> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> return 0;
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index b35fb83..519b919 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -23,6 +23,7 @@
> >>
> >> #include "vgic.h"
> >> #include "vgic-mmio.h"
> >> +#include "sys_regs.h"
> >>
> >> /* extract @num bytes at @offset bytes offset in data */
> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> break;
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 reg, id;
> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> + struct kvm_vcpu *vcpu;
> >> +
> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> +
> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> +
> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> + if (!vcpu)
> >> + return -EINVAL;
> >> +
> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> + }
> >> default:
> >> return -ENXIO;
> >> }
> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> new file mode 100644
> >> index 0000000..69d8597
> >> --- /dev/null
> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >
> > Shouldn't we have a GPL header here?
> >
> >> @@ -0,0 +1,324 @@
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> +#include <linux/kvm.h>
> >> +#include <linux/kvm_host.h>
> >> +#include <kvm/iodev.h>
> >> +#include <kvm/arm_vgic.h>
> >> +#include <asm/kvm_emulate.h>
> >> +#include <asm/kvm_arm.h>
> >> +#include <asm/kvm_mmu.h>
> >> +
> >> +#include "vgic.h"
> >> +#include "vgic-mmio.h"
> >> +#include "sys_regs.h"
> >> +
> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> + struct vgic_vmcr vmcr;
> >> + u64 val;
> >> + u32 num_pri_bits, num_id_bits;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + val = p->regval;
> >> +
> >> + /*
> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> + * guest programmed ID and PRI bits
> >> + */
> >
> > I would suggest rewording this comment:
> > Disallow restoring VM state not supported by this hardware.
> >
> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> + return false;
> >> +
> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >
> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> > understand which effect this is intended to have?
> >
> > Sure, it may limit what you do with other registers later, but since
> > there's no ordering requirement that the ctlr be restored first, I'm not
> > sure it makes sense.
> >
> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> > situation during runtime after a GICv3 restore where the
> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> > which is never the case if you didn't do a save/restore.
>
> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> than HW supported
> value.
>
So answer my question: What is the intended effect of writing this
value? Is it just so that if you migrate this platform back again, then
you're checking compatibility with what the guest would potentially do,
or should you maintain the num_pri_bits limitation during runtime
somehow?
> >
> > Finally, should we somehow ensure that this field is set to the same
> > value across VCPUs or is that not an architectural requirement?
> >
> Yes it is nice to have it same across VCPUs. But should be ok as
> we are ensuring value is not greater than HW supported value.
Does the architecture allow having a different number of priority bits
supported across CPUs? If not, you shouldn't allow a VM programming
things that way either.
> There is no single point of place where we can make such a check
>
> >> +
> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> >> + return false;
> >> +
> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
> >
> > same questions
> >
> >> +
> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> >> + ICH_VMCR_EOIM_SHIFT;
> >
> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> > format or in the VMCR format? I would assume the former, since
> > otherwise I don't get the point with this indirection, and for GICv2
> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> > into VMCR values.
> >
> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> > ring.
>
> I will check and fix it.
> >
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >
> > Should we check compatibility between the source and destination for the
> > SEIS and A3V support here?
>
> Can be checked. But I feel A3V check makes more sense than checking for
> SEIS.
>
Please argue the *why* for whatever you end up doing with respect to
both bits in the commit message of your next patch revision.
> >
> >> + } else {
> >> + val = 0;
> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >> + val |= vgic_v3_cpu->num_id_bits <<
> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> >> + ICC_CTLR_EL1_SEIS_SHIFT;
> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> >> + ICC_CTLR_EL1_A3V_SHIFT;
> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >
> > again, these last two look weird to me.
> >
> >> +
> >> + p->regval = val;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >> + ICC_BPR0_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >> + ICC_BPR0_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + if (!p->is_write)
> >> + p->regval = 0;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> >> + if (p->is_write) {
> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >> + ICC_BPR1_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >> + ICC_BPR1_EL1_MASK;
> >> + }
> >> + } else {
> >> + if (!p->is_write)
> >> + p->regval = min((vmcr.bpr + 1), 7U);
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >> + ICC_IGRPEN0_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >> + ICC_IGRPEN0_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >> + ICC_IGRPEN1_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >> + ICC_IGRPEN1_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> >> + struct sys_reg_params *p, u8 apr, u8 idx)
> >> +{
> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> + uint32_t *ap_reg;
> >> +
> >> + if (apr)
> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
> >> + else
> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
> >> +
> >> + if (p->is_write)
> >> + *ap_reg = p->regval;
> >> + else
> >> + p->regval = *ap_reg;
> >> +}
> >> +
> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r, u8 apr)
> >> +{
> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> + u8 idx = r->Op2 & 3;
> >> +
> >> + switch (vgic_v3_cpu->num_pri_bits) {
> >> + case 7:
> >> + if (idx > 3)
> >> + goto err;
> >
> > idx cannot be higher than three given the mask above, right?
> >
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + break;
> >> + case 6:
> >> + if (idx > 1)
> >> + goto err;
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + break;
> >> + default:
> >> + if (idx > 0)
> >> + goto err;
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + }
> >
> > what's the rationale behind ignoring the case where userspace is using
> > unsupported priorities? Is it that this will be checked during
> > save/restore of the ctlr?
> >
> > This sort of thing just looks like the case that's impossible to debug,
> > because userspace could be scratching its head trying to understand why
> > the value it wrote isn't recorded anywhere...
> >
> > If there's a good rationale for doing it this way, then could we have a
> > comment to that effect?
>
> Accessing umplemented priority registers raised UNDEF exception.
> So userspace accesing should be ignored instead of recording unsupported
> values.
>
That's not what I asked.
I asked why it's silently ignored as opposed to raising an error visible
to user space?
> >
> >> +
> >> + return;
> >> +err:
> >> + if (!p->is_write)
> >> + p->regval = 0;
> >> +}
> >> +
> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + access_gic_aprn(vcpu, p, r, 0);
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + access_gic_aprn(vcpu, p, r, 1);
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> +
> >> + /* Validate SRE bit */
> >> + if (p->is_write) {
> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
> >> + return false;
> >> + } else {
> >> + p->regval = vgicv3->vgic_sre;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> >> + /* ICC_PMR_EL1 */
> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> >> + /* ICC_BPR0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> >> + /* ICC_AP0R0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> >> + /* ICC_AP0R1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> >> + /* ICC_AP0R2_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> >> + /* ICC_AP0R3_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> >> + /* ICC_AP1R0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> >> + /* ICC_AP1R1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> >> + /* ICC_AP1R2_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> >> + /* ICC_AP1R3_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> >> + /* ICC_BPR1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> >> + /* ICC_CTLR_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> >> + /* ICC_SRE_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> >> + /* ICC_IGRPEN0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> >> + /* ICC_GRPEN1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> >> +};
> >> +
> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg)
> >> +{
> >> + struct sys_reg_params params;
> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> +
> >> + params.regval = *reg;
> >> + params.is_write = is_write;
> >> + params.is_aarch32 = false;
> >> + params.is_32bit = false;
> >> +
> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> >> + return 0;
> >> +
> >> + return -ENXIO;
> >> +}
> >> +
> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg)
> >> +{
> >> + struct sys_reg_params params;
> >> + const struct sys_reg_desc *r;
> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> +
> >> + if (is_write)
> >> + params.regval = *reg;
> >> + params.is_write = is_write;
> >> + params.is_aarch32 = false;
> >> + params.is_32bit = false;
> >> +
> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> >> + if (!r)
> >> + return -ENXIO;
> >> +
> >> + if (!r->access(vcpu, ¶ms, r))
> >> + return -EINVAL;
> >
> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
> > how it can be used or allocate a new error code?
> How abt EACCES error code?
>
That would mean permission denied, which is a bit weird.
> >
> >> +
> >> + if (!is_write)
> >> + *reg = params.regval;
> >> +
> >> + return 0;
> >> +}
> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> >> index 967c295..1139971 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> >> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> >> vgic_v3->vgic_sre = 0;
> >> }
> >>
> >> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_ID_BITS_MASK) >>
> >> + ICH_VTR_ID_BITS_SHIFT;
> >> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_PRI_BITS_MASK) >>
> >> + ICH_VTR_PRI_BITS_SHIFT) + 1;
> >> +
> >> /* Get the show on the road... */
> >> vgic_v3->vgic_hcr = ICH_HCR_EN;
> >> }
> >> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
> >> */
> >> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
> >> kvm_vgic_global_state.can_emulate_gicv2 = false;
> >> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
> >>
> >> if (!info->vcpu.start) {
> >> kvm_info("GICv3: no GICV resource entry\n");
> >> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> >> index c461f6b..0e632d0 100644
> >> --- a/virt/kvm/arm/vgic/vgic.h
> >> +++ b/virt/kvm/arm/vgic/vgic.h
> >> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val);
> >> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val);
> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> + u64 id, u64 *val);
> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg);
> >> #else
> >> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> >> {
> >> --
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-17 16:09 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-17 16:09 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>
> >> VGICv3 CPU interface registers are accessed using
> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> is used to identify the cpu for registers access.
> >>
> >> The version of VGIC v3 specification is define here
> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>
> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> ---
> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> arch/arm64/kvm/Makefile | 1 +
> >> include/kvm/arm_vgic.h | 9 +
> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> 8 files changed, 395 insertions(+)
> >>
> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> index 56dc08d..91c7137 100644
> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> +
> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >>
> >> /* Device Control API on vcpu fd */
> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> index d50a82a..1a14e29 100644
> >> --- a/arch/arm64/kvm/Makefile
> >> +++ b/arch/arm64/kvm/Makefile
> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >
> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> > mean that either it is clearly only supported on AArch64 systems or it's
> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> > on AArch32.
>
> It supports both AArch64 and AArch64 in handling of system registers
> save/restore.
> All system registers that we save/restore are 32-bit for both aarch64
> and aarch32.
> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> are same. However the codes sent by qemu is matched and register
> are handled properly irrespective of AArch32 or AArch64.
>
> I don't have platform which support AArch32 guests to verify.
Actually this is not about the guest, it's about an ARMv8 AArch32 host
that has a GICv3.
I just tried to do a v7 compile with your patches, and it results in an
epic failure, so there's something for you to look at.
> >
> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 002f092..730a18a 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >>
> >> /* GIC system register CPU interface */
> >> struct static_key_false gicv3_cpuif;
> >> +
> >> + /* Cache ICH_VTR_EL2 reg value */
> >> + u32 ich_vtr_el2;
> >> };
> >>
> >> extern struct vgic_global kvm_vgic_global_state;
> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> u64 pendbaser;
> >>
> >> bool lpis_enabled;
> >> +
> >> + /* Cache guest priority bits */
> >> + u32 num_pri_bits;
> >> +
> >> + /* Cache guest interrupt ID bits */
> >> + u32 num_id_bits;
> >> };
> >>
> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> index 6c7d30c..da532d1 100644
> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> if (!is_write)
> >> *reg = tmp32;
> >> break;
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 regid;
> >> +
> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> + regid, reg);
> >> + break;
> >> + }
> >> default:
> >> ret = -EINVAL;
> >> break;
> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> reg = tmp32;
> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> + u64 reg;
> >> +
> >> + if (get_user(reg, uaddr))
> >> + return -EFAULT;
> >> +
> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> + }
> >> }
> >> return -ENXIO;
> >> }
> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> tmp32 = reg;
> >> return put_user(tmp32, uaddr);
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> + u64 reg;
> >> +
> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> + if (ret)
> >> + return ret;
> >> + return put_user(reg, uaddr);
> >> + }
> >> }
> >>
> >> return -ENXIO;
> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> break;
> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> return vgic_v3_has_attr_regs(dev, attr);
> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> return 0;
> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> index b35fb83..519b919 100644
> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> @@ -23,6 +23,7 @@
> >>
> >> #include "vgic.h"
> >> #include "vgic-mmio.h"
> >> +#include "sys_regs.h"
> >>
> >> /* extract @num bytes at @offset bytes offset in data */
> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> break;
> >> }
> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> + u64 reg, id;
> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> + struct kvm_vcpu *vcpu;
> >> +
> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> +
> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> +
> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> + if (!vcpu)
> >> + return -EINVAL;
> >> +
> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> + }
> >> default:
> >> return -ENXIO;
> >> }
> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> new file mode 100644
> >> index 0000000..69d8597
> >> --- /dev/null
> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >
> > Shouldn't we have a GPL header here?
> >
> >> @@ -0,0 +1,324 @@
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> +#include <linux/kvm.h>
> >> +#include <linux/kvm_host.h>
> >> +#include <kvm/iodev.h>
> >> +#include <kvm/arm_vgic.h>
> >> +#include <asm/kvm_emulate.h>
> >> +#include <asm/kvm_arm.h>
> >> +#include <asm/kvm_mmu.h>
> >> +
> >> +#include "vgic.h"
> >> +#include "vgic-mmio.h"
> >> +#include "sys_regs.h"
> >> +
> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> + struct vgic_vmcr vmcr;
> >> + u64 val;
> >> + u32 num_pri_bits, num_id_bits;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + val = p->regval;
> >> +
> >> + /*
> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> + * guest programmed ID and PRI bits
> >> + */
> >
> > I would suggest rewording this comment:
> > Disallow restoring VM state not supported by this hardware.
> >
> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> + return false;
> >> +
> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >
> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> > understand which effect this is intended to have?
> >
> > Sure, it may limit what you do with other registers later, but since
> > there's no ordering requirement that the ctlr be restored first, I'm not
> > sure it makes sense.
> >
> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> > situation during runtime after a GICv3 restore where the
> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> > which is never the case if you didn't do a save/restore.
>
> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> than HW supported
> value.
>
So answer my question: What is the intended effect of writing this
value? Is it just so that if you migrate this platform back again, then
you're checking compatibility with what the guest would potentially do,
or should you maintain the num_pri_bits limitation during runtime
somehow?
> >
> > Finally, should we somehow ensure that this field is set to the same
> > value across VCPUs or is that not an architectural requirement?
> >
> Yes it is nice to have it same across VCPUs. But should be ok as
> we are ensuring value is not greater than HW supported value.
Does the architecture allow having a different number of priority bits
supported across CPUs? If not, you shouldn't allow a VM programming
things that way either.
> There is no single point of place where we can make such a check
>
> >> +
> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> >> + return false;
> >> +
> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
> >
> > same questions
> >
> >> +
> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> >> + ICH_VMCR_EOIM_SHIFT;
> >
> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> > format or in the VMCR format? I would assume the former, since
> > otherwise I don't get the point with this indirection, and for GICv2
> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> > into VMCR values.
> >
> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> > ring.
>
> I will check and fix it.
> >
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >
> > Should we check compatibility between the source and destination for the
> > SEIS and A3V support here?
>
> Can be checked. But I feel A3V check makes more sense than checking for
> SEIS.
>
Please argue the *why* for whatever you end up doing with respect to
both bits in the commit message of your next patch revision.
> >
> >> + } else {
> >> + val = 0;
> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >> + val |= vgic_v3_cpu->num_id_bits <<
> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> >> + ICC_CTLR_EL1_SEIS_SHIFT;
> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> >> + ICC_CTLR_EL1_A3V_SHIFT;
> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >
> > again, these last two look weird to me.
> >
> >> +
> >> + p->regval = val;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >> + ICC_BPR0_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >> + ICC_BPR0_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + if (!p->is_write)
> >> + p->regval = 0;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> >> + if (p->is_write) {
> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >> + ICC_BPR1_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >> + ICC_BPR1_EL1_MASK;
> >> + }
> >> + } else {
> >> + if (!p->is_write)
> >> + p->regval = min((vmcr.bpr + 1), 7U);
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >> + ICC_IGRPEN0_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >> + ICC_IGRPEN0_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_vmcr vmcr;
> >> +
> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> + if (p->is_write) {
> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >> + ICC_IGRPEN1_EL1_SHIFT;
> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> + } else {
> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >> + ICC_IGRPEN1_EL1_MASK;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> >> + struct sys_reg_params *p, u8 apr, u8 idx)
> >> +{
> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> + uint32_t *ap_reg;
> >> +
> >> + if (apr)
> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
> >> + else
> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
> >> +
> >> + if (p->is_write)
> >> + *ap_reg = p->regval;
> >> + else
> >> + p->regval = *ap_reg;
> >> +}
> >> +
> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r, u8 apr)
> >> +{
> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> + u8 idx = r->Op2 & 3;
> >> +
> >> + switch (vgic_v3_cpu->num_pri_bits) {
> >> + case 7:
> >> + if (idx > 3)
> >> + goto err;
> >
> > idx cannot be higher than three given the mask above, right?
> >
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + break;
> >> + case 6:
> >> + if (idx > 1)
> >> + goto err;
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + break;
> >> + default:
> >> + if (idx > 0)
> >> + goto err;
> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> + }
> >
> > what's the rationale behind ignoring the case where userspace is using
> > unsupported priorities? Is it that this will be checked during
> > save/restore of the ctlr?
> >
> > This sort of thing just looks like the case that's impossible to debug,
> > because userspace could be scratching its head trying to understand why
> > the value it wrote isn't recorded anywhere...
> >
> > If there's a good rationale for doing it this way, then could we have a
> > comment to that effect?
>
> Accessing umplemented priority registers raised UNDEF exception.
> So userspace accesing should be ignored instead of recording unsupported
> values.
>
That's not what I asked.
I asked why it's silently ignored as opposed to raising an error visible
to user space?
> >
> >> +
> >> + return;
> >> +err:
> >> + if (!p->is_write)
> >> + p->regval = 0;
> >> +}
> >> +
> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + access_gic_aprn(vcpu, p, r, 0);
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + access_gic_aprn(vcpu, p, r, 1);
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> + const struct sys_reg_desc *r)
> >> +{
> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> +
> >> + /* Validate SRE bit */
> >> + if (p->is_write) {
> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
> >> + return false;
> >> + } else {
> >> + p->regval = vgicv3->vgic_sre;
> >> + }
> >> +
> >> + return true;
> >> +}
> >> +
> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> >> + /* ICC_PMR_EL1 */
> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> >> + /* ICC_BPR0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> >> + /* ICC_AP0R0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> >> + /* ICC_AP0R1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> >> + /* ICC_AP0R2_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> >> + /* ICC_AP0R3_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> >> + /* ICC_AP1R0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> >> + /* ICC_AP1R1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> >> + /* ICC_AP1R2_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> >> + /* ICC_AP1R3_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> >> + /* ICC_BPR1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> >> + /* ICC_CTLR_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> >> + /* ICC_SRE_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> >> + /* ICC_IGRPEN0_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> >> + /* ICC_GRPEN1_EL1 */
> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> >> +};
> >> +
> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg)
> >> +{
> >> + struct sys_reg_params params;
> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> +
> >> + params.regval = *reg;
> >> + params.is_write = is_write;
> >> + params.is_aarch32 = false;
> >> + params.is_32bit = false;
> >> +
> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> >> + return 0;
> >> +
> >> + return -ENXIO;
> >> +}
> >> +
> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg)
> >> +{
> >> + struct sys_reg_params params;
> >> + const struct sys_reg_desc *r;
> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> +
> >> + if (is_write)
> >> + params.regval = *reg;
> >> + params.is_write = is_write;
> >> + params.is_aarch32 = false;
> >> + params.is_32bit = false;
> >> +
> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> >> + if (!r)
> >> + return -ENXIO;
> >> +
> >> + if (!r->access(vcpu, ¶ms, r))
> >> + return -EINVAL;
> >
> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
> > how it can be used or allocate a new error code?
> How abt EACCES error code?
>
That would mean permission denied, which is a bit weird.
> >
> >> +
> >> + if (!is_write)
> >> + *reg = params.regval;
> >> +
> >> + return 0;
> >> +}
> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
> >> index 967c295..1139971 100644
> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
> >> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
> >> vgic_v3->vgic_sre = 0;
> >> }
> >>
> >> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_ID_BITS_MASK) >>
> >> + ICH_VTR_ID_BITS_SHIFT;
> >> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
> >> + ICH_VTR_PRI_BITS_MASK) >>
> >> + ICH_VTR_PRI_BITS_SHIFT) + 1;
> >> +
> >> /* Get the show on the road... */
> >> vgic_v3->vgic_hcr = ICH_HCR_EN;
> >> }
> >> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
> >> */
> >> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
> >> kvm_vgic_global_state.can_emulate_gicv2 = false;
> >> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
> >>
> >> if (!info->vcpu.start) {
> >> kvm_info("GICv3: no GICV resource entry\n");
> >> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> >> index c461f6b..0e632d0 100644
> >> --- a/virt/kvm/arm/vgic/vgic.h
> >> +++ b/virt/kvm/arm/vgic/vgic.h
> >> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val);
> >> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> int offset, u32 *val);
> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> >> + u64 id, u64 *val);
> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> + u64 *reg);
> >> #else
> >> static inline int vgic_register_its_iodevs(struct kvm *kvm)
> >> {
> >> --
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-17 16:09 ` Christoffer Dall
@ 2016-11-18 16:58 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-18 16:58 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
>> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>
>> >> VGICv3 CPU interface registers are accessed using
>> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> is used to identify the cpu for registers access.
>> >>
>> >> The version of VGIC v3 specification is define here
>> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>
>> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> ---
>> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> arch/arm64/kvm/Makefile | 1 +
>> >> include/kvm/arm_vgic.h | 9 +
>> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> 8 files changed, 395 insertions(+)
>> >>
>> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> index 56dc08d..91c7137 100644
>> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> +
>> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >>
>> >> /* Device Control API on vcpu fd */
>> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> index d50a82a..1a14e29 100644
>> >> --- a/arch/arm64/kvm/Makefile
>> >> +++ b/arch/arm64/kvm/Makefile
>> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >
>> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> > mean that either it is clearly only supported on AArch64 systems or it's
>> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> > on AArch32.
>>
>> It supports both AArch64 and AArch64 in handling of system registers
>> save/restore.
>> All system registers that we save/restore are 32-bit for both aarch64
>> and aarch32.
>> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> are same. However the codes sent by qemu is matched and register
>> are handled properly irrespective of AArch32 or AArch64.
>>
>> I don't have platform which support AArch32 guests to verify.
>
> Actually this is not about the guest, it's about an ARMv8 AArch32 host
> that has a GICv3.
>
> I just tried to do a v7 compile with your patches, and it results in an
> epic failure, so there's something for you to look at.
>
>> >
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> >> index 002f092..730a18a 100644
>> >> --- a/include/kvm/arm_vgic.h
>> >> +++ b/include/kvm/arm_vgic.h
>> >> @@ -71,6 +71,9 @@ struct vgic_global {
>> >>
>> >> /* GIC system register CPU interface */
>> >> struct static_key_false gicv3_cpuif;
>> >> +
>> >> + /* Cache ICH_VTR_EL2 reg value */
>> >> + u32 ich_vtr_el2;
>> >> };
>> >>
>> >> extern struct vgic_global kvm_vgic_global_state;
>> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> >> u64 pendbaser;
>> >>
>> >> bool lpis_enabled;
>> >> +
>> >> + /* Cache guest priority bits */
>> >> + u32 num_pri_bits;
>> >> +
>> >> + /* Cache guest interrupt ID bits */
>> >> + u32 num_id_bits;
>> >> };
>> >>
>> >> extern struct static_key_false vgic_v2_cpuif_trap;
>> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> index 6c7d30c..da532d1 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> >> if (!is_write)
>> >> *reg = tmp32;
>> >> break;
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 regid;
>> >> +
>> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> >> + regid, reg);
>> >> + break;
>> >> + }
>> >> default:
>> >> ret = -EINVAL;
>> >> break;
>> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> >> reg = tmp32;
>> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> + u64 reg;
>> >> +
>> >> + if (get_user(reg, uaddr))
>> >> + return -EFAULT;
>> >> +
>> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> + }
>> >> }
>> >> return -ENXIO;
>> >> }
>> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> >> tmp32 = reg;
>> >> return put_user(tmp32, uaddr);
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> + u64 reg;
>> >> +
>> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> >> + if (ret)
>> >> + return ret;
>> >> + return put_user(reg, uaddr);
>> >> + }
>> >> }
>> >>
>> >> return -ENXIO;
>> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> >> break;
>> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> >> return vgic_v3_has_attr_regs(dev, attr);
>> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> >> return 0;
>> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> index b35fb83..519b919 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> @@ -23,6 +23,7 @@
>> >>
>> >> #include "vgic.h"
>> >> #include "vgic-mmio.h"
>> >> +#include "sys_regs.h"
>> >>
>> >> /* extract @num bytes at @offset bytes offset in data */
>> >> unsigned long extract_bytes(u64 data, unsigned int offset,
>> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> >> break;
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 reg, id;
>> >> + unsigned long vgic_mpidr, mpidr_reg;
>> >> + struct kvm_vcpu *vcpu;
>> >> +
>> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> >> +
>> >> + /* Convert plain mpidr value to MPIDR reg format */
>> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> >> +
>> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> >> + if (!vcpu)
>> >> + return -EINVAL;
>> >> +
>> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> >> + }
>> >> default:
>> >> return -ENXIO;
>> >> }
>> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> new file mode 100644
>> >> index 0000000..69d8597
>> >> --- /dev/null
>> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >
>> > Shouldn't we have a GPL header here?
>> >
>> >> @@ -0,0 +1,324 @@
>> >> +#include <linux/irqchip/arm-gic-v3.h>
>> >> +#include <linux/kvm.h>
>> >> +#include <linux/kvm_host.h>
>> >> +#include <kvm/iodev.h>
>> >> +#include <kvm/arm_vgic.h>
>> >> +#include <asm/kvm_emulate.h>
>> >> +#include <asm/kvm_arm.h>
>> >> +#include <asm/kvm_mmu.h>
>> >> +
>> >> +#include "vgic.h"
>> >> +#include "vgic-mmio.h"
>> >> +#include "sys_regs.h"
>> >> +
>> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> + struct vgic_vmcr vmcr;
>> >> + u64 val;
>> >> + u32 num_pri_bits, num_id_bits;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + val = p->regval;
>> >> +
>> >> + /*
>> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> >> + * guest programmed ID and PRI bits
>> >> + */
>> >
>> > I would suggest rewording this comment:
>> > Disallow restoring VM state not supported by this hardware.
>> >
>> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> >> + return false;
>> >> +
>> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>> >
>> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
>> > understand which effect this is intended to have?
>> >
>> > Sure, it may limit what you do with other registers later, but since
>> > there's no ordering requirement that the ctlr be restored first, I'm not
>> > sure it makes sense.
>> >
>> > Also, since this field is RO in the ICH_VTR, we'll have a strange
>> > situation during runtime after a GICv3 restore where the
>> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
>> > which is never the case if you didn't do a save/restore.
>>
>> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
>> than HW supported
>> value.
>>
>
> So answer my question: What is the intended effect of writing this
> value? Is it just so that if you migrate this platform back again, then
> you're checking compatibility with what the guest would potentially do,
Yes and also to limit the valid aprn registers access as you said above.
But that has ordering restriction. Which I think we should follow.
> or should you maintain the num_pri_bits limitation during runtime
> somehow?
Once after checking compatibility, at runtime it is not updated
and this value is not used at all in VGIC further
>
>> >
>> > Finally, should we somehow ensure that this field is set to the same
>> > value across VCPUs or is that not an architectural requirement?
>> >
>> Yes it is nice to have it same across VCPUs. But should be ok as
>> we are ensuring value is not greater than HW supported value.
>
> Does the architecture allow having a different number of priority bits
> supported across CPUs? If not, you shouldn't allow a VM programming
> things that way either.
AFAIK, architecturally it is not mentioned any where in the spec that priority
bits should be same across CPUs.
>
>> There is no single point of place where we can make such a check
>>
>> >> +
>> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
>> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
>> >> + return false;
>> >> +
>> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
>> >
>> > same questions
>> >
>> >> +
>> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
>> >> + ICH_VMCR_EOIM_SHIFT;
>> >
>> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
>> > format or in the VMCR format? I would assume the former, since
>> > otherwise I don't get the point with this indirection, and for GICv2
>> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
>> > into VMCR values.
>> >
>> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
>> > ring.
>>
>> I will check and fix it.
>> >
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >
>> > Should we check compatibility between the source and destination for the
>> > SEIS and A3V support here?
>>
>> Can be checked. But I feel A3V check makes more sense than checking for
>> SEIS.
>>
>
> Please argue the *why* for whatever you end up doing with respect to
> both bits in the commit message of your next patch revision.
>
>> >
>> >> + } else {
>> >> + val = 0;
>> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
>> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> >> + val |= vgic_v3_cpu->num_id_bits <<
>> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
>> >> + ICC_CTLR_EL1_SEIS_SHIFT;
>> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
>> >> + ICC_CTLR_EL1_A3V_SHIFT;
>> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>> >
>> > again, these last two look weird to me.
>> >
>> >> +
>> >> + p->regval = val;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> >> + ICC_BPR0_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> >> + ICC_BPR0_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + if (!p->is_write)
>> >> + p->regval = 0;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
>> >> + if (p->is_write) {
>> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> >> + ICC_BPR1_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> >> + ICC_BPR1_EL1_MASK;
>> >> + }
>> >> + } else {
>> >> + if (!p->is_write)
>> >> + p->regval = min((vmcr.bpr + 1), 7U);
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> >> + ICC_IGRPEN0_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> >> + ICC_IGRPEN0_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> >> + ICC_IGRPEN1_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> >> + ICC_IGRPEN1_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> >> + struct sys_reg_params *p, u8 apr, u8 idx)
>> >> +{
>> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> >> + uint32_t *ap_reg;
>> >> +
>> >> + if (apr)
>> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
>> >> + else
>> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
>> >> +
>> >> + if (p->is_write)
>> >> + *ap_reg = p->regval;
>> >> + else
>> >> + p->regval = *ap_reg;
>> >> +}
>> >> +
>> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r, u8 apr)
>> >> +{
>> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> + u8 idx = r->Op2 & 3;
>> >> +
>> >> + switch (vgic_v3_cpu->num_pri_bits) {
>> >> + case 7:
>> >> + if (idx > 3)
>> >> + goto err;
>> >
>> > idx cannot be higher than three given the mask above, right?
>> >
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + break;
>> >> + case 6:
>> >> + if (idx > 1)
>> >> + goto err;
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + break;
>> >> + default:
>> >> + if (idx > 0)
>> >> + goto err;
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + }
>> >
>> > what's the rationale behind ignoring the case where userspace is using
>> > unsupported priorities? Is it that this will be checked during
>> > save/restore of the ctlr?
>> >
>> > This sort of thing just looks like the case that's impossible to debug,
>> > because userspace could be scratching its head trying to understand why
>> > the value it wrote isn't recorded anywhere...
>> >
>> > If there's a good rationale for doing it this way, then could we have a
>> > comment to that effect?
>>
>> Accessing umplemented priority registers raised UNDEF exception.
>> So userspace accesing should be ignored instead of recording unsupported
>> values.
>>
>
> That's not what I asked.
>
> I asked why it's silently ignored as opposed to raising an error visible
> to user space?
>
>> >
>> >> +
>> >> + return;
>> >> +err:
>> >> + if (!p->is_write)
>> >> + p->regval = 0;
>> >> +}
>> >> +
>> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + access_gic_aprn(vcpu, p, r, 0);
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + access_gic_aprn(vcpu, p, r, 1);
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> >> +
>> >> + /* Validate SRE bit */
>> >> + if (p->is_write) {
>> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
>> >> + return false;
>> >> + } else {
>> >> + p->regval = vgicv3->vgic_sre;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
>> >> + /* ICC_PMR_EL1 */
>> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
>> >> + /* ICC_BPR0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
>> >> + /* ICC_AP0R0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
>> >> + /* ICC_AP0R1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
>> >> + /* ICC_AP0R2_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
>> >> + /* ICC_AP0R3_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
>> >> + /* ICC_AP1R0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
>> >> + /* ICC_AP1R1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
>> >> + /* ICC_AP1R2_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
>> >> + /* ICC_AP1R3_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
>> >> + /* ICC_BPR1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
>> >> + /* ICC_CTLR_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
>> >> + /* ICC_SRE_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
>> >> + /* ICC_IGRPEN0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
>> >> + /* ICC_GRPEN1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
>> >> +};
>> >> +
>> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg)
>> >> +{
>> >> + struct sys_reg_params params;
>> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> >> +
>> >> + params.regval = *reg;
>> >> + params.is_write = is_write;
>> >> + params.is_aarch32 = false;
>> >> + params.is_32bit = false;
>> >> +
>> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
>> >> + return 0;
>> >> +
>> >> + return -ENXIO;
>> >> +}
>> >> +
>> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg)
>> >> +{
>> >> + struct sys_reg_params params;
>> >> + const struct sys_reg_desc *r;
>> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> >> +
>> >> + if (is_write)
>> >> + params.regval = *reg;
>> >> + params.is_write = is_write;
>> >> + params.is_aarch32 = false;
>> >> + params.is_32bit = false;
>> >> +
>> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
>> >> + if (!r)
>> >> + return -ENXIO;
>> >> +
>> >> + if (!r->access(vcpu, ¶ms, r))
>> >> + return -EINVAL;
>> >
>> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
>> > how it can be used or allocate a new error code?
>> How abt EACCES error code?
>>
>
> That would mean permission denied, which is a bit weird.
Yes I agree, but you can suggest.
>
>> >
>> >> +
>> >> + if (!is_write)
>> >> + *reg = params.regval;
>> >> +
>> >> + return 0;
>> >> +}
>> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> >> index 967c295..1139971 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> >> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
>> >> vgic_v3->vgic_sre = 0;
>> >> }
>> >>
>> >> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_ID_BITS_MASK) >>
>> >> + ICH_VTR_ID_BITS_SHIFT;
>> >> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_PRI_BITS_MASK) >>
>> >> + ICH_VTR_PRI_BITS_SHIFT) + 1;
>> >> +
>> >> /* Get the show on the road... */
>> >> vgic_v3->vgic_hcr = ICH_HCR_EN;
>> >> }
>> >> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>> >> */
>> >> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
>> >> kvm_vgic_global_state.can_emulate_gicv2 = false;
>> >> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>> >>
>> >> if (!info->vcpu.start) {
>> >> kvm_info("GICv3: no GICV resource entry\n");
>> >> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> >> index c461f6b..0e632d0 100644
>> >> --- a/virt/kvm/arm/vgic/vgic.h
>> >> +++ b/virt/kvm/arm/vgic/vgic.h
>> >> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> int offset, u32 *val);
>> >> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> int offset, u32 *val);
>> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> + u64 id, u64 *val);
>> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg);
>> >> #else
>> >> static inline int vgic_register_its_iodevs(struct kvm *kvm)
>> >> {
>> >> --
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-18 16:58 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-18 16:58 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
>> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>
>> >> VGICv3 CPU interface registers are accessed using
>> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> is used to identify the cpu for registers access.
>> >>
>> >> The version of VGIC v3 specification is define here
>> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>
>> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> ---
>> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> arch/arm64/kvm/Makefile | 1 +
>> >> include/kvm/arm_vgic.h | 9 +
>> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> 8 files changed, 395 insertions(+)
>> >>
>> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> index 56dc08d..91c7137 100644
>> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> +
>> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >>
>> >> /* Device Control API on vcpu fd */
>> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> index d50a82a..1a14e29 100644
>> >> --- a/arch/arm64/kvm/Makefile
>> >> +++ b/arch/arm64/kvm/Makefile
>> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >
>> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> > mean that either it is clearly only supported on AArch64 systems or it's
>> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> > on AArch32.
>>
>> It supports both AArch64 and AArch64 in handling of system registers
>> save/restore.
>> All system registers that we save/restore are 32-bit for both aarch64
>> and aarch32.
>> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> are same. However the codes sent by qemu is matched and register
>> are handled properly irrespective of AArch32 or AArch64.
>>
>> I don't have platform which support AArch32 guests to verify.
>
> Actually this is not about the guest, it's about an ARMv8 AArch32 host
> that has a GICv3.
>
> I just tried to do a v7 compile with your patches, and it results in an
> epic failure, so there's something for you to look at.
>
>> >
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> >> index 002f092..730a18a 100644
>> >> --- a/include/kvm/arm_vgic.h
>> >> +++ b/include/kvm/arm_vgic.h
>> >> @@ -71,6 +71,9 @@ struct vgic_global {
>> >>
>> >> /* GIC system register CPU interface */
>> >> struct static_key_false gicv3_cpuif;
>> >> +
>> >> + /* Cache ICH_VTR_EL2 reg value */
>> >> + u32 ich_vtr_el2;
>> >> };
>> >>
>> >> extern struct vgic_global kvm_vgic_global_state;
>> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> >> u64 pendbaser;
>> >>
>> >> bool lpis_enabled;
>> >> +
>> >> + /* Cache guest priority bits */
>> >> + u32 num_pri_bits;
>> >> +
>> >> + /* Cache guest interrupt ID bits */
>> >> + u32 num_id_bits;
>> >> };
>> >>
>> >> extern struct static_key_false vgic_v2_cpuif_trap;
>> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> index 6c7d30c..da532d1 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> >> if (!is_write)
>> >> *reg = tmp32;
>> >> break;
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 regid;
>> >> +
>> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> >> + regid, reg);
>> >> + break;
>> >> + }
>> >> default:
>> >> ret = -EINVAL;
>> >> break;
>> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> >> reg = tmp32;
>> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> + u64 reg;
>> >> +
>> >> + if (get_user(reg, uaddr))
>> >> + return -EFAULT;
>> >> +
>> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> + }
>> >> }
>> >> return -ENXIO;
>> >> }
>> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> >> tmp32 = reg;
>> >> return put_user(tmp32, uaddr);
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> + u64 reg;
>> >> +
>> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> >> + if (ret)
>> >> + return ret;
>> >> + return put_user(reg, uaddr);
>> >> + }
>> >> }
>> >>
>> >> return -ENXIO;
>> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> >> break;
>> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> >> return vgic_v3_has_attr_regs(dev, attr);
>> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> >> return 0;
>> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> index b35fb83..519b919 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> @@ -23,6 +23,7 @@
>> >>
>> >> #include "vgic.h"
>> >> #include "vgic-mmio.h"
>> >> +#include "sys_regs.h"
>> >>
>> >> /* extract @num bytes at @offset bytes offset in data */
>> >> unsigned long extract_bytes(u64 data, unsigned int offset,
>> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> >> break;
>> >> }
>> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> + u64 reg, id;
>> >> + unsigned long vgic_mpidr, mpidr_reg;
>> >> + struct kvm_vcpu *vcpu;
>> >> +
>> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> >> +
>> >> + /* Convert plain mpidr value to MPIDR reg format */
>> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> >> +
>> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> >> + if (!vcpu)
>> >> + return -EINVAL;
>> >> +
>> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> >> + }
>> >> default:
>> >> return -ENXIO;
>> >> }
>> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> new file mode 100644
>> >> index 0000000..69d8597
>> >> --- /dev/null
>> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >
>> > Shouldn't we have a GPL header here?
>> >
>> >> @@ -0,0 +1,324 @@
>> >> +#include <linux/irqchip/arm-gic-v3.h>
>> >> +#include <linux/kvm.h>
>> >> +#include <linux/kvm_host.h>
>> >> +#include <kvm/iodev.h>
>> >> +#include <kvm/arm_vgic.h>
>> >> +#include <asm/kvm_emulate.h>
>> >> +#include <asm/kvm_arm.h>
>> >> +#include <asm/kvm_mmu.h>
>> >> +
>> >> +#include "vgic.h"
>> >> +#include "vgic-mmio.h"
>> >> +#include "sys_regs.h"
>> >> +
>> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> + struct vgic_vmcr vmcr;
>> >> + u64 val;
>> >> + u32 num_pri_bits, num_id_bits;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + val = p->regval;
>> >> +
>> >> + /*
>> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> >> + * guest programmed ID and PRI bits
>> >> + */
>> >
>> > I would suggest rewording this comment:
>> > Disallow restoring VM state not supported by this hardware.
>> >
>> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> >> + return false;
>> >> +
>> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>> >
>> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
>> > understand which effect this is intended to have?
>> >
>> > Sure, it may limit what you do with other registers later, but since
>> > there's no ordering requirement that the ctlr be restored first, I'm not
>> > sure it makes sense.
>> >
>> > Also, since this field is RO in the ICH_VTR, we'll have a strange
>> > situation during runtime after a GICv3 restore where the
>> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
>> > which is never the case if you didn't do a save/restore.
>>
>> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
>> than HW supported
>> value.
>>
>
> So answer my question: What is the intended effect of writing this
> value? Is it just so that if you migrate this platform back again, then
> you're checking compatibility with what the guest would potentially do,
Yes and also to limit the valid aprn registers access as you said above.
But that has ordering restriction. Which I think we should follow.
> or should you maintain the num_pri_bits limitation during runtime
> somehow?
Once after checking compatibility, at runtime it is not updated
and this value is not used at all in VGIC further
>
>> >
>> > Finally, should we somehow ensure that this field is set to the same
>> > value across VCPUs or is that not an architectural requirement?
>> >
>> Yes it is nice to have it same across VCPUs. But should be ok as
>> we are ensuring value is not greater than HW supported value.
>
> Does the architecture allow having a different number of priority bits
> supported across CPUs? If not, you shouldn't allow a VM programming
> things that way either.
AFAIK, architecturally it is not mentioned any where in the spec that priority
bits should be same across CPUs.
>
>> There is no single point of place where we can make such a check
>>
>> >> +
>> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
>> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
>> >> + return false;
>> >> +
>> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
>> >
>> > same questions
>> >
>> >> +
>> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
>> >> + ICH_VMCR_EOIM_SHIFT;
>> >
>> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
>> > format or in the VMCR format? I would assume the former, since
>> > otherwise I don't get the point with this indirection, and for GICv2
>> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
>> > into VMCR values.
>> >
>> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
>> > ring.
>>
>> I will check and fix it.
>> >
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >
>> > Should we check compatibility between the source and destination for the
>> > SEIS and A3V support here?
>>
>> Can be checked. But I feel A3V check makes more sense than checking for
>> SEIS.
>>
>
> Please argue the *why* for whatever you end up doing with respect to
> both bits in the commit message of your next patch revision.
>
>> >
>> >> + } else {
>> >> + val = 0;
>> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
>> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> >> + val |= vgic_v3_cpu->num_id_bits <<
>> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
>> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
>> >> + ICC_CTLR_EL1_SEIS_SHIFT;
>> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
>> >> + ICC_CTLR_EL1_A3V_SHIFT;
>> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>> >
>> > again, these last two look weird to me.
>> >
>> >> +
>> >> + p->regval = val;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> >> + ICC_BPR0_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> >> + ICC_BPR0_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + if (!p->is_write)
>> >> + p->regval = 0;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
>> >> + if (p->is_write) {
>> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> >> + ICC_BPR1_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> >> + ICC_BPR1_EL1_MASK;
>> >> + }
>> >> + } else {
>> >> + if (!p->is_write)
>> >> + p->regval = min((vmcr.bpr + 1), 7U);
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> >> + ICC_IGRPEN0_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> >> + ICC_IGRPEN0_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_vmcr vmcr;
>> >> +
>> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> + if (p->is_write) {
>> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> >> + ICC_IGRPEN1_EL1_SHIFT;
>> >> + vgic_set_vmcr(vcpu, &vmcr);
>> >> + } else {
>> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> >> + ICC_IGRPEN1_EL1_MASK;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> >> + struct sys_reg_params *p, u8 apr, u8 idx)
>> >> +{
>> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> >> + uint32_t *ap_reg;
>> >> +
>> >> + if (apr)
>> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
>> >> + else
>> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
>> >> +
>> >> + if (p->is_write)
>> >> + *ap_reg = p->regval;
>> >> + else
>> >> + p->regval = *ap_reg;
>> >> +}
>> >> +
>> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r, u8 apr)
>> >> +{
>> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> + u8 idx = r->Op2 & 3;
>> >> +
>> >> + switch (vgic_v3_cpu->num_pri_bits) {
>> >> + case 7:
>> >> + if (idx > 3)
>> >> + goto err;
>> >
>> > idx cannot be higher than three given the mask above, right?
>> >
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + break;
>> >> + case 6:
>> >> + if (idx > 1)
>> >> + goto err;
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + break;
>> >> + default:
>> >> + if (idx > 0)
>> >> + goto err;
>> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
>> >> + }
>> >
>> > what's the rationale behind ignoring the case where userspace is using
>> > unsupported priorities? Is it that this will be checked during
>> > save/restore of the ctlr?
>> >
>> > This sort of thing just looks like the case that's impossible to debug,
>> > because userspace could be scratching its head trying to understand why
>> > the value it wrote isn't recorded anywhere...
>> >
>> > If there's a good rationale for doing it this way, then could we have a
>> > comment to that effect?
>>
>> Accessing umplemented priority registers raised UNDEF exception.
>> So userspace accesing should be ignored instead of recording unsupported
>> values.
>>
>
> That's not what I asked.
>
> I asked why it's silently ignored as opposed to raising an error visible
> to user space?
>
>> >
>> >> +
>> >> + return;
>> >> +err:
>> >> + if (!p->is_write)
>> >> + p->regval = 0;
>> >> +}
>> >> +
>> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + access_gic_aprn(vcpu, p, r, 0);
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + access_gic_aprn(vcpu, p, r, 1);
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> + const struct sys_reg_desc *r)
>> >> +{
>> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> >> +
>> >> + /* Validate SRE bit */
>> >> + if (p->is_write) {
>> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
>> >> + return false;
>> >> + } else {
>> >> + p->regval = vgicv3->vgic_sre;
>> >> + }
>> >> +
>> >> + return true;
>> >> +}
>> >> +
>> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
>> >> + /* ICC_PMR_EL1 */
>> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
>> >> + /* ICC_BPR0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
>> >> + /* ICC_AP0R0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
>> >> + /* ICC_AP0R1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
>> >> + /* ICC_AP0R2_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
>> >> + /* ICC_AP0R3_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
>> >> + /* ICC_AP1R0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
>> >> + /* ICC_AP1R1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
>> >> + /* ICC_AP1R2_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
>> >> + /* ICC_AP1R3_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
>> >> + /* ICC_BPR1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
>> >> + /* ICC_CTLR_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
>> >> + /* ICC_SRE_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
>> >> + /* ICC_IGRPEN0_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
>> >> + /* ICC_GRPEN1_EL1 */
>> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
>> >> +};
>> >> +
>> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg)
>> >> +{
>> >> + struct sys_reg_params params;
>> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> >> +
>> >> + params.regval = *reg;
>> >> + params.is_write = is_write;
>> >> + params.is_aarch32 = false;
>> >> + params.is_32bit = false;
>> >> +
>> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
>> >> + return 0;
>> >> +
>> >> + return -ENXIO;
>> >> +}
>> >> +
>> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg)
>> >> +{
>> >> + struct sys_reg_params params;
>> >> + const struct sys_reg_desc *r;
>> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
>> >> +
>> >> + if (is_write)
>> >> + params.regval = *reg;
>> >> + params.is_write = is_write;
>> >> + params.is_aarch32 = false;
>> >> + params.is_32bit = false;
>> >> +
>> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
>> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
>> >> + if (!r)
>> >> + return -ENXIO;
>> >> +
>> >> + if (!r->access(vcpu, ¶ms, r))
>> >> + return -EINVAL;
>> >
>> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
>> > how it can be used or allocate a new error code?
>> How abt EACCES error code?
>>
>
> That would mean permission denied, which is a bit weird.
Yes I agree, but you can suggest.
>
>> >
>> >> +
>> >> + if (!is_write)
>> >> + *reg = params.regval;
>> >> +
>> >> + return 0;
>> >> +}
>> >> diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
>> >> index 967c295..1139971 100644
>> >> --- a/virt/kvm/arm/vgic/vgic-v3.c
>> >> +++ b/virt/kvm/arm/vgic/vgic-v3.c
>> >> @@ -228,6 +228,13 @@ void vgic_v3_enable(struct kvm_vcpu *vcpu)
>> >> vgic_v3->vgic_sre = 0;
>> >> }
>> >>
>> >> + vcpu->arch.vgic_cpu.num_id_bits = (kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_ID_BITS_MASK) >>
>> >> + ICH_VTR_ID_BITS_SHIFT;
>> >> + vcpu->arch.vgic_cpu.num_pri_bits = ((kvm_vgic_global_state.ich_vtr_el2 &
>> >> + ICH_VTR_PRI_BITS_MASK) >>
>> >> + ICH_VTR_PRI_BITS_SHIFT) + 1;
>> >> +
>> >> /* Get the show on the road... */
>> >> vgic_v3->vgic_hcr = ICH_HCR_EN;
>> >> }
>> >> @@ -328,6 +335,7 @@ int vgic_v3_probe(const struct gic_kvm_info *info)
>> >> */
>> >> kvm_vgic_global_state.nr_lr = (ich_vtr_el2 & 0xf) + 1;
>> >> kvm_vgic_global_state.can_emulate_gicv2 = false;
>> >> + kvm_vgic_global_state.ich_vtr_el2 = ich_vtr_el2;
>> >>
>> >> if (!info->vcpu.start) {
>> >> kvm_info("GICv3: no GICV resource entry\n");
>> >> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
>> >> index c461f6b..0e632d0 100644
>> >> --- a/virt/kvm/arm/vgic/vgic.h
>> >> +++ b/virt/kvm/arm/vgic/vgic.h
>> >> @@ -126,6 +126,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> int offset, u32 *val);
>> >> int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> int offset, u32 *val);
>> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>> >> + u64 id, u64 *val);
>> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
>> >> + u64 *reg);
>> >> #else
>> >> static inline int vgic_register_its_iodevs(struct kvm *kvm)
>> >> {
>> >> --
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-17 16:09 ` Christoffer Dall
@ 2016-11-18 18:48 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-18 18:48 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
>> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>
>> >> VGICv3 CPU interface registers are accessed using
>> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> is used to identify the cpu for registers access.
>> >>
>> >> The version of VGIC v3 specification is define here
>> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>
>> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> ---
>> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> arch/arm64/kvm/Makefile | 1 +
>> >> include/kvm/arm_vgic.h | 9 +
>> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> 8 files changed, 395 insertions(+)
>> >>
>> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> index 56dc08d..91c7137 100644
>> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> +
>> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >>
>> >> /* Device Control API on vcpu fd */
>> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> index d50a82a..1a14e29 100644
>> >> --- a/arch/arm64/kvm/Makefile
>> >> +++ b/arch/arm64/kvm/Makefile
>> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >
>> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> > mean that either it is clearly only supported on AArch64 systems or it's
>> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> > on AArch32.
>>
>> It supports both AArch64 and AArch64 in handling of system registers
>> save/restore.
>> All system registers that we save/restore are 32-bit for both aarch64
>> and aarch32.
>> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> are same. However the codes sent by qemu is matched and register
>> are handled properly irrespective of AArch32 or AArch64.
>>
>> I don't have platform which support AArch32 guests to verify.
>
> Actually this is not about the guest, it's about an ARMv8 AArch32 host
> that has a GICv3.
>
> I just tried to do a v7 compile with your patches, and it results in an
> epic failure, so there's something for you to look at.
>
Could you please share you config file?. I tried with multi_v7 defconfig with
CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-18 18:48 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-18 18:48 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
>> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>
>> >> VGICv3 CPU interface registers are accessed using
>> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> is used to identify the cpu for registers access.
>> >>
>> >> The version of VGIC v3 specification is define here
>> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>
>> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> ---
>> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> arch/arm64/kvm/Makefile | 1 +
>> >> include/kvm/arm_vgic.h | 9 +
>> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> 8 files changed, 395 insertions(+)
>> >>
>> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> index 56dc08d..91c7137 100644
>> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> +
>> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >>
>> >> /* Device Control API on vcpu fd */
>> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> index d50a82a..1a14e29 100644
>> >> --- a/arch/arm64/kvm/Makefile
>> >> +++ b/arch/arm64/kvm/Makefile
>> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >
>> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> > mean that either it is clearly only supported on AArch64 systems or it's
>> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> > on AArch32.
>>
>> It supports both AArch64 and AArch64 in handling of system registers
>> save/restore.
>> All system registers that we save/restore are 32-bit for both aarch64
>> and aarch32.
>> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> are same. However the codes sent by qemu is matched and register
>> are handled properly irrespective of AArch32 or AArch64.
>>
>> I don't have platform which support AArch32 guests to verify.
>
> Actually this is not about the guest, it's about an ARMv8 AArch32 host
> that has a GICv3.
>
> I just tried to do a v7 compile with your patches, and it results in an
> epic failure, so there's something for you to look at.
>
Could you please share you config file?. I tried with multi_v7 defconfig with
CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-18 18:48 ` Vijay Kilari
@ 2016-11-20 13:20 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-20 13:20 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >>
> >> >> VGICv3 CPU interface registers are accessed using
> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> is used to identify the cpu for registers access.
> >> >>
> >> >> The version of VGIC v3 specification is define here
> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >>
> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> ---
> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> 8 files changed, 395 insertions(+)
> >> >>
> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> index 56dc08d..91c7137 100644
> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> +
> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >>
> >> >> /* Device Control API on vcpu fd */
> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> index d50a82a..1a14e29 100644
> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >
> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> > on AArch32.
> >>
> >> It supports both AArch64 and AArch64 in handling of system registers
> >> save/restore.
> >> All system registers that we save/restore are 32-bit for both aarch64
> >> and aarch32.
> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> are same. However the codes sent by qemu is matched and register
> >> are handled properly irrespective of AArch32 or AArch64.
> >>
> >> I don't have platform which support AArch32 guests to verify.
> >
> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> > that has a GICv3.
> >
> > I just tried to do a v7 compile with your patches, and it results in an
> > epic failure, so there's something for you to look at.
> >
>
> Could you please share you config file?. I tried with multi_v7 defconfig with
> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
I think this has to do with which branch you apply your patches to.
When applied to kvmarm/next, it fails.
Here's the integration I did:
https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
Here's the config:
https://transfer.sh/xkAxp/.config
Here's the compile output:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:26:22: fatal error: sys_regs.h: No such file or directory
#include "sys_regs.h"
^
compilation terminated.
make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_parse_attr’:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: error: ‘KVM_DEV_ARM_VGIC_V3_MPIDR_MASK’ undeclared (first use in this function)
vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: note: each undeclared identifier is reported only once for each function it appears in
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:439:9: error: ‘KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT’ undeclared (first use in this function)
KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:441:2: error: implicit declaration of function ‘MPIDR_LEVEL_SHIFT’ [-Werror=implicit-function-declaration]
mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_attr_regs_access’:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:497:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:505:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:508:25: error: ‘KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK’ undeclared (first use in this function)
regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:513:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:516:24: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK’ undeclared (first use in this function)
info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:517:4: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT’ undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:518:15: error: ‘VGIC_LEVEL_INFO_LINE_LEVEL’ undeclared (first use in this function)
if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:522:5: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK’ undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_set_attr’:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:554:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:565:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:574:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_get_attr’:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:600:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:611:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:620:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_has_attr’:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:647:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:648:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:652:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:653:22: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK’ undeclared (first use in this function)
if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:654:9: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT’ undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:655:9: error: ‘VGIC_LEVEL_INFO_LINE_LEVEL’ undeclared (first use in this function)
VGIC_LEVEL_INFO_LINE_LEVEL)
^
cc1: some warnings being treated as errors
make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.o] Error 1
make[1]: *** [arch/arm/kvm] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [sub-make] Error 2
Thanks,
-Christoffer
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-20 13:20 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-20 13:20 UTC (permalink / raw)
To: linux-arm-kernel
On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >>
> >> >> VGICv3 CPU interface registers are accessed using
> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> is used to identify the cpu for registers access.
> >> >>
> >> >> The version of VGIC v3 specification is define here
> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >>
> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> ---
> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> 8 files changed, 395 insertions(+)
> >> >>
> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> index 56dc08d..91c7137 100644
> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> +
> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >>
> >> >> /* Device Control API on vcpu fd */
> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> index d50a82a..1a14e29 100644
> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >
> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> > on AArch32.
> >>
> >> It supports both AArch64 and AArch64 in handling of system registers
> >> save/restore.
> >> All system registers that we save/restore are 32-bit for both aarch64
> >> and aarch32.
> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> are same. However the codes sent by qemu is matched and register
> >> are handled properly irrespective of AArch32 or AArch64.
> >>
> >> I don't have platform which support AArch32 guests to verify.
> >
> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> > that has a GICv3.
> >
> > I just tried to do a v7 compile with your patches, and it results in an
> > epic failure, so there's something for you to look at.
> >
>
> Could you please share you config file?. I tried with multi_v7 defconfig with
> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
I think this has to do with which branch you apply your patches to.
When applied to kvmarm/next, it fails.
Here's the integration I did:
https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
Here's the config:
https://transfer.sh/xkAxp/.config
Here's the compile output:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:26:22: fatal error: sys_regs.h: No such file or directory
#include "sys_regs.h"
^
compilation terminated.
make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_parse_attr?:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: error: ?KVM_DEV_ARM_VGIC_V3_MPIDR_MASK? undeclared (first use in this function)
vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: note: each undeclared identifier is reported only once for each function it appears in
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:439:9: error: ?KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT? undeclared (first use in this function)
KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:441:2: error: implicit declaration of function ?MPIDR_LEVEL_SHIFT? [-Werror=implicit-function-declaration]
mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_attr_regs_access?:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:497:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:505:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:508:25: error: ?KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK? undeclared (first use in this function)
regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:513:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:516:24: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK? undeclared (first use in this function)
info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:517:4: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT? undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:518:15: error: ?VGIC_LEVEL_INFO_LINE_LEVEL? undeclared (first use in this function)
if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:522:5: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK? undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_set_attr?:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:554:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:565:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:574:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_get_attr?:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:600:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:611:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:620:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_has_attr?:
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:647:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:648:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:652:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:653:22: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK? undeclared (first use in this function)
if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:654:9: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT? undeclared (first use in this function)
KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
^
/home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:655:9: error: ?VGIC_LEVEL_INFO_LINE_LEVEL? undeclared (first use in this function)
VGIC_LEVEL_INFO_LINE_LEVEL)
^
cc1: some warnings being treated as errors
make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.o] Error 1
make[1]: *** [arch/arm/kvm] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [sub-make] Error 2
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-18 16:58 ` Vijay Kilari
@ 2016-11-21 10:19 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 10:19 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >>
> >> >> VGICv3 CPU interface registers are accessed using
> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> is used to identify the cpu for registers access.
> >> >>
> >> >> The version of VGIC v3 specification is define here
> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >>
> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> ---
> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> 8 files changed, 395 insertions(+)
> >> >>
> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> index 56dc08d..91c7137 100644
> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> +
> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >>
> >> >> /* Device Control API on vcpu fd */
> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> index d50a82a..1a14e29 100644
> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >
> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> > on AArch32.
> >>
> >> It supports both AArch64 and AArch64 in handling of system registers
> >> save/restore.
> >> All system registers that we save/restore are 32-bit for both aarch64
> >> and aarch32.
> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> are same. However the codes sent by qemu is matched and register
> >> are handled properly irrespective of AArch32 or AArch64.
> >>
> >> I don't have platform which support AArch32 guests to verify.
> >
> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> > that has a GICv3.
> >
> > I just tried to do a v7 compile with your patches, and it results in an
> > epic failure, so there's something for you to look at.
> >
> >> >
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> >> index 002f092..730a18a 100644
> >> >> --- a/include/kvm/arm_vgic.h
> >> >> +++ b/include/kvm/arm_vgic.h
> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >> >>
> >> >> /* GIC system register CPU interface */
> >> >> struct static_key_false gicv3_cpuif;
> >> >> +
> >> >> + /* Cache ICH_VTR_EL2 reg value */
> >> >> + u32 ich_vtr_el2;
> >> >> };
> >> >>
> >> >> extern struct vgic_global kvm_vgic_global_state;
> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> >> u64 pendbaser;
> >> >>
> >> >> bool lpis_enabled;
> >> >> +
> >> >> + /* Cache guest priority bits */
> >> >> + u32 num_pri_bits;
> >> >> +
> >> >> + /* Cache guest interrupt ID bits */
> >> >> + u32 num_id_bits;
> >> >> };
> >> >>
> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> index 6c7d30c..da532d1 100644
> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> >> if (!is_write)
> >> >> *reg = tmp32;
> >> >> break;
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 regid;
> >> >> +
> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> >> + regid, reg);
> >> >> + break;
> >> >> + }
> >> >> default:
> >> >> ret = -EINVAL;
> >> >> break;
> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> >> reg = tmp32;
> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> + u64 reg;
> >> >> +
> >> >> + if (get_user(reg, uaddr))
> >> >> + return -EFAULT;
> >> >> +
> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> + }
> >> >> }
> >> >> return -ENXIO;
> >> >> }
> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> >> tmp32 = reg;
> >> >> return put_user(tmp32, uaddr);
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> + u64 reg;
> >> >> +
> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> >> + if (ret)
> >> >> + return ret;
> >> >> + return put_user(reg, uaddr);
> >> >> + }
> >> >> }
> >> >>
> >> >> return -ENXIO;
> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> >> break;
> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> >> return vgic_v3_has_attr_regs(dev, attr);
> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> >> return 0;
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> index b35fb83..519b919 100644
> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> @@ -23,6 +23,7 @@
> >> >>
> >> >> #include "vgic.h"
> >> >> #include "vgic-mmio.h"
> >> >> +#include "sys_regs.h"
> >> >>
> >> >> /* extract @num bytes at @offset bytes offset in data */
> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> >> break;
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 reg, id;
> >> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> >> + struct kvm_vcpu *vcpu;
> >> >> +
> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> >> +
> >> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> >> +
> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> >> + if (!vcpu)
> >> >> + return -EINVAL;
> >> >> +
> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> >> + }
> >> >> default:
> >> >> return -ENXIO;
> >> >> }
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> new file mode 100644
> >> >> index 0000000..69d8597
> >> >> --- /dev/null
> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >
> >> > Shouldn't we have a GPL header here?
> >> >
> >> >> @@ -0,0 +1,324 @@
> >> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> >> +#include <linux/kvm.h>
> >> >> +#include <linux/kvm_host.h>
> >> >> +#include <kvm/iodev.h>
> >> >> +#include <kvm/arm_vgic.h>
> >> >> +#include <asm/kvm_emulate.h>
> >> >> +#include <asm/kvm_arm.h>
> >> >> +#include <asm/kvm_mmu.h>
> >> >> +
> >> >> +#include "vgic.h"
> >> >> +#include "vgic-mmio.h"
> >> >> +#include "sys_regs.h"
> >> >> +
> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> + struct vgic_vmcr vmcr;
> >> >> + u64 val;
> >> >> + u32 num_pri_bits, num_id_bits;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + val = p->regval;
> >> >> +
> >> >> + /*
> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> >> + * guest programmed ID and PRI bits
> >> >> + */
> >> >
> >> > I would suggest rewording this comment:
> >> > Disallow restoring VM state not supported by this hardware.
> >> >
> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> >> + return false;
> >> >> +
> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >> >
> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> >> > understand which effect this is intended to have?
> >> >
> >> > Sure, it may limit what you do with other registers later, but since
> >> > there's no ordering requirement that the ctlr be restored first, I'm not
> >> > sure it makes sense.
> >> >
> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> >> > situation during runtime after a GICv3 restore where the
> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> >> > which is never the case if you didn't do a save/restore.
> >>
> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> >> than HW supported
> >> value.
> >>
> >
> > So answer my question: What is the intended effect of writing this
> > value? Is it just so that if you migrate this platform back again, then
> > you're checking compatibility with what the guest would potentially do,
>
> Yes
Then add a comment explaining that
> and also to limit the valid aprn registers access as you said above.
> But that has ordering restriction. Which I think we should follow.
>
I'm sorry, now I'm confused. Is there an ordering requirement in the
API, or how should we follow this?
> > or should you maintain the num_pri_bits limitation during runtime
> > somehow?
> Once after checking compatibility, at runtime it is not updated
> and this value is not used at all in VGIC further
> >
> >> >
> >> > Finally, should we somehow ensure that this field is set to the same
> >> > value across VCPUs or is that not an architectural requirement?
> >> >
> >> Yes it is nice to have it same across VCPUs. But should be ok as
> >> we are ensuring value is not greater than HW supported value.
> >
> > Does the architecture allow having a different number of priority bits
> > supported across CPUs? If not, you shouldn't allow a VM programming
> > things that way either.
>
> AFAIK, architecturally it is not mentioned any where in the spec that priority
> bits should be same across CPUs.
ok
> >
> >> There is no single point of place where we can make such a check
> >>
> >> >> +
> >> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> >> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> >> >> + return false;
> >> >> +
> >> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
> >> >
> >> > same questions
> >> >
> >> >> +
> >> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> >> >> + ICH_VMCR_EOIM_SHIFT;
> >> >
> >> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> >> > format or in the VMCR format? I would assume the former, since
> >> > otherwise I don't get the point with this indirection, and for GICv2
> >> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> >> > into VMCR values.
> >> >
> >> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> >> > ring.
> >>
> >> I will check and fix it.
> >> >
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >
> >> > Should we check compatibility between the source and destination for the
> >> > SEIS and A3V support here?
> >>
> >> Can be checked. But I feel A3V check makes more sense than checking for
> >> SEIS.
> >>
> >
> > Please argue the *why* for whatever you end up doing with respect to
> > both bits in the commit message of your next patch revision.
> >
> >> >
> >> >> + } else {
> >> >> + val = 0;
> >> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >> >> + val |= vgic_v3_cpu->num_id_bits <<
> >> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> >> >> + ICC_CTLR_EL1_SEIS_SHIFT;
> >> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> >> >> + ICC_CTLR_EL1_A3V_SHIFT;
> >> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >> >
> >> > again, these last two look weird to me.
> >> >
> >> >> +
> >> >> + p->regval = val;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >> >> + ICC_BPR0_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >> >> + ICC_BPR0_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + if (!p->is_write)
> >> >> + p->regval = 0;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> >> >> + if (p->is_write) {
> >> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >> >> + ICC_BPR1_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >> >> + ICC_BPR1_EL1_MASK;
> >> >> + }
> >> >> + } else {
> >> >> + if (!p->is_write)
> >> >> + p->regval = min((vmcr.bpr + 1), 7U);
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >> >> + ICC_IGRPEN0_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >> >> + ICC_IGRPEN0_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >> >> + ICC_IGRPEN1_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >> >> + ICC_IGRPEN1_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> >> >> + struct sys_reg_params *p, u8 apr, u8 idx)
> >> >> +{
> >> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> >> + uint32_t *ap_reg;
> >> >> +
> >> >> + if (apr)
> >> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
> >> >> + else
> >> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
> >> >> +
> >> >> + if (p->is_write)
> >> >> + *ap_reg = p->regval;
> >> >> + else
> >> >> + p->regval = *ap_reg;
> >> >> +}
> >> >> +
> >> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r, u8 apr)
> >> >> +{
> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> + u8 idx = r->Op2 & 3;
> >> >> +
> >> >> + switch (vgic_v3_cpu->num_pri_bits) {
> >> >> + case 7:
> >> >> + if (idx > 3)
> >> >> + goto err;
> >> >
> >> > idx cannot be higher than three given the mask above, right?
> >> >
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + break;
> >> >> + case 6:
> >> >> + if (idx > 1)
> >> >> + goto err;
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + break;
> >> >> + default:
> >> >> + if (idx > 0)
> >> >> + goto err;
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + }
> >> >
> >> > what's the rationale behind ignoring the case where userspace is using
> >> > unsupported priorities? Is it that this will be checked during
> >> > save/restore of the ctlr?
> >> >
> >> > This sort of thing just looks like the case that's impossible to debug,
> >> > because userspace could be scratching its head trying to understand why
> >> > the value it wrote isn't recorded anywhere...
> >> >
> >> > If there's a good rationale for doing it this way, then could we have a
> >> > comment to that effect?
> >>
> >> Accessing umplemented priority registers raised UNDEF exception.
> >> So userspace accesing should be ignored instead of recording unsupported
> >> values.
> >>
> >
> > That's not what I asked.
> >
> > I asked why it's silently ignored as opposed to raising an error visible
> > to user space?
> >
> >> >
> >> >> +
> >> >> + return;
> >> >> +err:
> >> >> + if (!p->is_write)
> >> >> + p->regval = 0;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + access_gic_aprn(vcpu, p, r, 0);
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + access_gic_aprn(vcpu, p, r, 1);
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> >> +
> >> >> + /* Validate SRE bit */
> >> >> + if (p->is_write) {
> >> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
> >> >> + return false;
> >> >> + } else {
> >> >> + p->regval = vgicv3->vgic_sre;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> >> >> + /* ICC_PMR_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> >> >> + /* ICC_BPR0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> >> >> + /* ICC_AP0R0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> >> >> + /* ICC_AP0R1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> >> >> + /* ICC_AP0R2_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> >> >> + /* ICC_AP0R3_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> >> >> + /* ICC_AP1R0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> >> >> + /* ICC_AP1R1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> >> >> + /* ICC_AP1R2_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> >> >> + /* ICC_AP1R3_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> >> >> + /* ICC_BPR1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> >> >> + /* ICC_CTLR_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> >> >> + /* ICC_SRE_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> >> >> + /* ICC_IGRPEN0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> >> >> + /* ICC_GRPEN1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> >> >> +};
> >> >> +
> >> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> >> + u64 *reg)
> >> >> +{
> >> >> + struct sys_reg_params params;
> >> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> >> +
> >> >> + params.regval = *reg;
> >> >> + params.is_write = is_write;
> >> >> + params.is_aarch32 = false;
> >> >> + params.is_32bit = false;
> >> >> +
> >> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> >> >> + return 0;
> >> >> +
> >> >> + return -ENXIO;
> >> >> +}
> >> >> +
> >> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> >> + u64 *reg)
> >> >> +{
> >> >> + struct sys_reg_params params;
> >> >> + const struct sys_reg_desc *r;
> >> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> >> +
> >> >> + if (is_write)
> >> >> + params.regval = *reg;
> >> >> + params.is_write = is_write;
> >> >> + params.is_aarch32 = false;
> >> >> + params.is_32bit = false;
> >> >> +
> >> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> >> >> + if (!r)
> >> >> + return -ENXIO;
> >> >> +
> >> >> + if (!r->access(vcpu, ¶ms, r))
> >> >> + return -EINVAL;
> >> >
> >> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
> >> > how it can be used or allocate a new error code?
> >> How abt EACCES error code?
> >>
> >
> > That would mean permission denied, which is a bit weird.
> Yes I agree, but you can suggest.
You could expand the meaning in the API doc for gicv3 and use EINVAL, or
you could expand on what ENXIO means.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-21 10:19 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 10:19 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >>
> >> >> VGICv3 CPU interface registers are accessed using
> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> is used to identify the cpu for registers access.
> >> >>
> >> >> The version of VGIC v3 specification is define here
> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >>
> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> ---
> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> 8 files changed, 395 insertions(+)
> >> >>
> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> index 56dc08d..91c7137 100644
> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> +
> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >>
> >> >> /* Device Control API on vcpu fd */
> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> index d50a82a..1a14e29 100644
> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >
> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> > on AArch32.
> >>
> >> It supports both AArch64 and AArch64 in handling of system registers
> >> save/restore.
> >> All system registers that we save/restore are 32-bit for both aarch64
> >> and aarch32.
> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> are same. However the codes sent by qemu is matched and register
> >> are handled properly irrespective of AArch32 or AArch64.
> >>
> >> I don't have platform which support AArch32 guests to verify.
> >
> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> > that has a GICv3.
> >
> > I just tried to do a v7 compile with your patches, and it results in an
> > epic failure, so there's something for you to look at.
> >
> >> >
> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> >> index 002f092..730a18a 100644
> >> >> --- a/include/kvm/arm_vgic.h
> >> >> +++ b/include/kvm/arm_vgic.h
> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >> >>
> >> >> /* GIC system register CPU interface */
> >> >> struct static_key_false gicv3_cpuif;
> >> >> +
> >> >> + /* Cache ICH_VTR_EL2 reg value */
> >> >> + u32 ich_vtr_el2;
> >> >> };
> >> >>
> >> >> extern struct vgic_global kvm_vgic_global_state;
> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> >> u64 pendbaser;
> >> >>
> >> >> bool lpis_enabled;
> >> >> +
> >> >> + /* Cache guest priority bits */
> >> >> + u32 num_pri_bits;
> >> >> +
> >> >> + /* Cache guest interrupt ID bits */
> >> >> + u32 num_id_bits;
> >> >> };
> >> >>
> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> index 6c7d30c..da532d1 100644
> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> >> if (!is_write)
> >> >> *reg = tmp32;
> >> >> break;
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 regid;
> >> >> +
> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> >> + regid, reg);
> >> >> + break;
> >> >> + }
> >> >> default:
> >> >> ret = -EINVAL;
> >> >> break;
> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> >> reg = tmp32;
> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> + u64 reg;
> >> >> +
> >> >> + if (get_user(reg, uaddr))
> >> >> + return -EFAULT;
> >> >> +
> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> + }
> >> >> }
> >> >> return -ENXIO;
> >> >> }
> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> >> tmp32 = reg;
> >> >> return put_user(tmp32, uaddr);
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> + u64 reg;
> >> >> +
> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> >> + if (ret)
> >> >> + return ret;
> >> >> + return put_user(reg, uaddr);
> >> >> + }
> >> >> }
> >> >>
> >> >> return -ENXIO;
> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> >> break;
> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> >> return vgic_v3_has_attr_regs(dev, attr);
> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> >> return 0;
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> index b35fb83..519b919 100644
> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> @@ -23,6 +23,7 @@
> >> >>
> >> >> #include "vgic.h"
> >> >> #include "vgic-mmio.h"
> >> >> +#include "sys_regs.h"
> >> >>
> >> >> /* extract @num bytes at @offset bytes offset in data */
> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> >> break;
> >> >> }
> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> + u64 reg, id;
> >> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> >> + struct kvm_vcpu *vcpu;
> >> >> +
> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> >> +
> >> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> >> +
> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> >> + if (!vcpu)
> >> >> + return -EINVAL;
> >> >> +
> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> >> + }
> >> >> default:
> >> >> return -ENXIO;
> >> >> }
> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> new file mode 100644
> >> >> index 0000000..69d8597
> >> >> --- /dev/null
> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >
> >> > Shouldn't we have a GPL header here?
> >> >
> >> >> @@ -0,0 +1,324 @@
> >> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> >> +#include <linux/kvm.h>
> >> >> +#include <linux/kvm_host.h>
> >> >> +#include <kvm/iodev.h>
> >> >> +#include <kvm/arm_vgic.h>
> >> >> +#include <asm/kvm_emulate.h>
> >> >> +#include <asm/kvm_arm.h>
> >> >> +#include <asm/kvm_mmu.h>
> >> >> +
> >> >> +#include "vgic.h"
> >> >> +#include "vgic-mmio.h"
> >> >> +#include "sys_regs.h"
> >> >> +
> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> + struct vgic_vmcr vmcr;
> >> >> + u64 val;
> >> >> + u32 num_pri_bits, num_id_bits;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + val = p->regval;
> >> >> +
> >> >> + /*
> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> >> + * guest programmed ID and PRI bits
> >> >> + */
> >> >
> >> > I would suggest rewording this comment:
> >> > Disallow restoring VM state not supported by this hardware.
> >> >
> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> >> + return false;
> >> >> +
> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >> >
> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> >> > understand which effect this is intended to have?
> >> >
> >> > Sure, it may limit what you do with other registers later, but since
> >> > there's no ordering requirement that the ctlr be restored first, I'm not
> >> > sure it makes sense.
> >> >
> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> >> > situation during runtime after a GICv3 restore where the
> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> >> > which is never the case if you didn't do a save/restore.
> >>
> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> >> than HW supported
> >> value.
> >>
> >
> > So answer my question: What is the intended effect of writing this
> > value? Is it just so that if you migrate this platform back again, then
> > you're checking compatibility with what the guest would potentially do,
>
> Yes
Then add a comment explaining that
> and also to limit the valid aprn registers access as you said above.
> But that has ordering restriction. Which I think we should follow.
>
I'm sorry, now I'm confused. Is there an ordering requirement in the
API, or how should we follow this?
> > or should you maintain the num_pri_bits limitation during runtime
> > somehow?
> Once after checking compatibility, at runtime it is not updated
> and this value is not used at all in VGIC further
> >
> >> >
> >> > Finally, should we somehow ensure that this field is set to the same
> >> > value across VCPUs or is that not an architectural requirement?
> >> >
> >> Yes it is nice to have it same across VCPUs. But should be ok as
> >> we are ensuring value is not greater than HW supported value.
> >
> > Does the architecture allow having a different number of priority bits
> > supported across CPUs? If not, you shouldn't allow a VM programming
> > things that way either.
>
> AFAIK, architecturally it is not mentioned any where in the spec that priority
> bits should be same across CPUs.
ok
> >
> >> There is no single point of place where we can make such a check
> >>
> >> >> +
> >> >> + num_id_bits = (val & ICC_CTLR_EL1_ID_BITS_MASK) >>
> >> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> >> + if (num_id_bits > vgic_v3_cpu->num_id_bits)
> >> >> + return false;
> >> >> +
> >> >> + vgic_v3_cpu->num_id_bits = num_id_bits;
> >> >
> >> > same questions
> >> >
> >> >> +
> >> >> + vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >> >> + ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >> >> + vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >> >> + ICC_CTLR_EL1_EOImode_SHIFT) <<
> >> >> + ICH_VMCR_EOIM_SHIFT;
> >> >
> >> > I'm really confused here. Is the vmcr.ctlr field in the ICC_CTLR_EL1
> >> > format or in the VMCR format? I would assume the former, since
> >> > otherwise I don't get the point with this indirection, and for GICv2
> >> > vmcr.ctlr captures the GICC_CTLR value and git_set_vmcr transforms this
> >> > into VMCR values.
> >> >
> >> > Having a line that says "ctlr &= ~ICH_VMCR" should make some alarm bells
> >> > ring.
> >>
> >> I will check and fix it.
> >> >
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >
> >> > Should we check compatibility between the source and destination for the
> >> > SEIS and A3V support here?
> >>
> >> Can be checked. But I feel A3V check makes more sense than checking for
> >> SEIS.
> >>
> >
> > Please argue the *why* for whatever you end up doing with respect to
> > both bits in the commit message of your next patch revision.
> >
> >> >
> >> >> + } else {
> >> >> + val = 0;
> >> >> + val |= (vgic_v3_cpu->num_pri_bits - 1) <<
> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >> >> + val |= vgic_v3_cpu->num_id_bits <<
> >> >> + ICC_CTLR_EL1_ID_BITS_SHIFT;
> >> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> >> + ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT) <<
> >> >> + ICC_CTLR_EL1_SEIS_SHIFT;
> >> >> + val |= ((kvm_vgic_global_state.ich_vtr_el2 &
> >> >> + ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT) <<
> >> >> + ICC_CTLR_EL1_A3V_SHIFT;
> >> >> + val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >> >> + ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >> >> + val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >> >> + ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >> >
> >> > again, these last two look weird to me.
> >> >
> >> >> +
> >> >> + p->regval = val;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >> >> + ICC_BPR0_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >> >> + ICC_BPR0_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + if (!p->is_write)
> >> >> + p->regval = 0;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (!((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT)) {
> >> >> + if (p->is_write) {
> >> >> + vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >> >> + ICC_BPR1_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >> >> + ICC_BPR1_EL1_MASK;
> >> >> + }
> >> >> + } else {
> >> >> + if (!p->is_write)
> >> >> + p->regval = min((vmcr.bpr + 1), 7U);
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >> >> + ICC_IGRPEN0_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >> >> + ICC_IGRPEN0_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_vmcr vmcr;
> >> >> +
> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> + if (p->is_write) {
> >> >> + vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >> >> + ICC_IGRPEN1_EL1_SHIFT;
> >> >> + vgic_set_vmcr(vcpu, &vmcr);
> >> >> + } else {
> >> >> + p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >> >> + ICC_IGRPEN1_EL1_MASK;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> >> >> + struct sys_reg_params *p, u8 apr, u8 idx)
> >> >> +{
> >> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> >> + uint32_t *ap_reg;
> >> >> +
> >> >> + if (apr)
> >> >> + ap_reg = &vgicv3->vgic_ap1r[idx];
> >> >> + else
> >> >> + ap_reg = &vgicv3->vgic_ap0r[idx];
> >> >> +
> >> >> + if (p->is_write)
> >> >> + *ap_reg = p->regval;
> >> >> + else
> >> >> + p->regval = *ap_reg;
> >> >> +}
> >> >> +
> >> >> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r, u8 apr)
> >> >> +{
> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> + u8 idx = r->Op2 & 3;
> >> >> +
> >> >> + switch (vgic_v3_cpu->num_pri_bits) {
> >> >> + case 7:
> >> >> + if (idx > 3)
> >> >> + goto err;
> >> >
> >> > idx cannot be higher than three given the mask above, right?
> >> >
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + break;
> >> >> + case 6:
> >> >> + if (idx > 1)
> >> >> + goto err;
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + break;
> >> >> + default:
> >> >> + if (idx > 0)
> >> >> + goto err;
> >> >> + vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> >> >> + }
> >> >
> >> > what's the rationale behind ignoring the case where userspace is using
> >> > unsupported priorities? Is it that this will be checked during
> >> > save/restore of the ctlr?
> >> >
> >> > This sort of thing just looks like the case that's impossible to debug,
> >> > because userspace could be scratching its head trying to understand why
> >> > the value it wrote isn't recorded anywhere...
> >> >
> >> > If there's a good rationale for doing it this way, then could we have a
> >> > comment to that effect?
> >>
> >> Accessing umplemented priority registers raised UNDEF exception.
> >> So userspace accesing should be ignored instead of recording unsupported
> >> values.
> >>
> >
> > That's not what I asked.
> >
> > I asked why it's silently ignored as opposed to raising an error visible
> > to user space?
> >
> >> >
> >> >> +
> >> >> + return;
> >> >> +err:
> >> >> + if (!p->is_write)
> >> >> + p->regval = 0;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + access_gic_aprn(vcpu, p, r, 0);
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + access_gic_aprn(vcpu, p, r, 1);
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> + const struct sys_reg_desc *r)
> >> >> +{
> >> >> + struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> >> >> +
> >> >> + /* Validate SRE bit */
> >> >> + if (p->is_write) {
> >> >> + if (!(p->regval & ICC_SRE_EL1_SRE))
> >> >> + return false;
> >> >> + } else {
> >> >> + p->regval = vgicv3->vgic_sre;
> >> >> + }
> >> >> +
> >> >> + return true;
> >> >> +}
> >> >> +
> >> >> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> >> >> + /* ICC_PMR_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> >> >> + /* ICC_BPR0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> >> >> + /* ICC_AP0R0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> >> >> + /* ICC_AP0R1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> >> >> + /* ICC_AP0R2_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> >> >> + /* ICC_AP0R3_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> >> >> + /* ICC_AP1R0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> >> >> + /* ICC_AP1R1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> >> >> + /* ICC_AP1R2_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> >> >> + /* ICC_AP1R3_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> >> >> + /* ICC_BPR1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> >> >> + /* ICC_CTLR_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> >> >> + /* ICC_SRE_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> >> >> + /* ICC_IGRPEN0_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> >> >> + /* ICC_GRPEN1_EL1 */
> >> >> + { Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> >> >> +};
> >> >> +
> >> >> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> >> + u64 *reg)
> >> >> +{
> >> >> + struct sys_reg_params params;
> >> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> >> +
> >> >> + params.regval = *reg;
> >> >> + params.is_write = is_write;
> >> >> + params.is_aarch32 = false;
> >> >> + params.is_32bit = false;
> >> >> +
> >> >> + if (find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> >> + ARRAY_SIZE(gic_v3_icc_reg_descs)))
> >> >> + return 0;
> >> >> +
> >> >> + return -ENXIO;
> >> >> +}
> >> >> +
> >> >> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> >> >> + u64 *reg)
> >> >> +{
> >> >> + struct sys_reg_params params;
> >> >> + const struct sys_reg_desc *r;
> >> >> + u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> >> >> +
> >> >> + if (is_write)
> >> >> + params.regval = *reg;
> >> >> + params.is_write = is_write;
> >> >> + params.is_aarch32 = false;
> >> >> + params.is_32bit = false;
> >> >> +
> >> >> + r = find_reg_by_id(sysreg, ¶ms, gic_v3_icc_reg_descs,
> >> >> + ARRAY_SIZE(gic_v3_icc_reg_descs));
> >> >> + if (!r)
> >> >> + return -ENXIO;
> >> >> +
> >> >> + if (!r->access(vcpu, ¶ms, r))
> >> >> + return -EINVAL;
> >> >
> >> > According to the API, EINVAL meansinvalid mpidr. Should we expand on
> >> > how it can be used or allocate a new error code?
> >> How abt EACCES error code?
> >>
> >
> > That would mean permission denied, which is a bit weird.
> Yes I agree, but you can suggest.
You could expand the meaning in the API doc for gicv3 and use EINVAL, or
you could expand on what ENXIO means.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-21 10:19 ` Christoffer Dall
@ 2016-11-21 13:26 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-21 13:26 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Mon, Nov 21, 2016 at 3:49 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> >> <christoffer.dall@linaro.org> wrote:
>> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
>> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >>
>> >> >> VGICv3 CPU interface registers are accessed using
>> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> >> is used to identify the cpu for registers access.
>> >> >>
>> >> >> The version of VGIC v3 specification is define here
>> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >> >>
>> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >> ---
>> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> >> arch/arm64/kvm/Makefile | 1 +
>> >> >> include/kvm/arm_vgic.h | 9 +
>> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> >> 8 files changed, 395 insertions(+)
>> >> >>
>> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> index 56dc08d..91c7137 100644
>> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> >> +
>> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >> >>
>> >> >> /* Device Control API on vcpu fd */
>> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> >> index d50a82a..1a14e29 100644
>> >> >> --- a/arch/arm64/kvm/Makefile
>> >> >> +++ b/arch/arm64/kvm/Makefile
>> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >> >
>> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> >> > mean that either it is clearly only supported on AArch64 systems or it's
>> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> >> > on AArch32.
>> >>
>> >> It supports both AArch64 and AArch64 in handling of system registers
>> >> save/restore.
>> >> All system registers that we save/restore are 32-bit for both aarch64
>> >> and aarch32.
>> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> >> are same. However the codes sent by qemu is matched and register
>> >> are handled properly irrespective of AArch32 or AArch64.
>> >>
>> >> I don't have platform which support AArch32 guests to verify.
>> >
>> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
>> > that has a GICv3.
>> >
>> > I just tried to do a v7 compile with your patches, and it results in an
>> > epic failure, so there's something for you to look at.
>> >
>> >> >
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> >> >> index 002f092..730a18a 100644
>> >> >> --- a/include/kvm/arm_vgic.h
>> >> >> +++ b/include/kvm/arm_vgic.h
>> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
>> >> >>
>> >> >> /* GIC system register CPU interface */
>> >> >> struct static_key_false gicv3_cpuif;
>> >> >> +
>> >> >> + /* Cache ICH_VTR_EL2 reg value */
>> >> >> + u32 ich_vtr_el2;
>> >> >> };
>> >> >>
>> >> >> extern struct vgic_global kvm_vgic_global_state;
>> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> >> >> u64 pendbaser;
>> >> >>
>> >> >> bool lpis_enabled;
>> >> >> +
>> >> >> + /* Cache guest priority bits */
>> >> >> + u32 num_pri_bits;
>> >> >> +
>> >> >> + /* Cache guest interrupt ID bits */
>> >> >> + u32 num_id_bits;
>> >> >> };
>> >> >>
>> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> index 6c7d30c..da532d1 100644
>> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> >> >> if (!is_write)
>> >> >> *reg = tmp32;
>> >> >> break;
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 regid;
>> >> >> +
>> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> >> >> + regid, reg);
>> >> >> + break;
>> >> >> + }
>> >> >> default:
>> >> >> ret = -EINVAL;
>> >> >> break;
>> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> >> >> reg = tmp32;
>> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> >> + u64 reg;
>> >> >> +
>> >> >> + if (get_user(reg, uaddr))
>> >> >> + return -EFAULT;
>> >> >> +
>> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> >> + }
>> >> >> }
>> >> >> return -ENXIO;
>> >> >> }
>> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> >> >> tmp32 = reg;
>> >> >> return put_user(tmp32, uaddr);
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> >> + u64 reg;
>> >> >> +
>> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> >> >> + if (ret)
>> >> >> + return ret;
>> >> >> + return put_user(reg, uaddr);
>> >> >> + }
>> >> >> }
>> >> >>
>> >> >> return -ENXIO;
>> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> >> >> break;
>> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> >> >> return vgic_v3_has_attr_regs(dev, attr);
>> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> >> >> return 0;
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> index b35fb83..519b919 100644
>> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> @@ -23,6 +23,7 @@
>> >> >>
>> >> >> #include "vgic.h"
>> >> >> #include "vgic-mmio.h"
>> >> >> +#include "sys_regs.h"
>> >> >>
>> >> >> /* extract @num bytes at @offset bytes offset in data */
>> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
>> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> >> >> break;
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 reg, id;
>> >> >> + unsigned long vgic_mpidr, mpidr_reg;
>> >> >> + struct kvm_vcpu *vcpu;
>> >> >> +
>> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> >> >> +
>> >> >> + /* Convert plain mpidr value to MPIDR reg format */
>> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> >> >> +
>> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> >> >> + if (!vcpu)
>> >> >> + return -EINVAL;
>> >> >> +
>> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> >> >> + }
>> >> >> default:
>> >> >> return -ENXIO;
>> >> >> }
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> >> new file mode 100644
>> >> >> index 0000000..69d8597
>> >> >> --- /dev/null
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> >
>> >> > Shouldn't we have a GPL header here?
>> >> >
>> >> >> @@ -0,0 +1,324 @@
>> >> >> +#include <linux/irqchip/arm-gic-v3.h>
>> >> >> +#include <linux/kvm.h>
>> >> >> +#include <linux/kvm_host.h>
>> >> >> +#include <kvm/iodev.h>
>> >> >> +#include <kvm/arm_vgic.h>
>> >> >> +#include <asm/kvm_emulate.h>
>> >> >> +#include <asm/kvm_arm.h>
>> >> >> +#include <asm/kvm_mmu.h>
>> >> >> +
>> >> >> +#include "vgic.h"
>> >> >> +#include "vgic-mmio.h"
>> >> >> +#include "sys_regs.h"
>> >> >> +
>> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> >> + const struct sys_reg_desc *r)
>> >> >> +{
>> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> >> + struct vgic_vmcr vmcr;
>> >> >> + u64 val;
>> >> >> + u32 num_pri_bits, num_id_bits;
>> >> >> +
>> >> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> >> + if (p->is_write) {
>> >> >> + val = p->regval;
>> >> >> +
>> >> >> + /*
>> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> >> >> + * guest programmed ID and PRI bits
>> >> >> + */
>> >> >
>> >> > I would suggest rewording this comment:
>> >> > Disallow restoring VM state not supported by this hardware.
>> >> >
>> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> >> >> + return false;
>> >> >> +
>> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>> >> >
>> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
>> >> > understand which effect this is intended to have?
>> >> >
>> >> > Sure, it may limit what you do with other registers later, but since
>> >> > there's no ordering requirement that the ctlr be restored first, I'm not
>> >> > sure it makes sense.
>> >> >
>> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
>> >> > situation during runtime after a GICv3 restore where the
>> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
>> >> > which is never the case if you didn't do a save/restore.
>> >>
>> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
>> >> than HW supported
>> >> value.
>> >>
>> >
>> > So answer my question: What is the intended effect of writing this
>> > value? Is it just so that if you migrate this platform back again, then
>> > you're checking compatibility with what the guest would potentially do,
>>
>> Yes
>
> Then add a comment explaining that
>
>> and also to limit the valid aprn registers access as you said above.
>> But that has ordering restriction. Which I think we should follow.
>>
>
> I'm sorry, now I'm confused. Is there an ordering requirement in the
> API, or how should we follow this?
There is no ordering requirement mentioned in the API doc.
However the APRn registers depends on num_pri_bits. Hence first
ICC_CTLR_EL1 should be restored before APRn restore.
If ordering is not followed then APRn registers restore is allowed
as per hw supported num_pri_bits.
This should be mentioned in doc.
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-21 13:26 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-21 13:26 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 21, 2016 at 3:49 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> >> <christoffer.dall@linaro.org> wrote:
>> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
>> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >>
>> >> >> VGICv3 CPU interface registers are accessed using
>> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> >> is used to identify the cpu for registers access.
>> >> >>
>> >> >> The version of VGIC v3 specification is define here
>> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >> >>
>> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >> ---
>> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> >> arch/arm64/kvm/Makefile | 1 +
>> >> >> include/kvm/arm_vgic.h | 9 +
>> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> >> 8 files changed, 395 insertions(+)
>> >> >>
>> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> index 56dc08d..91c7137 100644
>> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> >> +
>> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >> >>
>> >> >> /* Device Control API on vcpu fd */
>> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> >> index d50a82a..1a14e29 100644
>> >> >> --- a/arch/arm64/kvm/Makefile
>> >> >> +++ b/arch/arm64/kvm/Makefile
>> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >> >
>> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> >> > mean that either it is clearly only supported on AArch64 systems or it's
>> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> >> > on AArch32.
>> >>
>> >> It supports both AArch64 and AArch64 in handling of system registers
>> >> save/restore.
>> >> All system registers that we save/restore are 32-bit for both aarch64
>> >> and aarch32.
>> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> >> are same. However the codes sent by qemu is matched and register
>> >> are handled properly irrespective of AArch32 or AArch64.
>> >>
>> >> I don't have platform which support AArch32 guests to verify.
>> >
>> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
>> > that has a GICv3.
>> >
>> > I just tried to do a v7 compile with your patches, and it results in an
>> > epic failure, so there's something for you to look at.
>> >
>> >> >
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> >> >> index 002f092..730a18a 100644
>> >> >> --- a/include/kvm/arm_vgic.h
>> >> >> +++ b/include/kvm/arm_vgic.h
>> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
>> >> >>
>> >> >> /* GIC system register CPU interface */
>> >> >> struct static_key_false gicv3_cpuif;
>> >> >> +
>> >> >> + /* Cache ICH_VTR_EL2 reg value */
>> >> >> + u32 ich_vtr_el2;
>> >> >> };
>> >> >>
>> >> >> extern struct vgic_global kvm_vgic_global_state;
>> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
>> >> >> u64 pendbaser;
>> >> >>
>> >> >> bool lpis_enabled;
>> >> >> +
>> >> >> + /* Cache guest priority bits */
>> >> >> + u32 num_pri_bits;
>> >> >> +
>> >> >> + /* Cache guest interrupt ID bits */
>> >> >> + u32 num_id_bits;
>> >> >> };
>> >> >>
>> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> index 6c7d30c..da532d1 100644
>> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>> >> >> if (!is_write)
>> >> >> *reg = tmp32;
>> >> >> break;
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 regid;
>> >> >> +
>> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> >> >> + regid, reg);
>> >> >> + break;
>> >> >> + }
>> >> >> default:
>> >> >> ret = -EINVAL;
>> >> >> break;
>> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>> >> >> reg = tmp32;
>> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> >> + u64 reg;
>> >> >> +
>> >> >> + if (get_user(reg, uaddr))
>> >> >> + return -EFAULT;
>> >> >> +
>> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
>> >> >> + }
>> >> >> }
>> >> >> return -ENXIO;
>> >> >> }
>> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>> >> >> tmp32 = reg;
>> >> >> return put_user(tmp32, uaddr);
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> >> >> + u64 reg;
>> >> >> +
>> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
>> >> >> + if (ret)
>> >> >> + return ret;
>> >> >> + return put_user(reg, uaddr);
>> >> >> + }
>> >> >> }
>> >> >>
>> >> >> return -ENXIO;
>> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>> >> >> break;
>> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>> >> >> return vgic_v3_has_attr_regs(dev, attr);
>> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>> >> >> return 0;
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> index b35fb83..519b919 100644
>> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> >> >> @@ -23,6 +23,7 @@
>> >> >>
>> >> >> #include "vgic.h"
>> >> >> #include "vgic-mmio.h"
>> >> >> +#include "sys_regs.h"
>> >> >>
>> >> >> /* extract @num bytes at @offset bytes offset in data */
>> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
>> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>> >> >> break;
>> >> >> }
>> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> >> >> + u64 reg, id;
>> >> >> + unsigned long vgic_mpidr, mpidr_reg;
>> >> >> + struct kvm_vcpu *vcpu;
>> >> >> +
>> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> >> >> +
>> >> >> + /* Convert plain mpidr value to MPIDR reg format */
>> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
>> >> >> +
>> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
>> >> >> + if (!vcpu)
>> >> >> + return -EINVAL;
>> >> >> +
>> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
>> >> >> + }
>> >> >> default:
>> >> >> return -ENXIO;
>> >> >> }
>> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> >> new file mode 100644
>> >> >> index 0000000..69d8597
>> >> >> --- /dev/null
>> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >> >
>> >> > Shouldn't we have a GPL header here?
>> >> >
>> >> >> @@ -0,0 +1,324 @@
>> >> >> +#include <linux/irqchip/arm-gic-v3.h>
>> >> >> +#include <linux/kvm.h>
>> >> >> +#include <linux/kvm_host.h>
>> >> >> +#include <kvm/iodev.h>
>> >> >> +#include <kvm/arm_vgic.h>
>> >> >> +#include <asm/kvm_emulate.h>
>> >> >> +#include <asm/kvm_arm.h>
>> >> >> +#include <asm/kvm_mmu.h>
>> >> >> +
>> >> >> +#include "vgic.h"
>> >> >> +#include "vgic-mmio.h"
>> >> >> +#include "sys_regs.h"
>> >> >> +
>> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >> >> + const struct sys_reg_desc *r)
>> >> >> +{
>> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
>> >> >> + struct vgic_vmcr vmcr;
>> >> >> + u64 val;
>> >> >> + u32 num_pri_bits, num_id_bits;
>> >> >> +
>> >> >> + vgic_get_vmcr(vcpu, &vmcr);
>> >> >> + if (p->is_write) {
>> >> >> + val = p->regval;
>> >> >> +
>> >> >> + /*
>> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
>> >> >> + * guest programmed ID and PRI bits
>> >> >> + */
>> >> >
>> >> > I would suggest rewording this comment:
>> >> > Disallow restoring VM state not supported by this hardware.
>> >> >
>> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
>> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
>> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
>> >> >> + return false;
>> >> >> +
>> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
>> >> >
>> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
>> >> > understand which effect this is intended to have?
>> >> >
>> >> > Sure, it may limit what you do with other registers later, but since
>> >> > there's no ordering requirement that the ctlr be restored first, I'm not
>> >> > sure it makes sense.
>> >> >
>> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
>> >> > situation during runtime after a GICv3 restore where the
>> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
>> >> > which is never the case if you didn't do a save/restore.
>> >>
>> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
>> >> than HW supported
>> >> value.
>> >>
>> >
>> > So answer my question: What is the intended effect of writing this
>> > value? Is it just so that if you migrate this platform back again, then
>> > you're checking compatibility with what the guest would potentially do,
>>
>> Yes
>
> Then add a comment explaining that
>
>> and also to limit the valid aprn registers access as you said above.
>> But that has ordering restriction. Which I think we should follow.
>>
>
> I'm sorry, now I'm confused. Is there an ordering requirement in the
> API, or how should we follow this?
There is no ordering requirement mentioned in the API doc.
However the APRn registers depends on num_pri_bits. Hence first
ICC_CTLR_EL1 should be restored before APRn restore.
If ordering is not followed then APRn registers restore is allowed
as per hw supported num_pri_bits.
This should be mentioned in doc.
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-20 13:20 ` Christoffer Dall
@ 2016-11-21 13:32 ` Vijay Kilari
-1 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-21 13:32 UTC (permalink / raw)
To: Christoffer Dall; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Sun, Nov 20, 2016 at 6:50 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> >> <christoffer.dall@linaro.org> wrote:
>> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
>> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >>
>> >> >> VGICv3 CPU interface registers are accessed using
>> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> >> is used to identify the cpu for registers access.
>> >> >>
>> >> >> The version of VGIC v3 specification is define here
>> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >> >>
>> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >> ---
>> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> >> arch/arm64/kvm/Makefile | 1 +
>> >> >> include/kvm/arm_vgic.h | 9 +
>> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> >> 8 files changed, 395 insertions(+)
>> >> >>
>> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> index 56dc08d..91c7137 100644
>> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> >> +
>> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >> >>
>> >> >> /* Device Control API on vcpu fd */
>> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> >> index d50a82a..1a14e29 100644
>> >> >> --- a/arch/arm64/kvm/Makefile
>> >> >> +++ b/arch/arm64/kvm/Makefile
>> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >> >
>> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> >> > mean that either it is clearly only supported on AArch64 systems or it's
>> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> >> > on AArch32.
>> >>
>> >> It supports both AArch64 and AArch64 in handling of system registers
>> >> save/restore.
>> >> All system registers that we save/restore are 32-bit for both aarch64
>> >> and aarch32.
>> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> >> are same. However the codes sent by qemu is matched and register
>> >> are handled properly irrespective of AArch32 or AArch64.
>> >>
>> >> I don't have platform which support AArch32 guests to verify.
>> >
>> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
>> > that has a GICv3.
>> >
>> > I just tried to do a v7 compile with your patches, and it results in an
>> > epic failure, so there's something for you to look at.
>> >
>>
>> Could you please share you config file?. I tried with multi_v7 defconfig with
>> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
>
> I think this has to do with which branch you apply your patches to.
> When applied to kvmarm/next, it fails.
>
> Here's the integration I did:
> https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
>
> Here's the config:
> https://transfer.sh/xkAxp/.config
>
Thanks for shareing the details, I could reproduce them.
However virt/kvm/arm/vgic/vgic-sys-reg-v3.c is written with
sys_regs_desc for AArch64.
For AArch32/v7, it has be to coproc_reg. I propose to add separate file for arm
which handles ICC* reg save/restore using coproc_reg.
> Here's the compile output:
>
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:26:22: fatal error: sys_regs.h: No such file or directory
> #include "sys_regs.h"
> ^
> compilation terminated.
> make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_parse_attr’:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: error: ‘KVM_DEV_ARM_VGIC_V3_MPIDR_MASK’ undeclared (first use in this function)
> vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: note: each undeclared identifier is reported only once for each function it appears in
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:439:9: error: ‘KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT’ undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:441:2: error: implicit declaration of function ‘MPIDR_LEVEL_SHIFT’ [-Werror=implicit-function-declaration]
> mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_attr_regs_access’:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:497:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:505:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:508:25: error: ‘KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK’ undeclared (first use in this function)
> regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:513:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:516:24: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK’ undeclared (first use in this function)
> info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:517:4: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT’ undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:518:15: error: ‘VGIC_LEVEL_INFO_LINE_LEVEL’ undeclared (first use in this function)
> if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:522:5: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK’ undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_set_attr’:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:554:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:565:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:574:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_get_attr’:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:600:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:611:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:620:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ‘vgic_v3_has_attr’:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:647:7: error: ‘KVM_DEV_ARM_VGIC_GRP_REDIST_REGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:648:7: error: ‘KVM_DEV_ARM_VGIC_CPU_SYSREGS’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:652:7: error: ‘KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO’ undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:653:22: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK’ undeclared (first use in this function)
> if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:654:9: error: ‘KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT’ undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:655:9: error: ‘VGIC_LEVEL_INFO_LINE_LEVEL’ undeclared (first use in this function)
> VGIC_LEVEL_INFO_LINE_LEVEL)
> ^
> cc1: some warnings being treated as errors
> make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.o] Error 1
> make[1]: *** [arch/arm/kvm] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [sub-make] Error 2
>
> Thanks,
> -Christoffer
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-21 13:32 ` Vijay Kilari
0 siblings, 0 replies; 62+ messages in thread
From: Vijay Kilari @ 2016-11-21 13:32 UTC (permalink / raw)
To: linux-arm-kernel
On Sun, Nov 20, 2016 at 6:50 PM, Christoffer Dall
<christoffer.dall@linaro.org> wrote:
> On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
>> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
>> <christoffer.dall@linaro.org> wrote:
>> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
>> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
>> >> <christoffer.dall@linaro.org> wrote:
>> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
>> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >>
>> >> >> VGICv3 CPU interface registers are accessed using
>> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >> >> is used to identify the cpu for registers access.
>> >> >>
>> >> >> The version of VGIC v3 specification is define here
>> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >> >>
>> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >> >> ---
>> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
>> >> >> arch/arm64/kvm/Makefile | 1 +
>> >> >> include/kvm/arm_vgic.h | 9 +
>> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
>> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
>> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
>> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
>> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
>> >> >> 8 files changed, 395 insertions(+)
>> >> >>
>> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> index 56dc08d..91c7137 100644
>> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
>> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
>> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
>> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
>> >> >> +
>> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
>> >> >>
>> >> >> /* Device Control API on vcpu fd */
>> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> >> >> index d50a82a..1a14e29 100644
>> >> >> --- a/arch/arm64/kvm/Makefile
>> >> >> +++ b/arch/arm64/kvm/Makefile
>> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>> >> >
>> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
>> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
>> >> > mean that either it is clearly only supported on AArch64 systems or it's
>> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
>> >> > on AArch32.
>> >>
>> >> It supports both AArch64 and AArch64 in handling of system registers
>> >> save/restore.
>> >> All system registers that we save/restore are 32-bit for both aarch64
>> >> and aarch32.
>> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
>> >> are same. However the codes sent by qemu is matched and register
>> >> are handled properly irrespective of AArch32 or AArch64.
>> >>
>> >> I don't have platform which support AArch32 guests to verify.
>> >
>> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
>> > that has a GICv3.
>> >
>> > I just tried to do a v7 compile with your patches, and it results in an
>> > epic failure, so there's something for you to look at.
>> >
>>
>> Could you please share you config file?. I tried with multi_v7 defconfig with
>> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
>
> I think this has to do with which branch you apply your patches to.
> When applied to kvmarm/next, it fails.
>
> Here's the integration I did:
> https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
>
> Here's the config:
> https://transfer.sh/xkAxp/.config
>
Thanks for shareing the details, I could reproduce them.
However virt/kvm/arm/vgic/vgic-sys-reg-v3.c is written with
sys_regs_desc for AArch64.
For AArch32/v7, it has be to coproc_reg. I propose to add separate file for arm
which handles ICC* reg save/restore using coproc_reg.
> Here's the compile output:
>
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.c:26:22: fatal error: sys_regs.h: No such file or directory
> #include "sys_regs.h"
> ^
> compilation terminated.
> make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-mmio-v3.o] Error 1
> make[2]: *** Waiting for unfinished jobs....
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_parse_attr?:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: error: ?KVM_DEV_ARM_VGIC_V3_MPIDR_MASK? undeclared (first use in this function)
> vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:438:29: note: each undeclared identifier is reported only once for each function it appears in
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:439:9: error: ?KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT? undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:441:2: error: implicit declaration of function ?MPIDR_LEVEL_SHIFT? [-Werror=implicit-function-declaration]
> mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_attr_regs_access?:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:497:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:505:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:508:25: error: ?KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK? undeclared (first use in this function)
> regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:513:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:516:24: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK? undeclared (first use in this function)
> info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:517:4: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT? undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:518:15: error: ?VGIC_LEVEL_INFO_LINE_LEVEL? undeclared (first use in this function)
> if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:522:5: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK? undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_set_attr?:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:554:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:565:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:574:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_get_attr?:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:600:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:611:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:620:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c: In function ?vgic_v3_has_attr?:
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:647:7: error: ?KVM_DEV_ARM_VGIC_GRP_REDIST_REGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:648:7: error: ?KVM_DEV_ARM_VGIC_CPU_SYSREGS? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:652:7: error: ?KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO? undeclared (first use in this function)
> case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:653:22: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK? undeclared (first use in this function)
> if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:654:9: error: ?KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT? undeclared (first use in this function)
> KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
> ^
> /home/christoffer/src/kvmarm/linux/arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.c:655:9: error: ?VGIC_LEVEL_INFO_LINE_LEVEL? undeclared (first use in this function)
> VGIC_LEVEL_INFO_LINE_LEVEL)
> ^
> cc1: some warnings being treated as errors
> make[2]: *** [arch/arm/kvm/../../../virt/kvm/arm/vgic/vgic-kvm-device.o] Error 1
> make[1]: *** [arch/arm/kvm] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [sub-make] Error 2
>
> Thanks,
> -Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-21 13:32 ` Vijay Kilari
@ 2016-11-21 13:41 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 13:41 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Mon, Nov 21, 2016 at 07:02:36PM +0530, Vijay Kilari wrote:
> On Sun, Nov 20, 2016 at 6:50 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> >> <christoffer.dall@linaro.org> wrote:
> >> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> >> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >>
> >> >> >> VGICv3 CPU interface registers are accessed using
> >> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> >> is used to identify the cpu for registers access.
> >> >> >>
> >> >> >> The version of VGIC v3 specification is define here
> >> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >> >>
> >> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >> ---
> >> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> >> 8 files changed, 395 insertions(+)
> >> >> >>
> >> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> index 56dc08d..91c7137 100644
> >> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> >> +
> >> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >> >>
> >> >> >> /* Device Control API on vcpu fd */
> >> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> >> index d50a82a..1a14e29 100644
> >> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >> >
> >> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> >> > on AArch32.
> >> >>
> >> >> It supports both AArch64 and AArch64 in handling of system registers
> >> >> save/restore.
> >> >> All system registers that we save/restore are 32-bit for both aarch64
> >> >> and aarch32.
> >> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> >> are same. However the codes sent by qemu is matched and register
> >> >> are handled properly irrespective of AArch32 or AArch64.
> >> >>
> >> >> I don't have platform which support AArch32 guests to verify.
> >> >
> >> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> >> > that has a GICv3.
> >> >
> >> > I just tried to do a v7 compile with your patches, and it results in an
> >> > epic failure, so there's something for you to look at.
> >> >
> >>
> >> Could you please share you config file?. I tried with multi_v7 defconfig with
> >> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
> >
> > I think this has to do with which branch you apply your patches to.
> > When applied to kvmarm/next, it fails.
> >
> > Here's the integration I did:
> > https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
> >
> > Here's the config:
> > https://transfer.sh/xkAxp/.config
> >
>
> Thanks for shareing the details, I could reproduce them.
> However virt/kvm/arm/vgic/vgic-sys-reg-v3.c is written with
> sys_regs_desc for AArch64.
> For AArch32/v7, it has be to coproc_reg. I propose to add separate file for arm
> which handles ICC* reg save/restore using coproc_reg.
That might make sense. In that case they want to be moved into
arch/arm/kvm/ and arch/arm64/kvm/
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-21 13:41 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 13:41 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 21, 2016 at 07:02:36PM +0530, Vijay Kilari wrote:
> On Sun, Nov 20, 2016 at 6:50 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Sat, Nov 19, 2016 at 12:18:53AM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> >> <christoffer.dall@linaro.org> wrote:
> >> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> >> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >>
> >> >> >> VGICv3 CPU interface registers are accessed using
> >> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> >> is used to identify the cpu for registers access.
> >> >> >>
> >> >> >> The version of VGIC v3 specification is define here
> >> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >> >>
> >> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >> ---
> >> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> >> 8 files changed, 395 insertions(+)
> >> >> >>
> >> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> index 56dc08d..91c7137 100644
> >> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> >> +
> >> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >> >>
> >> >> >> /* Device Control API on vcpu fd */
> >> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> >> index d50a82a..1a14e29 100644
> >> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >> >
> >> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> >> > on AArch32.
> >> >>
> >> >> It supports both AArch64 and AArch64 in handling of system registers
> >> >> save/restore.
> >> >> All system registers that we save/restore are 32-bit for both aarch64
> >> >> and aarch32.
> >> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> >> are same. However the codes sent by qemu is matched and register
> >> >> are handled properly irrespective of AArch32 or AArch64.
> >> >>
> >> >> I don't have platform which support AArch32 guests to verify.
> >> >
> >> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> >> > that has a GICv3.
> >> >
> >> > I just tried to do a v7 compile with your patches, and it results in an
> >> > epic failure, so there's something for you to look at.
> >> >
> >>
> >> Could you please share you config file?. I tried with multi_v7 defconfig with
> >> CONFIG KVM and CONFIG_KVM_ARM_HOST enabled. it compiled for me.
> >
> > I think this has to do with which branch you apply your patches to.
> > When applied to kvmarm/next, it fails.
> >
> > Here's the integration I did:
> > https://git.linaro.org/people/christoffer.dall/linux-kvm-arm.git tmp-gicv3-migrate-v8
> >
> > Here's the config:
> > https://transfer.sh/xkAxp/.config
> >
>
> Thanks for shareing the details, I could reproduce them.
> However virt/kvm/arm/vgic/vgic-sys-reg-v3.c is written with
> sys_regs_desc for AArch64.
> For AArch32/v7, it has be to coproc_reg. I propose to add separate file for arm
> which handles ICC* reg save/restore using coproc_reg.
That might make sense. In that case they want to be moved into
arch/arm/kvm/ and arch/arm64/kvm/
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
2016-11-21 13:26 ` Vijay Kilari
@ 2016-11-21 13:43 ` Christoffer Dall
-1 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 13:43 UTC (permalink / raw)
To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel
On Mon, Nov 21, 2016 at 06:56:08PM +0530, Vijay Kilari wrote:
> On Mon, Nov 21, 2016 at 3:49 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> >> <christoffer.dall@linaro.org> wrote:
> >> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari@gmail.com wrote:
> >> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >>
> >> >> >> VGICv3 CPU interface registers are accessed using
> >> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> >> is used to identify the cpu for registers access.
> >> >> >>
> >> >> >> The version of VGIC v3 specification is define here
> >> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >> >>
> >> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >> ---
> >> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> >> 8 files changed, 395 insertions(+)
> >> >> >>
> >> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> index 56dc08d..91c7137 100644
> >> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> >> +
> >> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >> >>
> >> >> >> /* Device Control API on vcpu fd */
> >> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> >> index d50a82a..1a14e29 100644
> >> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >> >
> >> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> >> > on AArch32.
> >> >>
> >> >> It supports both AArch64 and AArch64 in handling of system registers
> >> >> save/restore.
> >> >> All system registers that we save/restore are 32-bit for both aarch64
> >> >> and aarch32.
> >> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> >> are same. However the codes sent by qemu is matched and register
> >> >> are handled properly irrespective of AArch32 or AArch64.
> >> >>
> >> >> I don't have platform which support AArch32 guests to verify.
> >> >
> >> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> >> > that has a GICv3.
> >> >
> >> > I just tried to do a v7 compile with your patches, and it results in an
> >> > epic failure, so there's something for you to look at.
> >> >
> >> >> >
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> >> >> index 002f092..730a18a 100644
> >> >> >> --- a/include/kvm/arm_vgic.h
> >> >> >> +++ b/include/kvm/arm_vgic.h
> >> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >> >> >>
> >> >> >> /* GIC system register CPU interface */
> >> >> >> struct static_key_false gicv3_cpuif;
> >> >> >> +
> >> >> >> + /* Cache ICH_VTR_EL2 reg value */
> >> >> >> + u32 ich_vtr_el2;
> >> >> >> };
> >> >> >>
> >> >> >> extern struct vgic_global kvm_vgic_global_state;
> >> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> >> >> u64 pendbaser;
> >> >> >>
> >> >> >> bool lpis_enabled;
> >> >> >> +
> >> >> >> + /* Cache guest priority bits */
> >> >> >> + u32 num_pri_bits;
> >> >> >> +
> >> >> >> + /* Cache guest interrupt ID bits */
> >> >> >> + u32 num_id_bits;
> >> >> >> };
> >> >> >>
> >> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> index 6c7d30c..da532d1 100644
> >> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> >> >> if (!is_write)
> >> >> >> *reg = tmp32;
> >> >> >> break;
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 regid;
> >> >> >> +
> >> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> >> >> + regid, reg);
> >> >> >> + break;
> >> >> >> + }
> >> >> >> default:
> >> >> >> ret = -EINVAL;
> >> >> >> break;
> >> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> >> >> reg = tmp32;
> >> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> >> + u64 reg;
> >> >> >> +
> >> >> >> + if (get_user(reg, uaddr))
> >> >> >> + return -EFAULT;
> >> >> >> +
> >> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> >> + }
> >> >> >> }
> >> >> >> return -ENXIO;
> >> >> >> }
> >> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> >> >> tmp32 = reg;
> >> >> >> return put_user(tmp32, uaddr);
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> >> + u64 reg;
> >> >> >> +
> >> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> >> >> + if (ret)
> >> >> >> + return ret;
> >> >> >> + return put_user(reg, uaddr);
> >> >> >> + }
> >> >> >> }
> >> >> >>
> >> >> >> return -ENXIO;
> >> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> >> >> break;
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> >> >> return vgic_v3_has_attr_regs(dev, attr);
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> >> >> return 0;
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> index b35fb83..519b919 100644
> >> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> @@ -23,6 +23,7 @@
> >> >> >>
> >> >> >> #include "vgic.h"
> >> >> >> #include "vgic-mmio.h"
> >> >> >> +#include "sys_regs.h"
> >> >> >>
> >> >> >> /* extract @num bytes at @offset bytes offset in data */
> >> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> >> >> break;
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 reg, id;
> >> >> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> >> >> + struct kvm_vcpu *vcpu;
> >> >> >> +
> >> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> >> >> +
> >> >> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> >> >> +
> >> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> >> >> + if (!vcpu)
> >> >> >> + return -EINVAL;
> >> >> >> +
> >> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> >> >> + }
> >> >> >> default:
> >> >> >> return -ENXIO;
> >> >> >> }
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> >> new file mode 100644
> >> >> >> index 0000000..69d8597
> >> >> >> --- /dev/null
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> >
> >> >> > Shouldn't we have a GPL header here?
> >> >> >
> >> >> >> @@ -0,0 +1,324 @@
> >> >> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> >> >> +#include <linux/kvm.h>
> >> >> >> +#include <linux/kvm_host.h>
> >> >> >> +#include <kvm/iodev.h>
> >> >> >> +#include <kvm/arm_vgic.h>
> >> >> >> +#include <asm/kvm_emulate.h>
> >> >> >> +#include <asm/kvm_arm.h>
> >> >> >> +#include <asm/kvm_mmu.h>
> >> >> >> +
> >> >> >> +#include "vgic.h"
> >> >> >> +#include "vgic-mmio.h"
> >> >> >> +#include "sys_regs.h"
> >> >> >> +
> >> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> >> + const struct sys_reg_desc *r)
> >> >> >> +{
> >> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> >> + struct vgic_vmcr vmcr;
> >> >> >> + u64 val;
> >> >> >> + u32 num_pri_bits, num_id_bits;
> >> >> >> +
> >> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> >> + if (p->is_write) {
> >> >> >> + val = p->regval;
> >> >> >> +
> >> >> >> + /*
> >> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> >> >> + * guest programmed ID and PRI bits
> >> >> >> + */
> >> >> >
> >> >> > I would suggest rewording this comment:
> >> >> > Disallow restoring VM state not supported by this hardware.
> >> >> >
> >> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> >> >> + return false;
> >> >> >> +
> >> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >> >> >
> >> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> >> >> > understand which effect this is intended to have?
> >> >> >
> >> >> > Sure, it may limit what you do with other registers later, but since
> >> >> > there's no ordering requirement that the ctlr be restored first, I'm not
> >> >> > sure it makes sense.
> >> >> >
> >> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> >> >> > situation during runtime after a GICv3 restore where the
> >> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> >> >> > which is never the case if you didn't do a save/restore.
> >> >>
> >> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> >> >> than HW supported
> >> >> value.
> >> >>
> >> >
> >> > So answer my question: What is the intended effect of writing this
> >> > value? Is it just so that if you migrate this platform back again, then
> >> > you're checking compatibility with what the guest would potentially do,
> >>
> >> Yes
> >
> > Then add a comment explaining that
> >
> >> and also to limit the valid aprn registers access as you said above.
> >> But that has ordering restriction. Which I think we should follow.
> >>
> >
> > I'm sorry, now I'm confused. Is there an ordering requirement in the
> > API, or how should we follow this?
>
> There is no ordering requirement mentioned in the API doc.
> However the APRn registers depends on num_pri_bits. Hence first
> ICC_CTLR_EL1 should be restored before APRn restore.
>
> If ordering is not followed then APRn registers restore is allowed
> as per hw supported num_pri_bits.
>
> This should be mentioned in doc.
How about just having a consistency check function that you call from
uaccess updates to both functions, and in that way avoid requireing any
ordering which is likely to not be followed etc.?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access
@ 2016-11-21 13:43 ` Christoffer Dall
0 siblings, 0 replies; 62+ messages in thread
From: Christoffer Dall @ 2016-11-21 13:43 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Nov 21, 2016 at 06:56:08PM +0530, Vijay Kilari wrote:
> On Mon, Nov 21, 2016 at 3:49 PM, Christoffer Dall
> <christoffer.dall@linaro.org> wrote:
> > On Fri, Nov 18, 2016 at 10:28:34PM +0530, Vijay Kilari wrote:
> >> On Thu, Nov 17, 2016 at 9:39 PM, Christoffer Dall
> >> <christoffer.dall@linaro.org> wrote:
> >> > On Thu, Nov 17, 2016 at 09:25:59PM +0530, Vijay Kilari wrote:
> >> >> On Thu, Nov 17, 2016 at 12:22 AM, Christoffer Dall
> >> >> <christoffer.dall@linaro.org> wrote:
> >> >> > On Fri, Nov 04, 2016 at 04:43:32PM +0530, vijay.kilari at gmail.com wrote:
> >> >> >> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >>
> >> >> >> VGICv3 CPU interface registers are accessed using
> >> >> >> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >> >> >> as 64-bit. The cpu MPIDR value is passed along with register id.
> >> >> >> is used to identify the cpu for registers access.
> >> >> >>
> >> >> >> The version of VGIC v3 specification is define here
> >> >> >> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >> >> >>
> >> >> >> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >> >> >> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >> >> >> ---
> >> >> >> arch/arm64/include/uapi/asm/kvm.h | 3 +
> >> >> >> arch/arm64/kvm/Makefile | 1 +
> >> >> >> include/kvm/arm_vgic.h | 9 +
> >> >> >> virt/kvm/arm/vgic/vgic-kvm-device.c | 27 +++
> >> >> >> virt/kvm/arm/vgic/vgic-mmio-v3.c | 19 +++
> >> >> >> virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 324 ++++++++++++++++++++++++++++++++++++
> >> >> >> virt/kvm/arm/vgic/vgic-v3.c | 8 +
> >> >> >> virt/kvm/arm/vgic/vgic.h | 4 +
> >> >> >> 8 files changed, 395 insertions(+)
> >> >> >>
> >> >> >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> index 56dc08d..91c7137 100644
> >> >> >> --- a/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> +++ b/arch/arm64/include/uapi/asm/kvm.h
> >> >> >> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
> >> >> >> (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >> >> >> #define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> >> >> >> +#define KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_CTRL 4
> >> >> >> #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> >> >> >> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS 6
> >> >> >> +
> >> >> >> #define KVM_DEV_ARM_VGIC_CTRL_INIT 0
> >> >> >>
> >> >> >> /* Device Control API on vcpu fd */
> >> >> >> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> >> >> >> index d50a82a..1a14e29 100644
> >> >> >> --- a/arch/arm64/kvm/Makefile
> >> >> >> +++ b/arch/arm64/kvm/Makefile
> >> >> >> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> >> >> >> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
> >> >> >
> >> >> > Thi is making me wonder: Are we properly handling GICv3 save/restore
> >> >> > for AArch32 now that we have GICv3 support for AArch32? By properly I
> >> >> > mean that either it is clearly only supported on AArch64 systems or it's
> >> >> > supported on both AArch64 and AArch32, but it shouldn't break randomly
> >> >> > on AArch32.
> >> >>
> >> >> It supports both AArch64 and AArch64 in handling of system registers
> >> >> save/restore.
> >> >> All system registers that we save/restore are 32-bit for both aarch64
> >> >> and aarch32.
> >> >> Though opcode op0 should be zero for aarch32, the remaining Op and CRn codes
> >> >> are same. However the codes sent by qemu is matched and register
> >> >> are handled properly irrespective of AArch32 or AArch64.
> >> >>
> >> >> I don't have platform which support AArch32 guests to verify.
> >> >
> >> > Actually this is not about the guest, it's about an ARMv8 AArch32 host
> >> > that has a GICv3.
> >> >
> >> > I just tried to do a v7 compile with your patches, and it results in an
> >> > epic failure, so there's something for you to look at.
> >> >
> >> >> >
> >> >> >> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
> >> >> >> kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> >> >> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> >> >> index 002f092..730a18a 100644
> >> >> >> --- a/include/kvm/arm_vgic.h
> >> >> >> +++ b/include/kvm/arm_vgic.h
> >> >> >> @@ -71,6 +71,9 @@ struct vgic_global {
> >> >> >>
> >> >> >> /* GIC system register CPU interface */
> >> >> >> struct static_key_false gicv3_cpuif;
> >> >> >> +
> >> >> >> + /* Cache ICH_VTR_EL2 reg value */
> >> >> >> + u32 ich_vtr_el2;
> >> >> >> };
> >> >> >>
> >> >> >> extern struct vgic_global kvm_vgic_global_state;
> >> >> >> @@ -269,6 +272,12 @@ struct vgic_cpu {
> >> >> >> u64 pendbaser;
> >> >> >>
> >> >> >> bool lpis_enabled;
> >> >> >> +
> >> >> >> + /* Cache guest priority bits */
> >> >> >> + u32 num_pri_bits;
> >> >> >> +
> >> >> >> + /* Cache guest interrupt ID bits */
> >> >> >> + u32 num_id_bits;
> >> >> >> };
> >> >> >>
> >> >> >> extern struct static_key_false vgic_v2_cpuif_trap;
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> index 6c7d30c..da532d1 100644
> >> >> >> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> >> >> >> @@ -504,6 +504,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
> >> >> >> if (!is_write)
> >> >> >> *reg = tmp32;
> >> >> >> break;
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 regid;
> >> >> >> +
> >> >> >> + regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> >> + ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> >> >> >> + regid, reg);
> >> >> >> + break;
> >> >> >> + }
> >> >> >> default:
> >> >> >> ret = -EINVAL;
> >> >> >> break;
> >> >> >> @@ -537,6 +545,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
> >> >> >> reg = tmp32;
> >> >> >> return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> >> + u64 reg;
> >> >> >> +
> >> >> >> + if (get_user(reg, uaddr))
> >> >> >> + return -EFAULT;
> >> >> >> +
> >> >> >> + return vgic_v3_attr_regs_access(dev, attr, ®, true);
> >> >> >> + }
> >> >> >> }
> >> >> >> return -ENXIO;
> >> >> >> }
> >> >> >> @@ -563,6 +580,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
> >> >> >> tmp32 = reg;
> >> >> >> return put_user(tmp32, uaddr);
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> >> >> >> + u64 reg;
> >> >> >> +
> >> >> >> + ret = vgic_v3_attr_regs_access(dev, attr, ®, false);
> >> >> >> + if (ret)
> >> >> >> + return ret;
> >> >> >> + return put_user(reg, uaddr);
> >> >> >> + }
> >> >> >> }
> >> >> >>
> >> >> >> return -ENXIO;
> >> >> >> @@ -581,6 +607,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
> >> >> >> break;
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
> >> >> >> return vgic_v3_has_attr_regs(dev, attr);
> >> >> >> case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
> >> >> >> return 0;
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> index b35fb83..519b919 100644
> >> >> >> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> >> >> >> @@ -23,6 +23,7 @@
> >> >> >>
> >> >> >> #include "vgic.h"
> >> >> >> #include "vgic-mmio.h"
> >> >> >> +#include "sys_regs.h"
> >> >> >>
> >> >> >> /* extract @num bytes at @offset bytes offset in data */
> >> >> >> unsigned long extract_bytes(u64 data, unsigned int offset,
> >> >> >> @@ -639,6 +640,24 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
> >> >> >> nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
> >> >> >> break;
> >> >> >> }
> >> >> >> + case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> >> >> >> + u64 reg, id;
> >> >> >> + unsigned long vgic_mpidr, mpidr_reg;
> >> >> >> + struct kvm_vcpu *vcpu;
> >> >> >> +
> >> >> >> + vgic_mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> >> >> >> + KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> >> >> >> +
> >> >> >> + /* Convert plain mpidr value to MPIDR reg format */
> >> >> >> + mpidr_reg = VGIC_TO_MPIDR(vgic_mpidr);
> >> >> >> +
> >> >> >> + vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr_reg);
> >> >> >> + if (!vcpu)
> >> >> >> + return -EINVAL;
> >> >> >> +
> >> >> >> + id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> >> >> >> + return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, ®);
> >> >> >> + }
> >> >> >> default:
> >> >> >> return -ENXIO;
> >> >> >> }
> >> >> >> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> >> new file mode 100644
> >> >> >> index 0000000..69d8597
> >> >> >> --- /dev/null
> >> >> >> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >> >> >
> >> >> > Shouldn't we have a GPL header here?
> >> >> >
> >> >> >> @@ -0,0 +1,324 @@
> >> >> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> >> >> +#include <linux/kvm.h>
> >> >> >> +#include <linux/kvm_host.h>
> >> >> >> +#include <kvm/iodev.h>
> >> >> >> +#include <kvm/arm_vgic.h>
> >> >> >> +#include <asm/kvm_emulate.h>
> >> >> >> +#include <asm/kvm_arm.h>
> >> >> >> +#include <asm/kvm_mmu.h>
> >> >> >> +
> >> >> >> +#include "vgic.h"
> >> >> >> +#include "vgic-mmio.h"
> >> >> >> +#include "sys_regs.h"
> >> >> >> +
> >> >> >> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >> >> >> + const struct sys_reg_desc *r)
> >> >> >> +{
> >> >> >> + struct vgic_cpu *vgic_v3_cpu = &vcpu->arch.vgic_cpu;
> >> >> >> + struct vgic_vmcr vmcr;
> >> >> >> + u64 val;
> >> >> >> + u32 num_pri_bits, num_id_bits;
> >> >> >> +
> >> >> >> + vgic_get_vmcr(vcpu, &vmcr);
> >> >> >> + if (p->is_write) {
> >> >> >> + val = p->regval;
> >> >> >> +
> >> >> >> + /*
> >> >> >> + * Does not allow update of ICC_CTLR_EL1 if HW does not support
> >> >> >> + * guest programmed ID and PRI bits
> >> >> >> + */
> >> >> >
> >> >> > I would suggest rewording this comment:
> >> >> > Disallow restoring VM state not supported by this hardware.
> >> >> >
> >> >> >> + num_pri_bits = ((val & ICC_CTLR_EL1_PRI_BITS_MASK) >>
> >> >> >> + ICC_CTLR_EL1_PRI_BITS_SHIFT) + 1;
> >> >> >> + if (num_pri_bits > vgic_v3_cpu->num_pri_bits)
> >> >> >> + return false;
> >> >> >> +
> >> >> >> + vgic_v3_cpu->num_pri_bits = num_pri_bits;
> >> >> >
> >> >> > hmmm, this looks weird to me, because vgic_v3_cpu->num_pri_bits I don't
> >> >> > understand which effect this is intended to have?
> >> >> >
> >> >> > Sure, it may limit what you do with other registers later, but since
> >> >> > there's no ordering requirement that the ctlr be restored first, I'm not
> >> >> > sure it makes sense.
> >> >> >
> >> >> > Also, since this field is RO in the ICH_VTR, we'll have a strange
> >> >> > situation during runtime after a GICv3 restore where the
> >> >> > vgic_v3_cpu->num_pri_its differs from the hardware's ICH_VTR_EL2 field,
> >> >> > which is never the case if you didn't do a save/restore.
> >> >>
> >> >> Yes, but in any case, vgic_v3_cpu->num_pri_bits will be always less
> >> >> than HW supported
> >> >> value.
> >> >>
> >> >
> >> > So answer my question: What is the intended effect of writing this
> >> > value? Is it just so that if you migrate this platform back again, then
> >> > you're checking compatibility with what the guest would potentially do,
> >>
> >> Yes
> >
> > Then add a comment explaining that
> >
> >> and also to limit the valid aprn registers access as you said above.
> >> But that has ordering restriction. Which I think we should follow.
> >>
> >
> > I'm sorry, now I'm confused. Is there an ordering requirement in the
> > API, or how should we follow this?
>
> There is no ordering requirement mentioned in the API doc.
> However the APRn registers depends on num_pri_bits. Hence first
> ICC_CTLR_EL1 should be restored before APRn restore.
>
> If ordering is not followed then APRn registers restore is allowed
> as per hw supported num_pri_bits.
>
> This should be mentioned in doc.
How about just having a consistency check function that you call from
uaccess updates to both functions, and in that way avoid requireing any
ordering which is likely to not be followed etc.?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 62+ messages in thread
end of thread, other threads:[~2016-11-21 13:43 UTC | newest]
Thread overview: 62+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-04 11:13 [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-04 11:13 ` [PATCH v8 1/7] arm/arm64: vgic: Implement support for userspace access vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 18:52 ` Christoffer Dall
2016-11-17 11:26 ` Vijay Kilari
2016-11-17 11:26 ` Vijay Kilari
2016-11-17 11:40 ` Christoffer Dall
2016-11-17 11:40 ` Christoffer Dall
2016-11-04 11:13 ` [PATCH v8 2/7] arm/arm64: vgic: Add distributor and redistributor access vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 18:52 ` Christoffer Dall
2016-11-04 11:13 ` [PATCH v8 3/7] arm/arm64: vgic: Introduce find_reg_by_id() vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-04 11:13 ` [PATCH v8 4/7] irqchip/gic-v3: Add missing system register definitions vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-04 11:13 ` [PATCH v8 5/7] arm/arm64: vgic: Introduce VENG0 and VENG1 fields to vmcr struct vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 18:52 ` Christoffer Dall
2016-11-17 12:42 ` Vijay Kilari
2016-11-17 12:42 ` Vijay Kilari
2016-11-17 16:01 ` Christoffer Dall
2016-11-17 16:01 ` Christoffer Dall
2016-11-04 11:13 ` [PATCH v8 6/7] arm/arm64: vgic: Implement VGICv3 CPU interface access vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 18:52 ` Christoffer Dall
2016-11-17 15:55 ` Vijay Kilari
2016-11-17 15:55 ` Vijay Kilari
2016-11-17 16:09 ` Christoffer Dall
2016-11-17 16:09 ` Christoffer Dall
2016-11-18 16:58 ` Vijay Kilari
2016-11-18 16:58 ` Vijay Kilari
2016-11-21 10:19 ` Christoffer Dall
2016-11-21 10:19 ` Christoffer Dall
2016-11-21 13:26 ` Vijay Kilari
2016-11-21 13:26 ` Vijay Kilari
2016-11-21 13:43 ` Christoffer Dall
2016-11-21 13:43 ` Christoffer Dall
2016-11-18 18:48 ` Vijay Kilari
2016-11-18 18:48 ` Vijay Kilari
2016-11-20 13:20 ` Christoffer Dall
2016-11-20 13:20 ` Christoffer Dall
2016-11-21 13:32 ` Vijay Kilari
2016-11-21 13:32 ` Vijay Kilari
2016-11-21 13:41 ` Christoffer Dall
2016-11-21 13:41 ` Christoffer Dall
2016-11-04 11:13 ` [PATCH v8 7/7] arm/arm64: vgic: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl vijay.kilari
2016-11-04 11:13 ` vijay.kilari at gmail.com
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 18:52 ` Christoffer Dall
2016-11-16 11:47 ` [PATCH v8 0/7] arm/arm64: vgic: Implement API for vGICv3 live migration Christoffer Dall
2016-11-16 11:47 ` Christoffer Dall
2016-11-16 14:54 ` Vijay Kilari
2016-11-16 14:54 ` Vijay Kilari
2016-11-16 15:11 ` Christoffer Dall
2016-11-16 15:11 ` Christoffer Dall
2016-11-17 11:41 ` Christoffer Dall
2016-11-17 11:41 ` Christoffer Dall
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.