[PATCH v5 0/6] arm/arm64: vgic-new: Implement API for vGICv3 live migration

All of lore.kernel.org
 help / color / mirror / Atom feed

* [PATCH v5 0/6] arm/arm64: vgic-new: Implement API for vGICv3 live migration
@ 2016-09-16 12:20 ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

This patchset adds API for saving and restoring
of VGICv3 registers to support live migration with new vgic feature.
This API definition is as per version of VGICv3 specification
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Compatible live migration QEMU patches will be sent later.

The patch 3 & 4 are picked from the Pavel's previous implementation.
http://www.spinics.net/lists/kvm/msg122040.html

v4 => v5:
 - ICC_CTLR_EL1 access is updated to reflect HW values
 - Updated ICC reg access mask and shift macros
 - Introduced patch 4 for VMCR changes
 - Other minor fixes.
v3 => v4:
 - Rebased to latest code base
 - Moved vgic_uaccess() from vgic-mmio-v2.c to vgic-mmio.c
 - Dropped macro REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED_UACCESS
 - Dropped LE conversion for userspace access
 - Introduced vgic_uaccess_write_pending() for ISPENDR write
 - Change macro KVM_DEV_ARM_VGIC_V3_CPUID_MASK to KVM_DEV_ARM_VGIC_V3_MIDR_MASK
 - Refactored some code as common code.
 - Changed handing of ICC_* registers
 - Allowed ICC_SRE_EL1 read by userspace
 - Fixed KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_* macros

v2 => v3:
 - Implemented separate API for ISPENDR and ICPENDR to
   read soft_pending instead of pending for level triggerred interrupts
 - Implemented ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO to access line level
 - Rebased on top of Christoffer's patch set
   http://www.spinics.net/lists/kvm/msg136840.html

 NOTE: GICD_STATUSR and GICR_STATUSR are implemented as RAZ/WI.

v1 => v2:
 - The init sequence change patch is no more required.
   Fixed in patch 2 by using static vgic_io_dev regions structure instead
   of using dynamic allocation pointer.
 - Updated commit message of patch 4.
 - Dropped usage of union to manage 32-bit and 64-bit access in patch 1.
   Used local variable for 32-bit access.
 - Updated macro __ARM64_SYS_REG and ARM64_SYS_REG in
   arch/arm64/include/uapi/asm/kvm.h as per qemu requirements.


Vijaya Kumar K (6):
  arm/arm64: vgic-new: Implement support for userspace access
  arm/arm64: vgic-new: Add distributor and redistributor access
  arm/arm64: vgic-new: Introduce find_reg_by_id()
  arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
  arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  arm/arm64: vgic-new: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl

 arch/arm64/include/uapi/asm/kvm.h   |  13 ++
 arch/arm64/kvm/Makefile             |   1 +
 arch/arm64/kvm/sys_regs.c           |  22 ++-
 arch/arm64/kvm/sys_regs.h           |   4 +
 include/linux/irqchip/arm-gic-v3.h  |  45 +++++-
 virt/kvm/arm/vgic/vgic-kvm-device.c | 226 +++++++++++++++++++++++++--
 virt/kvm/arm/vgic/vgic-mmio-v2.c    |  57 +------
 virt/kvm/arm/vgic/vgic-mmio-v3.c    | 199 +++++++++++++++++++++---
 virt/kvm/arm/vgic/vgic-mmio.c       | 149 +++++++++++++++++-
 virt/kvm/arm/vgic/vgic-mmio.h       |  28 ++++
 virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic-v3.c         |  10 +-
 virt/kvm/arm/vgic/vgic.h            |  23 +++
 13 files changed, 969 insertions(+), 104 deletions(-)
 create mode 100644 virt/kvm/arm/vgic/vgic-sys-reg-v3.c

-- 
1.9.1

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH v5 0/6] arm/arm64: vgic-new: Implement API for vGICv3 live migration
@ 2016-09-16 12:20 ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

This patchset adds API for saving and restoring
of VGICv3 registers to support live migration with new vgic feature.
This API definition is as per version of VGICv3 specification
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Compatible live migration QEMU patches will be sent later.

The patch 3 & 4 are picked from the Pavel's previous implementation.
http://www.spinics.net/lists/kvm/msg122040.html

v4 => v5:
 - ICC_CTLR_EL1 access is updated to reflect HW values
 - Updated ICC reg access mask and shift macros
 - Introduced patch 4 for VMCR changes
 - Other minor fixes.
v3 => v4:
 - Rebased to latest code base
 - Moved vgic_uaccess() from vgic-mmio-v2.c to vgic-mmio.c
 - Dropped macro REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED_UACCESS
 - Dropped LE conversion for userspace access
 - Introduced vgic_uaccess_write_pending() for ISPENDR write
 - Change macro KVM_DEV_ARM_VGIC_V3_CPUID_MASK to KVM_DEV_ARM_VGIC_V3_MIDR_MASK
 - Refactored some code as common code.
 - Changed handing of ICC_* registers
 - Allowed ICC_SRE_EL1 read by userspace
 - Fixed KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_* macros

v2 => v3:
 - Implemented separate API for ISPENDR and ICPENDR to
   read soft_pending instead of pending for level triggerred interrupts
 - Implemented ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO to access line level
 - Rebased on top of Christoffer's patch set
   http://www.spinics.net/lists/kvm/msg136840.html

 NOTE: GICD_STATUSR and GICR_STATUSR are implemented as RAZ/WI.

v1 => v2:
 - The init sequence change patch is no more required.
   Fixed in patch 2 by using static vgic_io_dev regions structure instead
   of using dynamic allocation pointer.
 - Updated commit message of patch 4.
 - Dropped usage of union to manage 32-bit and 64-bit access in patch 1.
   Used local variable for 32-bit access.
 - Updated macro __ARM64_SYS_REG and ARM64_SYS_REG in
   arch/arm64/include/uapi/asm/kvm.h as per qemu requirements.


Vijaya Kumar K (6):
  arm/arm64: vgic-new: Implement support for userspace access
  arm/arm64: vgic-new: Add distributor and redistributor access
  arm/arm64: vgic-new: Introduce find_reg_by_id()
  arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
  arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  arm/arm64: vgic-new: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl

 arch/arm64/include/uapi/asm/kvm.h   |  13 ++
 arch/arm64/kvm/Makefile             |   1 +
 arch/arm64/kvm/sys_regs.c           |  22 ++-
 arch/arm64/kvm/sys_regs.h           |   4 +
 include/linux/irqchip/arm-gic-v3.h  |  45 +++++-
 virt/kvm/arm/vgic/vgic-kvm-device.c | 226 +++++++++++++++++++++++++--
 virt/kvm/arm/vgic/vgic-mmio-v2.c    |  57 +------
 virt/kvm/arm/vgic/vgic-mmio-v3.c    | 199 +++++++++++++++++++++---
 virt/kvm/arm/vgic/vgic-mmio.c       | 149 +++++++++++++++++-
 virt/kvm/arm/vgic/vgic-mmio.h       |  28 ++++
 virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic-v3.c         |  10 +-
 virt/kvm/arm/vgic/vgic.h            |  23 +++
 13 files changed, 969 insertions(+), 104 deletions(-)
 create mode 100644 virt/kvm/arm/vgic/vgic-sys-reg-v3.c

-- 
1.9.1

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH v5 1/6] arm/arm64: vgic-new: Implement support for userspace access
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

Read and write of some registers like ISPENDR and ICPENDR
from userspace requires special handling when compared to
guest access for these registers.

Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
for handling of ISPENDR, ICPENDR registers handling.

Add infrastructure to support guest and userspace read
and write for the required registers
Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
 virt/kvm/arm/vgic/vgic-mmio.c    | 78 ++++++++++++++++++++++++++++----
 virt/kvm/arm/vgic/vgic-mmio.h    | 19 ++++++++
 4 files changed, 169 insertions(+), 51 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index b44b359..0b32f40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 	return -ENXIO;
 }
 
-/*
- * When userland tries to access the VGIC register handlers, we need to
- * create a usable struct vgic_io_device to be passed to the handlers and we
- * have to set up a buffer similar to what would have happened if a guest MMIO
- * access occurred, including doing endian conversions on BE systems.
- */
-static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
-			bool is_write, int offset, u32 *val)
-{
-	unsigned int len = 4;
-	u8 buf[4];
-	int ret;
-
-	if (is_write) {
-		vgic_data_host_to_mmio_bus(buf, len, *val);
-		ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
-	} else {
-		ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
-		if (!ret)
-			*val = vgic_data_mmio_bus_to_host(buf, len);
-	}
-
-	return ret;
-}
-
 int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			  int offset, u32 *val)
 {
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 0d3c76a..edd3d40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static unsigned long vgic_uaccess_read_v3_pending(struct kvm_vcpu *vcpu,
+						  gpa_t addr, unsigned int len)
+{
+	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+	u32 value = 0;
+	int i;
+
+	/*
+	 * A level triggerred interrupt pending state is latched in both
+	 * "soft_pending" and "line_level" variables. Userspace will save
+	 * and restore soft_pending and line_level separately.
+	 * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
+	 * handling of ISPENDR and ICPENDR.
+	 */
+	for (i = 0; i < len * 8; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
+			value |= (1U << i);
+		if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
+			value |= (1U << i);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+
+	return value;
+}
+
+static void vgic_uaccess_write_v3_pending(struct kvm_vcpu *vcpu,
+					  gpa_t addr, unsigned int len,
+					  unsigned long val)
+{
+	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+	int i;
+
+	for (i = 0; i < len * 8; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		spin_lock(&irq->irq_lock);
+		if (test_bit(i, &val)) {
+			irq->pending = true;
+			irq->soft_pending = true;
+			vgic_queue_irq_unlock(vcpu->kvm, irq);
+		} else {
+			irq->soft_pending = false;
+			if (irq->config == VGIC_CONFIG_EDGE ||
+			    (irq->config == VGIC_CONFIG_LEVEL &&
+			    !irq->line_level))
+				irq->pending = false;
+			spin_unlock(&irq->irq_lock);
+		}
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+}
+
 /* We want to avoid outer shareable. */
 u64 vgic_sanitise_shareability(u64 field)
 {
@@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
  * We take some special care here to fix the calculation of the register
  * offset.
  */
-#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc)	\
+#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
 	{								\
 		.reg_offset = off,					\
 		.bits_per_irq = bpi,					\
@@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
 		.access_flags = acc,					\
 		.read = rd,						\
 		.write = wr,						\
+		.uaccess_read = ur,					\
+		.uaccess_write = uw,					\
 	}
 
 static const struct vgic_register_region vgic_v3_dist_registers[] = {
@@ -380,40 +438,42 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
 		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
-		vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
+		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
-		vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
+		vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
-		vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
+		vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
-		vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
+		vgic_mmio_read_pending, vgic_mmio_write_spending,
+		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
-		vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
+		vgic_mmio_read_pending, vgic_mmio_write_cpending,
+		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
-		vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
+		vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
-		vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
+		vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
-		vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
-		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+		vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
+		8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
-		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
 		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
-		vgic_mmio_read_config, vgic_mmio_write_config, 2,
+		vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
-		vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
-		vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
+		vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
 		vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
@@ -451,11 +511,13 @@ static const struct vgic_register_region vgic_v3_sgibase_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
 		vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
 		VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
-		vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
+	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
+		vgic_mmio_read_pending, vgic_mmio_write_spending,
+		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 4,
 		VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
-		vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
+	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
+		vgic_mmio_read_pending, vgic_mmio_write_cpending,
+		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 4,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
 		vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index e18b30d..31f85df 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
 	return false;
 }
 
+static const struct vgic_register_region *
+vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
+{
+	const struct vgic_register_region *region;
+
+	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
+				       addr - iodev->base_addr);
+	if (!region || !check_region(region, addr, len))
+		return NULL;
+
+	return region;
+}
+
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			     gpa_t addr, u32 *val)
+{
+	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+	const struct vgic_register_region *region;
+	struct kvm_vcpu *r_vcpu;
+
+	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+	if (!region) {
+		*val = 0;
+		return 0;
+	}
+
+	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+	if (region->uaccess_read)
+		*val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
+	else
+		*val = region->read(r_vcpu, addr, sizeof(u32));
+
+	return 0;
+}
+
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			      gpa_t addr, const u32 *val)
+{
+	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+	const struct vgic_register_region *region;
+	struct kvm_vcpu *r_vcpu;
+
+	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+	if (!region)
+		return 0;
+
+	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+	if (region->uaccess_write)
+		region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
+	else
+		region->write(r_vcpu, addr, sizeof(u32), *val);
+
+	return 0;
+}
+
+/*
+ * Userland access to VGIC registers.
+ */
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+		 bool is_write, int offset, u32 *val)
+{
+	if (is_write)
+		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+	else
+		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+}
+
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, void *val)
 {
@@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	const struct vgic_register_region *region;
 	unsigned long data = 0;
 
-	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
-				       addr - iodev->base_addr);
-	if (!region || !check_region(region, addr, len)) {
+	region = vgic_get_mmio_region(iodev, addr, len);
+	if (!region) {
 		memset(val, 0, len);
 		return 0;
 	}
@@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	const struct vgic_register_region *region;
 	unsigned long data = vgic_data_mmio_bus_to_host(val, len);
 
-	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
-				       addr - iodev->base_addr);
+	region = vgic_get_mmio_region(iodev, addr, len);
 	if (!region)
 		return 0;
 
-	if (!check_region(region, addr, len))
-		return 0;
-
 	switch (iodev->iodev_type) {
 	case IODEV_CPUIF:
 		region->write(vcpu, addr, len, data);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 4c34d39..97e6df7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -34,6 +34,10 @@ struct vgic_register_region {
 				  gpa_t addr, unsigned int len,
 				  unsigned long val);
 	};
+	unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
+				      unsigned int len);
+	void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
+			      unsigned int len, unsigned long val);
 };
 
 extern struct kvm_io_device_ops kvm_io_gic_ops;
@@ -86,6 +90,18 @@ extern struct kvm_io_device_ops kvm_io_gic_ops;
 		.write = wr,						\
 	}
 
+#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
+	{								\
+		.reg_offset = off,					\
+		.bits_per_irq = 0,					\
+		.len = length,						\
+		.access_flags = acc,					\
+		.read = rd,						\
+		.write = wr,						\
+		.uaccess_read = urd,					\
+		.uaccess_write = uwr,					\
+	}
+
 int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
 				  struct vgic_register_region *reg_desc,
 				  struct vgic_io_device *region,
@@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 			    gpa_t addr, unsigned int len,
 			    unsigned long val);
 
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+		 bool is_write, int offset, u32 *val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 1/6] arm/arm64: vgic-new: Implement support for userspace access
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

Read and write of some registers like ISPENDR and ICPENDR
from userspace requires special handling when compared to
guest access for these registers.

Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
for handling of ISPENDR, ICPENDR registers handling.

Add infrastructure to support guest and userspace read
and write for the required registers
Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
 virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
 virt/kvm/arm/vgic/vgic-mmio.c    | 78 ++++++++++++++++++++++++++++----
 virt/kvm/arm/vgic/vgic-mmio.h    | 19 ++++++++
 4 files changed, 169 insertions(+), 51 deletions(-)

diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index b44b359..0b32f40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 	return -ENXIO;
 }
 
-/*
- * When userland tries to access the VGIC register handlers, we need to
- * create a usable struct vgic_io_device to be passed to the handlers and we
- * have to set up a buffer similar to what would have happened if a guest MMIO
- * access occurred, including doing endian conversions on BE systems.
- */
-static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
-			bool is_write, int offset, u32 *val)
-{
-	unsigned int len = 4;
-	u8 buf[4];
-	int ret;
-
-	if (is_write) {
-		vgic_data_host_to_mmio_bus(buf, len, *val);
-		ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
-	} else {
-		ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
-		if (!ret)
-			*val = vgic_data_mmio_bus_to_host(buf, len);
-	}
-
-	return ret;
-}
-
 int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			  int offset, u32 *val)
 {
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 0d3c76a..edd3d40 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+static unsigned long vgic_uaccess_read_v3_pending(struct kvm_vcpu *vcpu,
+						  gpa_t addr, unsigned int len)
+{
+	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+	u32 value = 0;
+	int i;
+
+	/*
+	 * A level triggerred interrupt pending state is latched in both
+	 * "soft_pending" and "line_level" variables. Userspace will save
+	 * and restore soft_pending and line_level separately.
+	 * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
+	 * handling of ISPENDR and ICPENDR.
+	 */
+	for (i = 0; i < len * 8; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
+			value |= (1U << i);
+		if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
+			value |= (1U << i);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+
+	return value;
+}
+
+static void vgic_uaccess_write_v3_pending(struct kvm_vcpu *vcpu,
+					  gpa_t addr, unsigned int len,
+					  unsigned long val)
+{
+	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
+	int i;
+
+	for (i = 0; i < len * 8; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		spin_lock(&irq->irq_lock);
+		if (test_bit(i, &val)) {
+			irq->pending = true;
+			irq->soft_pending = true;
+			vgic_queue_irq_unlock(vcpu->kvm, irq);
+		} else {
+			irq->soft_pending = false;
+			if (irq->config == VGIC_CONFIG_EDGE ||
+			    (irq->config == VGIC_CONFIG_LEVEL &&
+			    !irq->line_level))
+				irq->pending = false;
+			spin_unlock(&irq->irq_lock);
+		}
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+}
+
 /* We want to avoid outer shareable. */
 u64 vgic_sanitise_shareability(u64 field)
 {
@@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
  * We take some special care here to fix the calculation of the register
  * offset.
  */
-#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc)	\
+#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
 	{								\
 		.reg_offset = off,					\
 		.bits_per_irq = bpi,					\
@@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
 		.access_flags = acc,					\
 		.read = rd,						\
 		.write = wr,						\
+		.uaccess_read = ur,					\
+		.uaccess_write = uw,					\
 	}
 
 static const struct vgic_register_region vgic_v3_dist_registers[] = {
@@ -380,40 +438,42 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
 		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
-		vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
+		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
-		vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
+		vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
-		vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
+		vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
-		vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
+		vgic_mmio_read_pending, vgic_mmio_write_spending,
+		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
-		vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
+		vgic_mmio_read_pending, vgic_mmio_write_cpending,
+		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
-		vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
+		vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
-		vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
+		vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
-		vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
-		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
+		vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
+		8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
-		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
 		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
-		vgic_mmio_read_config, vgic_mmio_write_config, 2,
+		vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
-		vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
-		vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
+		vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
 		vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
@@ -451,11 +511,13 @@ static const struct vgic_register_region vgic_v3_sgibase_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
 		vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
 		VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
-		vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
+	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
+		vgic_mmio_read_pending, vgic_mmio_write_spending,
+		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 4,
 		VGIC_ACCESS_32bit),
-	REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
-		vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
+	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
+		vgic_mmio_read_pending, vgic_mmio_write_cpending,
+		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 4,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
 		vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index e18b30d..31f85df 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
 	return false;
 }
 
+static const struct vgic_register_region *
+vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
+{
+	const struct vgic_register_region *region;
+
+	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
+				       addr - iodev->base_addr);
+	if (!region || !check_region(region, addr, len))
+		return NULL;
+
+	return region;
+}
+
+static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			     gpa_t addr, u32 *val)
+{
+	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+	const struct vgic_register_region *region;
+	struct kvm_vcpu *r_vcpu;
+
+	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+	if (!region) {
+		*val = 0;
+		return 0;
+	}
+
+	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+	if (region->uaccess_read)
+		*val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
+	else
+		*val = region->read(r_vcpu, addr, sizeof(u32));
+
+	return 0;
+}
+
+static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
+			      gpa_t addr, const u32 *val)
+{
+	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
+	const struct vgic_register_region *region;
+	struct kvm_vcpu *r_vcpu;
+
+	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
+	if (!region)
+		return 0;
+
+	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
+	if (region->uaccess_write)
+		region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
+	else
+		region->write(r_vcpu, addr, sizeof(u32), *val);
+
+	return 0;
+}
+
+/*
+ * Userland access to VGIC registers.
+ */
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+		 bool is_write, int offset, u32 *val)
+{
+	if (is_write)
+		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
+	else
+		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
+}
+
 static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, void *val)
 {
@@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	const struct vgic_register_region *region;
 	unsigned long data = 0;
 
-	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
-				       addr - iodev->base_addr);
-	if (!region || !check_region(region, addr, len)) {
+	region = vgic_get_mmio_region(iodev, addr, len);
+	if (!region) {
 		memset(val, 0, len);
 		return 0;
 	}
@@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 	const struct vgic_register_region *region;
 	unsigned long data = vgic_data_mmio_bus_to_host(val, len);
 
-	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
-				       addr - iodev->base_addr);
+	region = vgic_get_mmio_region(iodev, addr, len);
 	if (!region)
 		return 0;
 
-	if (!check_region(region, addr, len))
-		return 0;
-
 	switch (iodev->iodev_type) {
 	case IODEV_CPUIF:
 		region->write(vcpu, addr, len, data);
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 4c34d39..97e6df7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -34,6 +34,10 @@ struct vgic_register_region {
 				  gpa_t addr, unsigned int len,
 				  unsigned long val);
 	};
+	unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
+				      unsigned int len);
+	void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
+			      unsigned int len, unsigned long val);
 };
 
 extern struct kvm_io_device_ops kvm_io_gic_ops;
@@ -86,6 +90,18 @@ extern struct kvm_io_device_ops kvm_io_gic_ops;
 		.write = wr,						\
 	}
 
+#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
+	{								\
+		.reg_offset = off,					\
+		.bits_per_irq = 0,					\
+		.len = length,						\
+		.access_flags = acc,					\
+		.read = rd,						\
+		.write = wr,						\
+		.uaccess_read = urd,					\
+		.uaccess_write = uwr,					\
+	}
+
 int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
 				  struct vgic_register_region *reg_desc,
 				  struct vgic_io_device *region,
@@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 			    gpa_t addr, unsigned int len,
 			    unsigned long val);
 
+int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
+		 bool is_write, int offset, u32 *val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 2/6] arm/arm64: vgic-new: Add distributor and redistributor access
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

VGICv3 Distributor and Redistributor registers are accessed using
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
These registers are accessed as 32-bit and cpu mpidr
value passed along with register offset is used to identify the
cpu for redistributor registers access.

The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |   4 +
 virt/kvm/arm/vgic/vgic-kvm-device.c | 151 +++++++++++++++++++++++++++++++++---
 virt/kvm/arm/vgic/vgic-mmio-v2.c    |  16 +---
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |  72 +++++++++++++++++
 virt/kvm/arm/vgic/vgic-mmio.c       |  22 ++++++
 virt/kvm/arm/vgic/vgic-mmio.h       |   4 +
 virt/kvm/arm/vgic/vgic.h            |   5 ++
 7 files changed, 250 insertions(+), 24 deletions(-)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 3051f86..56dc08d 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
 #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS	2
 #define   KVM_DEV_ARM_VGIC_CPUID_SHIFT	32
 #define   KVM_DEV_ARM_VGIC_CPUID_MASK	(0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define   KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
+#define   KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
+			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
+#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
 /* Device Control API on vcpu fd */
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index ce1f4ed..a4656fc 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -237,7 +237,7 @@ struct vgic_reg_attr {
 	gpa_t addr;
 };
 
-static int parse_vgic_v2_attr(struct kvm_device *dev,
+static int vgic_v2_parse_attr(struct kvm_device *dev,
 			      struct kvm_device_attr *attr,
 			      struct vgic_reg_attr *reg_attr)
 {
@@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
 }
 
 /**
- * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
+ * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
  *
  * @dev:      kvm device handle
  * @attr:     kvm device attribute
  * @reg:      address the value is read or written
  * @is_write: true if userspace is writing a register
  */
-static int vgic_attr_regs_access_v2(struct kvm_device *dev,
+static int vgic_v2_attr_regs_access(struct kvm_device *dev,
 				    struct kvm_device_attr *attr,
 				    u32 *reg, bool is_write)
 {
@@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
 	struct kvm_vcpu *vcpu;
 	int ret;
 
-	ret = parse_vgic_v2_attr(dev, attr, &reg_attr);
+	ret = vgic_v2_parse_attr(dev, attr, &reg_attr);
 	if (ret)
 		return ret;
 
@@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
 
 	mutex_lock(&dev->kvm->lock);
 
-	ret = vgic_init(dev->kvm);
-	if (ret)
+	if (unlikely(!vgic_initialized(dev->kvm))) {
+		ret = -EBUSY;
 		goto out;
+	}
 
 	if (!lock_all_vcpus(dev->kvm)) {
 		ret = -EBUSY;
@@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
 		if (get_user(reg, uaddr))
 			return -EFAULT;
 
-		return vgic_attr_regs_access_v2(dev, attr, &reg, true);
+		return vgic_v2_attr_regs_access(dev, attr, &reg, true);
 	}
 	}
 
@@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
 		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
 		u32 reg = 0;
 
-		ret = vgic_attr_regs_access_v2(dev, attr, &reg, false);
+		ret = vgic_v2_attr_regs_access(dev, attr, &reg, false);
 		if (ret)
 			return ret;
 		return put_user(reg, uaddr);
@@ -430,16 +431,143 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 	.has_attr = vgic_v2_has_attr,
 };
 
+static int vgic_v3_parse_attr(struct kvm_device *dev,
+			      struct kvm_device_attr *attr,
+			      struct vgic_reg_attr *reg_attr)
+{
+	unsigned long mpidr;
+
+	mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+		 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+	reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
+	if (!reg_attr->vcpu)
+		return -EINVAL;
+
+	if (reg_attr->vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
+		return -EINVAL;
+
+	reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+	return 0;
+}
+
+/*
+ * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
+ *
+ * @dev:      kvm device handle
+ * @attr:     kvm device attribute
+ * @reg:      address the value is read or written
+ * @is_write: true if userspace is writing a register
+ */
+static int vgic_v3_attr_regs_access(struct kvm_device *dev,
+				    struct kvm_device_attr *attr,
+				    u64 *reg, bool is_write)
+{
+	struct vgic_reg_attr reg_attr;
+	gpa_t addr;
+	struct kvm_vcpu *vcpu;
+	int ret;
+	u32 tmp32;
+
+	ret = vgic_v3_parse_attr(dev, attr, &reg_attr);
+	if (ret)
+		return ret;
+
+	vcpu = reg_attr.vcpu;
+	addr = reg_attr.addr;
+
+	mutex_lock(&dev->kvm->lock);
+
+	if (unlikely(!vgic_initialized(dev->kvm))) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	if (!lock_all_vcpus(dev->kvm)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+		if (is_write)
+			tmp32 = *reg;
+
+		ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
+		if (!is_write)
+			*reg = tmp32;
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+		if (is_write)
+			tmp32 = *reg;
+
+		ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
+		if (!is_write)
+			*reg = tmp32;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	unlock_all_vcpus(dev->kvm);
+out:
+	mutex_unlock(&dev->kvm->lock);
+	return ret;
+}
+
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
-	return vgic_set_common_attr(dev, attr);
+	int ret;
+
+	ret = vgic_set_common_attr(dev, attr);
+	if (ret != -ENXIO)
+		return ret;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u32 tmp32;
+		u64 reg;
+
+		if (get_user(tmp32, uaddr))
+			return -EFAULT;
+
+		reg = tmp32;
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
+	}
+	return -ENXIO;
 }
 
 static int vgic_v3_get_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
-	return vgic_get_common_attr(dev, attr);
+	int ret;
+
+	ret = vgic_get_common_attr(dev, attr);
+	if (ret != -ENXIO)
+		return ret;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
+
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		tmp32 = reg;
+		return put_user(tmp32, uaddr);
+	}
+	}
+
+	return -ENXIO;
 }
 
 static int vgic_v3_has_attr(struct kvm_device *dev,
@@ -453,6 +581,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 			return 0;
 		}
 		break;
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
 	case KVM_DEV_ARM_VGIC_GRP_CTRL:
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 0b32f40..2cb04b7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
 
 int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 {
-	int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
 	const struct vgic_register_region *regions;
 	gpa_t addr;
-	int nr_regions, i, len;
+	int nr_regions;
 
 	addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
 
@@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 	if (addr & 3)
 		return -ENXIO;
 
-	for (i = 0; i < nr_regions; i++) {
-		if (regions[i].bits_per_irq)
-			len = (regions[i].bits_per_irq * nr_irqs) / 8;
-		else
-			len = regions[i].len;
-
-		if (regions[i].reg_offset <= addr &&
-		    regions[i].reg_offset + len > addr)
-			return 0;
-	}
-
-	return -ENXIO;
+	return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
 }
 
 int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index edd3d40..83dece8 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -18,6 +18,8 @@
 #include <kvm/arm_vgic.h>
 
 #include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
 
 #include "vgic.h"
 #include "vgic-mmio.h"
@@ -437,6 +439,9 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
 		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
 		VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
+		vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
 		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
@@ -484,12 +489,18 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
 		vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
 		VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
 		vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
 		vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
 		vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
@@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
 	return ret;
 }
 
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+	const struct vgic_register_region *regions;
+	gpa_t addr;
+	int nr_regions;
+
+	addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+		regions = vgic_v3_dist_registers;
+		nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
+		regions = vgic_v3_rdbase_registers;
+		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
+		break;
+	}
+	default:
+		return -ENXIO;
+	}
+
+	/* We only support aligned 32-bit accesses. */
+	if (addr & 3)
+		return -ENXIO;
+
+	return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
+}
 /*
  * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
  * generation register ICC_SGI1R_EL1) with a given VCPU.
@@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 		vgic_put_irq(vcpu->kvm, irq);
 	}
 }
+
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val)
+{
+	struct vgic_io_device dev = {
+		.regions = vgic_v3_dist_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
+	};
+
+	return vgic_uaccess(vcpu, &dev, is_write, offset, val);
+}
+
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			   int offset, u32 *val)
+{
+	struct vgic_io_device rd_dev = {
+		.regions = vgic_v3_rdbase_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
+	};
+
+	struct vgic_io_device sgi_dev = {
+		.regions = vgic_v3_sgibase_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
+	};
+
+	/* SGI_base is the next 64K frame after RD_base */
+	if (offset >= SZ_64K)
+		return vgic_uaccess(vcpu, &sgi_dev, is_write,
+				    offset - SZ_64K, val);
+	else
+		return vgic_uaccess(vcpu, &rd_dev, is_write,
+				    offset, val);
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 31f85df..9939d1d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -394,6 +394,28 @@ vgic_find_mmio_region(const struct vgic_register_region *region, int nr_regions,
 		       sizeof(region[0]), match_region);
 }
 
+/* Check if address falls within the region */
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+				   const struct vgic_register_region *regions,
+				   int nr_regions, gpa_t addr)
+{
+	int i, len;
+	int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+	for (i = 0; i < nr_regions; i++) {
+		if (regions[i].bits_per_irq)
+			len = (regions[i].bits_per_irq * nr_irqs) / 8;
+		else
+			len = regions[i].len;
+
+		if (regions[i].reg_offset <= addr &&
+		    regions[i].reg_offset + len > addr)
+			return 0;
+	}
+
+	return -ENXIO;
+}
+
 /*
  * kvm_mmio_read_buf() returns a value in a format where it can be converted
  * to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 97e6df7..acbf99e 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val);
 
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+				   const struct vgic_register_region *regions,
+				   int nr_regions, gpa_t addr);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9d9e014..0f74396 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -90,6 +90,11 @@ bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val);
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val);
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 2/6] arm/arm64: vgic-new: Add distributor and redistributor access
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

VGICv3 Distributor and Redistributor registers are accessed using
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_DIST_REGS
with KVM_SET_DEVICE_ATTR and KVM_GET_DEVICE_ATTR ioctls.
These registers are accessed as 32-bit and cpu mpidr
value passed along with register offset is used to identify the
cpu for redistributor registers access.

The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |   4 +
 virt/kvm/arm/vgic/vgic-kvm-device.c | 151 +++++++++++++++++++++++++++++++++---
 virt/kvm/arm/vgic/vgic-mmio-v2.c    |  16 +---
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |  72 +++++++++++++++++
 virt/kvm/arm/vgic/vgic-mmio.c       |  22 ++++++
 virt/kvm/arm/vgic/vgic-mmio.h       |   4 +
 virt/kvm/arm/vgic/vgic.h            |   5 ++
 7 files changed, 250 insertions(+), 24 deletions(-)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 3051f86..56dc08d 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -201,10 +201,14 @@ struct kvm_arch_memory_slot {
 #define KVM_DEV_ARM_VGIC_GRP_CPU_REGS	2
 #define   KVM_DEV_ARM_VGIC_CPUID_SHIFT	32
 #define   KVM_DEV_ARM_VGIC_CPUID_MASK	(0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define   KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT 32
+#define   KVM_DEV_ARM_VGIC_V3_MPIDR_MASK \
+			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
+#define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
 /* Device Control API on vcpu fd */
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index ce1f4ed..a4656fc 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -237,7 +237,7 @@ struct vgic_reg_attr {
 	gpa_t addr;
 };
 
-static int parse_vgic_v2_attr(struct kvm_device *dev,
+static int vgic_v2_parse_attr(struct kvm_device *dev,
 			      struct kvm_device_attr *attr,
 			      struct vgic_reg_attr *reg_attr)
 {
@@ -294,14 +294,14 @@ static bool lock_all_vcpus(struct kvm *kvm)
 }
 
 /**
- * vgic_attr_regs_access_v2 - allows user space to access VGIC v2 state
+ * vgic_v2_attr_regs_access - allows user space to access VGIC v2 state
  *
  * @dev:      kvm device handle
  * @attr:     kvm device attribute
  * @reg:      address the value is read or written
  * @is_write: true if userspace is writing a register
  */
-static int vgic_attr_regs_access_v2(struct kvm_device *dev,
+static int vgic_v2_attr_regs_access(struct kvm_device *dev,
 				    struct kvm_device_attr *attr,
 				    u32 *reg, bool is_write)
 {
@@ -310,7 +310,7 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
 	struct kvm_vcpu *vcpu;
 	int ret;
 
-	ret = parse_vgic_v2_attr(dev, attr, &reg_attr);
+	ret = vgic_v2_parse_attr(dev, attr, &reg_attr);
 	if (ret)
 		return ret;
 
@@ -319,9 +319,10 @@ static int vgic_attr_regs_access_v2(struct kvm_device *dev,
 
 	mutex_lock(&dev->kvm->lock);
 
-	ret = vgic_init(dev->kvm);
-	if (ret)
+	if (unlikely(!vgic_initialized(dev->kvm))) {
+		ret = -EBUSY;
 		goto out;
+	}
 
 	if (!lock_all_vcpus(dev->kvm)) {
 		ret = -EBUSY;
@@ -364,7 +365,7 @@ static int vgic_v2_set_attr(struct kvm_device *dev,
 		if (get_user(reg, uaddr))
 			return -EFAULT;
 
-		return vgic_attr_regs_access_v2(dev, attr, &reg, true);
+		return vgic_v2_attr_regs_access(dev, attr, &reg, true);
 	}
 	}
 
@@ -386,7 +387,7 @@ static int vgic_v2_get_attr(struct kvm_device *dev,
 		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
 		u32 reg = 0;
 
-		ret = vgic_attr_regs_access_v2(dev, attr, &reg, false);
+		ret = vgic_v2_attr_regs_access(dev, attr, &reg, false);
 		if (ret)
 			return ret;
 		return put_user(reg, uaddr);
@@ -430,16 +431,143 @@ struct kvm_device_ops kvm_arm_vgic_v2_ops = {
 	.has_attr = vgic_v2_has_attr,
 };
 
+static int vgic_v3_parse_attr(struct kvm_device *dev,
+			      struct kvm_device_attr *attr,
+			      struct vgic_reg_attr *reg_attr)
+{
+	unsigned long mpidr;
+
+	mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+		 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+	reg_attr->vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
+	if (!reg_attr->vcpu)
+		return -EINVAL;
+
+	if (reg_attr->vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
+		return -EINVAL;
+
+	reg_attr->addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+	return 0;
+}
+
+/*
+ * vgic_v3_attr_regs_access - allows user space to access VGIC v3 state
+ *
+ * @dev:      kvm device handle
+ * @attr:     kvm device attribute
+ * @reg:      address the value is read or written
+ * @is_write: true if userspace is writing a register
+ */
+static int vgic_v3_attr_regs_access(struct kvm_device *dev,
+				    struct kvm_device_attr *attr,
+				    u64 *reg, bool is_write)
+{
+	struct vgic_reg_attr reg_attr;
+	gpa_t addr;
+	struct kvm_vcpu *vcpu;
+	int ret;
+	u32 tmp32;
+
+	ret = vgic_v3_parse_attr(dev, attr, &reg_attr);
+	if (ret)
+		return ret;
+
+	vcpu = reg_attr.vcpu;
+	addr = reg_attr.addr;
+
+	mutex_lock(&dev->kvm->lock);
+
+	if (unlikely(!vgic_initialized(dev->kvm))) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	if (!lock_all_vcpus(dev->kvm)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+		if (is_write)
+			tmp32 = *reg;
+
+		ret = vgic_v3_dist_uaccess(vcpu, is_write, addr, &tmp32);
+		if (!is_write)
+			*reg = tmp32;
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+		if (is_write)
+			tmp32 = *reg;
+
+		ret = vgic_v3_redist_uaccess(vcpu, is_write, addr, &tmp32);
+		if (!is_write)
+			*reg = tmp32;
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	unlock_all_vcpus(dev->kvm);
+out:
+	mutex_unlock(&dev->kvm->lock);
+	return ret;
+}
+
 static int vgic_v3_set_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
-	return vgic_set_common_attr(dev, attr);
+	int ret;
+
+	ret = vgic_set_common_attr(dev, attr);
+	if (ret != -ENXIO)
+		return ret;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u32 tmp32;
+		u64 reg;
+
+		if (get_user(tmp32, uaddr))
+			return -EFAULT;
+
+		reg = tmp32;
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
+	}
+	return -ENXIO;
 }
 
 static int vgic_v3_get_attr(struct kvm_device *dev,
 			    struct kvm_device_attr *attr)
 {
-	return vgic_get_common_attr(dev, attr);
+	int ret;
+
+	ret = vgic_get_common_attr(dev, attr);
+	if (ret != -ENXIO)
+		return ret;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
+
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		tmp32 = reg;
+		return put_user(tmp32, uaddr);
+	}
+	}
+
+	return -ENXIO;
 }
 
 static int vgic_v3_has_attr(struct kvm_device *dev,
@@ -453,6 +581,9 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 			return 0;
 		}
 		break;
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
 	case KVM_DEV_ARM_VGIC_GRP_CTRL:
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 0b32f40..2cb04b7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -368,10 +368,9 @@ unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev)
 
 int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 {
-	int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
 	const struct vgic_register_region *regions;
 	gpa_t addr;
-	int nr_regions, i, len;
+	int nr_regions;
 
 	addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
 
@@ -392,18 +391,7 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 	if (addr & 3)
 		return -ENXIO;
 
-	for (i = 0; i < nr_regions; i++) {
-		if (regions[i].bits_per_irq)
-			len = (regions[i].bits_per_irq * nr_irqs) / 8;
-		else
-			len = regions[i].len;
-
-		if (regions[i].reg_offset <= addr &&
-		    regions[i].reg_offset + len > addr)
-			return 0;
-	}
-
-	return -ENXIO;
+	return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
 }
 
 int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index edd3d40..83dece8 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -18,6 +18,8 @@
 #include <kvm/arm_vgic.h>
 
 #include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
 
 #include "vgic.h"
 #include "vgic-mmio.h"
@@ -437,6 +439,9 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICD_CTLR,
 		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
 		VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICD_STATUSR,
+		vgic_mmio_read_rao, vgic_mmio_write_wi, 4,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
 		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
 		VGIC_ACCESS_32bit),
@@ -484,12 +489,18 @@ static const struct vgic_register_region vgic_v3_rdbase_registers[] = {
 	REGISTER_DESC_WITH_LENGTH(GICR_CTLR,
 		vgic_mmio_read_v3r_ctlr, vgic_mmio_write_v3r_ctlr, 4,
 		VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICR_STATUSR,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, 4,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_IIDR,
 		vgic_mmio_read_v3r_iidr, vgic_mmio_write_wi, 4,
 		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_TYPER,
 		vgic_mmio_read_v3r_typer, vgic_mmio_write_wi, 8,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
+	REGISTER_DESC_WITH_LENGTH(GICR_WAKER,
+		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
+		VGIC_ACCESS_32bit),
 	REGISTER_DESC_WITH_LENGTH(GICR_PROPBASER,
 		vgic_mmio_read_propbase, vgic_mmio_write_propbase, 8,
 		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
@@ -610,6 +621,34 @@ int vgic_register_redist_iodevs(struct kvm *kvm, gpa_t redist_base_address)
 	return ret;
 }
 
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+	const struct vgic_register_region *regions;
+	gpa_t addr;
+	int nr_regions;
+
+	addr = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+
+	switch (attr->group) {
+	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+		regions = vgic_v3_dist_registers;
+		nr_regions = ARRAY_SIZE(vgic_v3_dist_registers);
+		break;
+	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:{
+		regions = vgic_v3_rdbase_registers;
+		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
+		break;
+	}
+	default:
+		return -ENXIO;
+	}
+
+	/* We only support aligned 32-bit accesses. */
+	if (addr & 3)
+		return -ENXIO;
+
+	return vgic_validate_mmio_region_addr(dev, regions, nr_regions, addr);
+}
 /*
  * Compare a given affinity (level 1-3 and a level 0 mask, from the SGI
  * generation register ICC_SGI1R_EL1) with a given VCPU.
@@ -716,3 +755,36 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg)
 		vgic_put_irq(vcpu->kvm, irq);
 	}
 }
+
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val)
+{
+	struct vgic_io_device dev = {
+		.regions = vgic_v3_dist_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_dist_registers),
+	};
+
+	return vgic_uaccess(vcpu, &dev, is_write, offset, val);
+}
+
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			   int offset, u32 *val)
+{
+	struct vgic_io_device rd_dev = {
+		.regions = vgic_v3_rdbase_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers),
+	};
+
+	struct vgic_io_device sgi_dev = {
+		.regions = vgic_v3_sgibase_registers,
+		.nr_regions = ARRAY_SIZE(vgic_v3_sgibase_registers),
+	};
+
+	/* SGI_base is the next 64K frame after RD_base */
+	if (offset >= SZ_64K)
+		return vgic_uaccess(vcpu, &sgi_dev, is_write,
+				    offset - SZ_64K, val);
+	else
+		return vgic_uaccess(vcpu, &rd_dev, is_write,
+				    offset, val);
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 31f85df..9939d1d 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -394,6 +394,28 @@ vgic_find_mmio_region(const struct vgic_register_region *region, int nr_regions,
 		       sizeof(region[0]), match_region);
 }
 
+/* Check if address falls within the region */
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+				   const struct vgic_register_region *regions,
+				   int nr_regions, gpa_t addr)
+{
+	int i, len;
+	int nr_irqs = dev->kvm->arch.vgic.nr_spis + VGIC_NR_PRIVATE_IRQS;
+
+	for (i = 0; i < nr_regions; i++) {
+		if (regions[i].bits_per_irq)
+			len = (regions[i].bits_per_irq * nr_irqs) / 8;
+		else
+			len = regions[i].len;
+
+		if (regions[i].reg_offset <= addr &&
+		    regions[i].reg_offset + len > addr)
+			return 0;
+	}
+
+	return -ENXIO;
+}
+
 /*
  * kvm_mmio_read_buf() returns a value in a format where it can be converted
  * to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index 97e6df7..acbf99e 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -177,6 +177,10 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
 		 bool is_write, int offset, u32 *val);
 
+int vgic_validate_mmio_region_addr(struct kvm_device *dev,
+				   const struct vgic_register_region *regions,
+				   int nr_regions, gpa_t addr);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9d9e014..0f74396 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -90,6 +90,11 @@ bool vgic_has_its(struct kvm *kvm);
 int kvm_vgic_register_its_device(void);
 void vgic_enable_lpis(struct kvm_vcpu *vcpu);
 int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi);
+int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr);
+int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val);
+int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 int offset, u32 *val);
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 3/6] arm/arm64: vgic-new: Introduce find_reg_by_id()
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

In order to implement vGICv3 CPU interface access, we will need to perform
table lookup of system registers. We would need both index_to_params() and
find_reg() exported for that purpose, but instead we export a single
function which combines them both.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
 arch/arm64/kvm/sys_regs.c | 22 +++++++++++++++-------
 arch/arm64/kvm/sys_regs.h |  4 ++++
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e51367d..87ebe35 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1794,6 +1794,17 @@ static bool index_to_params(u64 id, struct sys_reg_params *params)
 	}
 }
 
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+					  struct sys_reg_params *params,
+					  const struct sys_reg_desc table[],
+					  unsigned int num)
+{
+	if (!index_to_params(id, params))
+		return NULL;
+
+	return find_reg(params, table, num);
+}
+
 /* Decode an index value, and find the sys_reg_desc entry. */
 static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
 						    u64 id)
@@ -1921,10 +1932,8 @@ static int get_invariant_sys_reg(u64 id, void __user *uaddr)
 	struct sys_reg_params params;
 	const struct sys_reg_desc *r;
 
-	if (!index_to_params(id, &params))
-		return -ENOENT;
-
-	r = find_reg(&params, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+	r = find_reg_by_id(id, &params, invariant_sys_regs,
+			   ARRAY_SIZE(invariant_sys_regs));
 	if (!r)
 		return -ENOENT;
 
@@ -1938,9 +1947,8 @@ static int set_invariant_sys_reg(u64 id, void __user *uaddr)
 	int err;
 	u64 val = 0; /* Make sure high bits are 0 for 32-bit regs */
 
-	if (!index_to_params(id, &params))
-		return -ENOENT;
-	r = find_reg(&params, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+	r = find_reg_by_id(id, &params, invariant_sys_regs,
+			   ARRAY_SIZE(invariant_sys_regs));
 	if (!r)
 		return -ENOENT;
 
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index dbbb01c..9c6ffd0 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -136,6 +136,10 @@ static inline int cmp_sys_reg(const struct sys_reg_desc *i1,
 	return i1->Op2 - i2->Op2;
 }
 
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+					  struct sys_reg_params *params,
+					  const struct sys_reg_desc table[],
+					  unsigned int num);
 
 #define Op0(_x) 	.Op0 = _x
 #define Op1(_x) 	.Op1 = _x
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 3/6] arm/arm64: vgic-new: Introduce find_reg_by_id()
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

In order to implement vGICv3 CPU interface access, we will need to perform
table lookup of system registers. We would need both index_to_params() and
find_reg() exported for that purpose, but instead we export a single
function which combines them both.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
---
 arch/arm64/kvm/sys_regs.c | 22 +++++++++++++++-------
 arch/arm64/kvm/sys_regs.h |  4 ++++
 2 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index e51367d..87ebe35 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -1794,6 +1794,17 @@ static bool index_to_params(u64 id, struct sys_reg_params *params)
 	}
 }
 
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+					  struct sys_reg_params *params,
+					  const struct sys_reg_desc table[],
+					  unsigned int num)
+{
+	if (!index_to_params(id, params))
+		return NULL;
+
+	return find_reg(params, table, num);
+}
+
 /* Decode an index value, and find the sys_reg_desc entry. */
 static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
 						    u64 id)
@@ -1921,10 +1932,8 @@ static int get_invariant_sys_reg(u64 id, void __user *uaddr)
 	struct sys_reg_params params;
 	const struct sys_reg_desc *r;
 
-	if (!index_to_params(id, &params))
-		return -ENOENT;
-
-	r = find_reg(&params, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+	r = find_reg_by_id(id, &params, invariant_sys_regs,
+			   ARRAY_SIZE(invariant_sys_regs));
 	if (!r)
 		return -ENOENT;
 
@@ -1938,9 +1947,8 @@ static int set_invariant_sys_reg(u64 id, void __user *uaddr)
 	int err;
 	u64 val = 0; /* Make sure high bits are 0 for 32-bit regs */
 
-	if (!index_to_params(id, &params))
-		return -ENOENT;
-	r = find_reg(&params, invariant_sys_regs, ARRAY_SIZE(invariant_sys_regs));
+	r = find_reg_by_id(id, &params, invariant_sys_regs,
+			   ARRAY_SIZE(invariant_sys_regs));
 	if (!r)
 		return -ENOENT;
 
diff --git a/arch/arm64/kvm/sys_regs.h b/arch/arm64/kvm/sys_regs.h
index dbbb01c..9c6ffd0 100644
--- a/arch/arm64/kvm/sys_regs.h
+++ b/arch/arm64/kvm/sys_regs.h
@@ -136,6 +136,10 @@ static inline int cmp_sys_reg(const struct sys_reg_desc *i1,
 	return i1->Op2 - i2->Op2;
 }
 
+const struct sys_reg_desc *find_reg_by_id(u64 id,
+					  struct sys_reg_params *params,
+					  const struct sys_reg_desc table[],
+					  unsigned int num);
 
 #define Op0(_x) 	.Op0 = _x
 #define Op1(_x) 	.Op1 = _x
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 4/6] arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
variables to struct vmcr to support read and write of these fields.

ICH_VMCR_CTLR_MASK is changed to mask only ICC_CTLR_EL1 fields.
Also refactor vgic_set_vmcr and vgic_get_vmcr() code.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 include/linux/irqchip/arm-gic-v3.h | 15 +++++++++++----
 virt/kvm/arm/vgic/vgic-mmio-v2.c   | 16 ----------------
 virt/kvm/arm/vgic/vgic-mmio.c      | 16 ++++++++++++++++
 virt/kvm/arm/vgic/vgic-v3.c        | 10 ++++++++--
 virt/kvm/arm/vgic/vgic.h           |  5 +++++
 5 files changed, 40 insertions(+), 22 deletions(-)

diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 99ac022..88d83d3 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -352,8 +352,9 @@
 /*
  * CPU interface registers
  */
-#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << 1)
-#define ICC_CTLR_EL1_EOImode_drop	(1U << 1)
+#define ICC_CTLR_EL1_EOImode_SHIFT	(1)
+#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
 #define ICC_SRE_EL1_SRE			(1U << 0)
 
 /*
@@ -382,14 +383,20 @@
 #define ICH_HCR_EN			(1 << 0)
 #define ICH_HCR_UIE			(1 << 1)
 
-#define ICH_VMCR_CTLR_SHIFT		0
-#define ICH_VMCR_CTLR_MASK		(0x21f << ICH_VMCR_CTLR_SHIFT)
+#define ICH_VMCR_CBPR_SHIFT		4
+#define ICH_VMCR_CBPR_MASK		(1 << ICH_VMCR_CBPR_SHIFT)
+#define ICH_VMCR_EOIM_SHIFT		9
+#define ICH_VMCR_EOIM_MASK		(1 << ICH_VMCR_EOIM_SHIFT)
 #define ICH_VMCR_BPR1_SHIFT		18
 #define ICH_VMCR_BPR1_MASK		(7 << ICH_VMCR_BPR1_SHIFT)
 #define ICH_VMCR_BPR0_SHIFT		21
 #define ICH_VMCR_BPR0_MASK		(7 << ICH_VMCR_BPR0_SHIFT)
 #define ICH_VMCR_PMR_SHIFT		24
 #define ICH_VMCR_PMR_MASK		(0xffUL << ICH_VMCR_PMR_SHIFT)
+#define ICH_VMCR_ENG0_SHIFT		0
+#define ICH_VMCR_ENG0_MASK		(1 << ICH_VMCR_ENG0_SHIFT)
+#define ICH_VMCR_ENG1_SHIFT		1
+#define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
 
 #define ICC_IAR1_EL1_SPURIOUS		0x3ff
 
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 2cb04b7..ad353b5 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
 	}
 }
 
-static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-	if (kvm_vgic_global_state.type == VGIC_V2)
-		vgic_v2_set_vmcr(vcpu, vmcr);
-	else
-		vgic_v3_set_vmcr(vcpu, vmcr);
-}
-
-static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-	if (kvm_vgic_global_state.type == VGIC_V2)
-		vgic_v2_get_vmcr(vcpu, vmcr);
-	else
-		vgic_v3_get_vmcr(vcpu, vmcr);
-}
-
 #define GICC_ARCH_VERSION_V2	0x2
 
 /* These are for userland accesses only, there is no guest-facing emulation. */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 9939d1d..173d6f0 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
 	return -ENXIO;
 }
 
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+	if (kvm_vgic_global_state.type == VGIC_V2)
+		vgic_v2_set_vmcr(vcpu, vmcr);
+	else
+		vgic_v3_set_vmcr(vcpu, vmcr);
+}
+
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+	if (kvm_vgic_global_state.type == VGIC_V2)
+		vgic_v2_get_vmcr(vcpu, vmcr);
+	else
+		vgic_v3_get_vmcr(vcpu, vmcr);
+}
+
 /*
  * kvm_mmio_read_buf() returns a value in a format where it can be converted
  * to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 9f0dae3..967c295 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
 {
 	u32 vmcr;
 
-	vmcr  = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
+	vmcr  = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
+	vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
 	vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
 	vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
 	vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
+	vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
+	vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
 
 	vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
 }
@@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
 {
 	u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
 
-	vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
+	vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
+	vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
 	vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
 	vmcrp->bpr  = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
 	vmcrp->pmr  = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
+	vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
+	vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
 }
 
 #define INITIAL_PENDBASER_VALUE						  \
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 0f74396..9ee54e3 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -35,6 +35,9 @@ struct vgic_vmcr {
 	u32	abpr;
 	u32	bpr;
 	u32	pmr;
+	/* Below member variable are valid only for GICv3 */
+	u32	grpen0;
+	u32	grpen1;
 };
 
 struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
@@ -122,6 +125,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
 #endif
 
 int kvm_register_vgic_device(unsigned long type);
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
 int vgic_lazy_init(struct kvm *kvm);
 int vgic_init(struct kvm *kvm);
 
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 4/6] arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
variables to struct vmcr to support read and write of these fields.

ICH_VMCR_CTLR_MASK is changed to mask only ICC_CTLR_EL1 fields.
Also refactor vgic_set_vmcr and vgic_get_vmcr() code.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 include/linux/irqchip/arm-gic-v3.h | 15 +++++++++++----
 virt/kvm/arm/vgic/vgic-mmio-v2.c   | 16 ----------------
 virt/kvm/arm/vgic/vgic-mmio.c      | 16 ++++++++++++++++
 virt/kvm/arm/vgic/vgic-v3.c        | 10 ++++++++--
 virt/kvm/arm/vgic/vgic.h           |  5 +++++
 5 files changed, 40 insertions(+), 22 deletions(-)

diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 99ac022..88d83d3 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -352,8 +352,9 @@
 /*
  * CPU interface registers
  */
-#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << 1)
-#define ICC_CTLR_EL1_EOImode_drop	(1U << 1)
+#define ICC_CTLR_EL1_EOImode_SHIFT	(1)
+#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
 #define ICC_SRE_EL1_SRE			(1U << 0)
 
 /*
@@ -382,14 +383,20 @@
 #define ICH_HCR_EN			(1 << 0)
 #define ICH_HCR_UIE			(1 << 1)
 
-#define ICH_VMCR_CTLR_SHIFT		0
-#define ICH_VMCR_CTLR_MASK		(0x21f << ICH_VMCR_CTLR_SHIFT)
+#define ICH_VMCR_CBPR_SHIFT		4
+#define ICH_VMCR_CBPR_MASK		(1 << ICH_VMCR_CBPR_SHIFT)
+#define ICH_VMCR_EOIM_SHIFT		9
+#define ICH_VMCR_EOIM_MASK		(1 << ICH_VMCR_EOIM_SHIFT)
 #define ICH_VMCR_BPR1_SHIFT		18
 #define ICH_VMCR_BPR1_MASK		(7 << ICH_VMCR_BPR1_SHIFT)
 #define ICH_VMCR_BPR0_SHIFT		21
 #define ICH_VMCR_BPR0_MASK		(7 << ICH_VMCR_BPR0_SHIFT)
 #define ICH_VMCR_PMR_SHIFT		24
 #define ICH_VMCR_PMR_MASK		(0xffUL << ICH_VMCR_PMR_SHIFT)
+#define ICH_VMCR_ENG0_SHIFT		0
+#define ICH_VMCR_ENG0_MASK		(1 << ICH_VMCR_ENG0_SHIFT)
+#define ICH_VMCR_ENG1_SHIFT		1
+#define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
 
 #define ICC_IAR1_EL1_SPURIOUS		0x3ff
 
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
index 2cb04b7..ad353b5 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
@@ -212,22 +212,6 @@ static void vgic_mmio_write_sgipends(struct kvm_vcpu *vcpu,
 	}
 }
 
-static void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-	if (kvm_vgic_global_state.type == VGIC_V2)
-		vgic_v2_set_vmcr(vcpu, vmcr);
-	else
-		vgic_v3_set_vmcr(vcpu, vmcr);
-}
-
-static void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
-{
-	if (kvm_vgic_global_state.type == VGIC_V2)
-		vgic_v2_get_vmcr(vcpu, vmcr);
-	else
-		vgic_v3_get_vmcr(vcpu, vmcr);
-}
-
 #define GICC_ARCH_VERSION_V2	0x2
 
 /* These are for userland accesses only, there is no guest-facing emulation. */
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 9939d1d..173d6f0 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -416,6 +416,22 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
 	return -ENXIO;
 }
 
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+	if (kvm_vgic_global_state.type == VGIC_V2)
+		vgic_v2_set_vmcr(vcpu, vmcr);
+	else
+		vgic_v3_set_vmcr(vcpu, vmcr);
+}
+
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr)
+{
+	if (kvm_vgic_global_state.type == VGIC_V2)
+		vgic_v2_get_vmcr(vcpu, vmcr);
+	else
+		vgic_v3_get_vmcr(vcpu, vmcr);
+}
+
 /*
  * kvm_mmio_read_buf() returns a value in a format where it can be converted
  * to a byte array and be directly observed as the guest wanted it to appear
diff --git a/virt/kvm/arm/vgic/vgic-v3.c b/virt/kvm/arm/vgic/vgic-v3.c
index 9f0dae3..967c295 100644
--- a/virt/kvm/arm/vgic/vgic-v3.c
+++ b/virt/kvm/arm/vgic/vgic-v3.c
@@ -175,10 +175,13 @@ void vgic_v3_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
 {
 	u32 vmcr;
 
-	vmcr  = (vmcrp->ctlr << ICH_VMCR_CTLR_SHIFT) & ICH_VMCR_CTLR_MASK;
+	vmcr  = (vmcrp->ctlr << ICH_VMCR_CBPR_SHIFT) & ICH_VMCR_CBPR_MASK;
+	vmcr |= (vmcrp->ctlr << ICH_VMCR_EOIM_SHIFT) & ICH_VMCR_EOIM_MASK;
 	vmcr |= (vmcrp->abpr << ICH_VMCR_BPR1_SHIFT) & ICH_VMCR_BPR1_MASK;
 	vmcr |= (vmcrp->bpr << ICH_VMCR_BPR0_SHIFT) & ICH_VMCR_BPR0_MASK;
 	vmcr |= (vmcrp->pmr << ICH_VMCR_PMR_SHIFT) & ICH_VMCR_PMR_MASK;
+	vmcr |= (vmcrp->grpen0 << ICH_VMCR_ENG0_SHIFT) & ICH_VMCR_ENG0_MASK;
+	vmcr |= (vmcrp->grpen1 << ICH_VMCR_ENG1_SHIFT) & ICH_VMCR_ENG1_MASK;
 
 	vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr = vmcr;
 }
@@ -187,10 +190,13 @@ void vgic_v3_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcrp)
 {
 	u32 vmcr = vcpu->arch.vgic_cpu.vgic_v3.vgic_vmcr;
 
-	vmcrp->ctlr = (vmcr & ICH_VMCR_CTLR_MASK) >> ICH_VMCR_CTLR_SHIFT;
+	vmcrp->ctlr = (vmcr & ICH_VMCR_CBPR_MASK) >> ICH_VMCR_CBPR_SHIFT;
+	vmcrp->ctlr |= (vmcr & ICH_VMCR_EOIM_MASK) >> ICH_VMCR_EOIM_SHIFT;
 	vmcrp->abpr = (vmcr & ICH_VMCR_BPR1_MASK) >> ICH_VMCR_BPR1_SHIFT;
 	vmcrp->bpr  = (vmcr & ICH_VMCR_BPR0_MASK) >> ICH_VMCR_BPR0_SHIFT;
 	vmcrp->pmr  = (vmcr & ICH_VMCR_PMR_MASK) >> ICH_VMCR_PMR_SHIFT;
+	vmcrp->grpen0 = (vmcr & ICH_VMCR_ENG0_MASK) >> ICH_VMCR_ENG0_SHIFT;
+	vmcrp->grpen1 = (vmcr & ICH_VMCR_ENG1_MASK) >> ICH_VMCR_ENG1_SHIFT;
 }
 
 #define INITIAL_PENDBASER_VALUE						  \
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 0f74396..9ee54e3 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -35,6 +35,9 @@ struct vgic_vmcr {
 	u32	abpr;
 	u32	bpr;
 	u32	pmr;
+	/* Below member variable are valid only for GICv3 */
+	u32	grpen0;
+	u32	grpen1;
 };
 
 struct vgic_irq *vgic_get_irq(struct kvm *kvm, struct kvm_vcpu *vcpu,
@@ -122,6 +125,8 @@ static inline int vgic_its_inject_msi(struct kvm *kvm, struct kvm_msi *msi)
 #endif
 
 int kvm_register_vgic_device(unsigned long type);
+void vgic_set_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
+void vgic_get_vmcr(struct kvm_vcpu *vcpu, struct vgic_vmcr *vmcr);
 int vgic_lazy_init(struct kvm *kvm);
 int vgic_init(struct kvm *kvm);
 
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

VGICv3 CPU interface registers are accessed using
KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
as 64-bit. The cpu MPIDR value is passed along with register id.
is used to identify the cpu for registers access.

The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |   3 +
 arch/arm64/kvm/Makefile             |   1 +
 include/linux/irqchip/arm-gic-v3.h  |  30 ++++
 virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
 virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.h            |  10 ++
 7 files changed, 385 insertions(+)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 56dc08d..91c7137 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
 			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
+#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
 #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
+#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
+
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
 /* Device Control API on vcpu fd */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d50a82a..1a14e29 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
 kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 88d83d3..d4e9c7d 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -355,6 +355,27 @@
 #define ICC_CTLR_EL1_EOImode_SHIFT	(1)
 #define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
 #define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_MASK	(1 << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_CBPR_SHIFT		(0)
+#define ICC_CTLR_EL1_CBPR_MASK		(1 << ICC_CTLR_EL1_CBPR_SHIFT)
+#define ICC_CTLR_EL1_PRI_BITS_SHIFT	(8)
+#define ICC_CTLR_EL1_PRI_BITS_MASK	(0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
+#define ICC_CTLR_EL1_ID_BITS_SHIFT	(11)
+#define ICC_CTLR_EL1_ID_BITS_MASK	(0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
+#define ICC_CTLR_EL1_SEIS_SHIFT		(14)
+#define ICC_CTLR_EL1_SEIS_MASK		(0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
+#define ICC_CTLR_EL1_A3V_SHIFT		(15)
+#define ICC_CTLR_EL1_A3V_MASK		(0x1 << ICC_CTLR_EL1_A3V_SHIFT)
+#define ICC_PMR_EL1_SHIFT		(0)
+#define ICC_PMR_EL1_MASK		(0xff << ICC_PMR_EL1_SHIFT)
+#define ICC_BPR0_EL1_SHIFT		(0)
+#define ICC_BPR0_EL1_MASK		(0x7 << ICC_BPR0_EL1_SHIFT)
+#define ICC_BPR1_EL1_SHIFT		(0)
+#define ICC_BPR1_EL1_MASK		(0x7 << ICC_BPR1_EL1_SHIFT)
+#define ICC_IGRPEN0_EL1_SHIFT		(0)
+#define ICC_IGRPEN0_EL1_MASK		(1 << ICC_IGRPEN0_EL1_SHIFT)
+#define ICC_IGRPEN1_EL1_SHIFT		(0)
+#define ICC_IGRPEN1_EL1_MASK		(1 << ICC_IGRPEN1_EL1_SHIFT)
 #define ICC_SRE_EL1_SRE			(1U << 0)
 
 /*
@@ -398,6 +419,15 @@
 #define ICH_VMCR_ENG1_SHIFT		1
 #define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
 
+#define ICH_VTR_PRI_BITS_SHIFT		29
+#define ICH_VTR_PRI_BITS_MASK		(7 << ICH_VTR_PRI_BITS_SHIFT)
+#define ICH_VTR_ID_BITS_SHIFT		23
+#define ICH_VTR_ID_BITS_MASK		(7 << ICH_VTR_ID_BITS_SHIFT)
+#define ICH_VTR_SEIS_SHIFT		22
+#define ICH_VTR_SEIS_MASK		(1 << ICH_VTR_SEIS_SHIFT)
+#define ICH_VTR_A3V_SHIFT		21
+#define ICH_VTR_A3V_MASK		(1 << ICH_VTR_A3V_SHIFT)
+
 #define ICC_IAR1_EL1_SPURIOUS		0x3ff
 
 #define ICC_SRE_EL2_SRE			(1 << 0)
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index a4656fc..a48101f 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
 		if (!is_write)
 			*reg = tmp32;
 		break;
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 regid;
+
+		regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+		ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
+						  regid, reg);
+		break;
+	}
 	default:
 		ret = -EINVAL;
 		break;
@@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
 		reg = tmp32;
 		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+
+		if (get_user(reg, uaddr))
+			return -EFAULT;
+
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
 	}
 	return -ENXIO;
 }
@@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
 		tmp32 = reg;
 		return put_user(tmp32, uaddr);
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		return put_user(reg, uaddr);
+	}
 	}
 
 	return -ENXIO;
@@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 		break;
 	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
 	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
 		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 83dece8..af748d7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,6 +23,7 @@
 
 #include "vgic.h"
 #include "vgic-mmio.h"
+#include "sys_regs.h"
 
 /* extract @num bytes at @offset bytes offset in data */
 unsigned long extract_bytes(u64 data, unsigned int offset,
@@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
 		break;
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 reg, id;
+		unsigned long mpidr;
+		struct kvm_vcpu *vcpu;
+
+		mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+			 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+		vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
+		if (!vcpu)
+			return -EINVAL;
+		if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
+			return -EINVAL;
+
+		id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+		return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
+	}
 	default:
 		return -ENXIO;
 	}
diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
new file mode 100644
index 0000000..8e4f403
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
@@ -0,0 +1,296 @@
+#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <kvm/iodev.h>
+#include <kvm/arm_vgic.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+#include "sys_regs.h"
+
+static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+	u64 val;
+	u32 ich_vtr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		val = p->regval;
+		vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
+		vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
+			      ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
+		vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
+			     ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
+
+		val = 0;
+		val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
+			ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
+			ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
+			<< ICC_CTLR_EL1_SEIS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
+			<< ICC_CTLR_EL1_A3V_SHIFT;
+		val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
+			ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
+		val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
+			ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
+
+		p->regval = val;
+	}
+
+	return true;
+}
+
+static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			   const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
+			    ICC_BPR0_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
+			     ICC_BPR0_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
+			     ICC_BPR1_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
+			     ICC_BPR1_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			      const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
+				      ICC_IGRPEN0_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
+			     ICC_IGRPEN0_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			      const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
+				      ICC_IGRPEN1_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
+			     ICC_IGRPEN1_EL1_MASK;
+	}
+
+	return true;
+}
+
+static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
+				   struct sys_reg_params *p, u8 apr, u8 idx)
+{
+	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+	uint32_t *ap_reg;
+
+	if (apr)
+		ap_reg = &vgicv3->vgic_ap1r[idx];
+	else
+		ap_reg = &vgicv3->vgic_ap0r[idx];
+
+	if (p->is_write)
+		*ap_reg = p->regval;
+	else
+		p->regval = *ap_reg;
+}
+
+static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r, u8 apr)
+{
+	u8 num_pri_bits, idx;
+	u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
+
+
+	num_pri_bits = ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
+			ICH_VTR_PRI_BITS_SHIFT) + 1;
+	idx = r->Op2 & 3;
+
+	switch (num_pri_bits) {
+	case 7:
+		if (idx > 3)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+		break;
+	case 6:
+		if (idx > 1)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+		break;
+	default:
+		if (idx > 0)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+	}
+
+	return;
+err:
+	if (!p->is_write)
+		p->regval = 0;
+}
+
+static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	access_gic_aprn(vcpu, p, r, 0);
+
+	return true;
+}
+
+static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	access_gic_aprn(vcpu, p, r, 1);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			   const struct sys_reg_desc *r)
+{
+	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+
+	/* Read only. Write ignore */
+	if (!p->is_write)
+		p->regval = vgicv3->vgic_sre;
+
+	return true;
+}
+
+static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
+	/* ICC_PMR_EL1 */
+	{ Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
+	/* ICC_BPR0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
+	/* ICC_AP0R0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
+	/* ICC_AP0R1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
+	/* ICC_AP0R2_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
+	/* ICC_AP0R3_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
+	/* ICC_AP1R0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
+	/* ICC_AP1R1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
+	/* ICC_AP1R2_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
+	/* ICC_AP1R3_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
+	/* ICC_BPR1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
+	/* ICC_CTLR_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
+	/* ICC_SRE_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
+	/* ICC_IGRPEN0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
+	/* ICC_GRPEN1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
+};
+
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg)
+{
+	struct sys_reg_params params;
+	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+	params.regval = *reg;
+	params.is_write = is_write;
+	params.is_aarch32 = false;
+	params.is_32bit = false;
+
+	if (find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
+			      ARRAY_SIZE(gic_v3_icc_reg_descs)))
+		return 0;
+
+	return -ENXIO;
+}
+
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg)
+{
+	struct sys_reg_params params;
+	const struct sys_reg_desc *r;
+	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+	if (is_write)
+		params.regval = *reg;
+	params.is_write = is_write;
+	params.is_aarch32 = false;
+	params.is_32bit = false;
+
+	r = find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
+			   ARRAY_SIZE(gic_v3_icc_reg_descs));
+	if (!r)
+		return -ENXIO;
+
+	if (!r->access(vcpu, &params, r))
+		return -EINVAL;
+
+	if (!is_write)
+		*reg = params.regval;
+
+	return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9ee54e3..4c0076c 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -30,6 +30,12 @@
 
 #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
 
+#define   KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
+					KVM_REG_ARM64_SYSREG_OP1_MASK | \
+					KVM_REG_ARM64_SYSREG_CRN_MASK | \
+					KVM_REG_ARM64_SYSREG_CRM_MASK | \
+					KVM_REG_ARM64_SYSREG_OP2_MASK)
+
 struct vgic_vmcr {
 	u32	ctlr;
 	u32	abpr;
@@ -98,6 +104,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 int offset, u32 *val);
 int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 int offset, u32 *val);
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 u64 id, u64 *val);
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg);
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

VGICv3 CPU interface registers are accessed using
KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
as 64-bit. The cpu MPIDR value is passed along with register id.
is used to identify the cpu for registers access.

The version of VGIC v3 specification is define here
http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |   3 +
 arch/arm64/kvm/Makefile             |   1 +
 include/linux/irqchip/arm-gic-v3.h  |  30 ++++
 virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
 virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
 virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic.h            |  10 ++
 7 files changed, 385 insertions(+)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 56dc08d..91c7137 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
 			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
 #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
 #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
+#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
 #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
 #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
+#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
+
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
 /* Device Control API on vcpu fd */
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d50a82a..1a14e29 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
 kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
 kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 88d83d3..d4e9c7d 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -355,6 +355,27 @@
 #define ICC_CTLR_EL1_EOImode_SHIFT	(1)
 #define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
 #define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_EOImode_MASK	(1 << ICC_CTLR_EL1_EOImode_SHIFT)
+#define ICC_CTLR_EL1_CBPR_SHIFT		(0)
+#define ICC_CTLR_EL1_CBPR_MASK		(1 << ICC_CTLR_EL1_CBPR_SHIFT)
+#define ICC_CTLR_EL1_PRI_BITS_SHIFT	(8)
+#define ICC_CTLR_EL1_PRI_BITS_MASK	(0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
+#define ICC_CTLR_EL1_ID_BITS_SHIFT	(11)
+#define ICC_CTLR_EL1_ID_BITS_MASK	(0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
+#define ICC_CTLR_EL1_SEIS_SHIFT		(14)
+#define ICC_CTLR_EL1_SEIS_MASK		(0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
+#define ICC_CTLR_EL1_A3V_SHIFT		(15)
+#define ICC_CTLR_EL1_A3V_MASK		(0x1 << ICC_CTLR_EL1_A3V_SHIFT)
+#define ICC_PMR_EL1_SHIFT		(0)
+#define ICC_PMR_EL1_MASK		(0xff << ICC_PMR_EL1_SHIFT)
+#define ICC_BPR0_EL1_SHIFT		(0)
+#define ICC_BPR0_EL1_MASK		(0x7 << ICC_BPR0_EL1_SHIFT)
+#define ICC_BPR1_EL1_SHIFT		(0)
+#define ICC_BPR1_EL1_MASK		(0x7 << ICC_BPR1_EL1_SHIFT)
+#define ICC_IGRPEN0_EL1_SHIFT		(0)
+#define ICC_IGRPEN0_EL1_MASK		(1 << ICC_IGRPEN0_EL1_SHIFT)
+#define ICC_IGRPEN1_EL1_SHIFT		(0)
+#define ICC_IGRPEN1_EL1_MASK		(1 << ICC_IGRPEN1_EL1_SHIFT)
 #define ICC_SRE_EL1_SRE			(1U << 0)
 
 /*
@@ -398,6 +419,15 @@
 #define ICH_VMCR_ENG1_SHIFT		1
 #define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
 
+#define ICH_VTR_PRI_BITS_SHIFT		29
+#define ICH_VTR_PRI_BITS_MASK		(7 << ICH_VTR_PRI_BITS_SHIFT)
+#define ICH_VTR_ID_BITS_SHIFT		23
+#define ICH_VTR_ID_BITS_MASK		(7 << ICH_VTR_ID_BITS_SHIFT)
+#define ICH_VTR_SEIS_SHIFT		22
+#define ICH_VTR_SEIS_MASK		(1 << ICH_VTR_SEIS_SHIFT)
+#define ICH_VTR_A3V_SHIFT		21
+#define ICH_VTR_A3V_MASK		(1 << ICH_VTR_A3V_SHIFT)
+
 #define ICC_IAR1_EL1_SPURIOUS		0x3ff
 
 #define ICC_SRE_EL2_SRE			(1 << 0)
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index a4656fc..a48101f 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
 		if (!is_write)
 			*reg = tmp32;
 		break;
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 regid;
+
+		regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+		ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
+						  regid, reg);
+		break;
+	}
 	default:
 		ret = -EINVAL;
 		break;
@@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
 		reg = tmp32;
 		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+
+		if (get_user(reg, uaddr))
+			return -EFAULT;
+
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
 	}
 	return -ENXIO;
 }
@@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
 		tmp32 = reg;
 		return put_user(tmp32, uaddr);
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+		u64 reg;
+
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		return put_user(reg, uaddr);
+	}
 	}
 
 	return -ENXIO;
@@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 		break;
 	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
 	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
 		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 83dece8..af748d7 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -23,6 +23,7 @@
 
 #include "vgic.h"
 #include "vgic-mmio.h"
+#include "sys_regs.h"
 
 /* extract @num bytes at @offset bytes offset in data */
 unsigned long extract_bytes(u64 data, unsigned int offset,
@@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
 		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
 		break;
 	}
+	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
+		u64 reg, id;
+		unsigned long mpidr;
+		struct kvm_vcpu *vcpu;
+
+		mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
+			 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
+
+		vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
+		if (!vcpu)
+			return -EINVAL;
+		if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
+			return -EINVAL;
+
+		id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
+		return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
+	}
 	default:
 		return -ENXIO;
 	}
diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
new file mode 100644
index 0000000..8e4f403
--- /dev/null
+++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
@@ -0,0 +1,296 @@
+#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/kvm.h>
+#include <linux/kvm_host.h>
+#include <kvm/iodev.h>
+#include <kvm/arm_vgic.h>
+#include <asm/kvm_emulate.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_mmu.h>
+
+#include "vgic.h"
+#include "vgic-mmio.h"
+#include "sys_regs.h"
+
+static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+	u64 val;
+	u32 ich_vtr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		val = p->regval;
+		vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
+		vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
+			      ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
+		vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
+			     ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
+
+		val = 0;
+		val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
+			ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
+			ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
+			<< ICC_CTLR_EL1_SEIS_SHIFT;
+		val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
+			<< ICC_CTLR_EL1_A3V_SHIFT;
+		val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
+			ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
+		val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
+			ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
+
+		p->regval = val;
+	}
+
+	return true;
+}
+
+static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			   const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
+			    ICC_BPR0_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
+			     ICC_BPR0_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
+			     ICC_BPR1_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
+			     ICC_BPR1_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			      const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
+				      ICC_IGRPEN0_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
+			     ICC_IGRPEN0_EL1_MASK;
+	}
+
+	return true;
+}
+
+static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			      const struct sys_reg_desc *r)
+{
+	struct vgic_vmcr vmcr;
+
+	vgic_get_vmcr(vcpu, &vmcr);
+	if (p->is_write) {
+		vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
+				      ICC_IGRPEN1_EL1_SHIFT;
+		vgic_set_vmcr(vcpu, &vmcr);
+	} else {
+		p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
+			     ICC_IGRPEN1_EL1_MASK;
+	}
+
+	return true;
+}
+
+static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
+				   struct sys_reg_params *p, u8 apr, u8 idx)
+{
+	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+	uint32_t *ap_reg;
+
+	if (apr)
+		ap_reg = &vgicv3->vgic_ap1r[idx];
+	else
+		ap_reg = &vgicv3->vgic_ap0r[idx];
+
+	if (p->is_write)
+		*ap_reg = p->regval;
+	else
+		p->regval = *ap_reg;
+}
+
+static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r, u8 apr)
+{
+	u8 num_pri_bits, idx;
+	u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
+
+
+	num_pri_bits = ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
+			ICH_VTR_PRI_BITS_SHIFT) + 1;
+	idx = r->Op2 & 3;
+
+	switch (num_pri_bits) {
+	case 7:
+		if (idx > 3)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+		break;
+	case 6:
+		if (idx > 1)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+		break;
+	default:
+		if (idx > 0)
+			goto err;
+		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
+	}
+
+	return;
+err:
+	if (!p->is_write)
+		p->regval = 0;
+}
+
+static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	access_gic_aprn(vcpu, p, r, 0);
+
+	return true;
+}
+
+static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			    const struct sys_reg_desc *r)
+{
+	access_gic_aprn(vcpu, p, r, 1);
+
+	return true;
+}
+
+static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
+			   const struct sys_reg_desc *r)
+{
+	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
+
+	/* Read only. Write ignore */
+	if (!p->is_write)
+		p->regval = vgicv3->vgic_sre;
+
+	return true;
+}
+
+static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
+	/* ICC_PMR_EL1 */
+	{ Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
+	/* ICC_BPR0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
+	/* ICC_AP0R0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
+	/* ICC_AP0R1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
+	/* ICC_AP0R2_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
+	/* ICC_AP0R3_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
+	/* ICC_AP1R0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
+	/* ICC_AP1R1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
+	/* ICC_AP1R2_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
+	/* ICC_AP1R3_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
+	/* ICC_BPR1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
+	/* ICC_CTLR_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
+	/* ICC_SRE_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
+	/* ICC_IGRPEN0_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
+	/* ICC_GRPEN1_EL1 */
+	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
+};
+
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg)
+{
+	struct sys_reg_params params;
+	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+	params.regval = *reg;
+	params.is_write = is_write;
+	params.is_aarch32 = false;
+	params.is_32bit = false;
+
+	if (find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
+			      ARRAY_SIZE(gic_v3_icc_reg_descs)))
+		return 0;
+
+	return -ENXIO;
+}
+
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg)
+{
+	struct sys_reg_params params;
+	const struct sys_reg_desc *r;
+	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
+
+	if (is_write)
+		params.regval = *reg;
+	params.is_write = is_write;
+	params.is_aarch32 = false;
+	params.is_32bit = false;
+
+	r = find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
+			   ARRAY_SIZE(gic_v3_icc_reg_descs));
+	if (!r)
+		return -ENXIO;
+
+	if (!r->access(vcpu, &params, r))
+		return -EINVAL;
+
+	if (!is_write)
+		*reg = params.regval;
+
+	return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 9ee54e3..4c0076c 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -30,6 +30,12 @@
 
 #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
 
+#define   KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
+					KVM_REG_ARM64_SYSREG_OP1_MASK | \
+					KVM_REG_ARM64_SYSREG_CRN_MASK | \
+					KVM_REG_ARM64_SYSREG_CRM_MASK | \
+					KVM_REG_ARM64_SYSREG_OP2_MASK)
+
 struct vgic_vmcr {
 	u32	ctlr;
 	u32	abpr;
@@ -98,6 +104,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 int offset, u32 *val);
 int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 int offset, u32 *val);
+int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+			 u64 id, u64 *val);
+int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
+				u64 *reg);
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 6/6] arm/arm64: vgic-new: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
  2016-09-16 12:20 ` vijay.kilari at gmail.com
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  -1 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari @ 2016-09-16 12:20 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

Userspace requires to store and restore of line_level for
level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |  6 +++++
 virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
 virt/kvm/arm/vgic/vgic-mmio-v3.c    | 11 ++++++++
 virt/kvm/arm/vgic/vgic-mmio.c       | 33 ++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic-mmio.h       |  5 ++++
 virt/kvm/arm/vgic/vgic.h            |  3 +++
 6 files changed, 107 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 91c7137..4100f8c 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
 #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
 #define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
+#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT	10
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
+			(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK	0x3ff
+#define VGIC_LEVEL_INFO_LINE_LEVEL	0
 
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index a48101f..de5d836 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -514,6 +514,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
 						  regid, reg);
 		break;
 	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		unsigned int info, intid;
+
+		info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+			KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
+		if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
+			if (is_write)
+				tmp32 = *reg;
+			intid = attr->attr &
+				KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
+			ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
+							      intid, &tmp32);
+			if (!is_write)
+				*reg = tmp32;
+		} else {
+			ret = -EINVAL;
+		}
+		break;
+	}
 	default:
 		ret = -EINVAL;
 		break;
@@ -556,6 +575,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
 
 		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
 	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
+
+		if (get_user(tmp32, uaddr))
+			return -EFAULT;
+
+		reg = tmp32;
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
 	}
 	return -ENXIO;
 }
@@ -591,8 +621,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
 			return ret;
 		return put_user(reg, uaddr);
 	}
-	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
 
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		tmp32 = reg;
+		return put_user(tmp32, uaddr);
+	}
+	}
 	return -ENXIO;
 }
 
@@ -613,11 +653,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+		      KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
+		      VGIC_LEVEL_INFO_LINE_LEVEL)
+			return 0;
+		break;
+	}
 	case KVM_DEV_ARM_VGIC_GRP_CTRL:
 		switch (attr->attr) {
 		case KVM_DEV_ARM_VGIC_CTRL_INIT:
 			return 0;
 		}
+		break;
 	}
 	return -ENXIO;
 }
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index af748d7..9070412 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -806,3 +806,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 		return vgic_uaccess(vcpu, &rd_dev, is_write,
 				    offset, val);
 }
+
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+				    u32 intid, u32 *val)
+{
+	if (is_write)
+		vgic_write_irq_line_level_info(vcpu, intid, *val);
+	else
+		*val = vgic_read_irq_line_level_info(vcpu, intid);
+
+	return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 173d6f0..fb018eb 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 	}
 }
 
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
+{
+	int i;
+	unsigned long val = 0;
+
+	for (i = 0; i < 32; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		if (irq->line_level)
+			val |= (1U << i);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+
+	return val;
+}
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+				    const unsigned long val)
+{
+	int i;
+
+	for_each_set_bit(i, &val, 32) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		spin_lock(&irq->irq_lock);
+		irq->line_level = true;
+		spin_unlock(&irq->irq_lock);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+}
+
 static int match_region(const void *key, const void *elt)
 {
 	const unsigned int offset = (unsigned long)key;
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index acbf99e..938702c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
 				   const struct vgic_register_region *regions,
 				   int nr_regions, gpa_t addr);
 
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+				    const unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 4c0076c..baf2edd 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -108,6 +108,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 u64 id, u64 *val);
 int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
 				u64 *reg);
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+				    u32 intid, u32 *val);
+
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* [PATCH v5 6/6] arm/arm64: vgic-new: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl
@ 2016-09-16 12:20   ` vijay.kilari at gmail.com
  0 siblings, 0 replies; 36+ messages in thread
From: vijay.kilari at gmail.com @ 2016-09-16 12:20 UTC (permalink / raw)
  To: linux-arm-kernel

From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>

Userspace requires to store and restore of line_level for
level triggered interrupts using ioctl KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO.

Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
---
 arch/arm64/include/uapi/asm/kvm.h   |  6 +++++
 virt/kvm/arm/vgic/vgic-kvm-device.c | 50 ++++++++++++++++++++++++++++++++++++-
 virt/kvm/arm/vgic/vgic-mmio-v3.c    | 11 ++++++++
 virt/kvm/arm/vgic/vgic-mmio.c       | 33 ++++++++++++++++++++++++
 virt/kvm/arm/vgic/vgic-mmio.h       |  5 ++++
 virt/kvm/arm/vgic/vgic.h            |  3 +++
 6 files changed, 107 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 91c7137..4100f8c 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -211,6 +211,12 @@ struct kvm_arch_memory_slot {
 #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
 #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
 #define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
+#define KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO 7
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT	10
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK \
+			(0x3fffffULL << KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT)
+#define KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK	0x3ff
+#define VGIC_LEVEL_INFO_LINE_LEVEL	0
 
 #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
 
diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
index a48101f..de5d836 100644
--- a/virt/kvm/arm/vgic/vgic-kvm-device.c
+++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
@@ -514,6 +514,25 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
 						  regid, reg);
 		break;
 	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		unsigned int info, intid;
+
+		info = (attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+			KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT;
+		if (info == VGIC_LEVEL_INFO_LINE_LEVEL) {
+			if (is_write)
+				tmp32 = *reg;
+			intid = attr->attr &
+				KVM_DEV_ARM_VGIC_LINE_LEVEL_INTID_MASK;
+			ret = vgic_v3_line_level_info_uaccess(vcpu, is_write,
+							      intid, &tmp32);
+			if (!is_write)
+				*reg = tmp32;
+		} else {
+			ret = -EINVAL;
+		}
+		break;
+	}
 	default:
 		ret = -EINVAL;
 		break;
@@ -556,6 +575,17 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
 
 		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
 	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
+
+		if (get_user(tmp32, uaddr))
+			return -EFAULT;
+
+		reg = tmp32;
+		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
+	}
 	}
 	return -ENXIO;
 }
@@ -591,8 +621,18 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
 			return ret;
 		return put_user(reg, uaddr);
 	}
-	}
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+		u64 reg;
+		u32 tmp32;
 
+		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
+		if (ret)
+			return ret;
+		tmp32 = reg;
+		return put_user(tmp32, uaddr);
+	}
+	}
 	return -ENXIO;
 }
 
@@ -613,11 +653,19 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
 		return vgic_v3_has_attr_regs(dev, attr);
 	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
 		return 0;
+	case KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO: {
+		if (((attr->attr & KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_MASK) >>
+		      KVM_DEV_ARM_VGIC_LINE_LEVEL_INFO_SHIFT) ==
+		      VGIC_LEVEL_INFO_LINE_LEVEL)
+			return 0;
+		break;
+	}
 	case KVM_DEV_ARM_VGIC_GRP_CTRL:
 		switch (attr->attr) {
 		case KVM_DEV_ARM_VGIC_CTRL_INIT:
 			return 0;
 		}
+		break;
 	}
 	return -ENXIO;
 }
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index af748d7..9070412 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -806,3 +806,14 @@ int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 		return vgic_uaccess(vcpu, &rd_dev, is_write,
 				    offset, val);
 }
+
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+				    u32 intid, u32 *val)
+{
+	if (is_write)
+		vgic_write_irq_line_level_info(vcpu, intid, *val);
+	else
+		*val = vgic_read_irq_line_level_info(vcpu, intid);
+
+	return 0;
+}
diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
index 173d6f0..fb018eb 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.c
+++ b/virt/kvm/arm/vgic/vgic-mmio.c
@@ -371,6 +371,39 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
 	}
 }
 
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid)
+{
+	int i;
+	unsigned long val = 0;
+
+	for (i = 0; i < 32; i++) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		if (irq->line_level)
+			val |= (1U << i);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+
+	return val;
+}
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+				    const unsigned long val)
+{
+	int i;
+
+	for_each_set_bit(i, &val, 32) {
+		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
+
+		spin_lock(&irq->irq_lock);
+		irq->line_level = true;
+		spin_unlock(&irq->irq_lock);
+
+		vgic_put_irq(vcpu->kvm, irq);
+	}
+}
+
 static int match_region(const void *key, const void *elt)
 {
 	const unsigned int offset = (unsigned long)key;
diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
index acbf99e..938702c 100644
--- a/virt/kvm/arm/vgic/vgic-mmio.h
+++ b/virt/kvm/arm/vgic/vgic-mmio.h
@@ -181,6 +181,11 @@ int vgic_validate_mmio_region_addr(struct kvm_device *dev,
 				   const struct vgic_register_region *regions,
 				   int nr_regions, gpa_t addr);
 
+unsigned long vgic_read_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid);
+
+void vgic_write_irq_line_level_info(struct kvm_vcpu *vcpu, u32 intid,
+				    const unsigned long val);
+
 unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
 
 unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 4c0076c..baf2edd 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -108,6 +108,9 @@ int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
 			 u64 id, u64 *val);
 int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
 				u64 *reg);
+int vgic_v3_line_level_info_uaccess(struct kvm_vcpu *vcpu, bool is_write,
+				    u32 intid, u32 *val);
+
 #else
 static inline int vgic_register_its_iodevs(struct kvm *kvm)
 {
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 36+ messages in thread

* Re: [PATCH v5 4/6] arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
  2016-09-16 12:20   ` vijay.kilari at gmail.com
@ 2016-09-16 13:54     ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 13:54 UTC (permalink / raw)
  To: vijay.kilari, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> variables to struct vmcr to support read and write of these fields.
> 
> ICH_VMCR_CTLR_MASK is changed to mask only ICC_CTLR_EL1 fields.

This comment doesn't match the patch.

> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  include/linux/irqchip/arm-gic-v3.h | 15 +++++++++++----
>  virt/kvm/arm/vgic/vgic-mmio-v2.c   | 16 ----------------
>  virt/kvm/arm/vgic/vgic-mmio.c      | 16 ++++++++++++++++
>  virt/kvm/arm/vgic/vgic-v3.c        | 10 ++++++++--
>  virt/kvm/arm/vgic/vgic.h           |  5 +++++
>  5 files changed, 40 insertions(+), 22 deletions(-)
> 
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index 99ac022..88d83d3 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -352,8 +352,9 @@
>  /*
>   * CPU interface registers
>   */
> -#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << 1)
> -#define ICC_CTLR_EL1_EOImode_drop	(1U << 1)
> +#define ICC_CTLR_EL1_EOImode_SHIFT	(1)
> +#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
>  #define ICC_SRE_EL1_SRE			(1U << 0)
>  
>  /*
> @@ -382,14 +383,20 @@
>  #define ICH_HCR_EN			(1 << 0)
>  #define ICH_HCR_UIE			(1 << 1)
>  
> -#define ICH_VMCR_CTLR_SHIFT		0
> -#define ICH_VMCR_CTLR_MASK		(0x21f << ICH_VMCR_CTLR_SHIFT)
> +#define ICH_VMCR_CBPR_SHIFT		4
> +#define ICH_VMCR_CBPR_MASK		(1 << ICH_VMCR_CBPR_SHIFT)
> +#define ICH_VMCR_EOIM_SHIFT		9
> +#define ICH_VMCR_EOIM_MASK		(1 << ICH_VMCR_EOIM_SHIFT)
>  #define ICH_VMCR_BPR1_SHIFT		18
>  #define ICH_VMCR_BPR1_MASK		(7 << ICH_VMCR_BPR1_SHIFT)
>  #define ICH_VMCR_BPR0_SHIFT		21
>  #define ICH_VMCR_BPR0_MASK		(7 << ICH_VMCR_BPR0_SHIFT)
>  #define ICH_VMCR_PMR_SHIFT		24
>  #define ICH_VMCR_PMR_MASK		(0xffUL << ICH_VMCR_PMR_SHIFT)
> +#define ICH_VMCR_ENG0_SHIFT		0
> +#define ICH_VMCR_ENG0_MASK		(1 << ICH_VMCR_ENG0_SHIFT)
> +#define ICH_VMCR_ENG1_SHIFT		1
> +#define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
>  
>  #define ICC_IAR1_EL1_SPURIOUS		0x3ff

I've asked you to make this a separate patch, which you've ignored.
Please make this change a separate patch, independent of any KVM change.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH v5 4/6] arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct
@ 2016-09-16 13:54     ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 13:54 UTC (permalink / raw)
  To: linux-arm-kernel

On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
> and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
> variables to struct vmcr to support read and write of these fields.
> 
> ICH_VMCR_CTLR_MASK is changed to mask only ICC_CTLR_EL1 fields.

This comment doesn't match the patch.

> Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  include/linux/irqchip/arm-gic-v3.h | 15 +++++++++++----
>  virt/kvm/arm/vgic/vgic-mmio-v2.c   | 16 ----------------
>  virt/kvm/arm/vgic/vgic-mmio.c      | 16 ++++++++++++++++
>  virt/kvm/arm/vgic/vgic-v3.c        | 10 ++++++++--
>  virt/kvm/arm/vgic/vgic.h           |  5 +++++
>  5 files changed, 40 insertions(+), 22 deletions(-)
> 
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index 99ac022..88d83d3 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -352,8 +352,9 @@
>  /*
>   * CPU interface registers
>   */
> -#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << 1)
> -#define ICC_CTLR_EL1_EOImode_drop	(1U << 1)
> +#define ICC_CTLR_EL1_EOImode_SHIFT	(1)
> +#define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
>  #define ICC_SRE_EL1_SRE			(1U << 0)
>  
>  /*
> @@ -382,14 +383,20 @@
>  #define ICH_HCR_EN			(1 << 0)
>  #define ICH_HCR_UIE			(1 << 1)
>  
> -#define ICH_VMCR_CTLR_SHIFT		0
> -#define ICH_VMCR_CTLR_MASK		(0x21f << ICH_VMCR_CTLR_SHIFT)
> +#define ICH_VMCR_CBPR_SHIFT		4
> +#define ICH_VMCR_CBPR_MASK		(1 << ICH_VMCR_CBPR_SHIFT)
> +#define ICH_VMCR_EOIM_SHIFT		9
> +#define ICH_VMCR_EOIM_MASK		(1 << ICH_VMCR_EOIM_SHIFT)
>  #define ICH_VMCR_BPR1_SHIFT		18
>  #define ICH_VMCR_BPR1_MASK		(7 << ICH_VMCR_BPR1_SHIFT)
>  #define ICH_VMCR_BPR0_SHIFT		21
>  #define ICH_VMCR_BPR0_MASK		(7 << ICH_VMCR_BPR0_SHIFT)
>  #define ICH_VMCR_PMR_SHIFT		24
>  #define ICH_VMCR_PMR_MASK		(0xffUL << ICH_VMCR_PMR_SHIFT)
> +#define ICH_VMCR_ENG0_SHIFT		0
> +#define ICH_VMCR_ENG0_MASK		(1 << ICH_VMCR_ENG0_SHIFT)
> +#define ICH_VMCR_ENG1_SHIFT		1
> +#define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
>  
>  #define ICC_IAR1_EL1_SPURIOUS		0x3ff

I've asked you to make this a separate patch, which you've ignored.
Please make this change a separate patch, independent of any KVM change.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 12:20   ` vijay.kilari at gmail.com
@ 2016-09-16 14:36     ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 14:36 UTC (permalink / raw)
  To: vijay.kilari, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> VGICv3 CPU interface registers are accessed using
> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> as 64-bit. The cpu MPIDR value is passed along with register id.
> is used to identify the cpu for registers access.
> 
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>  arch/arm64/kvm/Makefile             |   1 +
>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>  7 files changed, 385 insertions(+)
> 
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 56dc08d..91c7137 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>  			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>  #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
>  #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> +#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>  #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
>  #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
>  #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
> +
>  #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
>  
>  /* Device Control API on vcpu fd */
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index d50a82a..1a14e29 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>  kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index 88d83d3..d4e9c7d 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -355,6 +355,27 @@
>  #define ICC_CTLR_EL1_EOImode_SHIFT	(1)
>  #define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
>  #define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_EOImode_MASK	(1 << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_CBPR_SHIFT		(0)
> +#define ICC_CTLR_EL1_CBPR_MASK		(1 << ICC_CTLR_EL1_CBPR_SHIFT)
> +#define ICC_CTLR_EL1_PRI_BITS_SHIFT	(8)
> +#define ICC_CTLR_EL1_PRI_BITS_MASK	(0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
> +#define ICC_CTLR_EL1_ID_BITS_SHIFT	(11)
> +#define ICC_CTLR_EL1_ID_BITS_MASK	(0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
> +#define ICC_CTLR_EL1_SEIS_SHIFT		(14)
> +#define ICC_CTLR_EL1_SEIS_MASK		(0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
> +#define ICC_CTLR_EL1_A3V_SHIFT		(15)
> +#define ICC_CTLR_EL1_A3V_MASK		(0x1 << ICC_CTLR_EL1_A3V_SHIFT)
> +#define ICC_PMR_EL1_SHIFT		(0)
> +#define ICC_PMR_EL1_MASK		(0xff << ICC_PMR_EL1_SHIFT)
> +#define ICC_BPR0_EL1_SHIFT		(0)
> +#define ICC_BPR0_EL1_MASK		(0x7 << ICC_BPR0_EL1_SHIFT)
> +#define ICC_BPR1_EL1_SHIFT		(0)
> +#define ICC_BPR1_EL1_MASK		(0x7 << ICC_BPR1_EL1_SHIFT)
> +#define ICC_IGRPEN0_EL1_SHIFT		(0)
> +#define ICC_IGRPEN0_EL1_MASK		(1 << ICC_IGRPEN0_EL1_SHIFT)
> +#define ICC_IGRPEN1_EL1_SHIFT		(0)
> +#define ICC_IGRPEN1_EL1_MASK		(1 << ICC_IGRPEN1_EL1_SHIFT)
>  #define ICC_SRE_EL1_SRE			(1U << 0)
>  
>  /*
> @@ -398,6 +419,15 @@
>  #define ICH_VMCR_ENG1_SHIFT		1
>  #define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
>  
> +#define ICH_VTR_PRI_BITS_SHIFT		29
> +#define ICH_VTR_PRI_BITS_MASK		(7 << ICH_VTR_PRI_BITS_SHIFT)
> +#define ICH_VTR_ID_BITS_SHIFT		23
> +#define ICH_VTR_ID_BITS_MASK		(7 << ICH_VTR_ID_BITS_SHIFT)
> +#define ICH_VTR_SEIS_SHIFT		22
> +#define ICH_VTR_SEIS_MASK		(1 << ICH_VTR_SEIS_SHIFT)
> +#define ICH_VTR_A3V_SHIFT		21
> +#define ICH_VTR_A3V_MASK		(1 << ICH_VTR_A3V_SHIFT)
> +

Same thing here. Please group all the additions to this file into a
single, independent patch.

>  #define ICC_IAR1_EL1_SPURIOUS		0x3ff
>  
>  #define ICC_SRE_EL2_SRE			(1 << 0)
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index a4656fc..a48101f 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>  		if (!is_write)
>  			*reg = tmp32;
>  		break;
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 regid;
> +
> +		regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> +		ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> +						  regid, reg);
> +		break;
> +	}
>  	default:
>  		ret = -EINVAL;
>  		break;
> @@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>  		reg = tmp32;
>  		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> +		u64 reg;
> +
> +		if (get_user(reg, uaddr))
> +			return -EFAULT;
> +
> +		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
> +	}
>  	}
>  	return -ENXIO;
>  }
> @@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>  		tmp32 = reg;
>  		return put_user(tmp32, uaddr);
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> +		u64 reg;
> +
> +		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
> +		if (ret)
> +			return ret;
> +		return put_user(reg, uaddr);
> +	}
>  	}
>  
>  	return -ENXIO;
> @@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>  		break;
>  	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>  	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>  		return vgic_v3_has_attr_regs(dev, attr);
>  	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>  		return 0;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 83dece8..af748d7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,6 +23,7 @@
>  
>  #include "vgic.h"
>  #include "vgic-mmio.h"
> +#include "sys_regs.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
>  unsigned long extract_bytes(u64 data, unsigned int offset,
> @@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>  		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>  		break;
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 reg, id;
> +		unsigned long mpidr;
> +		struct kvm_vcpu *vcpu;
> +
> +		mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> +			 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> +		vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);

No. Your "mpidr" variable is not an MPIDR.  It represents the same
thing, but not with the right format.

> +		if (!vcpu)
> +			return -EINVAL;
> +		if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))

How can that happen?

> +			return -EINVAL;
> +
> +		id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> +		return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
> +	}
>  	default:
>  		return -ENXIO;
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> new file mode 100644
> index 0000000..8e4f403
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> @@ -0,0 +1,296 @@
> +#include <linux/irqchip/arm-gic-v3.h>
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +#include <kvm/iodev.h>
> +#include <kvm/arm_vgic.h>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
> +
> +#include "vgic.h"
> +#include "vgic-mmio.h"
> +#include "sys_regs.h"
> +
> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +	u64 val;
> +	u32 ich_vtr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		val = p->regval;
> +		vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> +		vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> +			      ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> +		vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> +			     ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);

You've ignored my comments again: "What if userspace writes something
that is incompatible with the current configuration? Wrong number of ID
bits, or number of priorities?"

> +	} else {
> +		ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
> +
> +		val = 0;
> +		val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> +			ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
> +			ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
> +			<< ICC_CTLR_EL1_SEIS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
> +			<< ICC_CTLR_EL1_A3V_SHIFT;
> +		val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> +			ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> +		val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> +			ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> +
> +		p->regval = val;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			   const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> +			    ICC_BPR0_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> +			     ICC_BPR0_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> +			     ICC_BPR1_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> +			     ICC_BPR1_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			      const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> +				      ICC_IGRPEN0_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> +			     ICC_IGRPEN0_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			      const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> +				      ICC_IGRPEN1_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> +			     ICC_IGRPEN1_EL1_MASK;

>From the previous review comments: "Shouldn't this account for the
ICC_CTLR_EL1.CBPR setting?"

> +	}
> +
> +	return true;
> +}
> +
> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> +				   struct sys_reg_params *p, u8 apr, u8 idx)
> +{
> +	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +	uint32_t *ap_reg;
> +
> +	if (apr)
> +		ap_reg = &vgicv3->vgic_ap1r[idx];
> +	else
> +		ap_reg = &vgicv3->vgic_ap0r[idx];
> +
> +	if (p->is_write)
> +		*ap_reg = p->regval;
> +	else
> +		p->regval = *ap_reg;
> +}
> +
> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r, u8 apr)
> +{
> +	u8 num_pri_bits, idx;
> +	u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);

Maybe you should cache this once and for all? It is fairly unlikely to
change over time...

> +
> +
> +	num_pri_bits = ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> +			ICH_VTR_PRI_BITS_SHIFT) + 1;
> +	idx = r->Op2 & 3;
> +
> +	switch (num_pri_bits) {
> +	case 7:
> +		if (idx > 3)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +		break;
> +	case 6:
> +		if (idx > 1)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +		break;
> +	default:
> +		if (idx > 0)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +	}
> +
> +	return;
> +err:
> +	if (!p->is_write)
> +		p->regval = 0;
> +}
> +
> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	access_gic_aprn(vcpu, p, r, 0);
> +
> +	return true;
> +}
> +
> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	access_gic_aprn(vcpu, p, r, 1);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			   const struct sys_reg_desc *r)
> +{
> +	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> +	/* Read only. Write ignore */
> +	if (!p->is_write)
> +		p->regval = vgicv3->vgic_sre;
> +
> +	return true;
> +}
> +
> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> +	/* ICC_PMR_EL1 */
> +	{ Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> +	/* ICC_BPR0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> +	/* ICC_AP0R0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> +	/* ICC_AP0R1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> +	/* ICC_AP0R2_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> +	/* ICC_AP0R3_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> +	/* ICC_AP1R0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> +	/* ICC_AP1R1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> +	/* ICC_AP1R2_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> +	/* ICC_AP1R3_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> +	/* ICC_BPR1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> +	/* ICC_CTLR_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> +	/* ICC_SRE_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> +	/* ICC_IGRPEN0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> +	/* ICC_GRPEN1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> +};
> +
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg)
> +{
> +	struct sys_reg_params params;
> +	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> +	params.regval = *reg;
> +	params.is_write = is_write;
> +	params.is_aarch32 = false;
> +	params.is_32bit = false;
> +
> +	if (find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
> +			      ARRAY_SIZE(gic_v3_icc_reg_descs)))
> +		return 0;
> +
> +	return -ENXIO;
> +}
> +
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg)
> +{
> +	struct sys_reg_params params;
> +	const struct sys_reg_desc *r;
> +	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> +	if (is_write)
> +		params.regval = *reg;
> +	params.is_write = is_write;
> +	params.is_aarch32 = false;
> +	params.is_32bit = false;
> +
> +	r = find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
> +			   ARRAY_SIZE(gic_v3_icc_reg_descs));
> +	if (!r)
> +		return -ENXIO;
> +
> +	if (!r->access(vcpu, &params, r))
> +		return -EINVAL;
> +
> +	if (!is_write)
> +		*reg = params.regval;
> +
> +	return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 9ee54e3..4c0076c 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -30,6 +30,12 @@
>  
>  #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
>  
> +#define   KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
> +					KVM_REG_ARM64_SYSREG_OP1_MASK | \
> +					KVM_REG_ARM64_SYSREG_CRN_MASK | \
> +					KVM_REG_ARM64_SYSREG_CRM_MASK | \
> +					KVM_REG_ARM64_SYSREG_OP2_MASK)
> +
>  struct vgic_vmcr {
>  	u32	ctlr;
>  	u32	abpr;
> @@ -98,6 +104,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			 int offset, u32 *val);
>  int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			 int offset, u32 *val);
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> +			 u64 id, u64 *val);
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg);
>  #else
>  static inline int vgic_register_its_iodevs(struct kvm *kvm)
>  {
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-16 14:36     ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 14:36 UTC (permalink / raw)
  To: linux-arm-kernel

On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> VGICv3 CPU interface registers are accessed using
> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> as 64-bit. The cpu MPIDR value is passed along with register id.
> is used to identify the cpu for registers access.
> 
> The version of VGIC v3 specification is define here
> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> 
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>  arch/arm64/kvm/Makefile             |   1 +
>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>  7 files changed, 385 insertions(+)
> 
> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
> index 56dc08d..91c7137 100644
> --- a/arch/arm64/include/uapi/asm/kvm.h
> +++ b/arch/arm64/include/uapi/asm/kvm.h
> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>  			(0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>  #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT	0
>  #define   KVM_DEV_ARM_VGIC_OFFSET_MASK	(0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
> +#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>  #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS	3
>  #define KVM_DEV_ARM_VGIC_GRP_CTRL	4
>  #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
> +
>  #define   KVM_DEV_ARM_VGIC_CTRL_INIT	0
>  
>  /* Device Control API on vcpu fd */
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index d50a82a..1a14e29 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>  kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
> index 88d83d3..d4e9c7d 100644
> --- a/include/linux/irqchip/arm-gic-v3.h
> +++ b/include/linux/irqchip/arm-gic-v3.h
> @@ -355,6 +355,27 @@
>  #define ICC_CTLR_EL1_EOImode_SHIFT	(1)
>  #define ICC_CTLR_EL1_EOImode_drop_dir	(0U << ICC_CTLR_EL1_EOImode_SHIFT)
>  #define ICC_CTLR_EL1_EOImode_drop	(1U << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_EOImode_MASK	(1 << ICC_CTLR_EL1_EOImode_SHIFT)
> +#define ICC_CTLR_EL1_CBPR_SHIFT		(0)
> +#define ICC_CTLR_EL1_CBPR_MASK		(1 << ICC_CTLR_EL1_CBPR_SHIFT)
> +#define ICC_CTLR_EL1_PRI_BITS_SHIFT	(8)
> +#define ICC_CTLR_EL1_PRI_BITS_MASK	(0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
> +#define ICC_CTLR_EL1_ID_BITS_SHIFT	(11)
> +#define ICC_CTLR_EL1_ID_BITS_MASK	(0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
> +#define ICC_CTLR_EL1_SEIS_SHIFT		(14)
> +#define ICC_CTLR_EL1_SEIS_MASK		(0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
> +#define ICC_CTLR_EL1_A3V_SHIFT		(15)
> +#define ICC_CTLR_EL1_A3V_MASK		(0x1 << ICC_CTLR_EL1_A3V_SHIFT)
> +#define ICC_PMR_EL1_SHIFT		(0)
> +#define ICC_PMR_EL1_MASK		(0xff << ICC_PMR_EL1_SHIFT)
> +#define ICC_BPR0_EL1_SHIFT		(0)
> +#define ICC_BPR0_EL1_MASK		(0x7 << ICC_BPR0_EL1_SHIFT)
> +#define ICC_BPR1_EL1_SHIFT		(0)
> +#define ICC_BPR1_EL1_MASK		(0x7 << ICC_BPR1_EL1_SHIFT)
> +#define ICC_IGRPEN0_EL1_SHIFT		(0)
> +#define ICC_IGRPEN0_EL1_MASK		(1 << ICC_IGRPEN0_EL1_SHIFT)
> +#define ICC_IGRPEN1_EL1_SHIFT		(0)
> +#define ICC_IGRPEN1_EL1_MASK		(1 << ICC_IGRPEN1_EL1_SHIFT)
>  #define ICC_SRE_EL1_SRE			(1U << 0)
>  
>  /*
> @@ -398,6 +419,15 @@
>  #define ICH_VMCR_ENG1_SHIFT		1
>  #define ICH_VMCR_ENG1_MASK		(1 << ICH_VMCR_ENG1_SHIFT)
>  
> +#define ICH_VTR_PRI_BITS_SHIFT		29
> +#define ICH_VTR_PRI_BITS_MASK		(7 << ICH_VTR_PRI_BITS_SHIFT)
> +#define ICH_VTR_ID_BITS_SHIFT		23
> +#define ICH_VTR_ID_BITS_MASK		(7 << ICH_VTR_ID_BITS_SHIFT)
> +#define ICH_VTR_SEIS_SHIFT		22
> +#define ICH_VTR_SEIS_MASK		(1 << ICH_VTR_SEIS_SHIFT)
> +#define ICH_VTR_A3V_SHIFT		21
> +#define ICH_VTR_A3V_MASK		(1 << ICH_VTR_A3V_SHIFT)
> +

Same thing here. Please group all the additions to this file into a
single, independent patch.

>  #define ICC_IAR1_EL1_SPURIOUS		0x3ff
>  
>  #define ICC_SRE_EL2_SRE			(1 << 0)
> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
> index a4656fc..a48101f 100644
> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
> @@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>  		if (!is_write)
>  			*reg = tmp32;
>  		break;
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 regid;
> +
> +		regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> +		ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
> +						  regid, reg);
> +		break;
> +	}
>  	default:
>  		ret = -EINVAL;
>  		break;
> @@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>  		reg = tmp32;
>  		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> +		u64 reg;
> +
> +		if (get_user(reg, uaddr))
> +			return -EFAULT;
> +
> +		return vgic_v3_attr_regs_access(dev, attr, &reg, true);
> +	}
>  	}
>  	return -ENXIO;
>  }
> @@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>  		tmp32 = reg;
>  		return put_user(tmp32, uaddr);
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 __user *uaddr = (u64 __user *)(long)attr->addr;
> +		u64 reg;
> +
> +		ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
> +		if (ret)
> +			return ret;
> +		return put_user(reg, uaddr);
> +	}
>  	}
>  
>  	return -ENXIO;
> @@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>  		break;
>  	case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>  	case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>  		return vgic_v3_has_attr_regs(dev, attr);
>  	case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>  		return 0;
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 83dece8..af748d7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -23,6 +23,7 @@
>  
>  #include "vgic.h"
>  #include "vgic-mmio.h"
> +#include "sys_regs.h"
>  
>  /* extract @num bytes at @offset bytes offset in data */
>  unsigned long extract_bytes(u64 data, unsigned int offset,
> @@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>  		nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>  		break;
>  	}
> +	case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
> +		u64 reg, id;
> +		unsigned long mpidr;
> +		struct kvm_vcpu *vcpu;
> +
> +		mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
> +			 KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
> +
> +		vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);

No. Your "mpidr" variable is not an MPIDR.  It represents the same
thing, but not with the right format.

> +		if (!vcpu)
> +			return -EINVAL;
> +		if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))

How can that happen?

> +			return -EINVAL;
> +
> +		id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
> +		return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
> +	}
>  	default:
>  		return -ENXIO;
>  	}
> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> new file mode 100644
> index 0000000..8e4f403
> --- /dev/null
> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> @@ -0,0 +1,296 @@
> +#include <linux/irqchip/arm-gic-v3.h>
> +#include <linux/kvm.h>
> +#include <linux/kvm_host.h>
> +#include <kvm/iodev.h>
> +#include <kvm/arm_vgic.h>
> +#include <asm/kvm_emulate.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_mmu.h>
> +
> +#include "vgic.h"
> +#include "vgic-mmio.h"
> +#include "sys_regs.h"
> +
> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +	u64 val;
> +	u32 ich_vtr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		val = p->regval;
> +		vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> +		vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> +			      ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> +		vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> +			     ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);

You've ignored my comments again: "What if userspace writes something
that is incompatible with the current configuration? Wrong number of ID
bits, or number of priorities?"

> +	} else {
> +		ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
> +
> +		val = 0;
> +		val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> +			ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
> +			ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
> +			<< ICC_CTLR_EL1_SEIS_SHIFT;
> +		val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
> +			<< ICC_CTLR_EL1_A3V_SHIFT;
> +		val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> +			ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> +		val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> +			ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> +
> +		p->regval = val;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			   const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> +			    ICC_BPR0_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> +			     ICC_BPR0_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> +			     ICC_BPR1_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> +			     ICC_BPR1_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			      const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> +				      ICC_IGRPEN0_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> +			     ICC_IGRPEN0_EL1_MASK;
> +	}
> +
> +	return true;
> +}
> +
> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			      const struct sys_reg_desc *r)
> +{
> +	struct vgic_vmcr vmcr;
> +
> +	vgic_get_vmcr(vcpu, &vmcr);
> +	if (p->is_write) {
> +		vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> +				      ICC_IGRPEN1_EL1_SHIFT;
> +		vgic_set_vmcr(vcpu, &vmcr);
> +	} else {
> +		p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> +			     ICC_IGRPEN1_EL1_MASK;

>From the previous review comments: "Shouldn't this account for the
ICC_CTLR_EL1.CBPR setting?"

> +	}
> +
> +	return true;
> +}
> +
> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
> +				   struct sys_reg_params *p, u8 apr, u8 idx)
> +{
> +	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +	uint32_t *ap_reg;
> +
> +	if (apr)
> +		ap_reg = &vgicv3->vgic_ap1r[idx];
> +	else
> +		ap_reg = &vgicv3->vgic_ap0r[idx];
> +
> +	if (p->is_write)
> +		*ap_reg = p->regval;
> +	else
> +		p->regval = *ap_reg;
> +}
> +
> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r, u8 apr)
> +{
> +	u8 num_pri_bits, idx;
> +	u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);

Maybe you should cache this once and for all? It is fairly unlikely to
change over time...

> +
> +
> +	num_pri_bits = ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> +			ICH_VTR_PRI_BITS_SHIFT) + 1;
> +	idx = r->Op2 & 3;
> +
> +	switch (num_pri_bits) {
> +	case 7:
> +		if (idx > 3)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +		break;
> +	case 6:
> +		if (idx > 1)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +		break;
> +	default:
> +		if (idx > 0)
> +			goto err;
> +		vgic_v3_access_apr_reg(vcpu, p, apr, idx);
> +	}
> +
> +	return;
> +err:
> +	if (!p->is_write)
> +		p->regval = 0;
> +}
> +
> +static bool access_gic_ap0r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	access_gic_aprn(vcpu, p, r, 0);
> +
> +	return true;
> +}
> +
> +static bool access_gic_ap1r(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			    const struct sys_reg_desc *r)
> +{
> +	access_gic_aprn(vcpu, p, r, 1);
> +
> +	return true;
> +}
> +
> +static bool access_gic_sre(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> +			   const struct sys_reg_desc *r)
> +{
> +	struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
> +
> +	/* Read only. Write ignore */
> +	if (!p->is_write)
> +		p->regval = vgicv3->vgic_sre;
> +
> +	return true;
> +}
> +
> +static const struct sys_reg_desc gic_v3_icc_reg_descs[] = {
> +	/* ICC_PMR_EL1 */
> +	{ Op0(3), Op1(0), CRn(4), CRm(6), Op2(0), access_gic_pmr },
> +	/* ICC_BPR0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(3), access_gic_bpr0 },
> +	/* ICC_AP0R0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(4), access_gic_ap0r },
> +	/* ICC_AP0R1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(5), access_gic_ap0r },
> +	/* ICC_AP0R2_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(6), access_gic_ap0r },
> +	/* ICC_AP0R3_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(8), Op2(7), access_gic_ap0r },
> +	/* ICC_AP1R0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(0), access_gic_ap1r },
> +	/* ICC_AP1R1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(1), access_gic_ap1r },
> +	/* ICC_AP1R2_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(2), access_gic_ap1r },
> +	/* ICC_AP1R3_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(9), Op2(3), access_gic_ap1r },
> +	/* ICC_BPR1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(3), access_gic_bpr1 },
> +	/* ICC_CTLR_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(4), access_gic_ctlr },
> +	/* ICC_SRE_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(5), access_gic_sre },
> +	/* ICC_IGRPEN0_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(6), access_gic_grpen0 },
> +	/* ICC_GRPEN1_EL1 */
> +	{ Op0(3), Op1(0), CRn(12), CRm(12), Op2(7), access_gic_grpen1 },
> +};
> +
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg)
> +{
> +	struct sys_reg_params params;
> +	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> +	params.regval = *reg;
> +	params.is_write = is_write;
> +	params.is_aarch32 = false;
> +	params.is_32bit = false;
> +
> +	if (find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
> +			      ARRAY_SIZE(gic_v3_icc_reg_descs)))
> +		return 0;
> +
> +	return -ENXIO;
> +}
> +
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg)
> +{
> +	struct sys_reg_params params;
> +	const struct sys_reg_desc *r;
> +	u64 sysreg = (id & KVM_DEV_ARM_VGIC_SYSREG_MASK) | KVM_REG_SIZE_U64;
> +
> +	if (is_write)
> +		params.regval = *reg;
> +	params.is_write = is_write;
> +	params.is_aarch32 = false;
> +	params.is_32bit = false;
> +
> +	r = find_reg_by_id(sysreg, &params, gic_v3_icc_reg_descs,
> +			   ARRAY_SIZE(gic_v3_icc_reg_descs));
> +	if (!r)
> +		return -ENXIO;
> +
> +	if (!r->access(vcpu, &params, r))
> +		return -EINVAL;
> +
> +	if (!is_write)
> +		*reg = params.regval;
> +
> +	return 0;
> +}
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 9ee54e3..4c0076c 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -30,6 +30,12 @@
>  
>  #define vgic_irq_is_sgi(intid) ((intid) < VGIC_NR_SGIS)
>  
> +#define   KVM_DEV_ARM_VGIC_SYSREG_MASK (KVM_REG_ARM64_SYSREG_OP0_MASK | \
> +					KVM_REG_ARM64_SYSREG_OP1_MASK | \
> +					KVM_REG_ARM64_SYSREG_CRN_MASK | \
> +					KVM_REG_ARM64_SYSREG_CRM_MASK | \
> +					KVM_REG_ARM64_SYSREG_OP2_MASK)
> +
>  struct vgic_vmcr {
>  	u32	ctlr;
>  	u32	abpr;
> @@ -98,6 +104,10 @@ int vgic_v3_dist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			 int offset, u32 *val);
>  int vgic_v3_redist_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			 int offset, u32 *val);
> +int vgic_v3_cpu_sysregs_uaccess(struct kvm_vcpu *vcpu, bool is_write,
> +			 u64 id, u64 *val);
> +int vgic_v3_has_cpu_sysregs_attr(struct kvm_vcpu *vcpu, bool is_write, u64 id,
> +				u64 *reg);
>  #else
>  static inline int vgic_register_its_iodevs(struct kvm *kvm)
>  {
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH v5 1/6] arm/arm64: vgic-new: Implement support for userspace access
  2016-09-16 12:20   ` vijay.kilari at gmail.com
@ 2016-09-16 16:31     ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 16:31 UTC (permalink / raw)
  To: vijay.kilari, christoffer.dall, peter.maydell
  Cc: kvmarm, linux-arm-kernel, Vijaya Kumar K

On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> Read and write of some registers like ISPENDR and ICPENDR
> from userspace requires special handling when compared to
> guest access for these registers.
> 
> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> for handling of ISPENDR, ICPENDR registers handling.
> 
> Add infrastructure to support guest and userspace read
> and write for the required registers
> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
>  virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
>  virt/kvm/arm/vgic/vgic-mmio.c    | 78 ++++++++++++++++++++++++++++----
>  virt/kvm/arm/vgic/vgic-mmio.h    | 19 ++++++++
>  4 files changed, 169 insertions(+), 51 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index b44b359..0b32f40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>  	return -ENXIO;
>  }
>  
> -/*
> - * When userland tries to access the VGIC register handlers, we need to
> - * create a usable struct vgic_io_device to be passed to the handlers and we
> - * have to set up a buffer similar to what would have happened if a guest MMIO
> - * access occurred, including doing endian conversions on BE systems.
> - */
> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> -			bool is_write, int offset, u32 *val)
> -{
> -	unsigned int len = 4;
> -	u8 buf[4];
> -	int ret;
> -
> -	if (is_write) {
> -		vgic_data_host_to_mmio_bus(buf, len, *val);
> -		ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> -	} else {
> -		ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> -		if (!ret)
> -			*val = vgic_data_mmio_bus_to_host(buf, len);
> -	}
> -
> -	return ret;
> -}
> -
>  int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			  int offset, u32 *val)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 0d3c76a..edd3d40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +static unsigned long vgic_uaccess_read_v3_pending(struct kvm_vcpu *vcpu,

Maming is completely inconsistent. This should read "vgic_v3_uaccess_read_pending".

> +						  gpa_t addr, unsigned int len)
> +{
> +	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> +	u32 value = 0;
> +	int i;
> +
> +	/*
> +	 * A level triggerred interrupt pending state is latched in both
> +	 * "soft_pending" and "line_level" variables. Userspace will save
> +	 * and restore soft_pending and line_level separately.
> +	 * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> +	 * handling of ISPENDR and ICPENDR.
> +	 */
> +	for (i = 0; i < len * 8; i++) {
> +		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> +		if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> +			value |= (1U << i);
> +		if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> +			value |= (1U << i);
> +
> +		vgic_put_irq(vcpu->kvm, irq);
> +	}
> +
> +	return value;
> +}
> +
> +static void vgic_uaccess_write_v3_pending(struct kvm_vcpu *vcpu,

Same here.

> +					  gpa_t addr, unsigned int len,
> +					  unsigned long val)
> +{
> +	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> +	int i;
> +
> +	for (i = 0; i < len * 8; i++) {
> +		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> +		spin_lock(&irq->irq_lock);
> +		if (test_bit(i, &val)) {
> +			irq->pending = true;
> +			irq->soft_pending = true;
> +			vgic_queue_irq_unlock(vcpu->kvm, irq);
> +		} else {
> +			irq->soft_pending = false;
> +			if (irq->config == VGIC_CONFIG_EDGE ||
> +			    (irq->config == VGIC_CONFIG_LEVEL &&
> +			    !irq->line_level))
> +				irq->pending = false;
> +			spin_unlock(&irq->irq_lock);
> +		}
> +
> +		vgic_put_irq(vcpu->kvm, irq);
> +	}
> +}
> +
>  /* We want to avoid outer shareable. */
>  u64 vgic_sanitise_shareability(u64 field)
>  {
> @@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
>   * We take some special care here to fix the calculation of the register
>   * offset.
>   */
> -#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc)	\
> +#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
>  	{								\
>  		.reg_offset = off,					\
>  		.bits_per_irq = bpi,					\
> @@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
>  		.access_flags = acc,					\
>  		.read = rd,						\
>  		.write = wr,						\
> +		.uaccess_read = ur,					\
> +		.uaccess_write = uw,					\
>  	}
>  
>  static const struct vgic_register_region vgic_v3_dist_registers[] = {
> @@ -380,40 +438,42 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
>  		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> -		vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
> +		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
> -		vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
> +		vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
> -		vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
> +		vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
> -		vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
> +		vgic_mmio_read_pending, vgic_mmio_write_spending,
> +		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
> -		vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
> +		vgic_mmio_read_pending, vgic_mmio_write_cpending,
> +		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 1,

>From the spec you quoted in the commit message:

+    Accesses to GICD_ICPENDR register region and GICR_ICPENDR0 registers have
+    RAZ/WI semantics, meaning that reads always return 0 and writes are always
+    ignored.

>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
> -		vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
> +		vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
> -		vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
> +		vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
> -		vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
> -		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> +		vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
> +		8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
> -		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> +		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
>  		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
> -		vgic_mmio_read_config, vgic_mmio_write_config, 2,
> +		vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
> -		vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> +		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
> -		vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
> +		vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
>  		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
>  		vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
> @@ -451,11 +511,13 @@ static const struct vgic_register_region vgic_v3_sgibase_registers[] = {
>  	REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
>  		vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
>  		VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
> -		vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
> +	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
> +		vgic_mmio_read_pending, vgic_mmio_write_spending,
> +		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 4,
>  		VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
> -		vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
> +	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
> +		vgic_mmio_read_pending, vgic_mmio_write_cpending,
> +		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 4,

Same issue.

>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
>  		vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index e18b30d..31f85df 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
>  	return false;
>  }
>  
> +static const struct vgic_register_region *
> +vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
> +{
> +	const struct vgic_register_region *region;
> +
> +	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> +				       addr - iodev->base_addr);
> +	if (!region || !check_region(region, addr, len))
> +		return NULL;
> +
> +	return region;
> +}
> +
> +static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +			     gpa_t addr, u32 *val)
> +{
> +	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> +	const struct vgic_register_region *region;
> +	struct kvm_vcpu *r_vcpu;
> +
> +	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> +	if (!region) {
> +		*val = 0;
> +		return 0;
> +	}
> +
> +	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> +	if (region->uaccess_read)
> +		*val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
> +	else
> +		*val = region->read(r_vcpu, addr, sizeof(u32));
> +
> +	return 0;
> +}
> +
> +static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +			      gpa_t addr, const u32 *val)
> +{
> +	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> +	const struct vgic_register_region *region;
> +	struct kvm_vcpu *r_vcpu;
> +
> +	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> +	if (!region)
> +		return 0;
> +
> +	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> +	if (region->uaccess_write)
> +		region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
> +	else
> +		region->write(r_vcpu, addr, sizeof(u32), *val);
> +
> +	return 0;
> +}
> +
> +/*
> + * Userland access to VGIC registers.
> + */
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> +		 bool is_write, int offset, u32 *val)
> +{
> +	if (is_write)
> +		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
> +	else
> +		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
> +}
> +
>  static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  			      gpa_t addr, int len, void *val)
>  {
> @@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  	const struct vgic_register_region *region;
>  	unsigned long data = 0;
>  
> -	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> -				       addr - iodev->base_addr);
> -	if (!region || !check_region(region, addr, len)) {
> +	region = vgic_get_mmio_region(iodev, addr, len);
> +	if (!region) {
>  		memset(val, 0, len);
>  		return 0;
>  	}
> @@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  	const struct vgic_register_region *region;
>  	unsigned long data = vgic_data_mmio_bus_to_host(val, len);
>  
> -	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> -				       addr - iodev->base_addr);
> +	region = vgic_get_mmio_region(iodev, addr, len);
>  	if (!region)
>  		return 0;
>  
> -	if (!check_region(region, addr, len))
> -		return 0;
> -
>  	switch (iodev->iodev_type) {
>  	case IODEV_CPUIF:
>  		region->write(vcpu, addr, len, data);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 4c34d39..97e6df7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -34,6 +34,10 @@ struct vgic_register_region {
>  				  gpa_t addr, unsigned int len,
>  				  unsigned long val);
>  	};
> +	unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
> +				      unsigned int len);
> +	void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
> +			      unsigned int len, unsigned long val);
>  };
>  
>  extern struct kvm_io_device_ops kvm_io_gic_ops;
> @@ -86,6 +90,18 @@ extern struct kvm_io_device_ops kvm_io_gic_ops;
>  		.write = wr,						\
>  	}
>  
> +#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
> +	{								\
> +		.reg_offset = off,					\
> +		.bits_per_irq = 0,					\
> +		.len = length,						\
> +		.access_flags = acc,					\
> +		.read = rd,						\
> +		.write = wr,						\
> +		.uaccess_read = urd,					\
> +		.uaccess_write = uwr,					\
> +	}
> +
>  int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
>  				  struct vgic_register_region *reg_desc,
>  				  struct vgic_io_device *region,
> @@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
>  			    gpa_t addr, unsigned int len,
>  			    unsigned long val);
>  
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> +		 bool is_write, int offset, u32 *val);
> +
>  unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH v5 1/6] arm/arm64: vgic-new: Implement support for userspace access
@ 2016-09-16 16:31     ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 16:31 UTC (permalink / raw)
  To: linux-arm-kernel

On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> 
> Read and write of some registers like ISPENDR and ICPENDR
> from userspace requires special handling when compared to
> guest access for these registers.
> 
> Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> for handling of ISPENDR, ICPENDR registers handling.
> 
> Add infrastructure to support guest and userspace read
> and write for the required registers
> Also moved vgic_uaccess from vgic-mmio-v2.c to vgic-mmio.c
> 
> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> ---
>  virt/kvm/arm/vgic/vgic-mmio-v2.c | 25 ----------
>  virt/kvm/arm/vgic/vgic-mmio-v3.c | 98 ++++++++++++++++++++++++++++++++--------
>  virt/kvm/arm/vgic/vgic-mmio.c    | 78 ++++++++++++++++++++++++++++----
>  virt/kvm/arm/vgic/vgic-mmio.h    | 19 ++++++++
>  4 files changed, 169 insertions(+), 51 deletions(-)
> 
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v2.c b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> index b44b359..0b32f40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v2.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v2.c
> @@ -406,31 +406,6 @@ int vgic_v2_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>  	return -ENXIO;
>  }
>  
> -/*
> - * When userland tries to access the VGIC register handlers, we need to
> - * create a usable struct vgic_io_device to be passed to the handlers and we
> - * have to set up a buffer similar to what would have happened if a guest MMIO
> - * access occurred, including doing endian conversions on BE systems.
> - */
> -static int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> -			bool is_write, int offset, u32 *val)
> -{
> -	unsigned int len = 4;
> -	u8 buf[4];
> -	int ret;
> -
> -	if (is_write) {
> -		vgic_data_host_to_mmio_bus(buf, len, *val);
> -		ret = kvm_io_gic_ops.write(vcpu, &dev->dev, offset, len, buf);
> -	} else {
> -		ret = kvm_io_gic_ops.read(vcpu, &dev->dev, offset, len, buf);
> -		if (!ret)
> -			*val = vgic_data_mmio_bus_to_host(buf, len);
> -	}
> -
> -	return ret;
> -}
> -
>  int vgic_v2_cpuif_uaccess(struct kvm_vcpu *vcpu, bool is_write,
>  			  int offset, u32 *val)
>  {
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 0d3c76a..edd3d40 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -209,6 +209,62 @@ static unsigned long vgic_mmio_read_v3_idregs(struct kvm_vcpu *vcpu,
>  	return 0;
>  }
>  
> +static unsigned long vgic_uaccess_read_v3_pending(struct kvm_vcpu *vcpu,

Maming is completely inconsistent. This should read "vgic_v3_uaccess_read_pending".

> +						  gpa_t addr, unsigned int len)
> +{
> +	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> +	u32 value = 0;
> +	int i;
> +
> +	/*
> +	 * A level triggerred interrupt pending state is latched in both
> +	 * "soft_pending" and "line_level" variables. Userspace will save
> +	 * and restore soft_pending and line_level separately.
> +	 * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> +	 * handling of ISPENDR and ICPENDR.
> +	 */
> +	for (i = 0; i < len * 8; i++) {
> +		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> +		if (irq->config == VGIC_CONFIG_LEVEL && irq->soft_pending)
> +			value |= (1U << i);
> +		if (irq->config == VGIC_CONFIG_EDGE && irq->pending)
> +			value |= (1U << i);
> +
> +		vgic_put_irq(vcpu->kvm, irq);
> +	}
> +
> +	return value;
> +}
> +
> +static void vgic_uaccess_write_v3_pending(struct kvm_vcpu *vcpu,

Same here.

> +					  gpa_t addr, unsigned int len,
> +					  unsigned long val)
> +{
> +	u32 intid = VGIC_ADDR_TO_INTID(addr, 1);
> +	int i;
> +
> +	for (i = 0; i < len * 8; i++) {
> +		struct vgic_irq *irq = vgic_get_irq(vcpu->kvm, vcpu, intid + i);
> +
> +		spin_lock(&irq->irq_lock);
> +		if (test_bit(i, &val)) {
> +			irq->pending = true;
> +			irq->soft_pending = true;
> +			vgic_queue_irq_unlock(vcpu->kvm, irq);
> +		} else {
> +			irq->soft_pending = false;
> +			if (irq->config == VGIC_CONFIG_EDGE ||
> +			    (irq->config == VGIC_CONFIG_LEVEL &&
> +			    !irq->line_level))
> +				irq->pending = false;
> +			spin_unlock(&irq->irq_lock);
> +		}
> +
> +		vgic_put_irq(vcpu->kvm, irq);
> +	}
> +}
> +
>  /* We want to avoid outer shareable. */
>  u64 vgic_sanitise_shareability(u64 field)
>  {
> @@ -358,7 +414,7 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
>   * We take some special care here to fix the calculation of the register
>   * offset.
>   */
> -#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, bpi, acc)	\
> +#define REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(off, rd, wr, ur, uw, bpi, acc) \
>  	{								\
>  		.reg_offset = off,					\
>  		.bits_per_irq = bpi,					\
> @@ -373,6 +429,8 @@ static void vgic_mmio_write_pendbase(struct kvm_vcpu *vcpu,
>  		.access_flags = acc,					\
>  		.read = rd,						\
>  		.write = wr,						\
> +		.uaccess_read = ur,					\
> +		.uaccess_write = uw,					\
>  	}
>  
>  static const struct vgic_register_region vgic_v3_dist_registers[] = {
> @@ -380,40 +438,42 @@ static const struct vgic_register_region vgic_v3_dist_registers[] = {
>  		vgic_mmio_read_v3_misc, vgic_mmio_write_v3_misc, 16,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGROUPR,
> -		vgic_mmio_read_rao, vgic_mmio_write_wi, 1,
> +		vgic_mmio_read_rao, vgic_mmio_write_wi, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISENABLER,
> -		vgic_mmio_read_enable, vgic_mmio_write_senable, 1,
> +		vgic_mmio_read_enable, vgic_mmio_write_senable, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICENABLER,
> -		vgic_mmio_read_enable, vgic_mmio_write_cenable, 1,
> +		vgic_mmio_read_enable, vgic_mmio_write_cenable, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISPENDR,
> -		vgic_mmio_read_pending, vgic_mmio_write_spending, 1,
> +		vgic_mmio_read_pending, vgic_mmio_write_spending,
> +		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICPENDR,
> -		vgic_mmio_read_pending, vgic_mmio_write_cpending, 1,
> +		vgic_mmio_read_pending, vgic_mmio_write_cpending,
> +		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 1,

>From the spec you quoted in the commit message:

+    Accesses to GICD_ICPENDR register region and GICR_ICPENDR0 registers have
+    RAZ/WI semantics, meaning that reads always return 0 and writes are always
+    ignored.

>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ISACTIVER,
> -		vgic_mmio_read_active, vgic_mmio_write_sactive, 1,
> +		vgic_mmio_read_active, vgic_mmio_write_sactive, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICACTIVER,
> -		vgic_mmio_read_active, vgic_mmio_write_cactive, 1,
> +		vgic_mmio_read_active, vgic_mmio_write_cactive, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IPRIORITYR,
> -		vgic_mmio_read_priority, vgic_mmio_write_priority, 8,
> -		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
> +		vgic_mmio_read_priority, vgic_mmio_write_priority, NULL, NULL,
> +		8, VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ITARGETSR,
> -		vgic_mmio_read_raz, vgic_mmio_write_wi, 8,
> +		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 8,
>  		VGIC_ACCESS_32bit | VGIC_ACCESS_8bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_ICFGR,
> -		vgic_mmio_read_config, vgic_mmio_write_config, 2,
> +		vgic_mmio_read_config, vgic_mmio_write_config, NULL, NULL, 2,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IGRPMODR,
> -		vgic_mmio_read_raz, vgic_mmio_write_wi, 1,
> +		vgic_mmio_read_raz, vgic_mmio_write_wi, NULL, NULL, 1,
>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_BITS_PER_IRQ_SHARED(GICD_IROUTER,
> -		vgic_mmio_read_irouter, vgic_mmio_write_irouter, 64,
> +		vgic_mmio_read_irouter, vgic_mmio_write_irouter, NULL, NULL, 64,
>  		VGIC_ACCESS_64bit | VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_LENGTH(GICD_IDREGS,
>  		vgic_mmio_read_v3_idregs, vgic_mmio_write_wi, 48,
> @@ -451,11 +511,13 @@ static const struct vgic_register_region vgic_v3_sgibase_registers[] = {
>  	REGISTER_DESC_WITH_LENGTH(GICR_ICENABLER0,
>  		vgic_mmio_read_enable, vgic_mmio_write_cenable, 4,
>  		VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICR_ISPENDR0,
> -		vgic_mmio_read_pending, vgic_mmio_write_spending, 4,
> +	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ISPENDR0,
> +		vgic_mmio_read_pending, vgic_mmio_write_spending,
> +		vgic_uaccess_read_v3_pending, vgic_uaccess_write_v3_pending, 4,
>  		VGIC_ACCESS_32bit),
> -	REGISTER_DESC_WITH_LENGTH(GICR_ICPENDR0,
> -		vgic_mmio_read_pending, vgic_mmio_write_cpending, 4,
> +	REGISTER_DESC_WITH_LENGTH_UACCESS(GICR_ICPENDR0,
> +		vgic_mmio_read_pending, vgic_mmio_write_cpending,
> +		vgic_uaccess_read_v3_pending, vgic_mmio_write_wi, 4,

Same issue.

>  		VGIC_ACCESS_32bit),
>  	REGISTER_DESC_WITH_LENGTH(GICR_ISACTIVER0,
>  		vgic_mmio_read_active, vgic_mmio_write_sactive, 4,
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.c b/virt/kvm/arm/vgic/vgic-mmio.c
> index e18b30d..31f85df 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio.c
> @@ -468,6 +468,73 @@ static bool check_region(const struct vgic_register_region *region,
>  	return false;
>  }
>  
> +static const struct vgic_register_region *
> +vgic_get_mmio_region(struct vgic_io_device *iodev, gpa_t addr, int len)
> +{
> +	const struct vgic_register_region *region;
> +
> +	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> +				       addr - iodev->base_addr);
> +	if (!region || !check_region(region, addr, len))
> +		return NULL;
> +
> +	return region;
> +}
> +
> +static int vgic_uaccess_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +			     gpa_t addr, u32 *val)
> +{
> +	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> +	const struct vgic_register_region *region;
> +	struct kvm_vcpu *r_vcpu;
> +
> +	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> +	if (!region) {
> +		*val = 0;
> +		return 0;
> +	}
> +
> +	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> +	if (region->uaccess_read)
> +		*val = region->uaccess_read(r_vcpu, addr, sizeof(u32));
> +	else
> +		*val = region->read(r_vcpu, addr, sizeof(u32));
> +
> +	return 0;
> +}
> +
> +static int vgic_uaccess_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
> +			      gpa_t addr, const u32 *val)
> +{
> +	struct vgic_io_device *iodev = kvm_to_vgic_iodev(dev);
> +	const struct vgic_register_region *region;
> +	struct kvm_vcpu *r_vcpu;
> +
> +	region = vgic_get_mmio_region(iodev, addr, sizeof(u32));
> +	if (!region)
> +		return 0;
> +
> +	r_vcpu = iodev->redist_vcpu ? iodev->redist_vcpu : vcpu;
> +	if (region->uaccess_write)
> +		region->uaccess_write(r_vcpu, addr, sizeof(u32), *val);
> +	else
> +		region->write(r_vcpu, addr, sizeof(u32), *val);
> +
> +	return 0;
> +}
> +
> +/*
> + * Userland access to VGIC registers.
> + */
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> +		 bool is_write, int offset, u32 *val)
> +{
> +	if (is_write)
> +		return vgic_uaccess_write(vcpu, &dev->dev, offset, val);
> +	else
> +		return vgic_uaccess_read(vcpu, &dev->dev, offset, val);
> +}
> +
>  static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  			      gpa_t addr, int len, void *val)
>  {
> @@ -475,9 +542,8 @@ static int dispatch_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  	const struct vgic_register_region *region;
>  	unsigned long data = 0;
>  
> -	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> -				       addr - iodev->base_addr);
> -	if (!region || !check_region(region, addr, len)) {
> +	region = vgic_get_mmio_region(iodev, addr, len);
> +	if (!region) {
>  		memset(val, 0, len);
>  		return 0;
>  	}
> @@ -508,14 +574,10 @@ static int dispatch_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
>  	const struct vgic_register_region *region;
>  	unsigned long data = vgic_data_mmio_bus_to_host(val, len);
>  
> -	region = vgic_find_mmio_region(iodev->regions, iodev->nr_regions,
> -				       addr - iodev->base_addr);
> +	region = vgic_get_mmio_region(iodev, addr, len);
>  	if (!region)
>  		return 0;
>  
> -	if (!check_region(region, addr, len))
> -		return 0;
> -
>  	switch (iodev->iodev_type) {
>  	case IODEV_CPUIF:
>  		region->write(vcpu, addr, len, data);
> diff --git a/virt/kvm/arm/vgic/vgic-mmio.h b/virt/kvm/arm/vgic/vgic-mmio.h
> index 4c34d39..97e6df7 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio.h
> +++ b/virt/kvm/arm/vgic/vgic-mmio.h
> @@ -34,6 +34,10 @@ struct vgic_register_region {
>  				  gpa_t addr, unsigned int len,
>  				  unsigned long val);
>  	};
> +	unsigned long (*uaccess_read)(struct kvm_vcpu *vcpu, gpa_t addr,
> +				      unsigned int len);
> +	void (*uaccess_write)(struct kvm_vcpu *vcpu, gpa_t addr,
> +			      unsigned int len, unsigned long val);
>  };
>  
>  extern struct kvm_io_device_ops kvm_io_gic_ops;
> @@ -86,6 +90,18 @@ extern struct kvm_io_device_ops kvm_io_gic_ops;
>  		.write = wr,						\
>  	}
>  
> +#define REGISTER_DESC_WITH_LENGTH_UACCESS(off, rd, wr, urd, uwr, length, acc) \
> +	{								\
> +		.reg_offset = off,					\
> +		.bits_per_irq = 0,					\
> +		.len = length,						\
> +		.access_flags = acc,					\
> +		.read = rd,						\
> +		.write = wr,						\
> +		.uaccess_read = urd,					\
> +		.uaccess_write = uwr,					\
> +	}
> +
>  int kvm_vgic_register_mmio_region(struct kvm *kvm, struct kvm_vcpu *vcpu,
>  				  struct vgic_register_region *reg_desc,
>  				  struct vgic_io_device *region,
> @@ -158,6 +174,9 @@ void vgic_mmio_write_config(struct kvm_vcpu *vcpu,
>  			    gpa_t addr, unsigned int len,
>  			    unsigned long val);
>  
> +int vgic_uaccess(struct kvm_vcpu *vcpu, struct vgic_io_device *dev,
> +		 bool is_write, int offset, u32 *val);
> +
>  unsigned int vgic_v2_init_dist_iodev(struct vgic_io_device *dev);
>  
>  unsigned int vgic_v3_init_dist_iodev(struct vgic_io_device *dev);
> 

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 14:36     ` Marc Zyngier
@ 2016-09-16 16:57       ` Vijay Kilari
  -1 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-16 16:57 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> VGICv3 CPU interface registers are accessed using
>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> is used to identify the cpu for registers access.
>>
>> The version of VGIC v3 specification is define here
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>  arch/arm64/kvm/Makefile             |   1 +
>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>  7 files changed, 385 insertions(+)
>>
>> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> index 56dc08d..91c7137 100644
>> --- a/arch/arm64/include/uapi/asm/kvm.h
>> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>>                       (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>>  #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT      0
>>  #define   KVM_DEV_ARM_VGIC_OFFSET_MASK       (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> +#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>>  #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>>  #define KVM_DEV_ARM_VGIC_GRP_CTRL    4
>>  #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
>> +
>>  #define   KVM_DEV_ARM_VGIC_CTRL_INIT 0
>>
>>  /* Device Control API on vcpu fd */
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index d50a82a..1a14e29 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>>  kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
>> index 88d83d3..d4e9c7d 100644
>> --- a/include/linux/irqchip/arm-gic-v3.h
>> +++ b/include/linux/irqchip/arm-gic-v3.h
>> @@ -355,6 +355,27 @@
>>  #define ICC_CTLR_EL1_EOImode_SHIFT   (1)
>>  #define ICC_CTLR_EL1_EOImode_drop_dir        (0U << ICC_CTLR_EL1_EOImode_SHIFT)
>>  #define ICC_CTLR_EL1_EOImode_drop    (1U << ICC_CTLR_EL1_EOImode_SHIFT)
>> +#define ICC_CTLR_EL1_EOImode_MASK    (1 << ICC_CTLR_EL1_EOImode_SHIFT)
>> +#define ICC_CTLR_EL1_CBPR_SHIFT              (0)
>> +#define ICC_CTLR_EL1_CBPR_MASK               (1 << ICC_CTLR_EL1_CBPR_SHIFT)
>> +#define ICC_CTLR_EL1_PRI_BITS_SHIFT  (8)
>> +#define ICC_CTLR_EL1_PRI_BITS_MASK   (0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
>> +#define ICC_CTLR_EL1_ID_BITS_SHIFT   (11)
>> +#define ICC_CTLR_EL1_ID_BITS_MASK    (0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
>> +#define ICC_CTLR_EL1_SEIS_SHIFT              (14)
>> +#define ICC_CTLR_EL1_SEIS_MASK               (0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
>> +#define ICC_CTLR_EL1_A3V_SHIFT               (15)
>> +#define ICC_CTLR_EL1_A3V_MASK                (0x1 << ICC_CTLR_EL1_A3V_SHIFT)
>> +#define ICC_PMR_EL1_SHIFT            (0)
>> +#define ICC_PMR_EL1_MASK             (0xff << ICC_PMR_EL1_SHIFT)
>> +#define ICC_BPR0_EL1_SHIFT           (0)
>> +#define ICC_BPR0_EL1_MASK            (0x7 << ICC_BPR0_EL1_SHIFT)
>> +#define ICC_BPR1_EL1_SHIFT           (0)
>> +#define ICC_BPR1_EL1_MASK            (0x7 << ICC_BPR1_EL1_SHIFT)
>> +#define ICC_IGRPEN0_EL1_SHIFT                (0)
>> +#define ICC_IGRPEN0_EL1_MASK         (1 << ICC_IGRPEN0_EL1_SHIFT)
>> +#define ICC_IGRPEN1_EL1_SHIFT                (0)
>> +#define ICC_IGRPEN1_EL1_MASK         (1 << ICC_IGRPEN1_EL1_SHIFT)
>>  #define ICC_SRE_EL1_SRE                      (1U << 0)
>>
>>  /*
>> @@ -398,6 +419,15 @@
>>  #define ICH_VMCR_ENG1_SHIFT          1
>>  #define ICH_VMCR_ENG1_MASK           (1 << ICH_VMCR_ENG1_SHIFT)
>>
>> +#define ICH_VTR_PRI_BITS_SHIFT               29
>> +#define ICH_VTR_PRI_BITS_MASK                (7 << ICH_VTR_PRI_BITS_SHIFT)
>> +#define ICH_VTR_ID_BITS_SHIFT                23
>> +#define ICH_VTR_ID_BITS_MASK         (7 << ICH_VTR_ID_BITS_SHIFT)
>> +#define ICH_VTR_SEIS_SHIFT           22
>> +#define ICH_VTR_SEIS_MASK            (1 << ICH_VTR_SEIS_SHIFT)
>> +#define ICH_VTR_A3V_SHIFT            21
>> +#define ICH_VTR_A3V_MASK             (1 << ICH_VTR_A3V_SHIFT)
>> +
>
> Same thing here. Please group all the additions to this file into a
> single, independent patch.
>
>>  #define ICC_IAR1_EL1_SPURIOUS                0x3ff
>>
>>  #define ICC_SRE_EL2_SRE                      (1 << 0)
>> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> index a4656fc..a48101f 100644
>> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> @@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>>               if (!is_write)
>>                       *reg = tmp32;
>>               break;
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 regid;
>> +
>> +             regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> +             ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> +                                               regid, reg);
>> +             break;
>> +     }
>>       default:
>>               ret = -EINVAL;
>>               break;
>> @@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>>               reg = tmp32;
>>               return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> +             u64 reg;
>> +
>> +             if (get_user(reg, uaddr))
>> +                     return -EFAULT;
>> +
>> +             return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>> +     }
>>       }
>>       return -ENXIO;
>>  }
>> @@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>>               tmp32 = reg;
>>               return put_user(tmp32, uaddr);
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> +             u64 reg;
>> +
>> +             ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
>> +             if (ret)
>> +                     return ret;
>> +             return put_user(reg, uaddr);
>> +     }
>>       }
>>
>>       return -ENXIO;
>> @@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>>               break;
>>       case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>>       case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>>               return vgic_v3_has_attr_regs(dev, attr);
>>       case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>>               return 0;
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index 83dece8..af748d7 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,6 +23,7 @@
>>
>>  #include "vgic.h"
>>  #include "vgic-mmio.h"
>> +#include "sys_regs.h"
>>
>>  /* extract @num bytes at @offset bytes offset in data */
>>  unsigned long extract_bytes(u64 data, unsigned int offset,
>> @@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>>               nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>>               break;
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 reg, id;
>> +             unsigned long mpidr;
>> +             struct kvm_vcpu *vcpu;
>> +
>> +             mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> +                      KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> +
>> +             vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
>
> No. Your "mpidr" variable is not an MPIDR.  It represents the same
> thing, but not with the right format.

Ok. I will format in the form of MPIDR reg.

>
>> +             if (!vcpu)
>> +                     return -EINVAL;
>> +             if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
>
> How can that happen?

OK. This check can be dropped as kvm_for_each_vcpu() already checks
for only online_vcpus.
>
>> +                     return -EINVAL;
>> +
>> +             id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> +             return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
>> +     }
>>       default:
>>               return -ENXIO;
>>       }
>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> new file mode 100644
>> index 0000000..8e4f403
>> --- /dev/null
>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> @@ -0,0 +1,296 @@
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +#include <linux/kvm.h>
>> +#include <linux/kvm_host.h>
>> +#include <kvm/iodev.h>
>> +#include <kvm/arm_vgic.h>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_arm.h>
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "vgic.h"
>> +#include "vgic-mmio.h"
>> +#include "sys_regs.h"
>> +
>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +     u64 val;
>> +     u32 ich_vtr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             val = p->regval;
>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>
> You've ignored my comments again: "What if userspace writes something
> that is incompatible with the current configuration? Wrong number of ID
> bits, or number of priorities?"

IMO, In case of incompatibility,
If ID bits and PRI bits are less than HW supported, it is ok.
If ID bits and PRI bits are greater than HW supported, then warn would be good
enough. Please suggest the behaviour that you think it should be.

>
>> +     } else {
>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>> +
>> +             val = 0;
>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>> +
>> +             p->regval = val;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                        const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> +                         ICC_BPR0_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> +                          ICC_BPR0_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> +                          ICC_BPR1_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> +                          ICC_BPR1_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                           const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> +                          ICC_IGRPEN0_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                           const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> +                          ICC_IGRPEN1_EL1_MASK;
>
> From the previous review comments: "Shouldn't this account for the
> ICC_CTLR_EL1.CBPR setting?"

 Ok. I think this comment is for ICC_BPR1_EL1 access.
I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.
>
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> +                                struct sys_reg_params *p, u8 apr, u8 idx)
>> +{
>> +     struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> +     uint32_t *ap_reg;
>> +
>> +     if (apr)
>> +             ap_reg = &vgicv3->vgic_ap1r[idx];
>> +     else
>> +             ap_reg = &vgicv3->vgic_ap0r[idx];
>> +
>> +     if (p->is_write)
>> +             *ap_reg = p->regval;
>> +     else
>> +             p->regval = *ap_reg;
>> +}
>> +
>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r, u8 apr)
>> +{
>> +     u8 num_pri_bits, idx;
>> +     u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>
> Maybe you should cache this once and for all? It is fairly unlikely to
> change over time...

Is it ok to cache in vgic_global struct?

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-16 16:57       ` Vijay Kilari
  0 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-16 16:57 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>
>> VGICv3 CPU interface registers are accessed using
>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> is used to identify the cpu for registers access.
>>
>> The version of VGIC v3 specification is define here
>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> ---
>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>  arch/arm64/kvm/Makefile             |   1 +
>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>  7 files changed, 385 insertions(+)
>>
>> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
>> index 56dc08d..91c7137 100644
>> --- a/arch/arm64/include/uapi/asm/kvm.h
>> +++ b/arch/arm64/include/uapi/asm/kvm.h
>> @@ -206,9 +206,12 @@ struct kvm_arch_memory_slot {
>>                       (0xffffffffULL << KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT)
>>  #define   KVM_DEV_ARM_VGIC_OFFSET_SHIFT      0
>>  #define   KVM_DEV_ARM_VGIC_OFFSET_MASK       (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>> +#define   KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK (0xffff)
>>  #define KVM_DEV_ARM_VGIC_GRP_NR_IRQS 3
>>  #define KVM_DEV_ARM_VGIC_GRP_CTRL    4
>>  #define KVM_DEV_ARM_VGIC_GRP_REDIST_REGS 5
>> +#define KVM_DEV_ARM_VGIC_CPU_SYSREGS    6
>> +
>>  #define   KVM_DEV_ARM_VGIC_CTRL_INIT 0
>>
>>  /* Device Control API on vcpu fd */
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index d50a82a..1a14e29 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -32,5 +32,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-mmio-v3.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-kvm-device.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-its.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/irqchip.o
>> +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic/vgic-sys-reg-v3.o
>>  kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
>>  kvm-$(CONFIG_KVM_ARM_PMU) += $(KVM)/arm/pmu.o
>> diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
>> index 88d83d3..d4e9c7d 100644
>> --- a/include/linux/irqchip/arm-gic-v3.h
>> +++ b/include/linux/irqchip/arm-gic-v3.h
>> @@ -355,6 +355,27 @@
>>  #define ICC_CTLR_EL1_EOImode_SHIFT   (1)
>>  #define ICC_CTLR_EL1_EOImode_drop_dir        (0U << ICC_CTLR_EL1_EOImode_SHIFT)
>>  #define ICC_CTLR_EL1_EOImode_drop    (1U << ICC_CTLR_EL1_EOImode_SHIFT)
>> +#define ICC_CTLR_EL1_EOImode_MASK    (1 << ICC_CTLR_EL1_EOImode_SHIFT)
>> +#define ICC_CTLR_EL1_CBPR_SHIFT              (0)
>> +#define ICC_CTLR_EL1_CBPR_MASK               (1 << ICC_CTLR_EL1_CBPR_SHIFT)
>> +#define ICC_CTLR_EL1_PRI_BITS_SHIFT  (8)
>> +#define ICC_CTLR_EL1_PRI_BITS_MASK   (0x7 << ICC_CTLR_EL1_PRI_BITS_SHIFT)
>> +#define ICC_CTLR_EL1_ID_BITS_SHIFT   (11)
>> +#define ICC_CTLR_EL1_ID_BITS_MASK    (0x7 << ICC_CTLR_EL1_ID_BITS_SHIFT)
>> +#define ICC_CTLR_EL1_SEIS_SHIFT              (14)
>> +#define ICC_CTLR_EL1_SEIS_MASK               (0x1 << ICC_CTLR_EL1_SEIS_SHIFT)
>> +#define ICC_CTLR_EL1_A3V_SHIFT               (15)
>> +#define ICC_CTLR_EL1_A3V_MASK                (0x1 << ICC_CTLR_EL1_A3V_SHIFT)
>> +#define ICC_PMR_EL1_SHIFT            (0)
>> +#define ICC_PMR_EL1_MASK             (0xff << ICC_PMR_EL1_SHIFT)
>> +#define ICC_BPR0_EL1_SHIFT           (0)
>> +#define ICC_BPR0_EL1_MASK            (0x7 << ICC_BPR0_EL1_SHIFT)
>> +#define ICC_BPR1_EL1_SHIFT           (0)
>> +#define ICC_BPR1_EL1_MASK            (0x7 << ICC_BPR1_EL1_SHIFT)
>> +#define ICC_IGRPEN0_EL1_SHIFT                (0)
>> +#define ICC_IGRPEN0_EL1_MASK         (1 << ICC_IGRPEN0_EL1_SHIFT)
>> +#define ICC_IGRPEN1_EL1_SHIFT                (0)
>> +#define ICC_IGRPEN1_EL1_MASK         (1 << ICC_IGRPEN1_EL1_SHIFT)
>>  #define ICC_SRE_EL1_SRE                      (1U << 0)
>>
>>  /*
>> @@ -398,6 +419,15 @@
>>  #define ICH_VMCR_ENG1_SHIFT          1
>>  #define ICH_VMCR_ENG1_MASK           (1 << ICH_VMCR_ENG1_SHIFT)
>>
>> +#define ICH_VTR_PRI_BITS_SHIFT               29
>> +#define ICH_VTR_PRI_BITS_MASK                (7 << ICH_VTR_PRI_BITS_SHIFT)
>> +#define ICH_VTR_ID_BITS_SHIFT                23
>> +#define ICH_VTR_ID_BITS_MASK         (7 << ICH_VTR_ID_BITS_SHIFT)
>> +#define ICH_VTR_SEIS_SHIFT           22
>> +#define ICH_VTR_SEIS_MASK            (1 << ICH_VTR_SEIS_SHIFT)
>> +#define ICH_VTR_A3V_SHIFT            21
>> +#define ICH_VTR_A3V_MASK             (1 << ICH_VTR_A3V_SHIFT)
>> +
>
> Same thing here. Please group all the additions to this file into a
> single, independent patch.
>
>>  #define ICC_IAR1_EL1_SPURIOUS                0x3ff
>>
>>  #define ICC_SRE_EL2_SRE                      (1 << 0)
>> diff --git a/virt/kvm/arm/vgic/vgic-kvm-device.c b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> index a4656fc..a48101f 100644
>> --- a/virt/kvm/arm/vgic/vgic-kvm-device.c
>> +++ b/virt/kvm/arm/vgic/vgic-kvm-device.c
>> @@ -506,6 +506,14 @@ static int vgic_v3_attr_regs_access(struct kvm_device *dev,
>>               if (!is_write)
>>                       *reg = tmp32;
>>               break;
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 regid;
>> +
>> +             regid = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> +             ret = vgic_v3_cpu_sysregs_uaccess(vcpu, is_write,
>> +                                               regid, reg);
>> +             break;
>> +     }
>>       default:
>>               ret = -EINVAL;
>>               break;
>> @@ -539,6 +547,15 @@ static int vgic_v3_set_attr(struct kvm_device *dev,
>>               reg = tmp32;
>>               return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> +             u64 reg;
>> +
>> +             if (get_user(reg, uaddr))
>> +                     return -EFAULT;
>> +
>> +             return vgic_v3_attr_regs_access(dev, attr, &reg, true);
>> +     }
>>       }
>>       return -ENXIO;
>>  }
>> @@ -565,6 +582,15 @@ static int vgic_v3_get_attr(struct kvm_device *dev,
>>               tmp32 = reg;
>>               return put_user(tmp32, uaddr);
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 __user *uaddr = (u64 __user *)(long)attr->addr;
>> +             u64 reg;
>> +
>> +             ret = vgic_v3_attr_regs_access(dev, attr, &reg, false);
>> +             if (ret)
>> +                     return ret;
>> +             return put_user(reg, uaddr);
>> +     }
>>       }
>>
>>       return -ENXIO;
>> @@ -583,6 +609,7 @@ static int vgic_v3_has_attr(struct kvm_device *dev,
>>               break;
>>       case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
>>       case KVM_DEV_ARM_VGIC_GRP_REDIST_REGS:
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS:
>>               return vgic_v3_has_attr_regs(dev, attr);
>>       case KVM_DEV_ARM_VGIC_GRP_NR_IRQS:
>>               return 0;
>> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> index 83dece8..af748d7 100644
>> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
>> @@ -23,6 +23,7 @@
>>
>>  #include "vgic.h"
>>  #include "vgic-mmio.h"
>> +#include "sys_regs.h"
>>
>>  /* extract @num bytes at @offset bytes offset in data */
>>  unsigned long extract_bytes(u64 data, unsigned int offset,
>> @@ -639,6 +640,23 @@ int vgic_v3_has_attr_regs(struct kvm_device *dev, struct kvm_device_attr *attr)
>>               nr_regions = ARRAY_SIZE(vgic_v3_rdbase_registers);
>>               break;
>>       }
>> +     case KVM_DEV_ARM_VGIC_CPU_SYSREGS: {
>> +             u64 reg, id;
>> +             unsigned long mpidr;
>> +             struct kvm_vcpu *vcpu;
>> +
>> +             mpidr = (attr->attr & KVM_DEV_ARM_VGIC_V3_MPIDR_MASK) >>
>> +                      KVM_DEV_ARM_VGIC_V3_MPIDR_SHIFT;
>> +
>> +             vcpu = kvm_mpidr_to_vcpu(dev->kvm, mpidr);
>
> No. Your "mpidr" variable is not an MPIDR.  It represents the same
> thing, but not with the right format.

Ok. I will format in the form of MPIDR reg.

>
>> +             if (!vcpu)
>> +                     return -EINVAL;
>> +             if (vcpu->vcpu_id >= atomic_read(&dev->kvm->online_vcpus))
>
> How can that happen?

OK. This check can be dropped as kvm_for_each_vcpu() already checks
for only online_vcpus.
>
>> +                     return -EINVAL;
>> +
>> +             id = (attr->attr & KVM_DEV_ARM_VGIC_SYSREG_INSTR_MASK);
>> +             return vgic_v3_has_cpu_sysregs_attr(vcpu, 0, id, &reg);
>> +     }
>>       default:
>>               return -ENXIO;
>>       }
>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> new file mode 100644
>> index 0000000..8e4f403
>> --- /dev/null
>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> @@ -0,0 +1,296 @@
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +#include <linux/kvm.h>
>> +#include <linux/kvm_host.h>
>> +#include <kvm/iodev.h>
>> +#include <kvm/arm_vgic.h>
>> +#include <asm/kvm_emulate.h>
>> +#include <asm/kvm_arm.h>
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "vgic.h"
>> +#include "vgic-mmio.h"
>> +#include "sys_regs.h"
>> +
>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +     u64 val;
>> +     u32 ich_vtr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             val = p->regval;
>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>
> You've ignored my comments again: "What if userspace writes something
> that is incompatible with the current configuration? Wrong number of ID
> bits, or number of priorities?"

IMO, In case of incompatibility,
If ID bits and PRI bits are less than HW supported, it is ok.
If ID bits and PRI bits are greater than HW supported, then warn would be good
enough. Please suggest the behaviour that you think it should be.

>
>> +     } else {
>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>> +
>> +             val = 0;
>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>> +
>> +             p->regval = val;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                        const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>> +                         ICC_BPR0_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>> +                          ICC_BPR0_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>> +                          ICC_BPR1_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>> +                          ICC_BPR1_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                           const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>> +                          ICC_IGRPEN0_EL1_MASK;
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                           const struct sys_reg_desc *r)
>> +{
>> +     struct vgic_vmcr vmcr;
>> +
>> +     vgic_get_vmcr(vcpu, &vmcr);
>> +     if (p->is_write) {
>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>> +             vgic_set_vmcr(vcpu, &vmcr);
>> +     } else {
>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>> +                          ICC_IGRPEN1_EL1_MASK;
>
> From the previous review comments: "Shouldn't this account for the
> ICC_CTLR_EL1.CBPR setting?"

 Ok. I think this comment is for ICC_BPR1_EL1 access.
I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.
>
>> +     }
>> +
>> +     return true;
>> +}
>> +
>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>> +                                struct sys_reg_params *p, u8 apr, u8 idx)
>> +{
>> +     struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>> +     uint32_t *ap_reg;
>> +
>> +     if (apr)
>> +             ap_reg = &vgicv3->vgic_ap1r[idx];
>> +     else
>> +             ap_reg = &vgicv3->vgic_ap0r[idx];
>> +
>> +     if (p->is_write)
>> +             *ap_reg = p->regval;
>> +     else
>> +             p->regval = *ap_reg;
>> +}
>> +
>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> +                         const struct sys_reg_desc *r, u8 apr)
>> +{
>> +     u8 num_pri_bits, idx;
>> +     u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>
> Maybe you should cache this once and for all? It is fairly unlikely to
> change over time...

Is it ok to cache in vgic_global struct?

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 16:57       ` Vijay Kilari
@ 2016-09-16 17:07         ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 17:07 UTC (permalink / raw)
  To: Vijay Kilari; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

On 16/09/16 17:57, Vijay Kilari wrote:
> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>
>>> VGICv3 CPU interface registers are accessed using
>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>> is used to identify the cpu for registers access.
>>>
>>> The version of VGIC v3 specification is define here
>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>
>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>> ---
>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>  arch/arm64/kvm/Makefile             |   1 +
>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>  7 files changed, 385 insertions(+)

[...]

>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>> new file mode 100644
>>> index 0000000..8e4f403
>>> --- /dev/null
>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>> @@ -0,0 +1,296 @@
>>> +#include <linux/irqchip/arm-gic-v3.h>
>>> +#include <linux/kvm.h>
>>> +#include <linux/kvm_host.h>
>>> +#include <kvm/iodev.h>
>>> +#include <kvm/arm_vgic.h>
>>> +#include <asm/kvm_emulate.h>
>>> +#include <asm/kvm_arm.h>
>>> +#include <asm/kvm_mmu.h>
>>> +
>>> +#include "vgic.h"
>>> +#include "vgic-mmio.h"
>>> +#include "sys_regs.h"
>>> +
>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +     u64 val;
>>> +     u32 ich_vtr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             val = p->regval;
>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>
>> You've ignored my comments again: "What if userspace writes something
>> that is incompatible with the current configuration? Wrong number of ID
>> bits, or number of priorities?"
> 
> IMO, In case of incompatibility,
> If ID bits and PRI bits are less than HW supported, it is ok.

Yes. But you also need to track of what the guest has programmed in
order to be able to migrate it back to its original configuration.

> If ID bits and PRI bits are greater than HW supported, then warn would be good
> enough. Please suggest the behaviour that you think it should be.

No, it is an error, plain and simple. You cannot run in this condition.

> 
>>
>>> +     } else {
>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>> +
>>> +             val = 0;
>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>>> +
>>> +             p->regval = val;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                        const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>>> +                         ICC_BPR0_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>>> +                          ICC_BPR0_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>>> +                          ICC_BPR1_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>>> +                          ICC_BPR1_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                           const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>>> +                          ICC_IGRPEN0_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                           const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>>> +                          ICC_IGRPEN1_EL1_MASK;
>>
>> From the previous review comments: "Shouldn't this account for the
>> ICC_CTLR_EL1.CBPR setting?"
> 
>  Ok. I think this comment is for ICC_BPR1_EL1 access.

Yes, sorry about the misplaced comment.

> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.

The reverse is also true: you also need to account the value of
ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.

>>
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>>> +                                struct sys_reg_params *p, u8 apr, u8 idx)
>>> +{
>>> +     struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>>> +     uint32_t *ap_reg;
>>> +
>>> +     if (apr)
>>> +             ap_reg = &vgicv3->vgic_ap1r[idx];
>>> +     else
>>> +             ap_reg = &vgicv3->vgic_ap0r[idx];
>>> +
>>> +     if (p->is_write)
>>> +             *ap_reg = p->regval;
>>> +     else
>>> +             p->regval = *ap_reg;
>>> +}
>>> +
>>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r, u8 apr)
>>> +{
>>> +     u8 num_pri_bits, idx;
>>> +     u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>
>> Maybe you should cache this once and for all? It is fairly unlikely to
>> change over time...
> 
> Is it ok to cache in vgic_global struct?

I can't see why not.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-16 17:07         ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-16 17:07 UTC (permalink / raw)
  To: linux-arm-kernel

On 16/09/16 17:57, Vijay Kilari wrote:
> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>
>>> VGICv3 CPU interface registers are accessed using
>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>> is used to identify the cpu for registers access.
>>>
>>> The version of VGIC v3 specification is define here
>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>
>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>> ---
>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>  arch/arm64/kvm/Makefile             |   1 +
>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>  7 files changed, 385 insertions(+)

[...]

>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>> new file mode 100644
>>> index 0000000..8e4f403
>>> --- /dev/null
>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>> @@ -0,0 +1,296 @@
>>> +#include <linux/irqchip/arm-gic-v3.h>
>>> +#include <linux/kvm.h>
>>> +#include <linux/kvm_host.h>
>>> +#include <kvm/iodev.h>
>>> +#include <kvm/arm_vgic.h>
>>> +#include <asm/kvm_emulate.h>
>>> +#include <asm/kvm_arm.h>
>>> +#include <asm/kvm_mmu.h>
>>> +
>>> +#include "vgic.h"
>>> +#include "vgic-mmio.h"
>>> +#include "sys_regs.h"
>>> +
>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +     u64 val;
>>> +     u32 ich_vtr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             val = p->regval;
>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>
>> You've ignored my comments again: "What if userspace writes something
>> that is incompatible with the current configuration? Wrong number of ID
>> bits, or number of priorities?"
> 
> IMO, In case of incompatibility,
> If ID bits and PRI bits are less than HW supported, it is ok.

Yes. But you also need to track of what the guest has programmed in
order to be able to migrate it back to its original configuration.

> If ID bits and PRI bits are greater than HW supported, then warn would be good
> enough. Please suggest the behaviour that you think it should be.

No, it is an error, plain and simple. You cannot run in this condition.

> 
>>
>>> +     } else {
>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>> +
>>> +             val = 0;
>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>>> +
>>> +             p->regval = val;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                        const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>>> +                         ICC_BPR0_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>>> +                          ICC_BPR0_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>>> +                          ICC_BPR1_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>>> +                          ICC_BPR1_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                           const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>>> +                          ICC_IGRPEN0_EL1_MASK;
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                           const struct sys_reg_desc *r)
>>> +{
>>> +     struct vgic_vmcr vmcr;
>>> +
>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>> +     if (p->is_write) {
>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>> +     } else {
>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>>> +                          ICC_IGRPEN1_EL1_MASK;
>>
>> From the previous review comments: "Shouldn't this account for the
>> ICC_CTLR_EL1.CBPR setting?"
> 
>  Ok. I think this comment is for ICC_BPR1_EL1 access.

Yes, sorry about the misplaced comment.

> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.

The reverse is also true: you also need to account the value of
ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.

>>
>>> +     }
>>> +
>>> +     return true;
>>> +}
>>> +
>>> +static void vgic_v3_access_apr_reg(struct kvm_vcpu *vcpu,
>>> +                                struct sys_reg_params *p, u8 apr, u8 idx)
>>> +{
>>> +     struct vgic_v3_cpu_if *vgicv3 = &vcpu->arch.vgic_cpu.vgic_v3;
>>> +     uint32_t *ap_reg;
>>> +
>>> +     if (apr)
>>> +             ap_reg = &vgicv3->vgic_ap1r[idx];
>>> +     else
>>> +             ap_reg = &vgicv3->vgic_ap0r[idx];
>>> +
>>> +     if (p->is_write)
>>> +             *ap_reg = p->regval;
>>> +     else
>>> +             p->regval = *ap_reg;
>>> +}
>>> +
>>> +static void access_gic_aprn(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>> +                         const struct sys_reg_desc *r, u8 apr)
>>> +{
>>> +     u8 num_pri_bits, idx;
>>> +     u32 ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>
>> Maybe you should cache this once and for all? It is fairly unlikely to
>> change over time...
> 
> Is it ok to cache in vgic_global struct?

I can't see why not.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 17:07         ` Marc Zyngier
@ 2016-09-17  6:28           ` Vijay Kilari
  -1 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-17  6:28 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 17:57, Vijay Kilari wrote:
>> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
>>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>>
>>>> VGICv3 CPU interface registers are accessed using
>>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>>> is used to identify the cpu for registers access.
>>>>
>>>> The version of VGIC v3 specification is define here
>>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>>
>>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>> ---
>>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>>  arch/arm64/kvm/Makefile             |   1 +
>>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>>  7 files changed, 385 insertions(+)
>
> [...]
>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> new file mode 100644
>>>> index 0000000..8e4f403
>>>> --- /dev/null
>>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> @@ -0,0 +1,296 @@
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>> +#include <linux/kvm.h>
>>>> +#include <linux/kvm_host.h>
>>>> +#include <kvm/iodev.h>
>>>> +#include <kvm/arm_vgic.h>
>>>> +#include <asm/kvm_emulate.h>
>>>> +#include <asm/kvm_arm.h>
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "vgic.h"
>>>> +#include "vgic-mmio.h"
>>>> +#include "sys_regs.h"
>>>> +
>>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +     u64 val;
>>>> +     u32 ich_vtr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             val = p->regval;
>>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>
>>> You've ignored my comments again: "What if userspace writes something
>>> that is incompatible with the current configuration? Wrong number of ID
>>> bits, or number of priorities?"
>>
>> IMO, In case of incompatibility,
>> If ID bits and PRI bits are less than HW supported, it is ok.
>
> Yes. But you also need to track of what the guest has programmed in
> order to be able to migrate it back to its original configuration.

You mean the vgic has to track/store the ID and PRI bits that guest
has programmed
and return the same when guest reads back instead of
returning HW supported value for ICC_CTLR_EL1 reg access?.

>
>> If ID bits and PRI bits are greater than HW supported, then warn would be good
>> enough. Please suggest the behaviour that you think it should be.
>
> No, it is an error, plain and simple. You cannot run in this condition.
>
[...]
>
> Thanks,
>
>         M.
> --
> Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-17  6:28           ` Vijay Kilari
  0 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-17  6:28 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 17:57, Vijay Kilari wrote:
>> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
>>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>>
>>>> VGICv3 CPU interface registers are accessed using
>>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>>> is used to identify the cpu for registers access.
>>>>
>>>> The version of VGIC v3 specification is define here
>>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>>
>>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>> ---
>>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>>  arch/arm64/kvm/Makefile             |   1 +
>>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>>  7 files changed, 385 insertions(+)
>
> [...]
>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> new file mode 100644
>>>> index 0000000..8e4f403
>>>> --- /dev/null
>>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> @@ -0,0 +1,296 @@
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>> +#include <linux/kvm.h>
>>>> +#include <linux/kvm_host.h>
>>>> +#include <kvm/iodev.h>
>>>> +#include <kvm/arm_vgic.h>
>>>> +#include <asm/kvm_emulate.h>
>>>> +#include <asm/kvm_arm.h>
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "vgic.h"
>>>> +#include "vgic-mmio.h"
>>>> +#include "sys_regs.h"
>>>> +
>>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +     u64 val;
>>>> +     u32 ich_vtr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             val = p->regval;
>>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>
>>> You've ignored my comments again: "What if userspace writes something
>>> that is incompatible with the current configuration? Wrong number of ID
>>> bits, or number of priorities?"
>>
>> IMO, In case of incompatibility,
>> If ID bits and PRI bits are less than HW supported, it is ok.
>
> Yes. But you also need to track of what the guest has programmed in
> order to be able to migrate it back to its original configuration.

You mean the vgic has to track/store the ID and PRI bits that guest
has programmed
and return the same when guest reads back instead of
returning HW supported value for ICC_CTLR_EL1 reg access?.

>
>> If ID bits and PRI bits are greater than HW supported, then warn would be good
>> enough. Please suggest the behaviour that you think it should be.
>
> No, it is an error, plain and simple. You cannot run in this condition.
>
[...]
>
> Thanks,
>
>         M.
> --
> Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-17  6:28           ` Vijay Kilari
@ 2016-09-17 11:37             ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-17 11:37 UTC (permalink / raw)
  To: Vijay Kilari; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

On Sat, 17 Sep 2016 11:58:48 +0530
Vijay Kilari <vijay.kilari@gmail.com> wrote:

> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 16/09/16 17:57, Vijay Kilari wrote:  
> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:  
> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>>
> >>>> VGICv3 CPU interface registers are accessed using
> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
> >>>> is used to identify the cpu for registers access.
> >>>>
> >>>> The version of VGIC v3 specification is define here
> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>>>
> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>> ---
> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
> >>>>  arch/arm64/kvm/Makefile             |   1 +
> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
> >>>>  7 files changed, 385 insertions(+)  
> >
> > [...]
> >  
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> new file mode 100644
> >>>> index 0000000..8e4f403
> >>>> --- /dev/null
> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> @@ -0,0 +1,296 @@
> >>>> +#include <linux/irqchip/arm-gic-v3.h>
> >>>> +#include <linux/kvm.h>
> >>>> +#include <linux/kvm_host.h>
> >>>> +#include <kvm/iodev.h>
> >>>> +#include <kvm/arm_vgic.h>
> >>>> +#include <asm/kvm_emulate.h>
> >>>> +#include <asm/kvm_arm.h>
> >>>> +#include <asm/kvm_mmu.h>
> >>>> +
> >>>> +#include "vgic.h"
> >>>> +#include "vgic-mmio.h"
> >>>> +#include "sys_regs.h"
> >>>> +
> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +     u64 val;
> >>>> +     u32 ich_vtr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             val = p->regval;
> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);  
> >>>
> >>> You've ignored my comments again: "What if userspace writes something
> >>> that is incompatible with the current configuration? Wrong number of ID
> >>> bits, or number of priorities?"  
> >>
> >> IMO, In case of incompatibility,
> >> If ID bits and PRI bits are less than HW supported, it is ok.  
> >
> > Yes. But you also need to track of what the guest has programmed in
> > order to be able to migrate it back to its original configuration.  
> 
> You mean the vgic has to track/store the ID and PRI bits that guest
> has programmed
> and return the same when guest reads back instead of
> returning HW supported value for ICC_CTLR_EL1 reg access?.

If you have two hosts (A and B), A having 5 bits of priority and B
having 7 bits, you should be able to migrate from A to B, and then from
B to A. Which means you have to preserve what the guest knows to be its
configuration, even if you run on a more capable system. Otherwise,
you're a bit stuck.

You probably won't be able to hide the discrepancy from inside the
guest though (the guest will be able to observe the change), but this
is better than nothing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-17 11:37             ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-17 11:37 UTC (permalink / raw)
  To: linux-arm-kernel

On Sat, 17 Sep 2016 11:58:48 +0530
Vijay Kilari <vijay.kilari@gmail.com> wrote:

> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 16/09/16 17:57, Vijay Kilari wrote:  
> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:  
> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>>
> >>>> VGICv3 CPU interface registers are accessed using
> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
> >>>> is used to identify the cpu for registers access.
> >>>>
> >>>> The version of VGIC v3 specification is define here
> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>>>
> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>> ---
> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
> >>>>  arch/arm64/kvm/Makefile             |   1 +
> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
> >>>>  7 files changed, 385 insertions(+)  
> >
> > [...]
> >  
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> new file mode 100644
> >>>> index 0000000..8e4f403
> >>>> --- /dev/null
> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> @@ -0,0 +1,296 @@
> >>>> +#include <linux/irqchip/arm-gic-v3.h>
> >>>> +#include <linux/kvm.h>
> >>>> +#include <linux/kvm_host.h>
> >>>> +#include <kvm/iodev.h>
> >>>> +#include <kvm/arm_vgic.h>
> >>>> +#include <asm/kvm_emulate.h>
> >>>> +#include <asm/kvm_arm.h>
> >>>> +#include <asm/kvm_mmu.h>
> >>>> +
> >>>> +#include "vgic.h"
> >>>> +#include "vgic-mmio.h"
> >>>> +#include "sys_regs.h"
> >>>> +
> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +     u64 val;
> >>>> +     u32 ich_vtr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             val = p->regval;
> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);  
> >>>
> >>> You've ignored my comments again: "What if userspace writes something
> >>> that is incompatible with the current configuration? Wrong number of ID
> >>> bits, or number of priorities?"  
> >>
> >> IMO, In case of incompatibility,
> >> If ID bits and PRI bits are less than HW supported, it is ok.  
> >
> > Yes. But you also need to track of what the guest has programmed in
> > order to be able to migrate it back to its original configuration.  
> 
> You mean the vgic has to track/store the ID and PRI bits that guest
> has programmed
> and return the same when guest reads back instead of
> returning HW supported value for ICC_CTLR_EL1 reg access?.

If you have two hosts (A and B), A having 5 bits of priority and B
having 7 bits, you should be able to migrate from A to B, and then from
B to A. Which means you have to preserve what the guest knows to be its
configuration, even if you run on a more capable system. Otherwise,
you're a bit stuck.

You probably won't be able to hide the discrepancy from inside the
guest though (the guest will be able to observe the change), but this
is better than nothing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-16 17:07         ` Marc Zyngier
@ 2016-09-18  6:30           ` Vijay Kilari
  -1 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-18  6:30 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

Hi Marc,

On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 17:57, Vijay Kilari wrote:
>> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
>>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>>
>>>> VGICv3 CPU interface registers are accessed using
>>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>>> is used to identify the cpu for registers access.
>>>>
>>>> The version of VGIC v3 specification is define here
>>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>>
>>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>> ---
>>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>>  arch/arm64/kvm/Makefile             |   1 +
>>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>>  7 files changed, 385 insertions(+)
>
> [...]
>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> new file mode 100644
>>>> index 0000000..8e4f403
>>>> --- /dev/null
>>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> @@ -0,0 +1,296 @@
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>> +#include <linux/kvm.h>
>>>> +#include <linux/kvm_host.h>
>>>> +#include <kvm/iodev.h>
>>>> +#include <kvm/arm_vgic.h>
>>>> +#include <asm/kvm_emulate.h>
>>>> +#include <asm/kvm_arm.h>
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "vgic.h"
>>>> +#include "vgic-mmio.h"
>>>> +#include "sys_regs.h"
>>>> +
>>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +     u64 val;
>>>> +     u32 ich_vtr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             val = p->regval;
>>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>
>>> You've ignored my comments again: "What if userspace writes something
>>> that is incompatible with the current configuration? Wrong number of ID
>>> bits, or number of priorities?"
>>
>> IMO, In case of incompatibility,
>> If ID bits and PRI bits are less than HW supported, it is ok.
>
> Yes. But you also need to track of what the guest has programmed in
> order to be able to migrate it back to its original configuration.
>
>> If ID bits and PRI bits are greater than HW supported, then warn would be good
>> enough. Please suggest the behaviour that you think it should be.
>
> No, it is an error, plain and simple. You cannot run in this condition.
>
>>
>>>
>>>> +     } else {
>>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>>> +
>>>> +             val = 0;
>>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>>>> +
>>>> +             p->regval = val;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                        const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>>>> +                         ICC_BPR0_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>>>> +                          ICC_BPR0_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>>>> +                          ICC_BPR1_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>>>> +                          ICC_BPR1_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                           const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>>>> +                          ICC_IGRPEN0_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                           const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>>>> +                          ICC_IGRPEN1_EL1_MASK;
>>>
>>> From the previous review comments: "Shouldn't this account for the
>>> ICC_CTLR_EL1.CBPR setting?"
>>
>>  Ok. I think this comment is for ICC_BPR1_EL1 access.
>
> Yes, sorry about the misplaced comment.
>
>> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.
>
> The reverse is also true: you also need to account the value of
> ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.
>

  ICC_CTLR_EL1 reg does not hold BPR1 value to account value of ICC_BPR1_EL1.

Regards
Vijay

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-18  6:30           ` Vijay Kilari
  0 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-18  6:30 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc,

On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On 16/09/16 17:57, Vijay Kilari wrote:
>> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
>>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>>
>>>> VGICv3 CPU interface registers are accessed using
>>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>>>> is used to identify the cpu for registers access.
>>>>
>>>> The version of VGIC v3 specification is define here
>>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>>>>
>>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>>>> ---
>>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>>>>  arch/arm64/kvm/Makefile             |   1 +
>>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>>>>  7 files changed, 385 insertions(+)
>
> [...]
>
>>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> new file mode 100644
>>>> index 0000000..8e4f403
>>>> --- /dev/null
>>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>>>> @@ -0,0 +1,296 @@
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>> +#include <linux/kvm.h>
>>>> +#include <linux/kvm_host.h>
>>>> +#include <kvm/iodev.h>
>>>> +#include <kvm/arm_vgic.h>
>>>> +#include <asm/kvm_emulate.h>
>>>> +#include <asm/kvm_arm.h>
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "vgic.h"
>>>> +#include "vgic-mmio.h"
>>>> +#include "sys_regs.h"
>>>> +
>>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +     u64 val;
>>>> +     u32 ich_vtr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             val = p->regval;
>>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>
>>> You've ignored my comments again: "What if userspace writes something
>>> that is incompatible with the current configuration? Wrong number of ID
>>> bits, or number of priorities?"
>>
>> IMO, In case of incompatibility,
>> If ID bits and PRI bits are less than HW supported, it is ok.
>
> Yes. But you also need to track of what the guest has programmed in
> order to be able to migrate it back to its original configuration.
>
>> If ID bits and PRI bits are greater than HW supported, then warn would be good
>> enough. Please suggest the behaviour that you think it should be.
>
> No, it is an error, plain and simple. You cannot run in this condition.
>
>>
>>>
>>>> +     } else {
>>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
>>>> +
>>>> +             val = 0;
>>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
>>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
>>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
>>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
>>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
>>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
>>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
>>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
>>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
>>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
>>>> +
>>>> +             p->regval = val;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                        const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
>>>> +                         ICC_BPR0_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
>>>> +                          ICC_BPR0_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                         const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
>>>> +                          ICC_BPR1_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
>>>> +                          ICC_BPR1_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                           const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
>>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
>>>> +                          ICC_IGRPEN0_EL1_MASK;
>>>> +     }
>>>> +
>>>> +     return true;
>>>> +}
>>>> +
>>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>>>> +                           const struct sys_reg_desc *r)
>>>> +{
>>>> +     struct vgic_vmcr vmcr;
>>>> +
>>>> +     vgic_get_vmcr(vcpu, &vmcr);
>>>> +     if (p->is_write) {
>>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
>>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
>>>> +             vgic_set_vmcr(vcpu, &vmcr);
>>>> +     } else {
>>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
>>>> +                          ICC_IGRPEN1_EL1_MASK;
>>>
>>> From the previous review comments: "Shouldn't this account for the
>>> ICC_CTLR_EL1.CBPR setting?"
>>
>>  Ok. I think this comment is for ICC_BPR1_EL1 access.
>
> Yes, sorry about the misplaced comment.
>
>> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.
>
> The reverse is also true: you also need to account the value of
> ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.
>

  ICC_CTLR_EL1 reg does not hold BPR1 value to account value of ICC_BPR1_EL1.

Regards
Vijay

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-18  6:30           ` Vijay Kilari
@ 2016-09-18  9:27             ` Marc Zyngier
  -1 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-18  9:27 UTC (permalink / raw)
  To: Vijay Kilari; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

On Sun, 18 Sep 2016 12:00:01 +0530
Vijay Kilari <vijay.kilari@gmail.com> wrote:

> Hi Marc,
> 
> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 16/09/16 17:57, Vijay Kilari wrote:  
> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:  
> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>>
> >>>> VGICv3 CPU interface registers are accessed using
> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
> >>>> is used to identify the cpu for registers access.
> >>>>
> >>>> The version of VGIC v3 specification is define here
> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>>>
> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>> ---
> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
> >>>>  arch/arm64/kvm/Makefile             |   1 +
> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
> >>>>  7 files changed, 385 insertions(+)  
> >
> > [...]
> >  
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> new file mode 100644
> >>>> index 0000000..8e4f403
> >>>> --- /dev/null
> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> @@ -0,0 +1,296 @@
> >>>> +#include <linux/irqchip/arm-gic-v3.h>
> >>>> +#include <linux/kvm.h>
> >>>> +#include <linux/kvm_host.h>
> >>>> +#include <kvm/iodev.h>
> >>>> +#include <kvm/arm_vgic.h>
> >>>> +#include <asm/kvm_emulate.h>
> >>>> +#include <asm/kvm_arm.h>
> >>>> +#include <asm/kvm_mmu.h>
> >>>> +
> >>>> +#include "vgic.h"
> >>>> +#include "vgic-mmio.h"
> >>>> +#include "sys_regs.h"
> >>>> +
> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +     u64 val;
> >>>> +     u32 ich_vtr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             val = p->regval;
> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);  
> >>>
> >>> You've ignored my comments again: "What if userspace writes something
> >>> that is incompatible with the current configuration? Wrong number of ID
> >>> bits, or number of priorities?"  
> >>
> >> IMO, In case of incompatibility,
> >> If ID bits and PRI bits are less than HW supported, it is ok.  
> >
> > Yes. But you also need to track of what the guest has programmed in
> > order to be able to migrate it back to its original configuration.
> >  
> >> If ID bits and PRI bits are greater than HW supported, then warn would be good
> >> enough. Please suggest the behaviour that you think it should be.  
> >
> > No, it is an error, plain and simple. You cannot run in this condition.
> >  
> >>  
> >>>  
> >>>> +     } else {
> >>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
> >>>> +
> >>>> +             val = 0;
> >>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> >>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
> >>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
> >>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
> >>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
> >>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >>>> +
> >>>> +             p->regval = val;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                        const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >>>> +                         ICC_BPR0_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >>>> +                          ICC_BPR0_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >>>> +                          ICC_BPR1_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >>>> +                          ICC_BPR1_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                           const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >>>> +                          ICC_IGRPEN0_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                           const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >>>> +                          ICC_IGRPEN1_EL1_MASK;  
> >>>
> >>> From the previous review comments: "Shouldn't this account for the
> >>> ICC_CTLR_EL1.CBPR setting?"  
> >>
> >>  Ok. I think this comment is for ICC_BPR1_EL1 access.  
> >
> > Yes, sorry about the misplaced comment.
> >  
> >> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.  
> >
> > The reverse is also true: you also need to account the value of
> > ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.
> >  
> 
>   ICC_CTLR_EL1 reg does not hold BPR1 value to account value of ICC_BPR1_EL1.

Oh please. What do you think ICC_CTLR_EL1.CPBR is for? It ties
ICC_BPR1_EL1 to ICC_BPR0_EL1. So if you have the following sequence:

	Set ICC_BPR0_EL1 to p0
	Set ICC_BPR1_EL1 to p1
	Set ICC_CTLR_EL1.CBPR to 1

you must ensure that ICC_BPR1_EL1 is now min(p0 + 1, 7), no matter what
p1 is. ICC_CTLR_EL1 may not hold the BPR1 value, but it directly
controls it.

Effectively, you must always consider BPR0, BPR1 and CBPR together.
Please read the architecture spec, it is one of the few things that are
easy actually to understand.

	M.
-- 
Jazz is not dead. It just smells funny.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-18  9:27             ` Marc Zyngier
  0 siblings, 0 replies; 36+ messages in thread
From: Marc Zyngier @ 2016-09-18  9:27 UTC (permalink / raw)
  To: linux-arm-kernel

On Sun, 18 Sep 2016 12:00:01 +0530
Vijay Kilari <vijay.kilari@gmail.com> wrote:

> Hi Marc,
> 
> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > On 16/09/16 17:57, Vijay Kilari wrote:  
> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:  
> >>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:  
> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>>
> >>>> VGICv3 CPU interface registers are accessed using
> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
> >>>> is used to identify the cpu for registers access.
> >>>>
> >>>> The version of VGIC v3 specification is define here
> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
> >>>>
> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
> >>>> ---
> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
> >>>>  arch/arm64/kvm/Makefile             |   1 +
> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
> >>>>  7 files changed, 385 insertions(+)  
> >
> > [...]
> >  
> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> new file mode 100644
> >>>> index 0000000..8e4f403
> >>>> --- /dev/null
> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
> >>>> @@ -0,0 +1,296 @@
> >>>> +#include <linux/irqchip/arm-gic-v3.h>
> >>>> +#include <linux/kvm.h>
> >>>> +#include <linux/kvm_host.h>
> >>>> +#include <kvm/iodev.h>
> >>>> +#include <kvm/arm_vgic.h>
> >>>> +#include <asm/kvm_emulate.h>
> >>>> +#include <asm/kvm_arm.h>
> >>>> +#include <asm/kvm_mmu.h>
> >>>> +
> >>>> +#include "vgic.h"
> >>>> +#include "vgic-mmio.h"
> >>>> +#include "sys_regs.h"
> >>>> +
> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +     u64 val;
> >>>> +     u32 ich_vtr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             val = p->regval;
> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);  
> >>>
> >>> You've ignored my comments again: "What if userspace writes something
> >>> that is incompatible with the current configuration? Wrong number of ID
> >>> bits, or number of priorities?"  
> >>
> >> IMO, In case of incompatibility,
> >> If ID bits and PRI bits are less than HW supported, it is ok.  
> >
> > Yes. But you also need to track of what the guest has programmed in
> > order to be able to migrate it back to its original configuration.
> >  
> >> If ID bits and PRI bits are greater than HW supported, then warn would be good
> >> enough. Please suggest the behaviour that you think it should be.  
> >
> > No, it is an error, plain and simple. You cannot run in this condition.
> >  
> >>  
> >>>  
> >>>> +     } else {
> >>>> +             ich_vtr = kvm_call_hyp(__vgic_v3_get_ich_vtr_el2);
> >>>> +
> >>>> +             val = 0;
> >>>> +             val |= ((ich_vtr & ICH_VTR_PRI_BITS_MASK) >>
> >>>> +                     ICH_VTR_PRI_BITS_SHIFT) << ICC_CTLR_EL1_PRI_BITS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_ID_BITS_MASK) >>
> >>>> +                     ICH_VTR_ID_BITS_SHIFT) << ICC_CTLR_EL1_ID_BITS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_SEIS_MASK) >> ICH_VTR_SEIS_SHIFT)
> >>>> +                     << ICC_CTLR_EL1_SEIS_SHIFT;
> >>>> +             val |= ((ich_vtr & ICH_VTR_A3V_MASK) >> ICH_VTR_A3V_SHIFT)
> >>>> +                     << ICC_CTLR_EL1_A3V_SHIFT;
> >>>> +             val |= ((vmcr.ctlr & ICH_VMCR_CBPR_MASK) >>
> >>>> +                     ICH_VMCR_CBPR_SHIFT) << ICC_CTLR_EL1_CBPR_SHIFT;
> >>>> +             val |= ((vmcr.ctlr & ICH_VMCR_EOIM_MASK) >>
> >>>> +                     ICH_VMCR_EOIM_SHIFT) << ICC_CTLR_EL1_EOImode_SHIFT;
> >>>> +
> >>>> +             p->regval = val;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_pmr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                        const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.pmr = (p->regval & ICC_PMR_EL1_MASK) >> ICC_PMR_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.pmr << ICC_PMR_EL1_SHIFT) & ICC_PMR_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_bpr0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.bpr = (p->regval & ICC_BPR0_EL1_MASK) >>
> >>>> +                         ICC_BPR0_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.bpr << ICC_BPR0_EL1_SHIFT) &
> >>>> +                          ICC_BPR0_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_bpr1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                         const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.abpr = (p->regval & ICC_BPR1_EL1_MASK) >>
> >>>> +                          ICC_BPR1_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.abpr << ICC_BPR1_EL1_SHIFT) &
> >>>> +                          ICC_BPR1_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_grpen0(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                           const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.grpen0 = (p->regval & ICC_IGRPEN0_EL1_MASK) >>
> >>>> +                                   ICC_IGRPEN0_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.grpen0 << ICC_IGRPEN0_EL1_SHIFT) &
> >>>> +                          ICC_IGRPEN0_EL1_MASK;
> >>>> +     }
> >>>> +
> >>>> +     return true;
> >>>> +}
> >>>> +
> >>>> +static bool access_gic_grpen1(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
> >>>> +                           const struct sys_reg_desc *r)
> >>>> +{
> >>>> +     struct vgic_vmcr vmcr;
> >>>> +
> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
> >>>> +     if (p->is_write) {
> >>>> +             vmcr.grpen1 = (p->regval & ICC_IGRPEN1_EL1_MASK) >>
> >>>> +                                   ICC_IGRPEN1_EL1_SHIFT;
> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
> >>>> +     } else {
> >>>> +             p->regval = (vmcr.grpen1 << ICC_IGRPEN1_EL1_SHIFT) &
> >>>> +                          ICC_IGRPEN1_EL1_MASK;  
> >>>
> >>> From the previous review comments: "Shouldn't this account for the
> >>> ICC_CTLR_EL1.CBPR setting?"  
> >>
> >>  Ok. I think this comment is for ICC_BPR1_EL1 access.  
> >
> > Yes, sorry about the misplaced comment.
> >  
> >> I will make a check on ICC_CTLR.EL1.CBPR for accessing ICC_BPR1_EL1.  
> >
> > The reverse is also true: you also need to account the value of
> > ICC_BPR1_EL1 when accessing ICC_CTLR_EL1.
> >  
> 
>   ICC_CTLR_EL1 reg does not hold BPR1 value to account value of ICC_BPR1_EL1.

Oh please. What do you think ICC_CTLR_EL1.CPBR is for? It ties
ICC_BPR1_EL1 to ICC_BPR0_EL1. So if you have the following sequence:

	Set ICC_BPR0_EL1 to p0
	Set ICC_BPR1_EL1 to p1
	Set ICC_CTLR_EL1.CBPR to 1

you must ensure that ICC_BPR1_EL1 is now min(p0 + 1, 7), no matter what
p1 is. ICC_CTLR_EL1 may not hold the BPR1 value, but it directly
controls it.

Effectively, you must always consider BPR0, BPR1 and CBPR together.
Please read the architecture spec, it is one of the few things that are
easy actually to understand.

	M.
-- 
Jazz is not dead. It just smells funny.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-17 11:37             ` Marc Zyngier
@ 2016-09-19  7:36               ` Vijay Kilari
  -1 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-19  7:36 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: Vijaya Kumar K, kvmarm, linux-arm-kernel

Hi Marc , Peter

On Sat, Sep 17, 2016 at 5:07 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Sat, 17 Sep 2016 11:58:48 +0530
> Vijay Kilari <vijay.kilari@gmail.com> wrote:
>
>> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> > On 16/09/16 17:57, Vijay Kilari wrote:
>> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> >>> On 16/09/16 13:20, vijay.kilari@gmail.com wrote:
>> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>>>
>> >>>> VGICv3 CPU interface registers are accessed using
>> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >>>> is used to identify the cpu for registers access.
>> >>>>
>> >>>> The version of VGIC v3 specification is define here
>> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>>>
>> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>>> ---
>> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>> >>>>  arch/arm64/kvm/Makefile             |   1 +
>> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>> >>>>  7 files changed, 385 insertions(+)
>> >
>> > [...]
>> >
>> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >>>> new file mode 100644
>> >>>> index 0000000..8e4f403
>> >>>> --- /dev/null
>> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >>>> @@ -0,0 +1,296 @@
>> >>>> +#include <linux/irqchip/arm-gic-v3.h>
>> >>>> +#include <linux/kvm.h>
>> >>>> +#include <linux/kvm_host.h>
>> >>>> +#include <kvm/iodev.h>
>> >>>> +#include <kvm/arm_vgic.h>
>> >>>> +#include <asm/kvm_emulate.h>
>> >>>> +#include <asm/kvm_arm.h>
>> >>>> +#include <asm/kvm_mmu.h>
>> >>>> +
>> >>>> +#include "vgic.h"
>> >>>> +#include "vgic-mmio.h"
>> >>>> +#include "sys_regs.h"
>> >>>> +
>> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >>>> +                         const struct sys_reg_desc *r)
>> >>>> +{
>> >>>> +     struct vgic_vmcr vmcr;
>> >>>> +     u64 val;
>> >>>> +     u32 ich_vtr;
>> >>>> +
>> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
>> >>>> +     if (p->is_write) {
>> >>>> +             val = p->regval;
>> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
>> >>>
>> >>> You've ignored my comments again: "What if userspace writes something
>> >>> that is incompatible with the current configuration? Wrong number of ID
>> >>> bits, or number of priorities?"
>> >>
>> >> IMO, In case of incompatibility,
>> >> If ID bits and PRI bits are less than HW supported, it is ok.
>> >
>> > Yes. But you also need to track of what the guest has programmed in
>> > order to be able to migrate it back to its original configuration.
>>
>> You mean the vgic has to track/store the ID and PRI bits that guest
>> has programmed
>> and return the same when guest reads back instead of
>> returning HW supported value for ICC_CTLR_EL1 reg access?.
>
> If you have two hosts (A and B), A having 5 bits of priority and B
> having 7 bits, you should be able to migrate from A to B, and then from
> B to A. Which means you have to preserve what the guest knows to be its
> configuration, even if you run on a more capable system. Otherwise,
> you're a bit stuck.
>
> You probably won't be able to hide the discrepancy from inside the
> guest though (the guest will be able to observe the change), but this
> is better than nothing.

In order to track the PRI and ID bits written by guest,
VGIC needs to store these values when ICC_CTRL_EL1 is updated.
However,  QEMU is reseting VGIC by writing 0's to all the
registers after VGIC initialization and hence the back up values are
always reset to 0 and hence when guest read back, VGIC returns wrong value.

One option is to drop VGIC reset from QEMU which is not doing much.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-19  7:36               ` Vijay Kilari
  0 siblings, 0 replies; 36+ messages in thread
From: Vijay Kilari @ 2016-09-19  7:36 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Marc , Peter

On Sat, Sep 17, 2016 at 5:07 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Sat, 17 Sep 2016 11:58:48 +0530
> Vijay Kilari <vijay.kilari@gmail.com> wrote:
>
>> On Fri, Sep 16, 2016 at 10:37 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> > On 16/09/16 17:57, Vijay Kilari wrote:
>> >> On Fri, Sep 16, 2016 at 8:06 PM, Marc Zyngier <marc.zyngier@arm.com> wrote:
>> >>> On 16/09/16 13:20, vijay.kilari at gmail.com wrote:
>> >>>> From: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>>>
>> >>>> VGICv3 CPU interface registers are accessed using
>> >>>> KVM_DEV_ARM_VGIC_CPU_SYSREGS ioctl. These registers are accessed
>> >>>> as 64-bit. The cpu MPIDR value is passed along with register id.
>> >>>> is used to identify the cpu for registers access.
>> >>>>
>> >>>> The version of VGIC v3 specification is define here
>> >>>> http://lists.infradead.org/pipermail/linux-arm-kernel/2016-July/445611.html
>> >>>>
>> >>>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> >>>> Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@cavium.com>
>> >>>> ---
>> >>>>  arch/arm64/include/uapi/asm/kvm.h   |   3 +
>> >>>>  arch/arm64/kvm/Makefile             |   1 +
>> >>>>  include/linux/irqchip/arm-gic-v3.h  |  30 ++++
>> >>>>  virt/kvm/arm/vgic/vgic-kvm-device.c |  27 ++++
>> >>>>  virt/kvm/arm/vgic/vgic-mmio-v3.c    |  18 +++
>> >>>>  virt/kvm/arm/vgic/vgic-sys-reg-v3.c | 296 ++++++++++++++++++++++++++++++++++++
>> >>>>  virt/kvm/arm/vgic/vgic.h            |  10 ++
>> >>>>  7 files changed, 385 insertions(+)
>> >
>> > [...]
>> >
>> >>>> diff --git a/virt/kvm/arm/vgic/vgic-sys-reg-v3.c b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >>>> new file mode 100644
>> >>>> index 0000000..8e4f403
>> >>>> --- /dev/null
>> >>>> +++ b/virt/kvm/arm/vgic/vgic-sys-reg-v3.c
>> >>>> @@ -0,0 +1,296 @@
>> >>>> +#include <linux/irqchip/arm-gic-v3.h>
>> >>>> +#include <linux/kvm.h>
>> >>>> +#include <linux/kvm_host.h>
>> >>>> +#include <kvm/iodev.h>
>> >>>> +#include <kvm/arm_vgic.h>
>> >>>> +#include <asm/kvm_emulate.h>
>> >>>> +#include <asm/kvm_arm.h>
>> >>>> +#include <asm/kvm_mmu.h>
>> >>>> +
>> >>>> +#include "vgic.h"
>> >>>> +#include "vgic-mmio.h"
>> >>>> +#include "sys_regs.h"
>> >>>> +
>> >>>> +static bool access_gic_ctlr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
>> >>>> +                         const struct sys_reg_desc *r)
>> >>>> +{
>> >>>> +     struct vgic_vmcr vmcr;
>> >>>> +     u64 val;
>> >>>> +     u32 ich_vtr;
>> >>>> +
>> >>>> +     vgic_get_vmcr(vcpu, &vmcr);
>> >>>> +     if (p->is_write) {
>> >>>> +             val = p->regval;
>> >>>> +             vmcr.ctlr &= ~(ICH_VMCR_CBPR_MASK | ICH_VMCR_EOIM_MASK);
>> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_CBPR_MASK) >>
>> >>>> +                           ICC_CTLR_EL1_CBPR_SHIFT) << ICH_VMCR_CBPR_SHIFT;
>> >>>> +             vmcr.ctlr |= ((val & ICC_CTLR_EL1_EOImode_MASK) >>
>> >>>> +                          ICC_CTLR_EL1_EOImode_SHIFT) << ICH_VMCR_EOIM_SHIFT;
>> >>>> +             vgic_set_vmcr(vcpu, &vmcr);
>> >>>
>> >>> You've ignored my comments again: "What if userspace writes something
>> >>> that is incompatible with the current configuration? Wrong number of ID
>> >>> bits, or number of priorities?"
>> >>
>> >> IMO, In case of incompatibility,
>> >> If ID bits and PRI bits are less than HW supported, it is ok.
>> >
>> > Yes. But you also need to track of what the guest has programmed in
>> > order to be able to migrate it back to its original configuration.
>>
>> You mean the vgic has to track/store the ID and PRI bits that guest
>> has programmed
>> and return the same when guest reads back instead of
>> returning HW supported value for ICC_CTLR_EL1 reg access?.
>
> If you have two hosts (A and B), A having 5 bits of priority and B
> having 7 bits, you should be able to migrate from A to B, and then from
> B to A. Which means you have to preserve what the guest knows to be its
> configuration, even if you run on a more capable system. Otherwise,
> you're a bit stuck.
>
> You probably won't be able to hide the discrepancy from inside the
> guest though (the guest will be able to observe the change), but this
> is better than nothing.

In order to track the PRI and ID bits written by guest,
VGIC needs to store these values when ICC_CTRL_EL1 is updated.
However,  QEMU is reseting VGIC by writing 0's to all the
registers after VGIC initialization and hence the back up values are
always reset to 0 and hence when guest read back, VGIC returns wrong value.

One option is to drop VGIC reset from QEMU which is not doing much.

^ permalink raw reply	[flat|nested] 36+ messages in thread

* Re: [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
  2016-09-19  7:36               ` Vijay Kilari
@ 2016-09-19 10:11                 ` Peter Maydell
  -1 siblings, 0 replies; 36+ messages in thread
From: Peter Maydell @ 2016-09-19 10:11 UTC (permalink / raw)
  To: Vijay Kilari; +Cc: Marc Zyngier, Vijaya Kumar K, kvmarm, linux-arm-kernel

On 19 September 2016 at 08:36, Vijay Kilari <vijay.kilari@gmail.com> wrote:
> In order to track the PRI and ID bits written by guest,
> VGIC needs to store these values when ICC_CTRL_EL1 is updated.
> However,  QEMU is reseting VGIC by writing 0's to all the
> registers after VGIC initialization and hence the back up values are
> always reset to 0 and hence when guest read back, VGIC returns wrong value.
>
> One option is to drop VGIC reset from QEMU which is not doing much.

No, if QEMU's reset is not resetting registers to the right value
the correct thing to do is fix the reset code. The way device
reset is supposed to work is that QEMU knows the right values
to use and it writes them to the kernel.

QEMU doesn't reset ICC_CTLR_EL1 to 0 for TCG:
    cs->icc_ctlr_el1[GICV3_NS] = ICC_CTLR_EL1_A3V |
        (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);

but I don't think that code gets run for the KVM VGIC.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 36+ messages in thread

* [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access
@ 2016-09-19 10:11                 ` Peter Maydell
  0 siblings, 0 replies; 36+ messages in thread
From: Peter Maydell @ 2016-09-19 10:11 UTC (permalink / raw)
  To: linux-arm-kernel

On 19 September 2016 at 08:36, Vijay Kilari <vijay.kilari@gmail.com> wrote:
> In order to track the PRI and ID bits written by guest,
> VGIC needs to store these values when ICC_CTRL_EL1 is updated.
> However,  QEMU is reseting VGIC by writing 0's to all the
> registers after VGIC initialization and hence the back up values are
> always reset to 0 and hence when guest read back, VGIC returns wrong value.
>
> One option is to drop VGIC reset from QEMU which is not doing much.

No, if QEMU's reset is not resetting registers to the right value
the correct thing to do is fix the reset code. The way device
reset is supposed to work is that QEMU knows the right values
to use and it writes them to the kernel.

QEMU doesn't reset ICC_CTLR_EL1 to 0 for TCG:
    cs->icc_ctlr_el1[GICV3_NS] = ICC_CTLR_EL1_A3V |
        (1 << ICC_CTLR_EL1_IDBITS_SHIFT) |
        (7 << ICC_CTLR_EL1_PRIBITS_SHIFT);

but I don't think that code gets run for the KVM VGIC.

thanks
-- PMM

^ permalink raw reply	[flat|nested] 36+ messages in thread

end of thread, other threads:[~2016-09-19 10:11 UTC | newest]

Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-16 12:20 [PATCH v5 0/6] arm/arm64: vgic-new: Implement API for vGICv3 live migration vijay.kilari
2016-09-16 12:20 ` vijay.kilari at gmail.com
2016-09-16 12:20 ` [PATCH v5 1/6] arm/arm64: vgic-new: Implement support for userspace access vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com
2016-09-16 16:31   ` Marc Zyngier
2016-09-16 16:31     ` Marc Zyngier
2016-09-16 12:20 ` [PATCH v5 2/6] arm/arm64: vgic-new: Add distributor and redistributor access vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com
2016-09-16 12:20 ` [PATCH v5 3/6] arm/arm64: vgic-new: Introduce find_reg_by_id() vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com
2016-09-16 12:20 ` [PATCH v5 4/6] arm/arm64: vgic-new: Introduce VENG0 and VENG1 fields to vmcr struct vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com
2016-09-16 13:54   ` Marc Zyngier
2016-09-16 13:54     ` Marc Zyngier
2016-09-16 12:20 ` [PATCH 5/6] arm/arm64: vgic-new: Implement VGICv3 CPU interface access vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com
2016-09-16 14:36   ` Marc Zyngier
2016-09-16 14:36     ` Marc Zyngier
2016-09-16 16:57     ` Vijay Kilari
2016-09-16 16:57       ` Vijay Kilari
2016-09-16 17:07       ` Marc Zyngier
2016-09-16 17:07         ` Marc Zyngier
2016-09-17  6:28         ` Vijay Kilari
2016-09-17  6:28           ` Vijay Kilari
2016-09-17 11:37           ` Marc Zyngier
2016-09-17 11:37             ` Marc Zyngier
2016-09-19  7:36             ` Vijay Kilari
2016-09-19  7:36               ` Vijay Kilari
2016-09-19 10:11               ` Peter Maydell
2016-09-19 10:11                 ` Peter Maydell
2016-09-18  6:30         ` Vijay Kilari
2016-09-18  6:30           ` Vijay Kilari
2016-09-18  9:27           ` Marc Zyngier
2016-09-18  9:27             ` Marc Zyngier
2016-09-16 12:20 ` [PATCH v5 6/6] arm/arm64: vgic-new: Implement KVM_DEV_ARM_VGIC_GRP_LEVEL_INFO ioctl vijay.kilari
2016-09-16 12:20   ` vijay.kilari at gmail.com

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.