* [PATCH v3 0/9] Support VGIC save/restore using device control API
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Implement save/restore of the VGIC state using the newer KVM Device
Control API. This requries some number of changes to existing code in
addition to actually supporting save/restore of the necessary state.
The first patches (01-03) support creating the VGIC using the Device
Control API. This change is necessary because there are no other
suitable KVM APIs that we can leverage to access the VGIC state from
user space and the device control API was crafted exactly for this
purpose.
Subsequent patches add the missing infrastructure and user space API
pieces necessary to actually save and restore the VGIC state. The GIC
v2.0 architecture specification already specifies registers that can be
used to save and restore the complete VGIC state for suspend/resume
purposes on real hardware, and we can resuse this interface for the
VGIC. The API is therefore based on the memory-mapped register accesses
defined in the specs. See the individual patches for details.
The patches are based on kvm-arm-next with the arch timers save/restore
patches applied:
git://git.linaro.org/people/cdall/linux-kvm-arm.git timer-migrate-v3
This patch series based on the above can be cloned from:
git://git.linaro.org/people/cdall/linux-kvm-arm.git vgic-migrate-v3
User space patches for QEMU have also been posted on the list, but an
updated version is underway. Tested on Versatile Express TC2.
Changelogs in the individual patches.
Christoffer Dall (9):
ARM: KVM: Allow creating the VGIC after VCPUs
KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
KVM: arm-vgic: Set base addr through device API
irqchip: arm-gic: Define additional MMIO offsets and masks
KVM: arm-vgic: Make vgic mmio functions more generic
KVM: arm-vgic: Add vgic reg access from dev attr
KVM: arm-vgic: Support unqueueing of LRs to the dist
KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
KVM: arm-vgic: Support CPU interface reg access
Documentation/virtual/kvm/api.txt | 7 +-
Documentation/virtual/kvm/devices/arm-vgic.txt | 73 +++
arch/arm/include/uapi/asm/kvm.h | 8 +
arch/arm/kvm/arm.c | 11 +-
include/kvm/arm_vgic.h | 2 +-
include/linux/irqchip/arm-gic.h | 14 +
include/linux/kvm_host.h | 1 +
include/uapi/linux/kvm.h | 1 +
virt/kvm/arm/vgic.c | 563 ++++++++++++++++++++++--
virt/kvm/kvm_main.c | 6 +-
10 files changed, 648 insertions(+), 38 deletions(-)
create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
--
1.7.10.4
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 0/9] Support VGIC save/restore using device control API
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Implement save/restore of the VGIC state using the newer KVM Device
Control API. This requries some number of changes to existing code in
addition to actually supporting save/restore of the necessary state.
The first patches (01-03) support creating the VGIC using the Device
Control API. This change is necessary because there are no other
suitable KVM APIs that we can leverage to access the VGIC state from
user space and the device control API was crafted exactly for this
purpose.
Subsequent patches add the missing infrastructure and user space API
pieces necessary to actually save and restore the VGIC state. The GIC
v2.0 architecture specification already specifies registers that can be
used to save and restore the complete VGIC state for suspend/resume
purposes on real hardware, and we can resuse this interface for the
VGIC. The API is therefore based on the memory-mapped register accesses
defined in the specs. See the individual patches for details.
The patches are based on kvm-arm-next with the arch timers save/restore
patches applied:
git://git.linaro.org/people/cdall/linux-kvm-arm.git timer-migrate-v3
This patch series based on the above can be cloned from:
git://git.linaro.org/people/cdall/linux-kvm-arm.git vgic-migrate-v3
User space patches for QEMU have also been posted on the list, but an
updated version is underway. Tested on Versatile Express TC2.
Changelogs in the individual patches.
Christoffer Dall (9):
ARM: KVM: Allow creating the VGIC after VCPUs
KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
KVM: arm-vgic: Set base addr through device API
irqchip: arm-gic: Define additional MMIO offsets and masks
KVM: arm-vgic: Make vgic mmio functions more generic
KVM: arm-vgic: Add vgic reg access from dev attr
KVM: arm-vgic: Support unqueueing of LRs to the dist
KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
KVM: arm-vgic: Support CPU interface reg access
Documentation/virtual/kvm/api.txt | 7 +-
Documentation/virtual/kvm/devices/arm-vgic.txt | 73 +++
arch/arm/include/uapi/asm/kvm.h | 8 +
arch/arm/kvm/arm.c | 11 +-
include/kvm/arm_vgic.h | 2 +-
include/linux/irqchip/arm-gic.h | 14 +
include/linux/kvm_host.h | 1 +
include/uapi/linux/kvm.h | 1 +
virt/kvm/arm/vgic.c | 563 ++++++++++++++++++++++--
virt/kvm/kvm_main.c | 6 +-
10 files changed, 648 insertions(+), 38 deletions(-)
create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
--
1.7.10.4
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 1/9] ARM: KVM: Allow creating the VGIC after VCPUs
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Rework the VGIC initialization slightly to allow initialization of the
vgic cpu-specific state even if the irqchip (the VGIC) hasn't been
created by user space yet. This is safe, because the vgic data
structures are already allocated when the CPU is allocated if VGIC
support is compiled into the kernel. Further, the init process does not
depend on any other information and the sacrifice is a slight
performance degradation for creating VMs in the no-VGIC case.
The reason is that the new device control API doesn't mandate creating
the VGIC before creating the VCPU and it is unreasonable to require user
space to create the VGIC before creating the VCPUs.
At the same time move the irqchip_in_kernel check out of
kvm_vcpu_first_run_init and into the init function to make the per-vcpu
and global init functions symmetric and add comments on the exported
functions making it a bit easier to understand the init flow by only
looking at vgic.c.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/arm.c | 7 ++++---
virt/kvm/arm/vgic.c | 22 +++++++++++++++++++---
2 files changed, 23 insertions(+), 6 deletions(-)
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 2bc2ec4..e80b6a1 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -464,6 +464,8 @@ static void update_vttbr(struct kvm *kvm)
static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
{
+ int ret;
+
if (likely(vcpu->arch.has_run_once))
return 0;
@@ -473,9 +475,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
* Initialize the VGIC before running a vcpu the first time on
* this VM.
*/
- if (irqchip_in_kernel(vcpu->kvm) &&
- unlikely(!vgic_initialized(vcpu->kvm))) {
- int ret = kvm_vgic_init(vcpu->kvm);
+ if (unlikely(!vgic_initialized(vcpu->kvm))) {
+ ret = kvm_vgic_init(vcpu->kvm);
if (ret)
return ret;
}
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 81e9481..5e9df47 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1243,15 +1243,19 @@ static irqreturn_t vgic_maintenance_handler(int irq, void *data)
return IRQ_HANDLED;
}
+/**
+ * kvm_vgic_vcpu_init - Initialize per-vcpu VGIC state
+ * @vcpu: pointer to the vcpu struct
+ *
+ * Initialize the vgic_cpu struct and vgic_dist struct fields pertaining to
+ * this vcpu and enable the VGIC for this VCPU
+ */
int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
int i;
- if (!irqchip_in_kernel(vcpu->kvm))
- return 0;
-
if (vcpu->vcpu_id >= VGIC_MAX_CPUS)
return -EBUSY;
@@ -1383,10 +1387,22 @@ out:
return ret;
}
+/**
+ * kvm_vgic_init - Initialize global VGIC state before running any VCPUs
+ * @kvm: pointer to the kvm struct
+ *
+ * Map the virtual CPU interface into the VM before running any VCPUs. We
+ * can't do this at creation time, because user space must first set the
+ * virtual CPU interface address in the guest physical address space. Also
+ * initialize the ITARGETSRn regs to 0 on the emulated distributor.
+ */
int kvm_vgic_init(struct kvm *kvm)
{
int ret = 0, i;
+ if (!irqchip_in_kernel(kvm))
+ return 0;
+
mutex_lock(&kvm->lock);
if (vgic_initialized(kvm))
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 1/9] ARM: KVM: Allow creating the VGIC after VCPUs
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Rework the VGIC initialization slightly to allow initialization of the
vgic cpu-specific state even if the irqchip (the VGIC) hasn't been
created by user space yet. This is safe, because the vgic data
structures are already allocated when the CPU is allocated if VGIC
support is compiled into the kernel. Further, the init process does not
depend on any other information and the sacrifice is a slight
performance degradation for creating VMs in the no-VGIC case.
The reason is that the new device control API doesn't mandate creating
the VGIC before creating the VCPU and it is unreasonable to require user
space to create the VGIC before creating the VCPUs.
At the same time move the irqchip_in_kernel check out of
kvm_vcpu_first_run_init and into the init function to make the per-vcpu
and global init functions symmetric and add comments on the exported
functions making it a bit easier to understand the init flow by only
looking at vgic.c.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
arch/arm/kvm/arm.c | 7 ++++---
virt/kvm/arm/vgic.c | 22 +++++++++++++++++++---
2 files changed, 23 insertions(+), 6 deletions(-)
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 2bc2ec4..e80b6a1 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -464,6 +464,8 @@ static void update_vttbr(struct kvm *kvm)
static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
{
+ int ret;
+
if (likely(vcpu->arch.has_run_once))
return 0;
@@ -473,9 +475,8 @@ static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
* Initialize the VGIC before running a vcpu the first time on
* this VM.
*/
- if (irqchip_in_kernel(vcpu->kvm) &&
- unlikely(!vgic_initialized(vcpu->kvm))) {
- int ret = kvm_vgic_init(vcpu->kvm);
+ if (unlikely(!vgic_initialized(vcpu->kvm))) {
+ ret = kvm_vgic_init(vcpu->kvm);
if (ret)
return ret;
}
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 81e9481..5e9df47 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1243,15 +1243,19 @@ static irqreturn_t vgic_maintenance_handler(int irq, void *data)
return IRQ_HANDLED;
}
+/**
+ * kvm_vgic_vcpu_init - Initialize per-vcpu VGIC state
+ * @vcpu: pointer to the vcpu struct
+ *
+ * Initialize the vgic_cpu struct and vgic_dist struct fields pertaining to
+ * this vcpu and enable the VGIC for this VCPU
+ */
int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
int i;
- if (!irqchip_in_kernel(vcpu->kvm))
- return 0;
-
if (vcpu->vcpu_id >= VGIC_MAX_CPUS)
return -EBUSY;
@@ -1383,10 +1387,22 @@ out:
return ret;
}
+/**
+ * kvm_vgic_init - Initialize global VGIC state before running any VCPUs
+ * @kvm: pointer to the kvm struct
+ *
+ * Map the virtual CPU interface into the VM before running any VCPUs. We
+ * can't do this at creation time, because user space must first set the
+ * virtual CPU interface address in the guest physical address space. Also
+ * initialize the ITARGETSRn regs to 0 on the emulated distributor.
+ */
int kvm_vgic_init(struct kvm *kvm)
{
int ret = 0, i;
+ if (!irqchip_in_kernel(kvm))
+ return 0;
+
mutex_lock(&kvm->lock);
if (vgic_initialized(kvm))
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
ioctl, which can then later be leveraged to use the
KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting addresses in
a more generic API than the ARM-specific one and is useful for
save/restore of VGIC state.
Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
Note that we change the check for creating a VGIC from bailing out if
any VCPUs were created, to bailing out if any VCPUs were ever run. This
is an important distinction that shouldn't break anything, but allows
creating the VGIC after the VCPUs have been created.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Prevent race in kvm_vgic_create by trying to take all the vcpu
locks before creating the vgic.
Changelog[v2]:
- None
---
Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
arch/arm/kvm/arm.c | 1 +
include/linux/kvm_host.h | 1 +
include/uapi/linux/kvm.h | 1 +
virt/kvm/arm/vgic.c | 58 +++++++++++++++++++++++-
virt/kvm/kvm_main.c | 6 ++-
6 files changed, 74 insertions(+), 3 deletions(-)
create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
new file mode 100644
index 0000000..38f27f7
--- /dev/null
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -0,0 +1,10 @@
+ARM Virtual Generic Interrupt Controller (VGIC)
+===============================================
+
+Device types supported:
+ KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
+
+Only one VGIC instance may be instantiated through either this API or the
+legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM interrupt
+controller, requiring emulated user-space devices to inject interrupts to the
+VGIC instead of directly to CPUs.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index e80b6a1..984908d 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_IRQCHIP:
r = vgic_present;
break;
+ case KVM_CAP_DEVICE_CTRL:
case KVM_CAP_USER_MEMORY:
case KVM_CAP_SYNC_MMU:
case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9523d2a..e68d9db 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct file *filp);
extern struct kvm_device_ops kvm_mpic_ops;
extern struct kvm_device_ops kvm_xics_ops;
extern struct kvm_device_ops kvm_vfio_ops;
+extern struct kvm_device_ops kvm_arm_vgic_ops;
#ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 902f124..b647c29 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -853,6 +853,7 @@ struct kvm_device_attr {
#define KVM_DEV_VFIO_GROUP 1
#define KVM_DEV_VFIO_GROUP_ADD 1
#define KVM_DEV_VFIO_GROUP_DEL 2
+#define KVM_DEV_TYPE_ARM_VGIC_V2 5
/*
* ioctls for VM fds
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 5e9df47..e3fbb5e 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1433,20 +1433,40 @@ out:
int kvm_vgic_create(struct kvm *kvm)
{
- int ret = 0;
+ int i, vcpu_lock_idx = -1, ret = 0;
+ struct kvm_vcpu *vcpu;
mutex_lock(&kvm->lock);
- if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
+ if (kvm->arch.vgic.vctrl_base) {
ret = -EEXIST;
goto out;
}
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!mutex_trylock(&vcpu->mutex))
+ goto out_unlock;
+ vcpu_lock_idx = i;
+ }
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vcpu->arch.has_run_once) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+ }
+
spin_lock_init(&kvm->arch.vgic.lock);
kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
+out_unlock:
+ for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
+ vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
+ mutex_unlock(&vcpu->mutex);
+ }
+
out:
mutex_unlock(&kvm->lock);
return ret;
@@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
mutex_unlock(&kvm->lock);
return r;
}
+
+static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static void vgic_destroy(struct kvm_device *dev)
+{
+ kfree(dev);
+}
+
+static int vgic_create(struct kvm_device *dev, u32 type)
+{
+ return kvm_vgic_create(dev->kvm);
+}
+
+struct kvm_device_ops kvm_arm_vgic_ops = {
+ .name = "kvm-arm-vgic",
+ .create = vgic_create,
+ .destroy = vgic_destroy,
+ .set_attr = vgic_set_attr,
+ .get_attr = vgic_get_attr,
+ .has_attr = vgic_has_attr,
+};
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 10015d6..fb8bb21 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct kvm *kvm,
#ifdef CONFIG_KVM_VFIO
case KVM_DEV_TYPE_VFIO:
ops = &kvm_vfio_ops;
- break;
+#endif
+#ifdef CONFIG_KVM_ARM_VGIC
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
+ ops = &kvm_arm_vgic_ops;
+ break;
#endif
default:
return -ENODEV;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
ioctl, which can then later be leveraged to use the
KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting addresses in
a more generic API than the ARM-specific one and is useful for
save/restore of VGIC state.
Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
Note that we change the check for creating a VGIC from bailing out if
any VCPUs were created, to bailing out if any VCPUs were ever run. This
is an important distinction that shouldn't break anything, but allows
creating the VGIC after the VCPUs have been created.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Prevent race in kvm_vgic_create by trying to take all the vcpu
locks before creating the vgic.
Changelog[v2]:
- None
---
Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
arch/arm/kvm/arm.c | 1 +
include/linux/kvm_host.h | 1 +
include/uapi/linux/kvm.h | 1 +
virt/kvm/arm/vgic.c | 58 +++++++++++++++++++++++-
virt/kvm/kvm_main.c | 6 ++-
6 files changed, 74 insertions(+), 3 deletions(-)
create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
new file mode 100644
index 0000000..38f27f7
--- /dev/null
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -0,0 +1,10 @@
+ARM Virtual Generic Interrupt Controller (VGIC)
+===============================================
+
+Device types supported:
+ KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
+
+Only one VGIC instance may be instantiated through either this API or the
+legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM interrupt
+controller, requiring emulated user-space devices to inject interrupts to the
+VGIC instead of directly to CPUs.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index e80b6a1..984908d 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_IRQCHIP:
r = vgic_present;
break;
+ case KVM_CAP_DEVICE_CTRL:
case KVM_CAP_USER_MEMORY:
case KVM_CAP_SYNC_MMU:
case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9523d2a..e68d9db 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct file *filp);
extern struct kvm_device_ops kvm_mpic_ops;
extern struct kvm_device_ops kvm_xics_ops;
extern struct kvm_device_ops kvm_vfio_ops;
+extern struct kvm_device_ops kvm_arm_vgic_ops;
#ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 902f124..b647c29 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -853,6 +853,7 @@ struct kvm_device_attr {
#define KVM_DEV_VFIO_GROUP 1
#define KVM_DEV_VFIO_GROUP_ADD 1
#define KVM_DEV_VFIO_GROUP_DEL 2
+#define KVM_DEV_TYPE_ARM_VGIC_V2 5
/*
* ioctls for VM fds
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 5e9df47..e3fbb5e 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1433,20 +1433,40 @@ out:
int kvm_vgic_create(struct kvm *kvm)
{
- int ret = 0;
+ int i, vcpu_lock_idx = -1, ret = 0;
+ struct kvm_vcpu *vcpu;
mutex_lock(&kvm->lock);
- if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
+ if (kvm->arch.vgic.vctrl_base) {
ret = -EEXIST;
goto out;
}
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (!mutex_trylock(&vcpu->mutex))
+ goto out_unlock;
+ vcpu_lock_idx = i;
+ }
+
+ kvm_for_each_vcpu(i, vcpu, kvm) {
+ if (vcpu->arch.has_run_once) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+ }
+
spin_lock_init(&kvm->arch.vgic.lock);
kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
+out_unlock:
+ for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
+ vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
+ mutex_unlock(&vcpu->mutex);
+ }
+
out:
mutex_unlock(&kvm->lock);
return ret;
@@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
mutex_unlock(&kvm->lock);
return r;
}
+
+static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
+{
+ return -ENXIO;
+}
+
+static void vgic_destroy(struct kvm_device *dev)
+{
+ kfree(dev);
+}
+
+static int vgic_create(struct kvm_device *dev, u32 type)
+{
+ return kvm_vgic_create(dev->kvm);
+}
+
+struct kvm_device_ops kvm_arm_vgic_ops = {
+ .name = "kvm-arm-vgic",
+ .create = vgic_create,
+ .destroy = vgic_destroy,
+ .set_attr = vgic_set_attr,
+ .get_attr = vgic_get_attr,
+ .has_attr = vgic_has_attr,
+};
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 10015d6..fb8bb21 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct kvm *kvm,
#ifdef CONFIG_KVM_VFIO
case KVM_DEV_TYPE_VFIO:
ops = &kvm_vfio_ops;
- break;
+#endif
+#ifdef CONFIG_KVM_ARM_VGIC
+ case KVM_DEV_TYPE_ARM_VGIC_V2:
+ ops = &kvm_arm_vgic_ops;
+ break;
#endif
default:
return -ENODEV;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Support setting the distributor and cpu interface base addresses in the
VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
in addition to the ARM specific API.
This has the added benefit of being able to share more code in user
space and do things in a uniform manner.
Also deprecate the older API at the same time, but backwards
compatibility will be maintained.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Spelling nits
- Fix error codes from vgic_get_attr
Changelog[v2]:
- None
---
Documentation/virtual/kvm/api.txt | 7 +-
Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
arch/arm/include/uapi/asm/kvm.h | 8 +++
arch/arm/kvm/arm.c | 2 +-
include/kvm/arm_vgic.h | 2 +-
virt/kvm/arm/vgic.c | 89 ++++++++++++++++++++----
6 files changed, 104 insertions(+), 15 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index a30035d..867112f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2391,7 +2391,8 @@ struct kvm_reg_list {
This ioctl returns the guest registers that are supported for the
KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
-4.85 KVM_ARM_SET_DEVICE_ADDR
+
+4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
Architectures: arm, arm64
@@ -2429,6 +2430,10 @@ must be called after calling KVM_CREATE_IRQCHIP, but before calling
KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of the
base addresses will return -EEXIST.
+Note, this IOCTL is deprecated and the more flexible SET/GET_DEVICE_ATTR API
+should be used instead.
+
+
4.86 KVM_PPC_RTAS_DEFINE_TOKEN
Capability: KVM_CAP_PPC_RTAS
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
index 38f27f7..c9febb2 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through either this API or the
legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM interrupt
controller, requiring emulated user-space devices to inject interrupts to the
VGIC instead of directly to CPUs.
+
+Groups:
+ KVM_DEV_ARM_VGIC_GRP_ADDR
+ Attributes:
+ KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
+ Base address in the guest physical address space of the GIC distributor
+ register mappings.
+
+ KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
+ Base address in the guest physical address space of the GIC virtual cpu
+ interface register mappings.
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index bd3b2f7..26a375c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
#define KVM_REG_ARM_VFP_FPINST 0x1009
#define KVM_REG_ARM_VFP_FPINST2 0x100A
+/* Device Control API: ARM VGIC */
+#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
+#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
+#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
+#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
+#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
+#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
/* KVM_IRQ_LINE irq field index values */
#define KVM_ARM_IRQ_TYPE_SHIFT 24
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 984908d..a0bf0d8 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -776,7 +776,7 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
case KVM_ARM_DEVICE_VGIC_V2:
if (!vgic_present)
return -ENXIO;
- return kvm_vgic_set_addr(kvm, type, dev_addr->addr);
+ return kvm_vgic_addr(kvm, type, &dev_addr->addr, true);
default:
return -ENODEV;
}
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7e2d158..be85127 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -144,7 +144,7 @@ struct kvm_run;
struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
-int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
+int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
int kvm_vgic_hyp_init(void);
int kvm_vgic_init(struct kvm *kvm);
int kvm_vgic_create(struct kvm *kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index e3fbb5e..edca754 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1490,6 +1490,12 @@ static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
{
int ret;
+ if (addr & ~KVM_PHYS_MASK)
+ return -E2BIG;
+
+ if (addr & (SZ_4K - 1))
+ return -EINVAL;
+
if (!IS_VGIC_ADDR_UNDEF(*ioaddr))
return -EEXIST;
if (addr + size < addr)
@@ -1502,26 +1508,41 @@ static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
return ret;
}
-int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+/**
+ * kvm_vgic_addr - set or get vgic VM base addresses
+ * @kvm: pointer to the vm struct
+ * @type: the VGIC addr type, one of KVM_VGIC_V2_ADDR_TYPE_XXX
+ * @addr: pointer to address value
+ * @write: if true set the address in the VM address space, if false read the
+ * address
+ *
+ * Set or get the vgic base addresses for the distributor and the virtual CPU
+ * interface in the VM physical address space. These addresses are properties
+ * of the emulated core/SoC and therefore user space initially knows this
+ * information.
+ */
+int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
{
int r = 0;
struct vgic_dist *vgic = &kvm->arch.vgic;
- if (addr & ~KVM_PHYS_MASK)
- return -E2BIG;
-
- if (addr & (SZ_4K - 1))
- return -EINVAL;
-
mutex_lock(&kvm->lock);
switch (type) {
case KVM_VGIC_V2_ADDR_TYPE_DIST:
- r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
- addr, KVM_VGIC_V2_DIST_SIZE);
+ if (write) {
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
+ *addr, KVM_VGIC_V2_DIST_SIZE);
+ } else {
+ *addr = vgic->vgic_dist_base;
+ }
break;
case KVM_VGIC_V2_ADDR_TYPE_CPU:
- r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
- addr, KVM_VGIC_V2_CPU_SIZE);
+ if (write) {
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
+ *addr, KVM_VGIC_V2_CPU_SIZE);
+ } else {
+ *addr = vgic->vgic_cpu_base;
+ }
break;
default:
r = -ENODEV;
@@ -1533,16 +1554,60 @@ int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
+ int r;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 addr;
+ unsigned long type = (unsigned long)attr->attr;
+
+ if (copy_from_user(&addr, uaddr, sizeof(addr)))
+ return -EFAULT;
+
+ r = kvm_vgic_addr(dev->kvm, type, &addr, true);
+ return (r == -ENODEV) ? -ENXIO : r;
+ }
+ }
+
return -ENXIO;
}
static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
- return -ENXIO;
+ int r = -ENXIO;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 addr;
+ unsigned long type = (unsigned long)attr->attr;
+
+ r = kvm_vgic_addr(dev->kvm, type, &addr, false);
+ if (r)
+ return (r == -ENODEV) ? -ENXIO : r;
+
+ if (copy_to_user(uaddr, &addr, sizeof(addr)))
+ return -EFAULT;
+ }
+ }
+
+ return r;
}
static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
+ phys_addr_t offset;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR:
+ switch (attr->attr) {
+ case KVM_VGIC_V2_ADDR_TYPE_DIST:
+ case KVM_VGIC_V2_ADDR_TYPE_CPU:
+ return 0;
+ }
+ break;
+ }
return -ENXIO;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Support setting the distributor and cpu interface base addresses in the
VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
in addition to the ARM specific API.
This has the added benefit of being able to share more code in user
space and do things in a uniform manner.
Also deprecate the older API at the same time, but backwards
compatibility will be maintained.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Spelling nits
- Fix error codes from vgic_get_attr
Changelog[v2]:
- None
---
Documentation/virtual/kvm/api.txt | 7 +-
Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
arch/arm/include/uapi/asm/kvm.h | 8 +++
arch/arm/kvm/arm.c | 2 +-
include/kvm/arm_vgic.h | 2 +-
virt/kvm/arm/vgic.c | 89 ++++++++++++++++++++----
6 files changed, 104 insertions(+), 15 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index a30035d..867112f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2391,7 +2391,8 @@ struct kvm_reg_list {
This ioctl returns the guest registers that are supported for the
KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
-4.85 KVM_ARM_SET_DEVICE_ADDR
+
+4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
Architectures: arm, arm64
@@ -2429,6 +2430,10 @@ must be called after calling KVM_CREATE_IRQCHIP, but before calling
KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of the
base addresses will return -EEXIST.
+Note, this IOCTL is deprecated and the more flexible SET/GET_DEVICE_ATTR API
+should be used instead.
+
+
4.86 KVM_PPC_RTAS_DEFINE_TOKEN
Capability: KVM_CAP_PPC_RTAS
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
index 38f27f7..c9febb2 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through either this API or the
legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM interrupt
controller, requiring emulated user-space devices to inject interrupts to the
VGIC instead of directly to CPUs.
+
+Groups:
+ KVM_DEV_ARM_VGIC_GRP_ADDR
+ Attributes:
+ KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
+ Base address in the guest physical address space of the GIC distributor
+ register mappings.
+
+ KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
+ Base address in the guest physical address space of the GIC virtual cpu
+ interface register mappings.
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index bd3b2f7..26a375c 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
#define KVM_REG_ARM_VFP_FPINST 0x1009
#define KVM_REG_ARM_VFP_FPINST2 0x100A
+/* Device Control API: ARM VGIC */
+#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
+#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
+#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
+#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
+#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL << KVM_DEV_ARM_VGIC_CPUID_SHIFT)
+#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
+#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL << KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
/* KVM_IRQ_LINE irq field index values */
#define KVM_ARM_IRQ_TYPE_SHIFT 24
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 984908d..a0bf0d8 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -776,7 +776,7 @@ static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
case KVM_ARM_DEVICE_VGIC_V2:
if (!vgic_present)
return -ENXIO;
- return kvm_vgic_set_addr(kvm, type, dev_addr->addr);
+ return kvm_vgic_addr(kvm, type, &dev_addr->addr, true);
default:
return -ENODEV;
}
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7e2d158..be85127 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -144,7 +144,7 @@ struct kvm_run;
struct kvm_exit_mmio;
#ifdef CONFIG_KVM_ARM_VGIC
-int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr);
+int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
int kvm_vgic_hyp_init(void);
int kvm_vgic_init(struct kvm *kvm);
int kvm_vgic_create(struct kvm *kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index e3fbb5e..edca754 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -1490,6 +1490,12 @@ static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
{
int ret;
+ if (addr & ~KVM_PHYS_MASK)
+ return -E2BIG;
+
+ if (addr & (SZ_4K - 1))
+ return -EINVAL;
+
if (!IS_VGIC_ADDR_UNDEF(*ioaddr))
return -EEXIST;
if (addr + size < addr)
@@ -1502,26 +1508,41 @@ static int vgic_ioaddr_assign(struct kvm *kvm, phys_addr_t *ioaddr,
return ret;
}
-int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
+/**
+ * kvm_vgic_addr - set or get vgic VM base addresses
+ * @kvm: pointer to the vm struct
+ * @type: the VGIC addr type, one of KVM_VGIC_V2_ADDR_TYPE_XXX
+ * @addr: pointer to address value
+ * @write: if true set the address in the VM address space, if false read the
+ * address
+ *
+ * Set or get the vgic base addresses for the distributor and the virtual CPU
+ * interface in the VM physical address space. These addresses are properties
+ * of the emulated core/SoC and therefore user space initially knows this
+ * information.
+ */
+int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
{
int r = 0;
struct vgic_dist *vgic = &kvm->arch.vgic;
- if (addr & ~KVM_PHYS_MASK)
- return -E2BIG;
-
- if (addr & (SZ_4K - 1))
- return -EINVAL;
-
mutex_lock(&kvm->lock);
switch (type) {
case KVM_VGIC_V2_ADDR_TYPE_DIST:
- r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
- addr, KVM_VGIC_V2_DIST_SIZE);
+ if (write) {
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_dist_base,
+ *addr, KVM_VGIC_V2_DIST_SIZE);
+ } else {
+ *addr = vgic->vgic_dist_base;
+ }
break;
case KVM_VGIC_V2_ADDR_TYPE_CPU:
- r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
- addr, KVM_VGIC_V2_CPU_SIZE);
+ if (write) {
+ r = vgic_ioaddr_assign(kvm, &vgic->vgic_cpu_base,
+ *addr, KVM_VGIC_V2_CPU_SIZE);
+ } else {
+ *addr = vgic->vgic_cpu_base;
+ }
break;
default:
r = -ENODEV;
@@ -1533,16 +1554,60 @@ int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
+ int r;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 addr;
+ unsigned long type = (unsigned long)attr->attr;
+
+ if (copy_from_user(&addr, uaddr, sizeof(addr)))
+ return -EFAULT;
+
+ r = kvm_vgic_addr(dev->kvm, type, &addr, true);
+ return (r == -ENODEV) ? -ENXIO : r;
+ }
+ }
+
return -ENXIO;
}
static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
- return -ENXIO;
+ int r = -ENXIO;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR: {
+ u64 __user *uaddr = (u64 __user *)(long)attr->addr;
+ u64 addr;
+ unsigned long type = (unsigned long)attr->attr;
+
+ r = kvm_vgic_addr(dev->kvm, type, &addr, false);
+ if (r)
+ return (r == -ENODEV) ? -ENXIO : r;
+
+ if (copy_to_user(uaddr, &addr, sizeof(addr)))
+ return -EFAULT;
+ }
+ }
+
+ return r;
}
static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
+ phys_addr_t offset;
+
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_ADDR:
+ switch (attr->attr) {
+ case KVM_VGIC_V2_ADDR_TYPE_DIST:
+ case KVM_VGIC_V2_ADDR_TYPE_CPU:
+ return 0;
+ }
+ break;
+ }
return -ENXIO;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 4/9] irqchip: arm-gic: Define additional MMIO offsets and masks
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall, Thomas Gleixner
Define CPU interface offsets for the GICC_ABPR, GICC_APR, and GICC_IIDR
registers. Define distributor registers for the GICD_SPENDSGIR and the
GICD_CPENDSGIR. KVM/ARM needs to know about these definitions to fully
support save/restore of the VGIC.
Also define some masks and shifts for the various GICH_VMCR fields.
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
include/linux/irqchip/arm-gic.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/include/linux/irqchip/arm-gic.h b/include/linux/irqchip/arm-gic.h
index 0e5d9ec..28b28fc 100644
--- a/include/linux/irqchip/arm-gic.h
+++ b/include/linux/irqchip/arm-gic.h
@@ -17,6 +17,9 @@
#define GIC_CPU_EOI 0x10
#define GIC_CPU_RUNNINGPRI 0x14
#define GIC_CPU_HIGHPRI 0x18
+#define GIC_CPU_ALIAS_BINPOINT 0x1c
+#define GIC_CPU_ACTIVEPRIO 0xd0
+#define GIC_CPU_IDENT 0xfc
#define GIC_DIST_CTRL 0x000
#define GIC_DIST_CTR 0x004
@@ -31,6 +34,8 @@
#define GIC_DIST_TARGET 0x800
#define GIC_DIST_CONFIG 0xc00
#define GIC_DIST_SOFTINT 0xf00
+#define GIC_DIST_SGI_CLEAR 0xf10
+#define GIC_DIST_SGI_SET 0xf20
#define GICH_HCR 0x0
#define GICH_VTR 0x4
@@ -54,6 +59,15 @@
#define GICH_LR_ACTIVE_BIT (1 << 29)
#define GICH_LR_EOI (1 << 19)
+#define GICH_VMCR_CTRL_SHIFT 0
+#define GICH_VMCR_CTRL_MASK (0x21f << GICH_VMCR_CTRL_SHIFT)
+#define GICH_VMCR_PRIMASK_SHIFT 27
+#define GICH_VMCR_PRIMASK_MASK (0x1f << GICH_VMCR_PRIMASK_SHIFT)
+#define GICH_VMCR_BINPOINT_SHIFT 21
+#define GICH_VMCR_BINPOINT_MASK (0x7 << GICH_VMCR_BINPOINT_SHIFT)
+#define GICH_VMCR_ALIAS_BINPOINT_SHIFT 18
+#define GICH_VMCR_ALIAS_BINPOINT_MASK (0x7 << GICH_VMCR_ALIAS_BINPOINT_SHIFT)
+
#define GICH_MISR_EOI (1 << 0)
#define GICH_MISR_U (1 << 1)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 4/9] irqchip: arm-gic: Define additional MMIO offsets and masks
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Define CPU interface offsets for the GICC_ABPR, GICC_APR, and GICC_IIDR
registers. Define distributor registers for the GICD_SPENDSGIR and the
GICD_CPENDSGIR. KVM/ARM needs to know about these definitions to fully
support save/restore of the VGIC.
Also define some masks and shifts for the various GICH_VMCR fields.
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
include/linux/irqchip/arm-gic.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/include/linux/irqchip/arm-gic.h b/include/linux/irqchip/arm-gic.h
index 0e5d9ec..28b28fc 100644
--- a/include/linux/irqchip/arm-gic.h
+++ b/include/linux/irqchip/arm-gic.h
@@ -17,6 +17,9 @@
#define GIC_CPU_EOI 0x10
#define GIC_CPU_RUNNINGPRI 0x14
#define GIC_CPU_HIGHPRI 0x18
+#define GIC_CPU_ALIAS_BINPOINT 0x1c
+#define GIC_CPU_ACTIVEPRIO 0xd0
+#define GIC_CPU_IDENT 0xfc
#define GIC_DIST_CTRL 0x000
#define GIC_DIST_CTR 0x004
@@ -31,6 +34,8 @@
#define GIC_DIST_TARGET 0x800
#define GIC_DIST_CONFIG 0xc00
#define GIC_DIST_SOFTINT 0xf00
+#define GIC_DIST_SGI_CLEAR 0xf10
+#define GIC_DIST_SGI_SET 0xf20
#define GICH_HCR 0x0
#define GICH_VTR 0x4
@@ -54,6 +59,15 @@
#define GICH_LR_ACTIVE_BIT (1 << 29)
#define GICH_LR_EOI (1 << 19)
+#define GICH_VMCR_CTRL_SHIFT 0
+#define GICH_VMCR_CTRL_MASK (0x21f << GICH_VMCR_CTRL_SHIFT)
+#define GICH_VMCR_PRIMASK_SHIFT 27
+#define GICH_VMCR_PRIMASK_MASK (0x1f << GICH_VMCR_PRIMASK_SHIFT)
+#define GICH_VMCR_BINPOINT_SHIFT 21
+#define GICH_VMCR_BINPOINT_MASK (0x7 << GICH_VMCR_BINPOINT_SHIFT)
+#define GICH_VMCR_ALIAS_BINPOINT_SHIFT 18
+#define GICH_VMCR_ALIAS_BINPOINT_MASK (0x7 << GICH_VMCR_ALIAS_BINPOINT_SHIFT)
+
#define GICH_MISR_EOI (1 << 0)
#define GICH_MISR_U (1 << 1)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 5/9] KVM: arm-vgic: Make vgic mmio functions more generic
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Rename the vgic_ranges array to vgic_dist_ranges to be more specific and
to prepare for handling CPU interface register access as well (for
save/restore of VGIC state).
Pass offset from distributor or interface MMIO base to
find_matching_range function instead of the physical address of the
access in the VM memory map. This allows other callers unaware of the
VM specifics, but with generic VGIC knowledge to reuse the function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/vgic.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index edca754..9b9fa20 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -602,7 +602,7 @@ struct mmio_range {
phys_addr_t offset);
};
-static const struct mmio_range vgic_ranges[] = {
+static const struct mmio_range vgic_dist_ranges[] = {
{
.base = GIC_DIST_CTRL,
.len = 12,
@@ -669,14 +669,13 @@ static const struct mmio_range vgic_ranges[] = {
static const
struct mmio_range *find_matching_range(const struct mmio_range *ranges,
struct kvm_exit_mmio *mmio,
- phys_addr_t base)
+ phys_addr_t offset)
{
const struct mmio_range *r = ranges;
- phys_addr_t addr = mmio->phys_addr - base;
while (r->len) {
- if (addr >= r->base &&
- (addr + mmio->len) <= (r->base + r->len))
+ if (offset >= r->base &&
+ (offset + mmio->len) <= (r->base + r->len))
return r;
r++;
}
@@ -713,7 +712,8 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
return true;
}
- range = find_matching_range(vgic_ranges, mmio, base);
+ offset = mmio->phys_addr - base;
+ range = find_matching_range(vgic_dist_ranges, mmio, offset);
if (unlikely(!range || !range->handle_mmio)) {
pr_warn("Unhandled access %d %08llx %d\n",
mmio->is_write, mmio->phys_addr, mmio->len);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 5/9] KVM: arm-vgic: Make vgic mmio functions more generic
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Rename the vgic_ranges array to vgic_dist_ranges to be more specific and
to prepare for handling CPU interface register access as well (for
save/restore of VGIC state).
Pass offset from distributor or interface MMIO base to
find_matching_range function instead of the physical address of the
access in the VM memory map. This allows other callers unaware of the
VM specifics, but with generic VGIC knowledge to reuse the function.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/vgic.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index edca754..9b9fa20 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -602,7 +602,7 @@ struct mmio_range {
phys_addr_t offset);
};
-static const struct mmio_range vgic_ranges[] = {
+static const struct mmio_range vgic_dist_ranges[] = {
{
.base = GIC_DIST_CTRL,
.len = 12,
@@ -669,14 +669,13 @@ static const struct mmio_range vgic_ranges[] = {
static const
struct mmio_range *find_matching_range(const struct mmio_range *ranges,
struct kvm_exit_mmio *mmio,
- phys_addr_t base)
+ phys_addr_t offset)
{
const struct mmio_range *r = ranges;
- phys_addr_t addr = mmio->phys_addr - base;
while (r->len) {
- if (addr >= r->base &&
- (addr + mmio->len) <= (r->base + r->len))
+ if (offset >= r->base &&
+ (offset + mmio->len) <= (r->base + r->len))
return r;
r++;
}
@@ -713,7 +712,8 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
return true;
}
- range = find_matching_range(vgic_ranges, mmio, base);
+ offset = mmio->phys_addr - base;
+ range = find_matching_range(vgic_dist_ranges, mmio, offset);
if (unlikely(!range || !range->handle_mmio)) {
pr_warn("Unhandled access %d %08llx %d\n",
mmio->is_write, mmio->phys_addr, mmio->len);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Add infrastructure to handle distributor and cpu interface register
accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding the
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS groups
and defining the semantics of the attr field to be the MMIO offset as
specified in the GICv2 specs.
Set the cpu field (physical CPU index) on the vcpu struct to -1 in
kvm_arch_vcpu_put so we have a method to check if the vcpu is running or
not without having to grab the VCPU mutexes.
Missing register accesses or other changes in individual register access
functions to support save/restore of the VGIC state is added in
subsequent patches.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Spelling and formatting nits
- Fill in the phys_addr in kvm_exit_mmio
- Put kvm_exit_mmio matching struct on stack
- Change if-else-if to switch statement in vgic_attr_regs_access
- Properly synchronize access to the VGIC with all VCPUs and the VM
structure, ensuring no VCPUs are running while user space is
accessing VGIC registers.
Changelog[v2]:
- Added implementation specific format for the GICC_APRn registers.
---
Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
arch/arm/kvm/arm.c | 1 +
virt/kvm/arm/vgic.c | 169 ++++++++++++++++++++++++
3 files changed, 222 insertions(+)
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
index c9febb2..7f4e91b 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -19,3 +19,55 @@ Groups:
KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
Base address in the guest physical address space of the GIC virtual cpu
interface register mappings.
+
+ KVM_DEV_ARM_VGIC_GRP_DIST_REGS
+ Attributes:
+ The attr field of kvm_device_attr encodes two values:
+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
+ values: | reserved | cpu id | offset |
+
+ All distributor regs are (rw, 32-bit)
+
+ The offset is relative to the "Distributor base address" as defined in the
+ GICv2 specs. Getting or setting such a register has the same effect as
+ reading or writing the register on the actual hardware from the cpu
+ specified with cpu id field. Note that most distributor fields are not
+ banked, but return the same value regardless of the cpu id used to access
+ the register.
+ Limitations:
+ - Priorities are not implemented, and registers are RAZ/WI
+ Errors:
+ -ENODEV: Getting or setting this register is not yet supported
+ -EBUSY: One or more VCPUs are running
+
+ KVM_DEV_ARM_VGIC_GRP_CPU_REGS
+ Attributes:
+ The attr field of kvm_device_attr encodes two values:
+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
+ values: | reserved | cpu id | offset |
+
+ All CPU interface regs are (rw, 32-bit)
+
+ The offset specifies the offset from the "CPU interface base address" as
+ defined in the GICv2 specs. Getting or setting such a register has the
+ same effect as reading or writing the register on the actual hardware.
+
+ The Active Priorities Registers APRn are implementation defined, so we set a
+ fixed format for our implementation that fits with the model of a "GICv2
+ implementation without the security extensions" which we present to the
+ guest. This interface always exposes four register APR[0-3] describing the
+ maximum possible 128 preemption levels. The semantics of the register
+ indicate if any interrupts in a given preemption level are in the active
+ state by setting the corresponding bit.
+
+ Thus, preemption level X has one or more active interrupts if and only if:
+
+ APRn[X mod 32] == 0b1, where n = X / 32
+
+ Bits for undefined preemption levels are RAZ/WI.
+
+ Limitations:
+ - Priorities are not implemented, and registers are RAZ/WI
+ Errors:
+ -ENODEV: Getting or setting this register is not yet supported
+ -EBUSY: One or more VCPUs are running
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index a0bf0d8..35acac8 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ vcpu->cpu = -1;
kvm_arm_set_running_vcpu(NULL);
}
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 9b9fa20..ecf6dcf 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
return false;
}
+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return false;
+}
+
+static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return false;
+}
+
/*
* I would have liked to use the kvm_bus_io_*() API instead, but it
* cannot cope with banked registers (only the VM pointer is passed
@@ -663,6 +677,16 @@ static const struct mmio_range vgic_dist_ranges[] = {
.len = 4,
.handle_mmio = handle_mmio_sgi_reg,
},
+ {
+ .base = GIC_DIST_SGI_CLEAR,
+ .len = VGIC_NR_SGIS,
+ .handle_mmio = handle_mmio_sgi_clear,
+ },
+ {
+ .base = GIC_DIST_SGI_SET,
+ .len = VGIC_NR_SGIS,
+ .handle_mmio = handle_mmio_sgi_set,
+ },
{}
};
@@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
return r;
}
+static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ return true;
+}
+
+static const struct mmio_range vgic_cpu_ranges[] = {
+ {
+ .base = GIC_CPU_CTRL,
+ .len = 12,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_ALIAS_BINPOINT,
+ .len = 4,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_ACTIVEPRIO,
+ .len = 16,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_IDENT,
+ .len = 4,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+};
+
+static int vgic_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u32 *reg, bool is_write)
+{
+ const struct mmio_range *r = NULL, *ranges;
+ phys_addr_t offset;
+ int ret, cpuid, c;
+ struct kvm_vcpu *vcpu, *tmp_vcpu;
+ struct vgic_dist *vgic;
+ struct kvm_exit_mmio mmio;
+
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
+ KVM_DEV_ARM_VGIC_CPUID_SHIFT;
+
+ mutex_lock(&dev->kvm->lock);
+
+ if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ vcpu = kvm_get_vcpu(dev->kvm, cpuid);
+ vgic = &dev->kvm->arch.vgic;
+
+ mmio.len = 4;
+ mmio.is_write = is_write;
+ if (is_write)
+ mmio_data_write(&mmio, ~0, *reg);
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ mmio.phys_addr = vgic->vgic_dist_base + offset;
+ ranges = vgic_dist_ranges;
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
+ mmio.phys_addr = vgic->vgic_cpu_base + offset;
+ ranges = vgic_cpu_ranges;
+ break;
+ default:
+ BUG();
+ }
+ r = find_matching_range(ranges, &mmio, offset);
+
+ if (unlikely(!r || !r->handle_mmio)) {
+ ret = -ENXIO;
+ goto out;
+ }
+
+
+ spin_lock(&vgic->lock);
+
+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
+ if (unlikely(tmp_vcpu->cpu != -1)) {
+ spin_unlock(&vgic->lock);
+ ret = -EBUSY;
+ goto out;
+ }
+ }
+
+ offset -= r->base;
+ r->handle_mmio(vcpu, &mmio, offset);
+ spin_unlock(&vgic->lock);
+
+ if (!is_write)
+ *reg = mmio_data_read(&mmio, ~0);
+
+ ret = 0;
+out:
+ mutex_unlock(&dev->kvm->lock);
+ return ret;
+}
+
static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
int r;
@@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
r = kvm_vgic_addr(dev->kvm, type, &addr, true);
return (r == -ENODEV) ? -ENXIO : r;
}
+
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 reg;
+
+ if (get_user(reg, uaddr))
+ return -EFAULT;
+
+ return vgic_attr_regs_access(dev, attr, ®, true);
+ }
+
}
return -ENXIO;
@@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
if (copy_to_user(uaddr, &addr, sizeof(addr)))
return -EFAULT;
+ break;
+ }
+
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 reg = 0;
+
+ r = vgic_attr_regs_access(dev, attr, ®, false);
+ if (r)
+ return r;
+ r = put_user(reg, uaddr);
+ break;
}
+
}
return r;
}
+static int vgic_has_attr_regs(const struct mmio_range *ranges,
+ phys_addr_t offset)
+{
+ struct kvm_exit_mmio dev_attr_mmio;
+
+ dev_attr_mmio.len = 4;
+ if (find_matching_range(ranges, &dev_attr_mmio, offset))
+ return 0;
+ else
+ return -ENXIO;
+}
+
static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
phys_addr_t offset;
@@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
return 0;
}
break;
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ return vgic_has_attr_regs(vgic_dist_ranges, offset);
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ return vgic_has_attr_regs(vgic_cpu_ranges, offset);
}
return -ENXIO;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Add infrastructure to handle distributor and cpu interface register
accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding the
KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS groups
and defining the semantics of the attr field to be the MMIO offset as
specified in the GICv2 specs.
Set the cpu field (physical CPU index) on the vcpu struct to -1 in
kvm_arch_vcpu_put so we have a method to check if the vcpu is running or
not without having to grab the VCPU mutexes.
Missing register accesses or other changes in individual register access
functions to support save/restore of the VGIC state is added in
subsequent patches.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Spelling and formatting nits
- Fill in the phys_addr in kvm_exit_mmio
- Put kvm_exit_mmio matching struct on stack
- Change if-else-if to switch statement in vgic_attr_regs_access
- Properly synchronize access to the VGIC with all VCPUs and the VM
structure, ensuring no VCPUs are running while user space is
accessing VGIC registers.
Changelog[v2]:
- Added implementation specific format for the GICC_APRn registers.
---
Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
arch/arm/kvm/arm.c | 1 +
virt/kvm/arm/vgic.c | 169 ++++++++++++++++++++++++
3 files changed, 222 insertions(+)
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virtual/kvm/devices/arm-vgic.txt
index c9febb2..7f4e91b 100644
--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
@@ -19,3 +19,55 @@ Groups:
KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
Base address in the guest physical address space of the GIC virtual cpu
interface register mappings.
+
+ KVM_DEV_ARM_VGIC_GRP_DIST_REGS
+ Attributes:
+ The attr field of kvm_device_attr encodes two values:
+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
+ values: | reserved | cpu id | offset |
+
+ All distributor regs are (rw, 32-bit)
+
+ The offset is relative to the "Distributor base address" as defined in the
+ GICv2 specs. Getting or setting such a register has the same effect as
+ reading or writing the register on the actual hardware from the cpu
+ specified with cpu id field. Note that most distributor fields are not
+ banked, but return the same value regardless of the cpu id used to access
+ the register.
+ Limitations:
+ - Priorities are not implemented, and registers are RAZ/WI
+ Errors:
+ -ENODEV: Getting or setting this register is not yet supported
+ -EBUSY: One or more VCPUs are running
+
+ KVM_DEV_ARM_VGIC_GRP_CPU_REGS
+ Attributes:
+ The attr field of kvm_device_attr encodes two values:
+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
+ values: | reserved | cpu id | offset |
+
+ All CPU interface regs are (rw, 32-bit)
+
+ The offset specifies the offset from the "CPU interface base address" as
+ defined in the GICv2 specs. Getting or setting such a register has the
+ same effect as reading or writing the register on the actual hardware.
+
+ The Active Priorities Registers APRn are implementation defined, so we set a
+ fixed format for our implementation that fits with the model of a "GICv2
+ implementation without the security extensions" which we present to the
+ guest. This interface always exposes four register APR[0-3] describing the
+ maximum possible 128 preemption levels. The semantics of the register
+ indicate if any interrupts in a given preemption level are in the active
+ state by setting the corresponding bit.
+
+ Thus, preemption level X has one or more active interrupts if and only if:
+
+ APRn[X mod 32] == 0b1, where n = X / 32
+
+ Bits for undefined preemption levels are RAZ/WI.
+
+ Limitations:
+ - Priorities are not implemented, and registers are RAZ/WI
+ Errors:
+ -ENODEV: Getting or setting this register is not yet supported
+ -EBUSY: One or more VCPUs are running
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index a0bf0d8..35acac8 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ vcpu->cpu = -1;
kvm_arm_set_running_vcpu(NULL);
}
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 9b9fa20..ecf6dcf 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
return false;
}
+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return false;
+}
+
+static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return false;
+}
+
/*
* I would have liked to use the kvm_bus_io_*() API instead, but it
* cannot cope with banked registers (only the VM pointer is passed
@@ -663,6 +677,16 @@ static const struct mmio_range vgic_dist_ranges[] = {
.len = 4,
.handle_mmio = handle_mmio_sgi_reg,
},
+ {
+ .base = GIC_DIST_SGI_CLEAR,
+ .len = VGIC_NR_SGIS,
+ .handle_mmio = handle_mmio_sgi_clear,
+ },
+ {
+ .base = GIC_DIST_SGI_SET,
+ .len = VGIC_NR_SGIS,
+ .handle_mmio = handle_mmio_sgi_set,
+ },
{}
};
@@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
return r;
}
+static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ return true;
+}
+
+static const struct mmio_range vgic_cpu_ranges[] = {
+ {
+ .base = GIC_CPU_CTRL,
+ .len = 12,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_ALIAS_BINPOINT,
+ .len = 4,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_ACTIVEPRIO,
+ .len = 16,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+ {
+ .base = GIC_CPU_IDENT,
+ .len = 4,
+ .handle_mmio = handle_cpu_mmio_misc,
+ },
+};
+
+static int vgic_attr_regs_access(struct kvm_device *dev,
+ struct kvm_device_attr *attr,
+ u32 *reg, bool is_write)
+{
+ const struct mmio_range *r = NULL, *ranges;
+ phys_addr_t offset;
+ int ret, cpuid, c;
+ struct kvm_vcpu *vcpu, *tmp_vcpu;
+ struct vgic_dist *vgic;
+ struct kvm_exit_mmio mmio;
+
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
+ KVM_DEV_ARM_VGIC_CPUID_SHIFT;
+
+ mutex_lock(&dev->kvm->lock);
+
+ if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ vcpu = kvm_get_vcpu(dev->kvm, cpuid);
+ vgic = &dev->kvm->arch.vgic;
+
+ mmio.len = 4;
+ mmio.is_write = is_write;
+ if (is_write)
+ mmio_data_write(&mmio, ~0, *reg);
+ switch (attr->group) {
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ mmio.phys_addr = vgic->vgic_dist_base + offset;
+ ranges = vgic_dist_ranges;
+ break;
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
+ mmio.phys_addr = vgic->vgic_cpu_base + offset;
+ ranges = vgic_cpu_ranges;
+ break;
+ default:
+ BUG();
+ }
+ r = find_matching_range(ranges, &mmio, offset);
+
+ if (unlikely(!r || !r->handle_mmio)) {
+ ret = -ENXIO;
+ goto out;
+ }
+
+
+ spin_lock(&vgic->lock);
+
+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
+ if (unlikely(tmp_vcpu->cpu != -1)) {
+ spin_unlock(&vgic->lock);
+ ret = -EBUSY;
+ goto out;
+ }
+ }
+
+ offset -= r->base;
+ r->handle_mmio(vcpu, &mmio, offset);
+ spin_unlock(&vgic->lock);
+
+ if (!is_write)
+ *reg = mmio_data_read(&mmio, ~0);
+
+ ret = 0;
+out:
+ mutex_unlock(&dev->kvm->lock);
+ return ret;
+}
+
static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
int r;
@@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
r = kvm_vgic_addr(dev->kvm, type, &addr, true);
return (r == -ENODEV) ? -ENXIO : r;
}
+
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 reg;
+
+ if (get_user(reg, uaddr))
+ return -EFAULT;
+
+ return vgic_attr_regs_access(dev, attr, ®, true);
+ }
+
}
return -ENXIO;
@@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
if (copy_to_user(uaddr, &addr, sizeof(addr)))
return -EFAULT;
+ break;
+ }
+
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
+ u32 reg = 0;
+
+ r = vgic_attr_regs_access(dev, attr, ®, false);
+ if (r)
+ return r;
+ r = put_user(reg, uaddr);
+ break;
}
+
}
return r;
}
+static int vgic_has_attr_regs(const struct mmio_range *ranges,
+ phys_addr_t offset)
+{
+ struct kvm_exit_mmio dev_attr_mmio;
+
+ dev_attr_mmio.len = 4;
+ if (find_matching_range(ranges, &dev_attr_mmio, offset))
+ return 0;
+ else
+ return -ENXIO;
+}
+
static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
phys_addr_t offset;
@@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
return 0;
}
break;
+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ return vgic_has_attr_regs(vgic_dist_ranges, offset);
+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
+ return vgic_has_attr_regs(vgic_cpu_ranges, offset);
}
return -ENXIO;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
To properly access the VGIC state from user space it is very unpractical
to have to loop through all the LRs in all register access functions.
Instead, support moving all pending state from LRs to the distributor,
but leave active state LRs alone.
Note that to accurately present the active and pending state to VCPUs
reading these distributor registers from a live VM, we would have to
stop all other VPUs than the calling VCPU and ask each CPU to unqueue
their LR state onto the distributor and add fields to track active state
on the distributor side as well. We don't have any users of such
functionality yet and there are other inaccuracies of the GIC emulation,
so don't provide accurate synchronized access to this state just yet.
However, when the time comes, having this function should help.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[3]:
- New patch in series
---
virt/kvm/arm/vgic.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 75 insertions(+), 5 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index ecf6dcf..44c669b 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
return false;
}
+#define LR_CPUID(lr) \
+ (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
+#define LR_IRQID(lr) \
+ ((lr) & GICH_LR_VIRTUALID)
+
+static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu *vgic_cpu)
+{
+ clear_bit(lr_nr, vgic_cpu->lr_used);
+ vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
+}
+
+/**
+ * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
+ * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
+ *
+ * Move any pending IRQs that have already been assigned to LRs back to the
+ * emulated distributor state so that the complete emulated state can be read
+ * from the main emulation structures without investigating the LRs.
+ *
+ * Note that IRQs in the active state in the LRs get their pending state moved
+ * to the distributor but the active state stays in the LRs, because we don't
+ * track the active state on the distributor side.
+ */
+static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ int vcpu_id = vcpu->vcpu_id;
+ int i, irq, source_cpu;
+ u32 *lr;
+
+ for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
+ lr = &vgic_cpu->vgic_lr[i];
+ irq = LR_IRQID(*lr);
+ source_cpu = LR_CPUID(*lr);
+
+ /*
+ * If the LR holds only an active interrupt (not pending) then
+ * just leave it alone.
+ */
+ if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
+ (unsigned long *)lr))
+ continue;
+
+ /*
+ * If the interrupt was only pending (not "active" or "pending
+ * and active") then we have effectively cleared the LR and it
+ * should be marked as free for other use.
+ */
+ if (!(*lr & GICH_LR_STATE))
+ vgic_retire_lr(i, irq, vgic_cpu);
+
+ /*
+ * Finally, reestablish the pending state on the distributor
+ * and the CPU interface. It may have already been pending,
+ * but that is fine, then we are only setting a few bits that
+ * were already set.
+ */
+ vgic_dist_irq_set(vcpu, irq);
+ if (irq < VGIC_NR_SGIS)
+ dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
+ vgic_update_state(vcpu->kvm);
+ }
+}
+
static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
@@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
}
}
-#define LR_CPUID(lr) \
- (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
#define MK_LR_PEND(src, irq) \
(GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
@@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu)
int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
if (!vgic_irq_is_enabled(vcpu, irq)) {
- vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
- clear_bit(lr, vgic_cpu->lr_used);
- vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
+ vgic_retire_lr(lr, irq, vgic_cpu);
if (vgic_irq_is_active(vcpu, irq))
vgic_irq_clear_active(vcpu, irq);
}
@@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
}
}
+ /*
+ * Move all pending IRQs from the LRs on all VCPUs so the pending
+ * state can be properly represented in the register state accessible
+ * through this API.
+ */
+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
+ vgic_unqueue_irqs(tmp_vcpu);
+
offset -= r->base;
r->handle_mmio(vcpu, &mmio, offset);
spin_unlock(&vgic->lock);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
To properly access the VGIC state from user space it is very unpractical
to have to loop through all the LRs in all register access functions.
Instead, support moving all pending state from LRs to the distributor,
but leave active state LRs alone.
Note that to accurately present the active and pending state to VCPUs
reading these distributor registers from a live VM, we would have to
stop all other VPUs than the calling VCPU and ask each CPU to unqueue
their LR state onto the distributor and add fields to track active state
on the distributor side as well. We don't have any users of such
functionality yet and there are other inaccuracies of the GIC emulation,
so don't provide accurate synchronized access to this state just yet.
However, when the time comes, having this function should help.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[3]:
- New patch in series
---
virt/kvm/arm/vgic.c | 80 +++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 75 insertions(+), 5 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index ecf6dcf..44c669b 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu *vcpu,
return false;
}
+#define LR_CPUID(lr) \
+ (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
+#define LR_IRQID(lr) \
+ ((lr) & GICH_LR_VIRTUALID)
+
+static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu *vgic_cpu)
+{
+ clear_bit(lr_nr, vgic_cpu->lr_used);
+ vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
+}
+
+/**
+ * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
+ * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
+ *
+ * Move any pending IRQs that have already been assigned to LRs back to the
+ * emulated distributor state so that the complete emulated state can be read
+ * from the main emulation structures without investigating the LRs.
+ *
+ * Note that IRQs in the active state in the LRs get their pending state moved
+ * to the distributor but the active state stays in the LRs, because we don't
+ * track the active state on the distributor side.
+ */
+static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ int vcpu_id = vcpu->vcpu_id;
+ int i, irq, source_cpu;
+ u32 *lr;
+
+ for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
+ lr = &vgic_cpu->vgic_lr[i];
+ irq = LR_IRQID(*lr);
+ source_cpu = LR_CPUID(*lr);
+
+ /*
+ * If the LR holds only an active interrupt (not pending) then
+ * just leave it alone.
+ */
+ if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
+ (unsigned long *)lr))
+ continue;
+
+ /*
+ * If the interrupt was only pending (not "active" or "pending
+ * and active") then we have effectively cleared the LR and it
+ * should be marked as free for other use.
+ */
+ if (!(*lr & GICH_LR_STATE))
+ vgic_retire_lr(i, irq, vgic_cpu);
+
+ /*
+ * Finally, reestablish the pending state on the distributor
+ * and the CPU interface. It may have already been pending,
+ * but that is fine, then we are only setting a few bits that
+ * were already set.
+ */
+ vgic_dist_irq_set(vcpu, irq);
+ if (irq < VGIC_NR_SGIS)
+ dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
+ vgic_update_state(vcpu->kvm);
+ }
+}
+
static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
@@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
}
}
-#define LR_CPUID(lr) \
- (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
#define MK_LR_PEND(src, irq) \
(GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) | (irq))
@@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu)
int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
if (!vgic_irq_is_enabled(vcpu, irq)) {
- vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
- clear_bit(lr, vgic_cpu->lr_used);
- vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
+ vgic_retire_lr(lr, irq, vgic_cpu);
if (vgic_irq_is_active(vcpu, irq))
vgic_irq_clear_active(vcpu, irq);
}
@@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
}
}
+ /*
+ * Move all pending IRQs from the LRs on all VCPUs so the pending
+ * state can be properly represented in the register state accessible
+ * through this API.
+ */
+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
+ vgic_unqueue_irqs(tmp_vcpu);
+
offset -= r->base;
r->handle_mmio(vcpu, &mmio, offset);
spin_unlock(&vgic->lock);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Handle MMIO accesses to the two registers which should support both the
case where the VMs want to read/write either of these registers and the
case where user space reads/writes these registers to do save/restore of
the VGIC state.
Note that the added complexity compared to simple set/clear enable
registers stems from the bookkeping of source cpu ids. It may be
possible to change the underlying data structure to simplify the
complexity, but since this is not in the critical path at all, this will
do.
Also note that reading this register from a live guest will not be
accurate compared to on hardware, because some state may be living on
the CPU LRs and the only way to give a consistent read would be to force
stop all the VCPUs and request them to unqueu the LR state onto the
distributor. Until we have an actual user of live reading this
register, we can live with the difference.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Renamed read/write SGI set/clear functions
- Rely on unqueuing of interrupts from LRs instead of reading LRs
directly
- Deduplicate code
Changelog[v2]:
- Use struct kvm_exit_mmio accessors for ->data field.
---
virt/kvm/arm/vgic.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 66 insertions(+), 4 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 44c669b..16053eb 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
}
}
-static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset)
+/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
+static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int sgi;
+ int min_sgi = (offset & ~0x3) * 4;
+ int max_sgi = min_sgi + 3;
+ int vcpu_id = vcpu->vcpu_id;
+ u32 reg = 0;
+
+ /* Copy source SGIs from distributor side */
+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
+ int shift = 8 * (sgi - min_sgi);
+ reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
+ }
+
+ mmio_data_write(mmio, ~0, reg);
return false;
}
+static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, bool set)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int sgi;
+ int min_sgi = (offset & ~0x3) * 4;
+ int max_sgi = min_sgi + 3;
+ int vcpu_id = vcpu->vcpu_id;
+ u32 reg;
+ bool updated = false;
+
+ reg = mmio_data_read(mmio, ~0);
+
+ /* Clear pending SGIs on the distributor */
+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
+ u8 mask = reg >> (8 * (sgi - min_sgi));
+ if (set) {
+ if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
+ updated = true;
+ dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
+ } else {
+ if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
+ updated = true;
+ dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
+ }
+ }
+
+ if (updated)
+ vgic_update_state(vcpu->kvm);
+
+ return updated;
+}
+
static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
{
- return false;
+ if (!mmio->is_write)
+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
+ else
+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
+}
+
+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ if (!mmio->is_write)
+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
+ else
+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
}
/*
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Handle MMIO accesses to the two registers which should support both the
case where the VMs want to read/write either of these registers and the
case where user space reads/writes these registers to do save/restore of
the VGIC state.
Note that the added complexity compared to simple set/clear enable
registers stems from the bookkeping of source cpu ids. It may be
possible to change the underlying data structure to simplify the
complexity, but since this is not in the critical path at all, this will
do.
Also note that reading this register from a live guest will not be
accurate compared to on hardware, because some state may be living on
the CPU LRs and the only way to give a consistent read would be to force
stop all the VCPUs and request them to unqueu the LR state onto the
distributor. Until we have an actual user of live reading this
register, we can live with the difference.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Changelog[v3]:
- Renamed read/write SGI set/clear functions
- Rely on unqueuing of interrupts from LRs instead of reading LRs
directly
- Deduplicate code
Changelog[v2]:
- Use struct kvm_exit_mmio accessors for ->data field.
---
virt/kvm/arm/vgic.c | 70 ++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 66 insertions(+), 4 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 44c669b..16053eb 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
}
}
-static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset)
+/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
+static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int sgi;
+ int min_sgi = (offset & ~0x3) * 4;
+ int max_sgi = min_sgi + 3;
+ int vcpu_id = vcpu->vcpu_id;
+ u32 reg = 0;
+
+ /* Copy source SGIs from distributor side */
+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
+ int shift = 8 * (sgi - min_sgi);
+ reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
+ }
+
+ mmio_data_write(mmio, ~0, reg);
return false;
}
+static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, bool set)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ int sgi;
+ int min_sgi = (offset & ~0x3) * 4;
+ int max_sgi = min_sgi + 3;
+ int vcpu_id = vcpu->vcpu_id;
+ u32 reg;
+ bool updated = false;
+
+ reg = mmio_data_read(mmio, ~0);
+
+ /* Clear pending SGIs on the distributor */
+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
+ u8 mask = reg >> (8 * (sgi - min_sgi));
+ if (set) {
+ if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
+ updated = true;
+ dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
+ } else {
+ if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
+ updated = true;
+ dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
+ }
+ }
+
+ if (updated)
+ vgic_update_state(vcpu->kvm);
+
+ return updated;
+}
+
static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
{
- return false;
+ if (!mmio->is_write)
+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
+ else
+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
+}
+
+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ if (!mmio->is_write)
+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
+ else
+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
}
/*
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 9/9] KVM: arm-vgic: Support CPU interface reg access
2013-11-17 4:30 ` Christoffer Dall
@ 2013-11-17 4:30 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: kvmarm, kvm, linux-arm-kernel; +Cc: patches, Christoffer Dall
Implement support for the CPU interface register access driven by MMIO
address offsets from the CPU interface base address. Useful for user
space to support save/restore of the VGIC state.
This commit adds support only for the same logic as the current VGIC
support, and no more. For example, the active priority registers are
handled as RAZ/WI, just like setting priorities on the emulated
distributor.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/vgic.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 73 insertions(+), 8 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 16053eb..a08b89a 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -71,6 +71,10 @@
#define VGIC_ADDR_UNDEF (-1)
#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
+#define PRODUCT_ID_KVM 0x4b /* ASCII code K */
+#define IMPLEMENTER_ARM 0x43b
+#define GICC_ARCH_VERSION_V2 0x2
+
/* Physical address of vgic virtual cpu interface */
static phys_addr_t vgic_vcpu_base;
@@ -312,7 +316,7 @@ static bool handle_mmio_misc(struct kvm_vcpu *vcpu,
u32 word_offset = offset & 3;
switch (offset & ~3) {
- case 0: /* CTLR */
+ case 0: /* GICD_CTLR */
reg = vcpu->kvm->arch.vgic.enabled;
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
@@ -323,15 +327,15 @@ static bool handle_mmio_misc(struct kvm_vcpu *vcpu,
}
break;
- case 4: /* TYPER */
+ case 4: /* GICD_TYPER */
reg = (atomic_read(&vcpu->kvm->online_vcpus) - 1) << 5;
reg |= (VGIC_NR_IRQS >> 5) - 1;
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
break;
- case 8: /* IIDR */
- reg = 0x4B00043B;
+ case 8: /* GICD_IIDR */
+ reg = (PRODUCT_ID_KVM << 24) | (IMPLEMENTER_ARM << 0);
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
break;
@@ -1703,9 +1707,70 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio, phys_addr_t offset)
{
- return true;
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ u32 reg, mask = 0, shift = 0;
+ bool updated = false;
+
+ switch (offset & ~0x3) {
+ case GIC_CPU_CTRL:
+ mask = GICH_VMCR_CTRL_MASK;
+ shift = GICH_VMCR_CTRL_SHIFT;
+ break;
+ case GIC_CPU_PRIMASK:
+ mask = GICH_VMCR_PRIMASK_MASK;
+ shift = GICH_VMCR_PRIMASK_SHIFT;
+ break;
+ case GIC_CPU_BINPOINT:
+ mask = GICH_VMCR_BINPOINT_MASK;
+ shift = GICH_VMCR_BINPOINT_SHIFT;
+ break;
+ case GIC_CPU_ALIAS_BINPOINT:
+ mask = GICH_VMCR_ALIAS_BINPOINT_MASK;
+ shift = GICH_VMCR_ALIAS_BINPOINT_SHIFT;
+ break;
+ }
+
+ if (!mmio->is_write) {
+ reg = (vgic_cpu->vgic_vmcr & mask) >> shift;
+ mmio_data_write(mmio, ~0, reg);
+ } else {
+ reg = mmio_data_read(mmio, ~0);
+ reg = (reg << shift) & mask;
+ if (reg != (vgic_cpu->vgic_vmcr & mask))
+ updated = true;
+ vgic_cpu->vgic_vmcr &= ~mask;
+ vgic_cpu->vgic_vmcr |= reg;
+ }
+ return updated;
+}
+
+static bool handle_mmio_abpr(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ return handle_cpu_mmio_misc(vcpu, mmio, GIC_CPU_ALIAS_BINPOINT);
}
+static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 reg;
+
+ if (mmio->is_write)
+ return false;
+
+ /* GICC_IIDR */
+ reg = (PRODUCT_ID_KVM << 20) |
+ (GICC_ARCH_VERSION_V2 << 16) |
+ (IMPLEMENTER_ARM << 0);
+ mmio_data_write(mmio, ~0, reg);
+ return false;
+}
+
+/*
+ * CPU Interface Register accesses - these are not accessed by the VM, but by
+ * user space for saving and restoring VGIC state.
+ */
static const struct mmio_range vgic_cpu_ranges[] = {
{
.base = GIC_CPU_CTRL,
@@ -1715,17 +1780,17 @@ static const struct mmio_range vgic_cpu_ranges[] = {
{
.base = GIC_CPU_ALIAS_BINPOINT,
.len = 4,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_mmio_abpr,
},
{
.base = GIC_CPU_ACTIVEPRIO,
.len = 16,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_mmio_raz_wi,
},
{
.base = GIC_CPU_IDENT,
.len = 4,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_cpu_mmio_ident,
},
};
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* [PATCH v3 9/9] KVM: arm-vgic: Support CPU interface reg access
@ 2013-11-17 4:30 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-11-17 4:30 UTC (permalink / raw)
To: linux-arm-kernel
Implement support for the CPU interface register access driven by MMIO
address offsets from the CPU interface base address. Useful for user
space to support save/restore of the VGIC state.
This commit adds support only for the same logic as the current VGIC
support, and no more. For example, the active priority registers are
handled as RAZ/WI, just like setting priorities on the emulated
distributor.
Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
---
virt/kvm/arm/vgic.c | 81 ++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 73 insertions(+), 8 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 16053eb..a08b89a 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -71,6 +71,10 @@
#define VGIC_ADDR_UNDEF (-1)
#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
+#define PRODUCT_ID_KVM 0x4b /* ASCII code K */
+#define IMPLEMENTER_ARM 0x43b
+#define GICC_ARCH_VERSION_V2 0x2
+
/* Physical address of vgic virtual cpu interface */
static phys_addr_t vgic_vcpu_base;
@@ -312,7 +316,7 @@ static bool handle_mmio_misc(struct kvm_vcpu *vcpu,
u32 word_offset = offset & 3;
switch (offset & ~3) {
- case 0: /* CTLR */
+ case 0: /* GICD_CTLR */
reg = vcpu->kvm->arch.vgic.enabled;
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
@@ -323,15 +327,15 @@ static bool handle_mmio_misc(struct kvm_vcpu *vcpu,
}
break;
- case 4: /* TYPER */
+ case 4: /* GICD_TYPER */
reg = (atomic_read(&vcpu->kvm->online_vcpus) - 1) << 5;
reg |= (VGIC_NR_IRQS >> 5) - 1;
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
break;
- case 8: /* IIDR */
- reg = 0x4B00043B;
+ case 8: /* GICD_IIDR */
+ reg = (PRODUCT_ID_KVM << 24) | (IMPLEMENTER_ARM << 0);
vgic_reg_access(mmio, ®, word_offset,
ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
break;
@@ -1703,9 +1707,70 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio, phys_addr_t offset)
{
- return true;
+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
+ u32 reg, mask = 0, shift = 0;
+ bool updated = false;
+
+ switch (offset & ~0x3) {
+ case GIC_CPU_CTRL:
+ mask = GICH_VMCR_CTRL_MASK;
+ shift = GICH_VMCR_CTRL_SHIFT;
+ break;
+ case GIC_CPU_PRIMASK:
+ mask = GICH_VMCR_PRIMASK_MASK;
+ shift = GICH_VMCR_PRIMASK_SHIFT;
+ break;
+ case GIC_CPU_BINPOINT:
+ mask = GICH_VMCR_BINPOINT_MASK;
+ shift = GICH_VMCR_BINPOINT_SHIFT;
+ break;
+ case GIC_CPU_ALIAS_BINPOINT:
+ mask = GICH_VMCR_ALIAS_BINPOINT_MASK;
+ shift = GICH_VMCR_ALIAS_BINPOINT_SHIFT;
+ break;
+ }
+
+ if (!mmio->is_write) {
+ reg = (vgic_cpu->vgic_vmcr & mask) >> shift;
+ mmio_data_write(mmio, ~0, reg);
+ } else {
+ reg = mmio_data_read(mmio, ~0);
+ reg = (reg << shift) & mask;
+ if (reg != (vgic_cpu->vgic_vmcr & mask))
+ updated = true;
+ vgic_cpu->vgic_vmcr &= ~mask;
+ vgic_cpu->vgic_vmcr |= reg;
+ }
+ return updated;
+}
+
+static bool handle_mmio_abpr(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
+{
+ return handle_cpu_mmio_misc(vcpu, mmio, GIC_CPU_ALIAS_BINPOINT);
}
+static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 reg;
+
+ if (mmio->is_write)
+ return false;
+
+ /* GICC_IIDR */
+ reg = (PRODUCT_ID_KVM << 20) |
+ (GICC_ARCH_VERSION_V2 << 16) |
+ (IMPLEMENTER_ARM << 0);
+ mmio_data_write(mmio, ~0, reg);
+ return false;
+}
+
+/*
+ * CPU Interface Register accesses - these are not accessed by the VM, but by
+ * user space for saving and restoring VGIC state.
+ */
static const struct mmio_range vgic_cpu_ranges[] = {
{
.base = GIC_CPU_CTRL,
@@ -1715,17 +1780,17 @@ static const struct mmio_range vgic_cpu_ranges[] = {
{
.base = GIC_CPU_ALIAS_BINPOINT,
.len = 4,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_mmio_abpr,
},
{
.base = GIC_CPU_ACTIVEPRIO,
.len = 16,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_mmio_raz_wi,
},
{
.base = GIC_CPU_IDENT,
.len = 4,
- .handle_mmio = handle_cpu_mmio_misc,
+ .handle_mmio = handle_cpu_mmio_ident,
},
};
--
1.7.10.4
^ permalink raw reply related [flat|nested] 48+ messages in thread
* Re: [PATCH v3 1/9] ARM: KVM: Allow creating the VGIC after VCPUs
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 15:11 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:11 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Rework the VGIC initialization slightly to allow initialization of
> the
> vgic cpu-specific state even if the irqchip (the VGIC) hasn't been
> created by user space yet. This is safe, because the vgic data
> structures are already allocated when the CPU is allocated if VGIC
> support is compiled into the kernel. Further, the init process does
> not
> depend on any other information and the sacrifice is a slight
> performance degradation for creating VMs in the no-VGIC case.
>
> The reason is that the new device control API doesn't mandate
> creating
> the VGIC before creating the VCPU and it is unreasonable to require
> user
> space to create the VGIC before creating the VCPUs.
>
> At the same time move the irqchip_in_kernel check out of
> kvm_vcpu_first_run_init and into the init function to make the
> per-vcpu
> and global init functions symmetric and add comments on the exported
> functions making it a bit easier to understand the init flow by only
> looking at vgic.c.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Looks good to me:
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
M.
> ---
> arch/arm/kvm/arm.c | 7 ++++---
> virt/kvm/arm/vgic.c | 22 +++++++++++++++++++---
> 2 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 2bc2ec4..e80b6a1 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -464,6 +464,8 @@ static void update_vttbr(struct kvm *kvm)
>
> static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
> {
> + int ret;
> +
> if (likely(vcpu->arch.has_run_once))
> return 0;
>
> @@ -473,9 +475,8 @@ static int kvm_vcpu_first_run_init(struct
> kvm_vcpu *vcpu)
> * Initialize the VGIC before running a vcpu the first time on
> * this VM.
> */
> - if (irqchip_in_kernel(vcpu->kvm) &&
> - unlikely(!vgic_initialized(vcpu->kvm))) {
> - int ret = kvm_vgic_init(vcpu->kvm);
> + if (unlikely(!vgic_initialized(vcpu->kvm))) {
> + ret = kvm_vgic_init(vcpu->kvm);
> if (ret)
> return ret;
> }
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 81e9481..5e9df47 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1243,15 +1243,19 @@ static irqreturn_t
> vgic_maintenance_handler(int irq, void *data)
> return IRQ_HANDLED;
> }
>
> +/**
> + * kvm_vgic_vcpu_init - Initialize per-vcpu VGIC state
> + * @vcpu: pointer to the vcpu struct
> + *
> + * Initialize the vgic_cpu struct and vgic_dist struct fields
> pertaining to
> + * this vcpu and enable the VGIC for this VCPU
> + */
> int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
> {
> struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> int i;
>
> - if (!irqchip_in_kernel(vcpu->kvm))
> - return 0;
> -
> if (vcpu->vcpu_id >= VGIC_MAX_CPUS)
> return -EBUSY;
>
> @@ -1383,10 +1387,22 @@ out:
> return ret;
> }
>
> +/**
> + * kvm_vgic_init - Initialize global VGIC state before running any
> VCPUs
> + * @kvm: pointer to the kvm struct
> + *
> + * Map the virtual CPU interface into the VM before running any
> VCPUs. We
> + * can't do this at creation time, because user space must first set
> the
> + * virtual CPU interface address in the guest physical address
> space. Also
> + * initialize the ITARGETSRn regs to 0 on the emulated distributor.
> + */
> int kvm_vgic_init(struct kvm *kvm)
> {
> int ret = 0, i;
>
> + if (!irqchip_in_kernel(kvm))
> + return 0;
> +
> mutex_lock(&kvm->lock);
>
> if (vgic_initialized(kvm))
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 1/9] ARM: KVM: Allow creating the VGIC after VCPUs
@ 2013-12-09 15:11 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:11 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Rework the VGIC initialization slightly to allow initialization of
> the
> vgic cpu-specific state even if the irqchip (the VGIC) hasn't been
> created by user space yet. This is safe, because the vgic data
> structures are already allocated when the CPU is allocated if VGIC
> support is compiled into the kernel. Further, the init process does
> not
> depend on any other information and the sacrifice is a slight
> performance degradation for creating VMs in the no-VGIC case.
>
> The reason is that the new device control API doesn't mandate
> creating
> the VGIC before creating the VCPU and it is unreasonable to require
> user
> space to create the VGIC before creating the VCPUs.
>
> At the same time move the irqchip_in_kernel check out of
> kvm_vcpu_first_run_init and into the init function to make the
> per-vcpu
> and global init functions symmetric and add comments on the exported
> functions making it a bit easier to understand the init flow by only
> looking at vgic.c.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Looks good to me:
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
M.
> ---
> arch/arm/kvm/arm.c | 7 ++++---
> virt/kvm/arm/vgic.c | 22 +++++++++++++++++++---
> 2 files changed, 23 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index 2bc2ec4..e80b6a1 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -464,6 +464,8 @@ static void update_vttbr(struct kvm *kvm)
>
> static int kvm_vcpu_first_run_init(struct kvm_vcpu *vcpu)
> {
> + int ret;
> +
> if (likely(vcpu->arch.has_run_once))
> return 0;
>
> @@ -473,9 +475,8 @@ static int kvm_vcpu_first_run_init(struct
> kvm_vcpu *vcpu)
> * Initialize the VGIC before running a vcpu the first time on
> * this VM.
> */
> - if (irqchip_in_kernel(vcpu->kvm) &&
> - unlikely(!vgic_initialized(vcpu->kvm))) {
> - int ret = kvm_vgic_init(vcpu->kvm);
> + if (unlikely(!vgic_initialized(vcpu->kvm))) {
> + ret = kvm_vgic_init(vcpu->kvm);
> if (ret)
> return ret;
> }
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 81e9481..5e9df47 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1243,15 +1243,19 @@ static irqreturn_t
> vgic_maintenance_handler(int irq, void *data)
> return IRQ_HANDLED;
> }
>
> +/**
> + * kvm_vgic_vcpu_init - Initialize per-vcpu VGIC state
> + * @vcpu: pointer to the vcpu struct
> + *
> + * Initialize the vgic_cpu struct and vgic_dist struct fields
> pertaining to
> + * this vcpu and enable the VGIC for this VCPU
> + */
> int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
> {
> struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> int i;
>
> - if (!irqchip_in_kernel(vcpu->kvm))
> - return 0;
> -
> if (vcpu->vcpu_id >= VGIC_MAX_CPUS)
> return -EBUSY;
>
> @@ -1383,10 +1387,22 @@ out:
> return ret;
> }
>
> +/**
> + * kvm_vgic_init - Initialize global VGIC state before running any
> VCPUs
> + * @kvm: pointer to the kvm struct
> + *
> + * Map the virtual CPU interface into the VM before running any
> VCPUs. We
> + * can't do this at creation time, because user space must first set
> the
> + * virtual CPU interface address in the guest physical address
> space. Also
> + * initialize the ITARGETSRn regs to 0 on the emulated distributor.
> + */
> int kvm_vgic_init(struct kvm *kvm)
> {
> int ret = 0, i;
>
> + if (!irqchip_in_kernel(kvm))
> + return 0;
> +
> mutex_lock(&kvm->lock);
>
> if (vgic_initialized(kvm))
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 15:19 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:19 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
> ioctl, which can then later be leveraged to use the
> KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting addresses
> in
> a more generic API than the ARM-specific one and is useful for
> save/restore of VGIC state.
>
> Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
>
> Note that we change the check for creating a VGIC from bailing out if
> any VCPUs were created, to bailing out if any VCPUs were ever run.
> This
> is an important distinction that shouldn't break anything, but allows
> creating the VGIC after the VCPUs have been created.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Prevent race in kvm_vgic_create by trying to take all the vcpu
> locks before creating the vgic.
>
> Changelog[v2]:
> - None
> ---
> Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
> arch/arm/kvm/arm.c | 1 +
> include/linux/kvm_host.h | 1 +
> include/uapi/linux/kvm.h | 1 +
> virt/kvm/arm/vgic.c | 58
> +++++++++++++++++++++++-
> virt/kvm/kvm_main.c | 6 ++-
> 6 files changed, 74 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
>
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> new file mode 100644
> index 0000000..38f27f7
> --- /dev/null
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -0,0 +1,10 @@
> +ARM Virtual Generic Interrupt Controller (VGIC)
> +===============================================
> +
> +Device types supported:
> + KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
> +
> +Only one VGIC instance may be instantiated through either this API
> or the
> +legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> interrupt
> +controller, requiring emulated user-space devices to inject
> interrupts to the
> +VGIC instead of directly to CPUs.
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index e80b6a1..984908d 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> case KVM_CAP_IRQCHIP:
> r = vgic_present;
> break;
> + case KVM_CAP_DEVICE_CTRL:
> case KVM_CAP_USER_MEMORY:
> case KVM_CAP_SYNC_MMU:
> case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 9523d2a..e68d9db 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct
> file *filp);
> extern struct kvm_device_ops kvm_mpic_ops;
> extern struct kvm_device_ops kvm_xics_ops;
> extern struct kvm_device_ops kvm_vfio_ops;
> +extern struct kvm_device_ops kvm_arm_vgic_ops;
Can we rename this to kvm_arm_vgic_v2_ops in order to be consistent
with KVM_DEV_TYPE_ARM_VGIC_V2?
>
> #ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
>
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 902f124..b647c29 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -853,6 +853,7 @@ struct kvm_device_attr {
> #define KVM_DEV_VFIO_GROUP 1
> #define KVM_DEV_VFIO_GROUP_ADD 1
> #define KVM_DEV_VFIO_GROUP_DEL 2
> +#define KVM_DEV_TYPE_ARM_VGIC_V2 5
>
> /*
> * ioctls for VM fds
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 5e9df47..e3fbb5e 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1433,20 +1433,40 @@ out:
>
> int kvm_vgic_create(struct kvm *kvm)
> {
> - int ret = 0;
> + int i, vcpu_lock_idx = -1, ret = 0;
> + struct kvm_vcpu *vcpu;
>
> mutex_lock(&kvm->lock);
>
> - if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
> + if (kvm->arch.vgic.vctrl_base) {
> ret = -EEXIST;
> goto out;
> }
>
> + kvm_for_each_vcpu(i, vcpu, kvm) {
> + if (!mutex_trylock(&vcpu->mutex))
> + goto out_unlock;
> + vcpu_lock_idx = i;
> + }
I think this deserves a comment explaining that vcpu->mutex is taken by
vcpu_load...
> + kvm_for_each_vcpu(i, vcpu, kvm) {
> + if (vcpu->arch.has_run_once) {
> + ret = -EBUSY;
> + goto out_unlock;
> + }
> + }
> +
> spin_lock_init(&kvm->arch.vgic.lock);
> kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
> kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
> kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
>
> +out_unlock:
> + for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
> + mutex_unlock(&vcpu->mutex);
> + }
> +
> out:
> mutex_unlock(&kvm->lock);
> return ret;
> @@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm,
> unsigned long type, u64 addr)
> mutex_unlock(&kvm->lock);
> return r;
> }
> +
> +static int vgic_set_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static int vgic_get_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static int vgic_has_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static void vgic_destroy(struct kvm_device *dev)
> +{
> + kfree(dev);
> +}
> +
> +static int vgic_create(struct kvm_device *dev, u32 type)
> +{
> + return kvm_vgic_create(dev->kvm);
> +}
> +
> +struct kvm_device_ops kvm_arm_vgic_ops = {
> + .name = "kvm-arm-vgic",
> + .create = vgic_create,
> + .destroy = vgic_destroy,
> + .set_attr = vgic_set_attr,
> + .get_attr = vgic_get_attr,
> + .has_attr = vgic_has_attr,
> +};
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 10015d6..fb8bb21 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct kvm
> *kvm,
> #ifdef CONFIG_KVM_VFIO
> case KVM_DEV_TYPE_VFIO:
> ops = &kvm_vfio_ops;
> - break;
> +#endif
> +#ifdef CONFIG_KVM_ARM_VGIC
> + case KVM_DEV_TYPE_ARM_VGIC_V2:
> + ops = &kvm_arm_vgic_ops;
> + break;
> #endif
> default:
> return -ENODEV;
Otherwise, looks good to me.
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
@ 2013-12-09 15:19 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:19 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
> ioctl, which can then later be leveraged to use the
> KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting addresses
> in
> a more generic API than the ARM-specific one and is useful for
> save/restore of VGIC state.
>
> Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
>
> Note that we change the check for creating a VGIC from bailing out if
> any VCPUs were created, to bailing out if any VCPUs were ever run.
> This
> is an important distinction that shouldn't break anything, but allows
> creating the VGIC after the VCPUs have been created.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Prevent race in kvm_vgic_create by trying to take all the vcpu
> locks before creating the vgic.
>
> Changelog[v2]:
> - None
> ---
> Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
> arch/arm/kvm/arm.c | 1 +
> include/linux/kvm_host.h | 1 +
> include/uapi/linux/kvm.h | 1 +
> virt/kvm/arm/vgic.c | 58
> +++++++++++++++++++++++-
> virt/kvm/kvm_main.c | 6 ++-
> 6 files changed, 74 insertions(+), 3 deletions(-)
> create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
>
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> new file mode 100644
> index 0000000..38f27f7
> --- /dev/null
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -0,0 +1,10 @@
> +ARM Virtual Generic Interrupt Controller (VGIC)
> +===============================================
> +
> +Device types supported:
> + KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
> +
> +Only one VGIC instance may be instantiated through either this API
> or the
> +legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> interrupt
> +controller, requiring emulated user-space devices to inject
> interrupts to the
> +VGIC instead of directly to CPUs.
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index e80b6a1..984908d 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> case KVM_CAP_IRQCHIP:
> r = vgic_present;
> break;
> + case KVM_CAP_DEVICE_CTRL:
> case KVM_CAP_USER_MEMORY:
> case KVM_CAP_SYNC_MMU:
> case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
> diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> index 9523d2a..e68d9db 100644
> --- a/include/linux/kvm_host.h
> +++ b/include/linux/kvm_host.h
> @@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct
> file *filp);
> extern struct kvm_device_ops kvm_mpic_ops;
> extern struct kvm_device_ops kvm_xics_ops;
> extern struct kvm_device_ops kvm_vfio_ops;
> +extern struct kvm_device_ops kvm_arm_vgic_ops;
Can we rename this to kvm_arm_vgic_v2_ops in order to be consistent
with KVM_DEV_TYPE_ARM_VGIC_V2?
>
> #ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
>
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 902f124..b647c29 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -853,6 +853,7 @@ struct kvm_device_attr {
> #define KVM_DEV_VFIO_GROUP 1
> #define KVM_DEV_VFIO_GROUP_ADD 1
> #define KVM_DEV_VFIO_GROUP_DEL 2
> +#define KVM_DEV_TYPE_ARM_VGIC_V2 5
>
> /*
> * ioctls for VM fds
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 5e9df47..e3fbb5e 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -1433,20 +1433,40 @@ out:
>
> int kvm_vgic_create(struct kvm *kvm)
> {
> - int ret = 0;
> + int i, vcpu_lock_idx = -1, ret = 0;
> + struct kvm_vcpu *vcpu;
>
> mutex_lock(&kvm->lock);
>
> - if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
> + if (kvm->arch.vgic.vctrl_base) {
> ret = -EEXIST;
> goto out;
> }
>
> + kvm_for_each_vcpu(i, vcpu, kvm) {
> + if (!mutex_trylock(&vcpu->mutex))
> + goto out_unlock;
> + vcpu_lock_idx = i;
> + }
I think this deserves a comment explaining that vcpu->mutex is taken by
vcpu_load...
> + kvm_for_each_vcpu(i, vcpu, kvm) {
> + if (vcpu->arch.has_run_once) {
> + ret = -EBUSY;
> + goto out_unlock;
> + }
> + }
> +
> spin_lock_init(&kvm->arch.vgic.lock);
> kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
> kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
> kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
>
> +out_unlock:
> + for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
> + vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
> + mutex_unlock(&vcpu->mutex);
> + }
> +
> out:
> mutex_unlock(&kvm->lock);
> return ret;
> @@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm,
> unsigned long type, u64 addr)
> mutex_unlock(&kvm->lock);
> return r;
> }
> +
> +static int vgic_set_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static int vgic_get_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static int vgic_has_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> +{
> + return -ENXIO;
> +}
> +
> +static void vgic_destroy(struct kvm_device *dev)
> +{
> + kfree(dev);
> +}
> +
> +static int vgic_create(struct kvm_device *dev, u32 type)
> +{
> + return kvm_vgic_create(dev->kvm);
> +}
> +
> +struct kvm_device_ops kvm_arm_vgic_ops = {
> + .name = "kvm-arm-vgic",
> + .create = vgic_create,
> + .destroy = vgic_destroy,
> + .set_attr = vgic_set_attr,
> + .get_attr = vgic_get_attr,
> + .has_attr = vgic_has_attr,
> +};
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 10015d6..fb8bb21 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct kvm
> *kvm,
> #ifdef CONFIG_KVM_VFIO
> case KVM_DEV_TYPE_VFIO:
> ops = &kvm_vfio_ops;
> - break;
> +#endif
> +#ifdef CONFIG_KVM_ARM_VGIC
> + case KVM_DEV_TYPE_ARM_VGIC_V2:
> + ops = &kvm_arm_vgic_ops;
> + break;
> #endif
> default:
> return -ENODEV;
Otherwise, looks good to me.
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 15:38 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:38 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Support setting the distributor and cpu interface base addresses in
> the
> VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
> in addition to the ARM specific API.
>
> This has the added benefit of being able to share more code in user
> space and do things in a uniform manner.
>
> Also deprecate the older API at the same time, but backwards
> compatibility will be maintained.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Spelling nits
> - Fix error codes from vgic_get_attr
>
> Changelog[v2]:
> - None
> ---
> Documentation/virtual/kvm/api.txt | 7 +-
> Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
> arch/arm/include/uapi/asm/kvm.h | 8 +++
> arch/arm/kvm/arm.c | 2 +-
> include/kvm/arm_vgic.h | 2 +-
> virt/kvm/arm/vgic.c | 89
> ++++++++++++++++++++----
> 6 files changed, 104 insertions(+), 15 deletions(-)
>
> diff --git a/Documentation/virtual/kvm/api.txt
> b/Documentation/virtual/kvm/api.txt
> index a30035d..867112f 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -2391,7 +2391,8 @@ struct kvm_reg_list {
> This ioctl returns the guest registers that are supported for the
> KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
>
> -4.85 KVM_ARM_SET_DEVICE_ADDR
> +
> +4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
>
> Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
> Architectures: arm, arm64
> @@ -2429,6 +2430,10 @@ must be called after calling
> KVM_CREATE_IRQCHIP, but before calling
> KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of
> the
> base addresses will return -EEXIST.
>
> +Note, this IOCTL is deprecated and the more flexible
> SET/GET_DEVICE_ATTR API
> +should be used instead.
> +
> +
> 4.86 KVM_PPC_RTAS_DEFINE_TOKEN
>
> Capability: KVM_CAP_PPC_RTAS
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> index 38f27f7..c9febb2 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through
> either this API or the
> legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> interrupt
> controller, requiring emulated user-space devices to inject
> interrupts to the
> VGIC instead of directly to CPUs.
> +
> +Groups:
> + KVM_DEV_ARM_VGIC_GRP_ADDR
> + Attributes:
> + KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
> + Base address in the guest physical address space of the GIC
> distributor
> + register mappings.
> +
> + KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> + Base address in the guest physical address space of the GIC
> virtual cpu
> + interface register mappings.
> diff --git a/arch/arm/include/uapi/asm/kvm.h
> b/arch/arm/include/uapi/asm/kvm.h
> index bd3b2f7..26a375c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
> #define KVM_REG_ARM_VFP_FPINST 0x1009
> #define KVM_REG_ARM_VFP_FPINST2 0x100A
>
> +/* Device Control API: ARM VGIC */
> +#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> +#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> +#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> +#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> +#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL <<
> KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> +#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> +#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL <<
> KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
What are these #defines for? I suppose they are used in a subsequent
patch? If so, can we please move them?
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
@ 2013-12-09 15:38 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:38 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Support setting the distributor and cpu interface base addresses in
> the
> VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
> in addition to the ARM specific API.
>
> This has the added benefit of being able to share more code in user
> space and do things in a uniform manner.
>
> Also deprecate the older API at the same time, but backwards
> compatibility will be maintained.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Spelling nits
> - Fix error codes from vgic_get_attr
>
> Changelog[v2]:
> - None
> ---
> Documentation/virtual/kvm/api.txt | 7 +-
> Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
> arch/arm/include/uapi/asm/kvm.h | 8 +++
> arch/arm/kvm/arm.c | 2 +-
> include/kvm/arm_vgic.h | 2 +-
> virt/kvm/arm/vgic.c | 89
> ++++++++++++++++++++----
> 6 files changed, 104 insertions(+), 15 deletions(-)
>
> diff --git a/Documentation/virtual/kvm/api.txt
> b/Documentation/virtual/kvm/api.txt
> index a30035d..867112f 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -2391,7 +2391,8 @@ struct kvm_reg_list {
> This ioctl returns the guest registers that are supported for the
> KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
>
> -4.85 KVM_ARM_SET_DEVICE_ADDR
> +
> +4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
>
> Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
> Architectures: arm, arm64
> @@ -2429,6 +2430,10 @@ must be called after calling
> KVM_CREATE_IRQCHIP, but before calling
> KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of
> the
> base addresses will return -EEXIST.
>
> +Note, this IOCTL is deprecated and the more flexible
> SET/GET_DEVICE_ATTR API
> +should be used instead.
> +
> +
> 4.86 KVM_PPC_RTAS_DEFINE_TOKEN
>
> Capability: KVM_CAP_PPC_RTAS
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> index 38f27f7..c9febb2 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through
> either this API or the
> legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> interrupt
> controller, requiring emulated user-space devices to inject
> interrupts to the
> VGIC instead of directly to CPUs.
> +
> +Groups:
> + KVM_DEV_ARM_VGIC_GRP_ADDR
> + Attributes:
> + KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
> + Base address in the guest physical address space of the GIC
> distributor
> + register mappings.
> +
> + KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> + Base address in the guest physical address space of the GIC
> virtual cpu
> + interface register mappings.
> diff --git a/arch/arm/include/uapi/asm/kvm.h
> b/arch/arm/include/uapi/asm/kvm.h
> index bd3b2f7..26a375c 100644
> --- a/arch/arm/include/uapi/asm/kvm.h
> +++ b/arch/arm/include/uapi/asm/kvm.h
> @@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
> #define KVM_REG_ARM_VFP_FPINST 0x1009
> #define KVM_REG_ARM_VFP_FPINST2 0x100A
>
> +/* Device Control API: ARM VGIC */
> +#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> +#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> +#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> +#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> +#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL <<
> KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> +#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> +#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL <<
> KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
What are these #defines for? I suppose they are used in a subsequent
patch? If so, can we please move them?
M.
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 4/9] irqchip: arm-gic: Define additional MMIO offsets and masks
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 15:40 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:40 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, Thomas Gleixner, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Define CPU interface offsets for the GICC_ABPR, GICC_APR, and
> GICC_IIDR
> registers. Define distributor registers for the GICD_SPENDSGIR and
> the
> GICD_CPENDSGIR. KVM/ARM needs to know about these definitions to
> fully
> support save/restore of the VGIC.
>
> Also define some masks and shifts for the various GICH_VMCR fields.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> include/linux/irqchip/arm-gic.h | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/include/linux/irqchip/arm-gic.h
> b/include/linux/irqchip/arm-gic.h
> index 0e5d9ec..28b28fc 100644
> --- a/include/linux/irqchip/arm-gic.h
> +++ b/include/linux/irqchip/arm-gic.h
> @@ -17,6 +17,9 @@
> #define GIC_CPU_EOI 0x10
> #define GIC_CPU_RUNNINGPRI 0x14
> #define GIC_CPU_HIGHPRI 0x18
> +#define GIC_CPU_ALIAS_BINPOINT 0x1c
> +#define GIC_CPU_ACTIVEPRIO 0xd0
> +#define GIC_CPU_IDENT 0xfc
>
> #define GIC_DIST_CTRL 0x000
> #define GIC_DIST_CTR 0x004
> @@ -31,6 +34,8 @@
> #define GIC_DIST_TARGET 0x800
> #define GIC_DIST_CONFIG 0xc00
> #define GIC_DIST_SOFTINT 0xf00
> +#define GIC_DIST_SGI_CLEAR 0xf10
> +#define GIC_DIST_SGI_SET 0xf20
>
> #define GICH_HCR 0x0
> #define GICH_VTR 0x4
> @@ -54,6 +59,15 @@
> #define GICH_LR_ACTIVE_BIT (1 << 29)
> #define GICH_LR_EOI (1 << 19)
>
> +#define GICH_VMCR_CTRL_SHIFT 0
> +#define GICH_VMCR_CTRL_MASK (0x21f << GICH_VMCR_CTRL_SHIFT)
> +#define GICH_VMCR_PRIMASK_SHIFT 27
> +#define GICH_VMCR_PRIMASK_MASK (0x1f << GICH_VMCR_PRIMASK_SHIFT)
> +#define GICH_VMCR_BINPOINT_SHIFT 21
> +#define GICH_VMCR_BINPOINT_MASK (0x7 << GICH_VMCR_BINPOINT_SHIFT)
> +#define GICH_VMCR_ALIAS_BINPOINT_SHIFT 18
> +#define GICH_VMCR_ALIAS_BINPOINT_MASK (0x7 <<
> GICH_VMCR_ALIAS_BINPOINT_SHIFT)
> +
> #define GICH_MISR_EOI (1 << 0)
> #define GICH_MISR_U (1 << 1)
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 4/9] irqchip: arm-gic: Define additional MMIO offsets and masks
@ 2013-12-09 15:40 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:40 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Define CPU interface offsets for the GICC_ABPR, GICC_APR, and
> GICC_IIDR
> registers. Define distributor registers for the GICD_SPENDSGIR and
> the
> GICD_CPENDSGIR. KVM/ARM needs to know about these definitions to
> fully
> support save/restore of the VGIC.
>
> Also define some masks and shifts for the various GICH_VMCR fields.
>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> include/linux/irqchip/arm-gic.h | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/include/linux/irqchip/arm-gic.h
> b/include/linux/irqchip/arm-gic.h
> index 0e5d9ec..28b28fc 100644
> --- a/include/linux/irqchip/arm-gic.h
> +++ b/include/linux/irqchip/arm-gic.h
> @@ -17,6 +17,9 @@
> #define GIC_CPU_EOI 0x10
> #define GIC_CPU_RUNNINGPRI 0x14
> #define GIC_CPU_HIGHPRI 0x18
> +#define GIC_CPU_ALIAS_BINPOINT 0x1c
> +#define GIC_CPU_ACTIVEPRIO 0xd0
> +#define GIC_CPU_IDENT 0xfc
>
> #define GIC_DIST_CTRL 0x000
> #define GIC_DIST_CTR 0x004
> @@ -31,6 +34,8 @@
> #define GIC_DIST_TARGET 0x800
> #define GIC_DIST_CONFIG 0xc00
> #define GIC_DIST_SOFTINT 0xf00
> +#define GIC_DIST_SGI_CLEAR 0xf10
> +#define GIC_DIST_SGI_SET 0xf20
>
> #define GICH_HCR 0x0
> #define GICH_VTR 0x4
> @@ -54,6 +59,15 @@
> #define GICH_LR_ACTIVE_BIT (1 << 29)
> #define GICH_LR_EOI (1 << 19)
>
> +#define GICH_VMCR_CTRL_SHIFT 0
> +#define GICH_VMCR_CTRL_MASK (0x21f << GICH_VMCR_CTRL_SHIFT)
> +#define GICH_VMCR_PRIMASK_SHIFT 27
> +#define GICH_VMCR_PRIMASK_MASK (0x1f << GICH_VMCR_PRIMASK_SHIFT)
> +#define GICH_VMCR_BINPOINT_SHIFT 21
> +#define GICH_VMCR_BINPOINT_MASK (0x7 << GICH_VMCR_BINPOINT_SHIFT)
> +#define GICH_VMCR_ALIAS_BINPOINT_SHIFT 18
> +#define GICH_VMCR_ALIAS_BINPOINT_MASK (0x7 <<
> GICH_VMCR_ALIAS_BINPOINT_SHIFT)
> +
> #define GICH_MISR_EOI (1 << 0)
> #define GICH_MISR_U (1 << 1)
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 5/9] KVM: arm-vgic: Make vgic mmio functions more generic
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 15:47 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:47 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Rename the vgic_ranges array to vgic_dist_ranges to be more specific
> and
> to prepare for handling CPU interface register access as well (for
> save/restore of VGIC state).
>
> Pass offset from distributor or interface MMIO base to
> find_matching_range function instead of the physical address of the
> access in the VM memory map. This allows other callers unaware of
> the
> VM specifics, but with generic VGIC knowledge to reuse the function.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> virt/kvm/arm/vgic.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index edca754..9b9fa20 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -602,7 +602,7 @@ struct mmio_range {
> phys_addr_t offset);
> };
>
> -static const struct mmio_range vgic_ranges[] = {
> +static const struct mmio_range vgic_dist_ranges[] = {
> {
> .base = GIC_DIST_CTRL,
> .len = 12,
> @@ -669,14 +669,13 @@ static const struct mmio_range vgic_ranges[] =
> {
> static const
> struct mmio_range *find_matching_range(const struct mmio_range
> *ranges,
> struct kvm_exit_mmio *mmio,
> - phys_addr_t base)
> + phys_addr_t offset)
> {
> const struct mmio_range *r = ranges;
> - phys_addr_t addr = mmio->phys_addr - base;
>
> while (r->len) {
> - if (addr >= r->base &&
> - (addr + mmio->len) <= (r->base + r->len))
> + if (offset >= r->base &&
> + (offset + mmio->len) <= (r->base + r->len))
> return r;
> r++;
> }
> @@ -713,7 +712,8 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu,
> struct kvm_run *run,
> return true;
> }
>
> - range = find_matching_range(vgic_ranges, mmio, base);
> + offset = mmio->phys_addr - base;
> + range = find_matching_range(vgic_dist_ranges, mmio, offset);
> if (unlikely(!range || !range->handle_mmio)) {
> pr_warn("Unhandled access %d %08llx %d\n",
> mmio->is_write, mmio->phys_addr, mmio->len);
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 5/9] KVM: arm-vgic: Make vgic mmio functions more generic
@ 2013-12-09 15:47 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 15:47 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Rename the vgic_ranges array to vgic_dist_ranges to be more specific
> and
> to prepare for handling CPU interface register access as well (for
> save/restore of VGIC state).
>
> Pass offset from distributor or interface MMIO base to
> find_matching_range function instead of the physical address of the
> access in the VM memory map. This allows other callers unaware of
> the
> VM specifics, but with generic VGIC knowledge to reuse the function.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> virt/kvm/arm/vgic.c | 12 ++++++------
> 1 file changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index edca754..9b9fa20 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -602,7 +602,7 @@ struct mmio_range {
> phys_addr_t offset);
> };
>
> -static const struct mmio_range vgic_ranges[] = {
> +static const struct mmio_range vgic_dist_ranges[] = {
> {
> .base = GIC_DIST_CTRL,
> .len = 12,
> @@ -669,14 +669,13 @@ static const struct mmio_range vgic_ranges[] =
> {
> static const
> struct mmio_range *find_matching_range(const struct mmio_range
> *ranges,
> struct kvm_exit_mmio *mmio,
> - phys_addr_t base)
> + phys_addr_t offset)
> {
> const struct mmio_range *r = ranges;
> - phys_addr_t addr = mmio->phys_addr - base;
>
> while (r->len) {
> - if (addr >= r->base &&
> - (addr + mmio->len) <= (r->base + r->len))
> + if (offset >= r->base &&
> + (offset + mmio->len) <= (r->base + r->len))
> return r;
> r++;
> }
> @@ -713,7 +712,8 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu,
> struct kvm_run *run,
> return true;
> }
>
> - range = find_matching_range(vgic_ranges, mmio, base);
> + offset = mmio->phys_addr - base;
> + range = find_matching_range(vgic_dist_ranges, mmio, offset);
> if (unlikely(!range || !range->handle_mmio)) {
> pr_warn("Unhandled access %d %08llx %d\n",
> mmio->is_write, mmio->phys_addr, mmio->len);
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 16:02 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:02 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Add infrastructure to handle distributor and cpu interface register
> accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding
> the
> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> groups
> and defining the semantics of the attr field to be the MMIO offset as
> specified in the GICv2 specs.
>
> Set the cpu field (physical CPU index) on the vcpu struct to -1 in
> kvm_arch_vcpu_put so we have a method to check if the vcpu is running
> or
> not without having to grab the VCPU mutexes.
>
> Missing register accesses or other changes in individual register
> access
> functions to support save/restore of the VGIC state is added in
> subsequent patches.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Spelling and formatting nits
> - Fill in the phys_addr in kvm_exit_mmio
> - Put kvm_exit_mmio matching struct on stack
> - Change if-else-if to switch statement in vgic_attr_regs_access
> - Properly synchronize access to the VGIC with all VCPUs and the VM
> structure, ensuring no VCPUs are running while user space is
> accessing VGIC registers.
I have some doubts about this last point, see below.
> Changelog[v2]:
> - Added implementation specific format for the GICC_APRn registers.
> ---
> Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
> arch/arm/kvm/arm.c | 1 +
> virt/kvm/arm/vgic.c | 169
> ++++++++++++++++++++++++
> 3 files changed, 222 insertions(+)
>
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> index c9febb2..7f4e91b 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -19,3 +19,55 @@ Groups:
> KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> Base address in the guest physical address space of the GIC
> virtual cpu
> interface register mappings.
> +
> + KVM_DEV_ARM_VGIC_GRP_DIST_REGS
> + Attributes:
> + The attr field of kvm_device_attr encodes two values:
> + bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> + values: | reserved | cpu id | offset |
> +
> + All distributor regs are (rw, 32-bit)
> +
> + The offset is relative to the "Distributor base address" as
> defined in the
> + GICv2 specs. Getting or setting such a register has the same
> effect as
> + reading or writing the register on the actual hardware from the
> cpu
> + specified with cpu id field. Note that most distributor fields
> are not
> + banked, but return the same value regardless of the cpu id used
> to access
> + the register.
> + Limitations:
> + - Priorities are not implemented, and registers are RAZ/WI
> + Errors:
> + -ENODEV: Getting or setting this register is not yet supported
> + -EBUSY: One or more VCPUs are running
> +
> + KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> + Attributes:
> + The attr field of kvm_device_attr encodes two values:
> + bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> + values: | reserved | cpu id | offset |
> +
> + All CPU interface regs are (rw, 32-bit)
> +
> + The offset specifies the offset from the "CPU interface base
> address" as
> + defined in the GICv2 specs. Getting or setting such a register
> has the
> + same effect as reading or writing the register on the actual
> hardware.
> +
> + The Active Priorities Registers APRn are implementation defined,
> so we set a
> + fixed format for our implementation that fits with the model of
> a "GICv2
> + implementation without the security extensions" which we present
> to the
> + guest. This interface always exposes four register APR[0-3]
> describing the
> + maximum possible 128 preemption levels. The semantics of the
> register
> + indicate if any interrupts in a given preemption level are in
> the active
> + state by setting the corresponding bit.
> +
> + Thus, preemption level X has one or more active interrupts if
> and only if:
> +
> + APRn[X mod 32] == 0b1, where n = X / 32
> +
> + Bits for undefined preemption levels are RAZ/WI.
> +
> + Limitations:
> + - Priorities are not implemented, and registers are RAZ/WI
> + Errors:
> + -ENODEV: Getting or setting this register is not yet supported
> + -EBUSY: One or more VCPUs are running
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index a0bf0d8..35acac8 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu,
> int cpu)
>
> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> {
> + vcpu->cpu = -1;
> kvm_arm_set_running_vcpu(NULL);
> }
This guy deserves to be in a separate patch, with a proper comment and
maybe a #define for the -1 value. Something like "KVM_VCPU_NOT_RUNNING"?
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 9b9fa20..ecf6dcf 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu
> *vcpu,
> return false;
> }
>
> +static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + return false;
> +}
> +
> +static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + return false;
> +}
> +
> /*
> * I would have liked to use the kvm_bus_io_*() API instead, but it
> * cannot cope with banked registers (only the VM pointer is passed
> @@ -663,6 +677,16 @@ static const struct mmio_range
> vgic_dist_ranges[] = {
> .len = 4,
> .handle_mmio = handle_mmio_sgi_reg,
> },
> + {
> + .base = GIC_DIST_SGI_CLEAR,
> + .len = VGIC_NR_SGIS,
> + .handle_mmio = handle_mmio_sgi_clear,
> + },
> + {
> + .base = GIC_DIST_SGI_SET,
> + .len = VGIC_NR_SGIS,
> + .handle_mmio = handle_mmio_sgi_set,
> + },
> {}
> };
>
> @@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> long type, u64 *addr, bool write)
> return r;
> }
>
> +static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio, phys_addr_t offset)
> +{
> + return true;
> +}
> +
> +static const struct mmio_range vgic_cpu_ranges[] = {
> + {
> + .base = GIC_CPU_CTRL,
> + .len = 12,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_ALIAS_BINPOINT,
> + .len = 4,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_ACTIVEPRIO,
> + .len = 16,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_IDENT,
> + .len = 4,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> +};
> +
> +static int vgic_attr_regs_access(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + u32 *reg, bool is_write)
> +{
> + const struct mmio_range *r = NULL, *ranges;
> + phys_addr_t offset;
> + int ret, cpuid, c;
> + struct kvm_vcpu *vcpu, *tmp_vcpu;
> + struct vgic_dist *vgic;
> + struct kvm_exit_mmio mmio;
> +
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
> + KVM_DEV_ARM_VGIC_CPUID_SHIFT;
> +
> + mutex_lock(&dev->kvm->lock);
> +
> + if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + vcpu = kvm_get_vcpu(dev->kvm, cpuid);
> + vgic = &dev->kvm->arch.vgic;
> +
> + mmio.len = 4;
> + mmio.is_write = is_write;
> + if (is_write)
> + mmio_data_write(&mmio, ~0, *reg);
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + mmio.phys_addr = vgic->vgic_dist_base + offset;
> + ranges = vgic_dist_ranges;
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> + mmio.phys_addr = vgic->vgic_cpu_base + offset;
> + ranges = vgic_cpu_ranges;
> + break;
> + default:
> + BUG();
> + }
> + r = find_matching_range(ranges, &mmio, offset);
> +
> + if (unlikely(!r || !r->handle_mmio)) {
> + ret = -ENXIO;
> + goto out;
> + }
> +
> +
> + spin_lock(&vgic->lock);
> +
> + kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
> + if (unlikely(tmp_vcpu->cpu != -1)) {
What guarantees that the vcpu is not going to be restarted behind your
back? I can't see anything that locks vcpu->mutex.
> + spin_unlock(&vgic->lock);
> + ret = -EBUSY;
> + goto out;
> + }
> + }
> +
> + offset -= r->base;
> + r->handle_mmio(vcpu, &mmio, offset);
> + spin_unlock(&vgic->lock);
> +
> + if (!is_write)
> + *reg = mmio_data_read(&mmio, ~0);
> +
> + ret = 0;
> +out:
How about moving the "spin_unlock(&vgic->lock);" here, and simplify the
exit case of the loop above?
> + mutex_unlock(&dev->kvm->lock);
> + return ret;
> +}
> +
> static int vgic_set_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> {
> int r;
> @@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
> r = kvm_vgic_addr(dev->kvm, type, &addr, true);
> return (r == -ENODEV) ? -ENXIO : r;
> }
> +
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 reg;
> +
> + if (get_user(reg, uaddr))
> + return -EFAULT;
> +
> + return vgic_attr_regs_access(dev, attr, ®, true);
> + }
> +
> }
>
> return -ENXIO;
> @@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
>
> if (copy_to_user(uaddr, &addr, sizeof(addr)))
> return -EFAULT;
> + break;
> + }
> +
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 reg = 0;
> +
> + r = vgic_attr_regs_access(dev, attr, ®, false);
> + if (r)
> + return r;
> + r = put_user(reg, uaddr);
> + break;
> }
> +
> }
>
> return r;
> }
>
> +static int vgic_has_attr_regs(const struct mmio_range *ranges,
> + phys_addr_t offset)
> +{
> + struct kvm_exit_mmio dev_attr_mmio;
> +
> + dev_attr_mmio.len = 4;
> + if (find_matching_range(ranges, &dev_attr_mmio, offset))
> + return 0;
> + else
> + return -ENXIO;
> +}
> +
> static int vgic_has_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> {
> phys_addr_t offset;
> @@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
> return 0;
> }
> break;
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + return vgic_has_attr_regs(vgic_dist_ranges, offset);
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + return vgic_has_attr_regs(vgic_cpu_ranges, offset);
> }
> return -ENXIO;
> }
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
@ 2013-12-09 16:02 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:02 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Add infrastructure to handle distributor and cpu interface register
> accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding
> the
> KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> groups
> and defining the semantics of the attr field to be the MMIO offset as
> specified in the GICv2 specs.
>
> Set the cpu field (physical CPU index) on the vcpu struct to -1 in
> kvm_arch_vcpu_put so we have a method to check if the vcpu is running
> or
> not without having to grab the VCPU mutexes.
>
> Missing register accesses or other changes in individual register
> access
> functions to support save/restore of the VGIC state is added in
> subsequent patches.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[v3]:
> - Spelling and formatting nits
> - Fill in the phys_addr in kvm_exit_mmio
> - Put kvm_exit_mmio matching struct on stack
> - Change if-else-if to switch statement in vgic_attr_regs_access
> - Properly synchronize access to the VGIC with all VCPUs and the VM
> structure, ensuring no VCPUs are running while user space is
> accessing VGIC registers.
I have some doubts about this last point, see below.
> Changelog[v2]:
> - Added implementation specific format for the GICC_APRn registers.
> ---
> Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
> arch/arm/kvm/arm.c | 1 +
> virt/kvm/arm/vgic.c | 169
> ++++++++++++++++++++++++
> 3 files changed, 222 insertions(+)
>
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> b/Documentation/virtual/kvm/devices/arm-vgic.txt
> index c9febb2..7f4e91b 100644
> --- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> +++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> @@ -19,3 +19,55 @@ Groups:
> KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> Base address in the guest physical address space of the GIC
> virtual cpu
> interface register mappings.
> +
> + KVM_DEV_ARM_VGIC_GRP_DIST_REGS
> + Attributes:
> + The attr field of kvm_device_attr encodes two values:
> + bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> + values: | reserved | cpu id | offset |
> +
> + All distributor regs are (rw, 32-bit)
> +
> + The offset is relative to the "Distributor base address" as
> defined in the
> + GICv2 specs. Getting or setting such a register has the same
> effect as
> + reading or writing the register on the actual hardware from the
> cpu
> + specified with cpu id field. Note that most distributor fields
> are not
> + banked, but return the same value regardless of the cpu id used
> to access
> + the register.
> + Limitations:
> + - Priorities are not implemented, and registers are RAZ/WI
> + Errors:
> + -ENODEV: Getting or setting this register is not yet supported
> + -EBUSY: One or more VCPUs are running
> +
> + KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> + Attributes:
> + The attr field of kvm_device_attr encodes two values:
> + bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> + values: | reserved | cpu id | offset |
> +
> + All CPU interface regs are (rw, 32-bit)
> +
> + The offset specifies the offset from the "CPU interface base
> address" as
> + defined in the GICv2 specs. Getting or setting such a register
> has the
> + same effect as reading or writing the register on the actual
> hardware.
> +
> + The Active Priorities Registers APRn are implementation defined,
> so we set a
> + fixed format for our implementation that fits with the model of
> a "GICv2
> + implementation without the security extensions" which we present
> to the
> + guest. This interface always exposes four register APR[0-3]
> describing the
> + maximum possible 128 preemption levels. The semantics of the
> register
> + indicate if any interrupts in a given preemption level are in
> the active
> + state by setting the corresponding bit.
> +
> + Thus, preemption level X has one or more active interrupts if
> and only if:
> +
> + APRn[X mod 32] == 0b1, where n = X / 32
> +
> + Bits for undefined preemption levels are RAZ/WI.
> +
> + Limitations:
> + - Priorities are not implemented, and registers are RAZ/WI
> + Errors:
> + -ENODEV: Getting or setting this register is not yet supported
> + -EBUSY: One or more VCPUs are running
> diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> index a0bf0d8..35acac8 100644
> --- a/arch/arm/kvm/arm.c
> +++ b/arch/arm/kvm/arm.c
> @@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu,
> int cpu)
>
> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> {
> + vcpu->cpu = -1;
> kvm_arm_set_running_vcpu(NULL);
> }
This guy deserves to be in a separate patch, with a proper comment and
maybe a #define for the -1 value. Something like "KVM_VCPU_NOT_RUNNING"?
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 9b9fa20..ecf6dcf 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu
> *vcpu,
> return false;
> }
>
> +static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + return false;
> +}
> +
> +static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + return false;
> +}
> +
> /*
> * I would have liked to use the kvm_bus_io_*() API instead, but it
> * cannot cope with banked registers (only the VM pointer is passed
> @@ -663,6 +677,16 @@ static const struct mmio_range
> vgic_dist_ranges[] = {
> .len = 4,
> .handle_mmio = handle_mmio_sgi_reg,
> },
> + {
> + .base = GIC_DIST_SGI_CLEAR,
> + .len = VGIC_NR_SGIS,
> + .handle_mmio = handle_mmio_sgi_clear,
> + },
> + {
> + .base = GIC_DIST_SGI_SET,
> + .len = VGIC_NR_SGIS,
> + .handle_mmio = handle_mmio_sgi_set,
> + },
> {}
> };
>
> @@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> long type, u64 *addr, bool write)
> return r;
> }
>
> +static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio, phys_addr_t offset)
> +{
> + return true;
> +}
> +
> +static const struct mmio_range vgic_cpu_ranges[] = {
> + {
> + .base = GIC_CPU_CTRL,
> + .len = 12,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_ALIAS_BINPOINT,
> + .len = 4,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_ACTIVEPRIO,
> + .len = 16,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> + {
> + .base = GIC_CPU_IDENT,
> + .len = 4,
> + .handle_mmio = handle_cpu_mmio_misc,
> + },
> +};
> +
> +static int vgic_attr_regs_access(struct kvm_device *dev,
> + struct kvm_device_attr *attr,
> + u32 *reg, bool is_write)
> +{
> + const struct mmio_range *r = NULL, *ranges;
> + phys_addr_t offset;
> + int ret, cpuid, c;
> + struct kvm_vcpu *vcpu, *tmp_vcpu;
> + struct vgic_dist *vgic;
> + struct kvm_exit_mmio mmio;
> +
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
> + KVM_DEV_ARM_VGIC_CPUID_SHIFT;
> +
> + mutex_lock(&dev->kvm->lock);
> +
> + if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + vcpu = kvm_get_vcpu(dev->kvm, cpuid);
> + vgic = &dev->kvm->arch.vgic;
> +
> + mmio.len = 4;
> + mmio.is_write = is_write;
> + if (is_write)
> + mmio_data_write(&mmio, ~0, *reg);
> + switch (attr->group) {
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + mmio.phys_addr = vgic->vgic_dist_base + offset;
> + ranges = vgic_dist_ranges;
> + break;
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> + mmio.phys_addr = vgic->vgic_cpu_base + offset;
> + ranges = vgic_cpu_ranges;
> + break;
> + default:
> + BUG();
> + }
> + r = find_matching_range(ranges, &mmio, offset);
> +
> + if (unlikely(!r || !r->handle_mmio)) {
> + ret = -ENXIO;
> + goto out;
> + }
> +
> +
> + spin_lock(&vgic->lock);
> +
> + kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
> + if (unlikely(tmp_vcpu->cpu != -1)) {
What guarantees that the vcpu is not going to be restarted behind your
back? I can't see anything that locks vcpu->mutex.
> + spin_unlock(&vgic->lock);
> + ret = -EBUSY;
> + goto out;
> + }
> + }
> +
> + offset -= r->base;
> + r->handle_mmio(vcpu, &mmio, offset);
> + spin_unlock(&vgic->lock);
> +
> + if (!is_write)
> + *reg = mmio_data_read(&mmio, ~0);
> +
> + ret = 0;
> +out:
How about moving the "spin_unlock(&vgic->lock);" here, and simplify the
exit case of the loop above?
> + mutex_unlock(&dev->kvm->lock);
> + return ret;
> +}
> +
> static int vgic_set_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> {
> int r;
> @@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
> r = kvm_vgic_addr(dev->kvm, type, &addr, true);
> return (r == -ENODEV) ? -ENXIO : r;
> }
> +
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 reg;
> +
> + if (get_user(reg, uaddr))
> + return -EFAULT;
> +
> + return vgic_attr_regs_access(dev, attr, ®, true);
> + }
> +
> }
>
> return -ENXIO;
> @@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
>
> if (copy_to_user(uaddr, &addr, sizeof(addr)))
> return -EFAULT;
> + break;
> + }
> +
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> + u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> + u32 reg = 0;
> +
> + r = vgic_attr_regs_access(dev, attr, ®, false);
> + if (r)
> + return r;
> + r = put_user(reg, uaddr);
> + break;
> }
> +
> }
>
> return r;
> }
>
> +static int vgic_has_attr_regs(const struct mmio_range *ranges,
> + phys_addr_t offset)
> +{
> + struct kvm_exit_mmio dev_attr_mmio;
> +
> + dev_attr_mmio.len = 4;
> + if (find_matching_range(ranges, &dev_attr_mmio, offset))
> + return 0;
> + else
> + return -ENXIO;
> +}
> +
> static int vgic_has_attr(struct kvm_device *dev, struct
> kvm_device_attr *attr)
> {
> phys_addr_t offset;
> @@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device
> *dev, struct kvm_device_attr *attr)
> return 0;
> }
> break;
> + case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + return vgic_has_attr_regs(vgic_dist_ranges, offset);
> + case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> + offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> + return vgic_has_attr_regs(vgic_cpu_ranges, offset);
> }
> return -ENXIO;
> }
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 16:38 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:38 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> To properly access the VGIC state from user space it is very
> unpractical
> to have to loop through all the LRs in all register access functions.
> Instead, support moving all pending state from LRs to the
> distributor,
> but leave active state LRs alone.
>
> Note that to accurately present the active and pending state to VCPUs
> reading these distributor registers from a live VM, we would have to
> stop all other VPUs than the calling VCPU and ask each CPU to unqueue
> their LR state onto the distributor and add fields to track active
> state
> on the distributor side as well. We don't have any users of such
> functionality yet and there are other inaccuracies of the GIC
> emulation,
> so don't provide accurate synchronized access to this state just yet.
> However, when the time comes, having this function should help.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[3]:
> - New patch in series
> ---
> virt/kvm/arm/vgic.c | 80
> +++++++++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 75 insertions(+), 5 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index ecf6dcf..44c669b 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu
> *vcpu,
> return false;
> }
>
> +#define LR_CPUID(lr) \
> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> +#define LR_IRQID(lr) \
> + ((lr) & GICH_LR_VIRTUALID)
> +
> +static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu
> *vgic_cpu)
> +{
> + clear_bit(lr_nr, vgic_cpu->lr_used);
> + vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
> + vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> +}
> +
> +/**
> + * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
> + * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
> + *
> + * Move any pending IRQs that have already been assigned to LRs back
> to the
> + * emulated distributor state so that the complete emulated state
> can be read
> + * from the main emulation structures without investigating the LRs.
> + *
> + * Note that IRQs in the active state in the LRs get their pending
> state moved
> + * to the distributor but the active state stays in the LRs, because
> we don't
> + * track the active state on the distributor side.
> + */
> +static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> + int vcpu_id = vcpu->vcpu_id;
> + int i, irq, source_cpu;
> + u32 *lr;
> +
> + for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
> + lr = &vgic_cpu->vgic_lr[i];
> + irq = LR_IRQID(*lr);
> + source_cpu = LR_CPUID(*lr);
> +
> + /*
> + * If the LR holds only an active interrupt (not pending) then
> + * just leave it alone.
> + */
> + if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
> + (unsigned long *)lr))
> + continue;
Now you got me confused. The comment talks about the active bit, but
you're actually clearing the pending bit. Surely that deserves a better
explanation.
> +
> + /*
> + * If the interrupt was only pending (not "active" or "pending
> + * and active") then we have effectively cleared the LR and it
> + * should be marked as free for other use.
> + */
> + if (!(*lr & GICH_LR_STATE))
> + vgic_retire_lr(i, irq, vgic_cpu);
Why not directly testing the active bit? It'd be more consistent with
the above if you used a similar method.
> + /*
> + * Finally, reestablish the pending state on the distributor
> + * and the CPU interface. It may have already been pending,
> + * but that is fine, then we are only setting a few bits that
> + * were already set.
> + */
> + vgic_dist_irq_set(vcpu, irq);
> + if (irq < VGIC_NR_SGIS)
> + dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
> + vgic_update_state(vcpu->kvm);
> + }
> +}
> +
> static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio,
> phys_addr_t offset)
> @@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
> }
> }
>
> -#define LR_CPUID(lr) \
> - (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> #define MK_LR_PEND(src, irq) \
> (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) |
> (irq))
>
> @@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct
> kvm_vcpu *vcpu)
> int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
>
> if (!vgic_irq_is_enabled(vcpu, irq)) {
> - vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> - clear_bit(lr, vgic_cpu->lr_used);
> - vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
> + vgic_retire_lr(lr, irq, vgic_cpu);
> if (vgic_irq_is_active(vcpu, irq))
> vgic_irq_clear_active(vcpu, irq);
> }
> @@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct
> kvm_device *dev,
> }
> }
>
> + /*
> + * Move all pending IRQs from the LRs on all VCPUs so the pending
> + * state can be properly represented in the register state
> accessible
> + * through this API.
> + */
> + kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
> + vgic_unqueue_irqs(tmp_vcpu);
> +
> offset -= r->base;
> r->handle_mmio(vcpu, &mmio, offset);
> spin_unlock(&vgic->lock);
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
@ 2013-12-09 16:38 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:38 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> To properly access the VGIC state from user space it is very
> unpractical
> to have to loop through all the LRs in all register access functions.
> Instead, support moving all pending state from LRs to the
> distributor,
> but leave active state LRs alone.
>
> Note that to accurately present the active and pending state to VCPUs
> reading these distributor registers from a live VM, we would have to
> stop all other VPUs than the calling VCPU and ask each CPU to unqueue
> their LR state onto the distributor and add fields to track active
> state
> on the distributor side as well. We don't have any users of such
> functionality yet and there are other inaccuracies of the GIC
> emulation,
> so don't provide accurate synchronized access to this state just yet.
> However, when the time comes, having this function should help.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Changelog[3]:
> - New patch in series
> ---
> virt/kvm/arm/vgic.c | 80
> +++++++++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 75 insertions(+), 5 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index ecf6dcf..44c669b 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct kvm_vcpu
> *vcpu,
> return false;
> }
>
> +#define LR_CPUID(lr) \
> + (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> +#define LR_IRQID(lr) \
> + ((lr) & GICH_LR_VIRTUALID)
> +
> +static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu
> *vgic_cpu)
> +{
> + clear_bit(lr_nr, vgic_cpu->lr_used);
> + vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
> + vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> +}
> +
> +/**
> + * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
> + * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
> + *
> + * Move any pending IRQs that have already been assigned to LRs back
> to the
> + * emulated distributor state so that the complete emulated state
> can be read
> + * from the main emulation structures without investigating the LRs.
> + *
> + * Note that IRQs in the active state in the LRs get their pending
> state moved
> + * to the distributor but the active state stays in the LRs, because
> we don't
> + * track the active state on the distributor side.
> + */
> +static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> + int vcpu_id = vcpu->vcpu_id;
> + int i, irq, source_cpu;
> + u32 *lr;
> +
> + for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
> + lr = &vgic_cpu->vgic_lr[i];
> + irq = LR_IRQID(*lr);
> + source_cpu = LR_CPUID(*lr);
> +
> + /*
> + * If the LR holds only an active interrupt (not pending) then
> + * just leave it alone.
> + */
> + if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
> + (unsigned long *)lr))
> + continue;
Now you got me confused. The comment talks about the active bit, but
you're actually clearing the pending bit. Surely that deserves a better
explanation.
> +
> + /*
> + * If the interrupt was only pending (not "active" or "pending
> + * and active") then we have effectively cleared the LR and it
> + * should be marked as free for other use.
> + */
> + if (!(*lr & GICH_LR_STATE))
> + vgic_retire_lr(i, irq, vgic_cpu);
Why not directly testing the active bit? It'd be more consistent with
the above if you used a similar method.
> + /*
> + * Finally, reestablish the pending state on the distributor
> + * and the CPU interface. It may have already been pending,
> + * but that is fine, then we are only setting a few bits that
> + * were already set.
> + */
> + vgic_dist_irq_set(vcpu, irq);
> + if (irq < VGIC_NR_SGIS)
> + dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
> + vgic_update_state(vcpu->kvm);
> + }
> +}
> +
> static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio,
> phys_addr_t offset)
> @@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
> }
> }
>
> -#define LR_CPUID(lr) \
> - (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> #define MK_LR_PEND(src, irq) \
> (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) |
> (irq))
>
> @@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct
> kvm_vcpu *vcpu)
> int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
>
> if (!vgic_irq_is_enabled(vcpu, irq)) {
> - vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> - clear_bit(lr, vgic_cpu->lr_used);
> - vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
> + vgic_retire_lr(lr, irq, vgic_cpu);
> if (vgic_irq_is_active(vcpu, irq))
> vgic_irq_clear_active(vcpu, irq);
> }
> @@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct
> kvm_device *dev,
> }
> }
>
> + /*
> + * Move all pending IRQs from the LRs on all VCPUs so the pending
> + * state can be properly represented in the register state
> accessible
> + * through this API.
> + */
> + kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
> + vgic_unqueue_irqs(tmp_vcpu);
> +
> offset -= r->base;
> r->handle_mmio(vcpu, &mmio, offset);
> spin_unlock(&vgic->lock);
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 16:49 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:49 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Handle MMIO accesses to the two registers which should support both
> the
> case where the VMs want to read/write either of these registers and
> the
> case where user space reads/writes these registers to do save/restore
> of
> the VGIC state.
>
> Note that the added complexity compared to simple set/clear enable
> registers stems from the bookkeping of source cpu ids. It may be
> possible to change the underlying data structure to simplify the
> complexity, but since this is not in the critical path at all, this
> will
> do.
>
> Also note that reading this register from a live guest will not be
> accurate compared to on hardware, because some state may be living on
> the CPU LRs and the only way to give a consistent read would be to
> force
> stop all the VCPUs and request them to unqueu the LR state onto the
> distributor. Until we have an actual user of live reading this
> register, we can live with the difference.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Looks pretty good to me. Small note below, but otherwise:
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>
> Changelog[v3]:
> - Renamed read/write SGI set/clear functions
> - Rely on unqueuing of interrupts from LRs instead of reading LRs
> directly
> - Deduplicate code
>
> Changelog[v2]:
> - Use struct kvm_exit_mmio accessors for ->data field.
> ---
> virt/kvm/arm/vgic.c | 70
> ++++++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 66 insertions(+), 4 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 44c669b..16053eb 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct kvm_vcpu
> *vcpu)
> }
> }
>
> -static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> - struct kvm_exit_mmio *mmio,
> - phys_addr_t offset)
> +/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
> +static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> {
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + int sgi;
> + int min_sgi = (offset & ~0x3) * 4;
> + int max_sgi = min_sgi + 3;
> + int vcpu_id = vcpu->vcpu_id;
> + u32 reg = 0;
> +
> + /* Copy source SGIs from distributor side */
> + for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> + int shift = 8 * (sgi - min_sgi);
> + reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
> + }
> +
> + mmio_data_write(mmio, ~0, reg);
> return false;
> }
>
> +static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset, bool set)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + int sgi;
> + int min_sgi = (offset & ~0x3) * 4;
> + int max_sgi = min_sgi + 3;
> + int vcpu_id = vcpu->vcpu_id;
> + u32 reg;
> + bool updated = false;
> +
> + reg = mmio_data_read(mmio, ~0);
> +
> + /* Clear pending SGIs on the distributor */
> + for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> + u8 mask = reg >> (8 * (sgi - min_sgi));
> + if (set) {
> + if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
> + updated = true;
> + dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
> + } else {
> + if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
> + updated = true;
> + dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
> + }
> + }
> +
> + if (updated)
> + vgic_update_state(vcpu->kvm);
So I realize we have that construct everywhere. Surely it'd be worth it
moving the mmio calls to vgic_update_state into vgic_handle_mmio. Or
have I missed something?
> + return updated;
> +}
> +
> static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio,
> phys_addr_t offset)
> {
> - return false;
> + if (!mmio->is_write)
> + return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> + else
> + return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
> +}
> +
> +static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + if (!mmio->is_write)
> + return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> + else
> + return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
> }
>
> /*
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
@ 2013-12-09 16:49 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 16:49 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Handle MMIO accesses to the two registers which should support both
> the
> case where the VMs want to read/write either of these registers and
> the
> case where user space reads/writes these registers to do save/restore
> of
> the VGIC state.
>
> Note that the added complexity compared to simple set/clear enable
> registers stems from the bookkeping of source cpu ids. It may be
> possible to change the underlying data structure to simplify the
> complexity, but since this is not in the critical path at all, this
> will
> do.
>
> Also note that reading this register from a live guest will not be
> accurate compared to on hardware, because some state may be living on
> the CPU LRs and the only way to give a consistent read would be to
> force
> stop all the VCPUs and request them to unqueu the LR state onto the
> distributor. Until we have an actual user of live reading this
> register, we can live with the difference.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Looks pretty good to me. Small note below, but otherwise:
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>
> Changelog[v3]:
> - Renamed read/write SGI set/clear functions
> - Rely on unqueuing of interrupts from LRs instead of reading LRs
> directly
> - Deduplicate code
>
> Changelog[v2]:
> - Use struct kvm_exit_mmio accessors for ->data field.
> ---
> virt/kvm/arm/vgic.c | 70
> ++++++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 66 insertions(+), 4 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 44c669b..16053eb 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct kvm_vcpu
> *vcpu)
> }
> }
>
> -static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> - struct kvm_exit_mmio *mmio,
> - phys_addr_t offset)
> +/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
> +static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> {
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + int sgi;
> + int min_sgi = (offset & ~0x3) * 4;
> + int max_sgi = min_sgi + 3;
> + int vcpu_id = vcpu->vcpu_id;
> + u32 reg = 0;
> +
> + /* Copy source SGIs from distributor side */
> + for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> + int shift = 8 * (sgi - min_sgi);
> + reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
> + }
> +
> + mmio_data_write(mmio, ~0, reg);
> return false;
> }
>
> +static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset, bool set)
> +{
> + struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> + int sgi;
> + int min_sgi = (offset & ~0x3) * 4;
> + int max_sgi = min_sgi + 3;
> + int vcpu_id = vcpu->vcpu_id;
> + u32 reg;
> + bool updated = false;
> +
> + reg = mmio_data_read(mmio, ~0);
> +
> + /* Clear pending SGIs on the distributor */
> + for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> + u8 mask = reg >> (8 * (sgi - min_sgi));
> + if (set) {
> + if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
> + updated = true;
> + dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
> + } else {
> + if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
> + updated = true;
> + dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
> + }
> + }
> +
> + if (updated)
> + vgic_update_state(vcpu->kvm);
So I realize we have that construct everywhere. Surely it'd be worth it
moving the mmio calls to vgic_update_state into vgic_handle_mmio. Or
have I missed something?
> + return updated;
> +}
> +
> static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio,
> phys_addr_t offset)
> {
> - return false;
> + if (!mmio->is_write)
> + return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> + else
> + return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
> +}
> +
> +static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + if (!mmio->is_write)
> + return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> + else
> + return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
> }
>
> /*
--
Fast, cheap, reliable. Pick two.
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 9/9] KVM: arm-vgic: Support CPU interface reg access
2013-11-17 4:30 ` Christoffer Dall
@ 2013-12-09 17:13 ` Marc Zyngier
-1 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 17:13 UTC (permalink / raw)
To: Christoffer Dall; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On 2013-11-17 04:30, Christoffer Dall wrote:
> Implement support for the CPU interface register access driven by
> MMIO
> address offsets from the CPU interface base address. Useful for user
> space to support save/restore of the VGIC state.
>
> This commit adds support only for the same logic as the current VGIC
> support, and no more. For example, the active priority registers are
> handled as RAZ/WI, just like setting priorities on the emulated
> distributor.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> virt/kvm/arm/vgic.c | 81
> ++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 73 insertions(+), 8 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 16053eb..a08b89a 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -71,6 +71,10 @@
> #define VGIC_ADDR_UNDEF (-1)
> #define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
>
> +#define PRODUCT_ID_KVM 0x4b /* ASCII code K */
> +#define IMPLEMENTER_ARM 0x43b
> +#define GICC_ARCH_VERSION_V2 0x2
> +
> /* Physical address of vgic virtual cpu interface */
> static phys_addr_t vgic_vcpu_base;
>
> @@ -312,7 +316,7 @@ static bool handle_mmio_misc(struct kvm_vcpu
> *vcpu,
> u32 word_offset = offset & 3;
>
> switch (offset & ~3) {
> - case 0: /* CTLR */
> + case 0: /* GICD_CTLR */
> reg = vcpu->kvm->arch.vgic.enabled;
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
> @@ -323,15 +327,15 @@ static bool handle_mmio_misc(struct kvm_vcpu
> *vcpu,
> }
> break;
>
> - case 4: /* TYPER */
> + case 4: /* GICD_TYPER */
> reg = (atomic_read(&vcpu->kvm->online_vcpus) - 1) << 5;
> reg |= (VGIC_NR_IRQS >> 5) - 1;
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
> break;
>
> - case 8: /* IIDR */
> - reg = 0x4B00043B;
> + case 8: /* GICD_IIDR */
> + reg = (PRODUCT_ID_KVM << 24) | (IMPLEMENTER_ARM << 0);
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
> break;
> @@ -1703,9 +1707,70 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> long type, u64 *addr, bool write)
> static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio, phys_addr_t offset)
> {
> - return true;
> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> + u32 reg, mask = 0, shift = 0;
> + bool updated = false;
> +
> + switch (offset & ~0x3) {
> + case GIC_CPU_CTRL:
> + mask = GICH_VMCR_CTRL_MASK;
> + shift = GICH_VMCR_CTRL_SHIFT;
> + break;
> + case GIC_CPU_PRIMASK:
> + mask = GICH_VMCR_PRIMASK_MASK;
> + shift = GICH_VMCR_PRIMASK_SHIFT;
> + break;
> + case GIC_CPU_BINPOINT:
> + mask = GICH_VMCR_BINPOINT_MASK;
> + shift = GICH_VMCR_BINPOINT_SHIFT;
> + break;
> + case GIC_CPU_ALIAS_BINPOINT:
> + mask = GICH_VMCR_ALIAS_BINPOINT_MASK;
> + shift = GICH_VMCR_ALIAS_BINPOINT_SHIFT;
> + break;
> + }
> +
> + if (!mmio->is_write) {
> + reg = (vgic_cpu->vgic_vmcr & mask) >> shift;
> + mmio_data_write(mmio, ~0, reg);
> + } else {
> + reg = mmio_data_read(mmio, ~0);
> + reg = (reg << shift) & mask;
> + if (reg != (vgic_cpu->vgic_vmcr & mask))
> + updated = true;
> + vgic_cpu->vgic_vmcr &= ~mask;
> + vgic_cpu->vgic_vmcr |= reg;
> + }
> + return updated;
> +}
> +
> +static bool handle_mmio_abpr(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio, phys_addr_t offset)
> +{
> + return handle_cpu_mmio_misc(vcpu, mmio, GIC_CPU_ALIAS_BINPOINT);
> }
>
> +static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + u32 reg;
> +
> + if (mmio->is_write)
> + return false;
> +
> + /* GICC_IIDR */
> + reg = (PRODUCT_ID_KVM << 20) |
> + (GICC_ARCH_VERSION_V2 << 16) |
> + (IMPLEMENTER_ARM << 0);
> + mmio_data_write(mmio, ~0, reg);
> + return false;
> +}
> +
> +/*
> + * CPU Interface Register accesses - these are not accessed by the
> VM, but by
> + * user space for saving and restoring VGIC state.
> + */
> static const struct mmio_range vgic_cpu_ranges[] = {
> {
> .base = GIC_CPU_CTRL,
> @@ -1715,17 +1780,17 @@ static const struct mmio_range
> vgic_cpu_ranges[] = {
> {
> .base = GIC_CPU_ALIAS_BINPOINT,
> .len = 4,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_mmio_abpr,
> },
> {
> .base = GIC_CPU_ACTIVEPRIO,
> .len = 16,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_mmio_raz_wi,
> },
> {
> .base = GIC_CPU_IDENT,
> .len = 4,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_cpu_mmio_ident,
> },
> };
--
Who you jivin' with that Cosmik Debris?
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 9/9] KVM: arm-vgic: Support CPU interface reg access
@ 2013-12-09 17:13 ` Marc Zyngier
0 siblings, 0 replies; 48+ messages in thread
From: Marc Zyngier @ 2013-12-09 17:13 UTC (permalink / raw)
To: linux-arm-kernel
On 2013-11-17 04:30, Christoffer Dall wrote:
> Implement support for the CPU interface register access driven by
> MMIO
> address offsets from the CPU interface base address. Useful for user
> space to support save/restore of the VGIC state.
>
> This commit adds support only for the same logic as the current VGIC
> support, and no more. For example, the active priority registers are
> handled as RAZ/WI, just like setting priorities on the emulated
> distributor.
>
> Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> virt/kvm/arm/vgic.c | 81
> ++++++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 73 insertions(+), 8 deletions(-)
>
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 16053eb..a08b89a 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -71,6 +71,10 @@
> #define VGIC_ADDR_UNDEF (-1)
> #define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
>
> +#define PRODUCT_ID_KVM 0x4b /* ASCII code K */
> +#define IMPLEMENTER_ARM 0x43b
> +#define GICC_ARCH_VERSION_V2 0x2
> +
> /* Physical address of vgic virtual cpu interface */
> static phys_addr_t vgic_vcpu_base;
>
> @@ -312,7 +316,7 @@ static bool handle_mmio_misc(struct kvm_vcpu
> *vcpu,
> u32 word_offset = offset & 3;
>
> switch (offset & ~3) {
> - case 0: /* CTLR */
> + case 0: /* GICD_CTLR */
> reg = vcpu->kvm->arch.vgic.enabled;
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_VALUE);
> @@ -323,15 +327,15 @@ static bool handle_mmio_misc(struct kvm_vcpu
> *vcpu,
> }
> break;
>
> - case 4: /* TYPER */
> + case 4: /* GICD_TYPER */
> reg = (atomic_read(&vcpu->kvm->online_vcpus) - 1) << 5;
> reg |= (VGIC_NR_IRQS >> 5) - 1;
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
> break;
>
> - case 8: /* IIDR */
> - reg = 0x4B00043B;
> + case 8: /* GICD_IIDR */
> + reg = (PRODUCT_ID_KVM << 24) | (IMPLEMENTER_ARM << 0);
> vgic_reg_access(mmio, ®, word_offset,
> ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
> break;
> @@ -1703,9 +1707,70 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> long type, u64 *addr, bool write)
> static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> struct kvm_exit_mmio *mmio, phys_addr_t offset)
> {
> - return true;
> + struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> + u32 reg, mask = 0, shift = 0;
> + bool updated = false;
> +
> + switch (offset & ~0x3) {
> + case GIC_CPU_CTRL:
> + mask = GICH_VMCR_CTRL_MASK;
> + shift = GICH_VMCR_CTRL_SHIFT;
> + break;
> + case GIC_CPU_PRIMASK:
> + mask = GICH_VMCR_PRIMASK_MASK;
> + shift = GICH_VMCR_PRIMASK_SHIFT;
> + break;
> + case GIC_CPU_BINPOINT:
> + mask = GICH_VMCR_BINPOINT_MASK;
> + shift = GICH_VMCR_BINPOINT_SHIFT;
> + break;
> + case GIC_CPU_ALIAS_BINPOINT:
> + mask = GICH_VMCR_ALIAS_BINPOINT_MASK;
> + shift = GICH_VMCR_ALIAS_BINPOINT_SHIFT;
> + break;
> + }
> +
> + if (!mmio->is_write) {
> + reg = (vgic_cpu->vgic_vmcr & mask) >> shift;
> + mmio_data_write(mmio, ~0, reg);
> + } else {
> + reg = mmio_data_read(mmio, ~0);
> + reg = (reg << shift) & mask;
> + if (reg != (vgic_cpu->vgic_vmcr & mask))
> + updated = true;
> + vgic_cpu->vgic_vmcr &= ~mask;
> + vgic_cpu->vgic_vmcr |= reg;
> + }
> + return updated;
> +}
> +
> +static bool handle_mmio_abpr(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio, phys_addr_t offset)
> +{
> + return handle_cpu_mmio_misc(vcpu, mmio, GIC_CPU_ALIAS_BINPOINT);
> }
>
> +static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
> + struct kvm_exit_mmio *mmio,
> + phys_addr_t offset)
> +{
> + u32 reg;
> +
> + if (mmio->is_write)
> + return false;
> +
> + /* GICC_IIDR */
> + reg = (PRODUCT_ID_KVM << 20) |
> + (GICC_ARCH_VERSION_V2 << 16) |
> + (IMPLEMENTER_ARM << 0);
> + mmio_data_write(mmio, ~0, reg);
> + return false;
> +}
> +
> +/*
> + * CPU Interface Register accesses - these are not accessed by the
> VM, but by
> + * user space for saving and restoring VGIC state.
> + */
> static const struct mmio_range vgic_cpu_ranges[] = {
> {
> .base = GIC_CPU_CTRL,
> @@ -1715,17 +1780,17 @@ static const struct mmio_range
> vgic_cpu_ranges[] = {
> {
> .base = GIC_CPU_ALIAS_BINPOINT,
> .len = 4,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_mmio_abpr,
> },
> {
> .base = GIC_CPU_ACTIVEPRIO,
> .len = 16,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_mmio_raz_wi,
> },
> {
> .base = GIC_CPU_IDENT,
> .len = 4,
> - .handle_mmio = handle_cpu_mmio_misc,
> + .handle_mmio = handle_cpu_mmio_ident,
> },
> };
--
Who you jivin' with that Cosmik Debris?
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
2013-12-09 15:19 ` Marc Zyngier
@ 2013-12-12 5:29 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On Mon, Dec 09, 2013 at 03:19:17PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
> >ioctl, which can then later be leveraged to use the
> >KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting
> >addresses in
> >a more generic API than the ARM-specific one and is useful for
> >save/restore of VGIC state.
> >
> >Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
> >
> >Note that we change the check for creating a VGIC from bailing out if
> >any VCPUs were created, to bailing out if any VCPUs were ever run.
> >This
> >is an important distinction that shouldn't break anything, but allows
> >creating the VGIC after the VCPUs have been created.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Prevent race in kvm_vgic_create by trying to take all the vcpu
> > locks before creating the vgic.
> >
> >Changelog[v2]:
> > - None
> >---
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
> > arch/arm/kvm/arm.c | 1 +
> > include/linux/kvm_host.h | 1 +
> > include/uapi/linux/kvm.h | 1 +
> > virt/kvm/arm/vgic.c | 58
> >+++++++++++++++++++++++-
> > virt/kvm/kvm_main.c | 6 ++-
> > 6 files changed, 74 insertions(+), 3 deletions(-)
> > create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
> >
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >new file mode 100644
> >index 0000000..38f27f7
> >--- /dev/null
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -0,0 +1,10 @@
> >+ARM Virtual Generic Interrupt Controller (VGIC)
> >+===============================================
> >+
> >+Device types supported:
> >+ KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
> >+
> >+Only one VGIC instance may be instantiated through either this
> >API or the
> >+legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> >interrupt
> >+controller, requiring emulated user-space devices to inject
> >interrupts to the
> >+VGIC instead of directly to CPUs.
> >diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> >index e80b6a1..984908d 100644
> >--- a/arch/arm/kvm/arm.c
> >+++ b/arch/arm/kvm/arm.c
> >@@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> > case KVM_CAP_IRQCHIP:
> > r = vgic_present;
> > break;
> >+ case KVM_CAP_DEVICE_CTRL:
> > case KVM_CAP_USER_MEMORY:
> > case KVM_CAP_SYNC_MMU:
> > case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
> >diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> >index 9523d2a..e68d9db 100644
> >--- a/include/linux/kvm_host.h
> >+++ b/include/linux/kvm_host.h
> >@@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct
> >file *filp);
> > extern struct kvm_device_ops kvm_mpic_ops;
> > extern struct kvm_device_ops kvm_xics_ops;
> > extern struct kvm_device_ops kvm_vfio_ops;
> >+extern struct kvm_device_ops kvm_arm_vgic_ops;
>
> Can we rename this to kvm_arm_vgic_v2_ops in order to be consistent
> with KVM_DEV_TYPE_ARM_VGIC_V2?
>
yes, we most certainly can.
> >
> > #ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
> >
> >diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> >index 902f124..b647c29 100644
> >--- a/include/uapi/linux/kvm.h
> >+++ b/include/uapi/linux/kvm.h
> >@@ -853,6 +853,7 @@ struct kvm_device_attr {
> > #define KVM_DEV_VFIO_GROUP 1
> > #define KVM_DEV_VFIO_GROUP_ADD 1
> > #define KVM_DEV_VFIO_GROUP_DEL 2
> >+#define KVM_DEV_TYPE_ARM_VGIC_V2 5
> >
> > /*
> > * ioctls for VM fds
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 5e9df47..e3fbb5e 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -1433,20 +1433,40 @@ out:
> >
> > int kvm_vgic_create(struct kvm *kvm)
> > {
> >- int ret = 0;
> >+ int i, vcpu_lock_idx = -1, ret = 0;
> >+ struct kvm_vcpu *vcpu;
> >
> > mutex_lock(&kvm->lock);
> >
> >- if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
> >+ if (kvm->arch.vgic.vctrl_base) {
> > ret = -EEXIST;
> > goto out;
> > }
> >
> >+ kvm_for_each_vcpu(i, vcpu, kvm) {
> >+ if (!mutex_trylock(&vcpu->mutex))
> >+ goto out_unlock;
> >+ vcpu_lock_idx = i;
> >+ }
>
> I think this deserves a comment explaining that vcpu->mutex is taken
> by vcpu_load...
>
I agree, I've added that.
> >+ kvm_for_each_vcpu(i, vcpu, kvm) {
> >+ if (vcpu->arch.has_run_once) {
> >+ ret = -EBUSY;
> >+ goto out_unlock;
> >+ }
> >+ }
> >+
> > spin_lock_init(&kvm->arch.vgic.lock);
> > kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
> > kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
> > kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
> >
> >+out_unlock:
> >+ for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
> >+ vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
> >+ mutex_unlock(&vcpu->mutex);
> >+ }
> >+
> > out:
> > mutex_unlock(&kvm->lock);
> > return ret;
> >@@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm,
> >unsigned long type, u64 addr)
> > mutex_unlock(&kvm->lock);
> > return r;
> > }
> >+
> >+static int vgic_set_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static int vgic_get_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static int vgic_has_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static void vgic_destroy(struct kvm_device *dev)
> >+{
> >+ kfree(dev);
> >+}
> >+
> >+static int vgic_create(struct kvm_device *dev, u32 type)
> >+{
> >+ return kvm_vgic_create(dev->kvm);
> >+}
> >+
> >+struct kvm_device_ops kvm_arm_vgic_ops = {
> >+ .name = "kvm-arm-vgic",
> >+ .create = vgic_create,
> >+ .destroy = vgic_destroy,
> >+ .set_attr = vgic_set_attr,
> >+ .get_attr = vgic_get_attr,
> >+ .has_attr = vgic_has_attr,
> >+};
> >diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> >index 10015d6..fb8bb21 100644
> >--- a/virt/kvm/kvm_main.c
> >+++ b/virt/kvm/kvm_main.c
> >@@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct
> >kvm *kvm,
> > #ifdef CONFIG_KVM_VFIO
> > case KVM_DEV_TYPE_VFIO:
> > ops = &kvm_vfio_ops;
> >- break;
> >+#endif
> >+#ifdef CONFIG_KVM_ARM_VGIC
> >+ case KVM_DEV_TYPE_ARM_VGIC_V2:
> >+ ops = &kvm_arm_vgic_ops;
> >+ break;
> > #endif
> > default:
> > return -ENODEV;
>
> Otherwise, looks good to me.
>
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC
@ 2013-12-12 5:29 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Dec 09, 2013 at 03:19:17PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Support creating the ARM VGIC device through the KVM_CREATE_DEVICE
> >ioctl, which can then later be leveraged to use the
> >KVM_{GET/SET}_DEVICE_ATTR, which is useful both for setting
> >addresses in
> >a more generic API than the ARM-specific one and is useful for
> >save/restore of VGIC state.
> >
> >Adds KVM_CAP_DEVICE_CTRL to ARM capabilities.
> >
> >Note that we change the check for creating a VGIC from bailing out if
> >any VCPUs were created, to bailing out if any VCPUs were ever run.
> >This
> >is an important distinction that shouldn't break anything, but allows
> >creating the VGIC after the VCPUs have been created.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Prevent race in kvm_vgic_create by trying to take all the vcpu
> > locks before creating the vgic.
> >
> >Changelog[v2]:
> > - None
> >---
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 10 ++++
> > arch/arm/kvm/arm.c | 1 +
> > include/linux/kvm_host.h | 1 +
> > include/uapi/linux/kvm.h | 1 +
> > virt/kvm/arm/vgic.c | 58
> >+++++++++++++++++++++++-
> > virt/kvm/kvm_main.c | 6 ++-
> > 6 files changed, 74 insertions(+), 3 deletions(-)
> > create mode 100644 Documentation/virtual/kvm/devices/arm-vgic.txt
> >
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >new file mode 100644
> >index 0000000..38f27f7
> >--- /dev/null
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -0,0 +1,10 @@
> >+ARM Virtual Generic Interrupt Controller (VGIC)
> >+===============================================
> >+
> >+Device types supported:
> >+ KVM_DEV_TYPE_ARM_VGIC_V2 ARM Generic Interrupt Controller v2.0
> >+
> >+Only one VGIC instance may be instantiated through either this
> >API or the
> >+legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> >interrupt
> >+controller, requiring emulated user-space devices to inject
> >interrupts to the
> >+VGIC instead of directly to CPUs.
> >diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> >index e80b6a1..984908d 100644
> >--- a/arch/arm/kvm/arm.c
> >+++ b/arch/arm/kvm/arm.c
> >@@ -190,6 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
> > case KVM_CAP_IRQCHIP:
> > r = vgic_present;
> > break;
> >+ case KVM_CAP_DEVICE_CTRL:
> > case KVM_CAP_USER_MEMORY:
> > case KVM_CAP_SYNC_MMU:
> > case KVM_CAP_DESTROY_MEMORY_REGION_WORKS:
> >diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
> >index 9523d2a..e68d9db 100644
> >--- a/include/linux/kvm_host.h
> >+++ b/include/linux/kvm_host.h
> >@@ -1076,6 +1076,7 @@ struct kvm_device *kvm_device_from_filp(struct
> >file *filp);
> > extern struct kvm_device_ops kvm_mpic_ops;
> > extern struct kvm_device_ops kvm_xics_ops;
> > extern struct kvm_device_ops kvm_vfio_ops;
> >+extern struct kvm_device_ops kvm_arm_vgic_ops;
>
> Can we rename this to kvm_arm_vgic_v2_ops in order to be consistent
> with KVM_DEV_TYPE_ARM_VGIC_V2?
>
yes, we most certainly can.
> >
> > #ifdef CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT
> >
> >diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> >index 902f124..b647c29 100644
> >--- a/include/uapi/linux/kvm.h
> >+++ b/include/uapi/linux/kvm.h
> >@@ -853,6 +853,7 @@ struct kvm_device_attr {
> > #define KVM_DEV_VFIO_GROUP 1
> > #define KVM_DEV_VFIO_GROUP_ADD 1
> > #define KVM_DEV_VFIO_GROUP_DEL 2
> >+#define KVM_DEV_TYPE_ARM_VGIC_V2 5
> >
> > /*
> > * ioctls for VM fds
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 5e9df47..e3fbb5e 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -1433,20 +1433,40 @@ out:
> >
> > int kvm_vgic_create(struct kvm *kvm)
> > {
> >- int ret = 0;
> >+ int i, vcpu_lock_idx = -1, ret = 0;
> >+ struct kvm_vcpu *vcpu;
> >
> > mutex_lock(&kvm->lock);
> >
> >- if (atomic_read(&kvm->online_vcpus) || kvm->arch.vgic.vctrl_base) {
> >+ if (kvm->arch.vgic.vctrl_base) {
> > ret = -EEXIST;
> > goto out;
> > }
> >
> >+ kvm_for_each_vcpu(i, vcpu, kvm) {
> >+ if (!mutex_trylock(&vcpu->mutex))
> >+ goto out_unlock;
> >+ vcpu_lock_idx = i;
> >+ }
>
> I think this deserves a comment explaining that vcpu->mutex is taken
> by vcpu_load...
>
I agree, I've added that.
> >+ kvm_for_each_vcpu(i, vcpu, kvm) {
> >+ if (vcpu->arch.has_run_once) {
> >+ ret = -EBUSY;
> >+ goto out_unlock;
> >+ }
> >+ }
> >+
> > spin_lock_init(&kvm->arch.vgic.lock);
> > kvm->arch.vgic.vctrl_base = vgic_vctrl_base;
> > kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF;
> > kvm->arch.vgic.vgic_cpu_base = VGIC_ADDR_UNDEF;
> >
> >+out_unlock:
> >+ for (; vcpu_lock_idx >= 0; vcpu_lock_idx--) {
> >+ vcpu = kvm_get_vcpu(kvm, vcpu_lock_idx);
> >+ mutex_unlock(&vcpu->mutex);
> >+ }
> >+
> > out:
> > mutex_unlock(&kvm->lock);
> > return ret;
> >@@ -1510,3 +1530,37 @@ int kvm_vgic_set_addr(struct kvm *kvm,
> >unsigned long type, u64 addr)
> > mutex_unlock(&kvm->lock);
> > return r;
> > }
> >+
> >+static int vgic_set_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static int vgic_get_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static int vgic_has_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> >+{
> >+ return -ENXIO;
> >+}
> >+
> >+static void vgic_destroy(struct kvm_device *dev)
> >+{
> >+ kfree(dev);
> >+}
> >+
> >+static int vgic_create(struct kvm_device *dev, u32 type)
> >+{
> >+ return kvm_vgic_create(dev->kvm);
> >+}
> >+
> >+struct kvm_device_ops kvm_arm_vgic_ops = {
> >+ .name = "kvm-arm-vgic",
> >+ .create = vgic_create,
> >+ .destroy = vgic_destroy,
> >+ .set_attr = vgic_set_attr,
> >+ .get_attr = vgic_get_attr,
> >+ .has_attr = vgic_has_attr,
> >+};
> >diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> >index 10015d6..fb8bb21 100644
> >--- a/virt/kvm/kvm_main.c
> >+++ b/virt/kvm/kvm_main.c
> >@@ -2278,7 +2278,11 @@ static int kvm_ioctl_create_device(struct
> >kvm *kvm,
> > #ifdef CONFIG_KVM_VFIO
> > case KVM_DEV_TYPE_VFIO:
> > ops = &kvm_vfio_ops;
> >- break;
> >+#endif
> >+#ifdef CONFIG_KVM_ARM_VGIC
> >+ case KVM_DEV_TYPE_ARM_VGIC_V2:
> >+ ops = &kvm_arm_vgic_ops;
> >+ break;
> > #endif
> > default:
> > return -ENODEV;
>
> Otherwise, looks good to me.
>
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
2013-12-09 15:38 ` Marc Zyngier
@ 2013-12-12 5:29 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On Mon, Dec 09, 2013 at 03:38:19PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Support setting the distributor and cpu interface base addresses
> >in the
> >VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
> >in addition to the ARM specific API.
> >
> >This has the added benefit of being able to share more code in user
> >space and do things in a uniform manner.
> >
> >Also deprecate the older API at the same time, but backwards
> >compatibility will be maintained.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Spelling nits
> > - Fix error codes from vgic_get_attr
> >
> >Changelog[v2]:
> > - None
> >---
> > Documentation/virtual/kvm/api.txt | 7 +-
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
> > arch/arm/include/uapi/asm/kvm.h | 8 +++
> > arch/arm/kvm/arm.c | 2 +-
> > include/kvm/arm_vgic.h | 2 +-
> > virt/kvm/arm/vgic.c | 89
> >++++++++++++++++++++----
> > 6 files changed, 104 insertions(+), 15 deletions(-)
> >
> >diff --git a/Documentation/virtual/kvm/api.txt
> >b/Documentation/virtual/kvm/api.txt
> >index a30035d..867112f 100644
> >--- a/Documentation/virtual/kvm/api.txt
> >+++ b/Documentation/virtual/kvm/api.txt
> >@@ -2391,7 +2391,8 @@ struct kvm_reg_list {
> > This ioctl returns the guest registers that are supported for the
> > KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
> >
> >-4.85 KVM_ARM_SET_DEVICE_ADDR
> >+
> >+4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
> >
> > Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
> > Architectures: arm, arm64
> >@@ -2429,6 +2430,10 @@ must be called after calling
> >KVM_CREATE_IRQCHIP, but before calling
> > KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of
> >the
> > base addresses will return -EEXIST.
> >
> >+Note, this IOCTL is deprecated and the more flexible
> >SET/GET_DEVICE_ATTR API
> >+should be used instead.
> >+
> >+
> > 4.86 KVM_PPC_RTAS_DEFINE_TOKEN
> >
> > Capability: KVM_CAP_PPC_RTAS
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >index 38f27f7..c9febb2 100644
> >--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through
> >either this API or the
> > legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> >interrupt
> > controller, requiring emulated user-space devices to inject
> >interrupts to the
> > VGIC instead of directly to CPUs.
> >+
> >+Groups:
> >+ KVM_DEV_ARM_VGIC_GRP_ADDR
> >+ Attributes:
> >+ KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
> >+ Base address in the guest physical address space of the GIC
> >distributor
> >+ register mappings.
> >+
> >+ KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> >+ Base address in the guest physical address space of the GIC
> >virtual cpu
> >+ interface register mappings.
> >diff --git a/arch/arm/include/uapi/asm/kvm.h
> >b/arch/arm/include/uapi/asm/kvm.h
> >index bd3b2f7..26a375c 100644
> >--- a/arch/arm/include/uapi/asm/kvm.h
> >+++ b/arch/arm/include/uapi/asm/kvm.h
> >@@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
> > #define KVM_REG_ARM_VFP_FPINST 0x1009
> > #define KVM_REG_ARM_VFP_FPINST2 0x100A
> >
> >+/* Device Control API: ARM VGIC */
> >+#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> >+#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> >+#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> >+#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> >+#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL <<
> >KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> >+#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >+#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL <<
> >KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>
> What are these #defines for? I suppose they are used in a subsequent
> patch? If so, can we please move them?
>
Yeah, they're used in the patch:
"KVM: arm-vgic: Add vgic reg access from dev attr"
There was also an unused variable warning on compiling this patch that I
fixed. I must have screwed something up in the rebasing.
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API
@ 2013-12-12 5:29 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Dec 09, 2013 at 03:38:19PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Support setting the distributor and cpu interface base addresses
> >in the
> >VM physical address space through the KVM_{SET,GET}_DEVICE_ATTR API
> >in addition to the ARM specific API.
> >
> >This has the added benefit of being able to share more code in user
> >space and do things in a uniform manner.
> >
> >Also deprecate the older API at the same time, but backwards
> >compatibility will be maintained.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Spelling nits
> > - Fix error codes from vgic_get_attr
> >
> >Changelog[v2]:
> > - None
> >---
> > Documentation/virtual/kvm/api.txt | 7 +-
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 11 +++
> > arch/arm/include/uapi/asm/kvm.h | 8 +++
> > arch/arm/kvm/arm.c | 2 +-
> > include/kvm/arm_vgic.h | 2 +-
> > virt/kvm/arm/vgic.c | 89
> >++++++++++++++++++++----
> > 6 files changed, 104 insertions(+), 15 deletions(-)
> >
> >diff --git a/Documentation/virtual/kvm/api.txt
> >b/Documentation/virtual/kvm/api.txt
> >index a30035d..867112f 100644
> >--- a/Documentation/virtual/kvm/api.txt
> >+++ b/Documentation/virtual/kvm/api.txt
> >@@ -2391,7 +2391,8 @@ struct kvm_reg_list {
> > This ioctl returns the guest registers that are supported for the
> > KVM_GET_ONE_REG/KVM_SET_ONE_REG calls.
> >
> >-4.85 KVM_ARM_SET_DEVICE_ADDR
> >+
> >+4.85 KVM_ARM_SET_DEVICE_ADDR (deprecated)
> >
> > Capability: KVM_CAP_ARM_SET_DEVICE_ADDR
> > Architectures: arm, arm64
> >@@ -2429,6 +2430,10 @@ must be called after calling
> >KVM_CREATE_IRQCHIP, but before calling
> > KVM_RUN on any of the VCPUs. Calling this ioctl twice for any of
> >the
> > base addresses will return -EEXIST.
> >
> >+Note, this IOCTL is deprecated and the more flexible
> >SET/GET_DEVICE_ATTR API
> >+should be used instead.
> >+
> >+
> > 4.86 KVM_PPC_RTAS_DEFINE_TOKEN
> >
> > Capability: KVM_CAP_PPC_RTAS
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >index 38f27f7..c9febb2 100644
> >--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -8,3 +8,14 @@ Only one VGIC instance may be instantiated through
> >either this API or the
> > legacy KVM_CREATE_IRQCHIP api. The created VGIC will act as the VM
> >interrupt
> > controller, requiring emulated user-space devices to inject
> >interrupts to the
> > VGIC instead of directly to CPUs.
> >+
> >+Groups:
> >+ KVM_DEV_ARM_VGIC_GRP_ADDR
> >+ Attributes:
> >+ KVM_VGIC_V2_ADDR_TYPE_DIST (rw, 64-bit)
> >+ Base address in the guest physical address space of the GIC
> >distributor
> >+ register mappings.
> >+
> >+ KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> >+ Base address in the guest physical address space of the GIC
> >virtual cpu
> >+ interface register mappings.
> >diff --git a/arch/arm/include/uapi/asm/kvm.h
> >b/arch/arm/include/uapi/asm/kvm.h
> >index bd3b2f7..26a375c 100644
> >--- a/arch/arm/include/uapi/asm/kvm.h
> >+++ b/arch/arm/include/uapi/asm/kvm.h
> >@@ -159,6 +159,14 @@ struct kvm_arch_memory_slot {
> > #define KVM_REG_ARM_VFP_FPINST 0x1009
> > #define KVM_REG_ARM_VFP_FPINST2 0x100A
> >
> >+/* Device Control API: ARM VGIC */
> >+#define KVM_DEV_ARM_VGIC_GRP_ADDR 0
> >+#define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1
> >+#define KVM_DEV_ARM_VGIC_GRP_CPU_REGS 2
> >+#define KVM_DEV_ARM_VGIC_CPUID_SHIFT 32
> >+#define KVM_DEV_ARM_VGIC_CPUID_MASK (0xffULL <<
> >KVM_DEV_ARM_VGIC_CPUID_SHIFT)
> >+#define KVM_DEV_ARM_VGIC_OFFSET_SHIFT 0
> >+#define KVM_DEV_ARM_VGIC_OFFSET_MASK (0xffffffffULL <<
> >KVM_DEV_ARM_VGIC_OFFSET_SHIFT)
>
> What are these #defines for? I suppose they are used in a subsequent
> patch? If so, can we please move them?
>
Yeah, they're used in the patch:
"KVM: arm-vgic: Add vgic reg access from dev attr"
There was also an unused variable warning on compiling this patch that I
fixed. I must have screwed something up in the rebasing.
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
2013-12-09 16:02 ` Marc Zyngier
@ 2013-12-12 5:29 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On Mon, Dec 09, 2013 at 04:02:13PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Add infrastructure to handle distributor and cpu interface register
> >accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding
> >the
> >KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> >groups
> >and defining the semantics of the attr field to be the MMIO offset as
> >specified in the GICv2 specs.
> >
> >Set the cpu field (physical CPU index) on the vcpu struct to -1 in
> >kvm_arch_vcpu_put so we have a method to check if the vcpu is
> >running or
> >not without having to grab the VCPU mutexes.
> >
> >Missing register accesses or other changes in individual register
> >access
> >functions to support save/restore of the VGIC state is added in
> >subsequent patches.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Spelling and formatting nits
> > - Fill in the phys_addr in kvm_exit_mmio
> > - Put kvm_exit_mmio matching struct on stack
> > - Change if-else-if to switch statement in vgic_attr_regs_access
> > - Properly synchronize access to the VGIC with all VCPUs and the VM
> > structure, ensuring no VCPUs are running while user space is
> > accessing VGIC registers.
>
> I have some doubts about this last point, see below.
>
> >Changelog[v2]:
> > - Added implementation specific format for the GICC_APRn registers.
> >---
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
> > arch/arm/kvm/arm.c | 1 +
> > virt/kvm/arm/vgic.c | 169
> >++++++++++++++++++++++++
> > 3 files changed, 222 insertions(+)
> >
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >index c9febb2..7f4e91b 100644
> >--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -19,3 +19,55 @@ Groups:
> > KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> > Base address in the guest physical address space of the GIC
> >virtual cpu
> > interface register mappings.
> >+
> >+ KVM_DEV_ARM_VGIC_GRP_DIST_REGS
> >+ Attributes:
> >+ The attr field of kvm_device_attr encodes two values:
> >+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> >+ values: | reserved | cpu id | offset |
> >+
> >+ All distributor regs are (rw, 32-bit)
> >+
> >+ The offset is relative to the "Distributor base address" as
> >defined in the
> >+ GICv2 specs. Getting or setting such a register has the same
> >effect as
> >+ reading or writing the register on the actual hardware from
> >the cpu
> >+ specified with cpu id field. Note that most distributor
> >fields are not
> >+ banked, but return the same value regardless of the cpu id used
> >to access
> >+ the register.
> >+ Limitations:
> >+ - Priorities are not implemented, and registers are RAZ/WI
> >+ Errors:
> >+ -ENODEV: Getting or setting this register is not yet supported
> >+ -EBUSY: One or more VCPUs are running
> >+
> >+ KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> >+ Attributes:
> >+ The attr field of kvm_device_attr encodes two values:
> >+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> >+ values: | reserved | cpu id | offset |
> >+
> >+ All CPU interface regs are (rw, 32-bit)
> >+
> >+ The offset specifies the offset from the "CPU interface base
> >address" as
> >+ defined in the GICv2 specs. Getting or setting such a
> >register has the
> >+ same effect as reading or writing the register on the actual
> >hardware.
> >+
> >+ The Active Priorities Registers APRn are implementation defined,
> >so we set a
> >+ fixed format for our implementation that fits with the model
> >of a "GICv2
> >+ implementation without the security extensions" which we
> >present to the
> >+ guest. This interface always exposes four register APR[0-3]
> >describing the
> >+ maximum possible 128 preemption levels. The semantics of the
> >register
> >+ indicate if any interrupts in a given preemption level are in
> >the active
> >+ state by setting the corresponding bit.
> >+
> >+ Thus, preemption level X has one or more active interrupts if
> >and only if:
> >+
> >+ APRn[X mod 32] == 0b1, where n = X / 32
> >+
> >+ Bits for undefined preemption levels are RAZ/WI.
> >+
> >+ Limitations:
> >+ - Priorities are not implemented, and registers are RAZ/WI
> >+ Errors:
> >+ -ENODEV: Getting or setting this register is not yet supported
> >+ -EBUSY: One or more VCPUs are running
> >diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> >index a0bf0d8..35acac8 100644
> >--- a/arch/arm/kvm/arm.c
> >+++ b/arch/arm/kvm/arm.c
> >@@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu,
> >int cpu)
> >
> > void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> > {
> >+ vcpu->cpu = -1;
> > kvm_arm_set_running_vcpu(NULL);
> > }
>
> This guy deserves to be in a separate patch, with a proper comment
> and maybe a #define for the -1 value. Something like
> "KVM_VCPU_NOT_RUNNING"?
>
fair enough, separate patch it is.
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 9b9fa20..ecf6dcf 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct
> >kvm_vcpu *vcpu,
> > return false;
> > }
> >
> >+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ return false;
> >+}
> >+
> >+static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ return false;
> >+}
> >+
> > /*
> > * I would have liked to use the kvm_bus_io_*() API instead, but it
> > * cannot cope with banked registers (only the VM pointer is passed
> >@@ -663,6 +677,16 @@ static const struct mmio_range
> >vgic_dist_ranges[] = {
> > .len = 4,
> > .handle_mmio = handle_mmio_sgi_reg,
> > },
> >+ {
> >+ .base = GIC_DIST_SGI_CLEAR,
> >+ .len = VGIC_NR_SGIS,
> >+ .handle_mmio = handle_mmio_sgi_clear,
> >+ },
> >+ {
> >+ .base = GIC_DIST_SGI_SET,
> >+ .len = VGIC_NR_SGIS,
> >+ .handle_mmio = handle_mmio_sgi_set,
> >+ },
> > {}
> > };
> >
> >@@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> >long type, u64 *addr, bool write)
> > return r;
> > }
> >
> >+static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
> >+{
> >+ return true;
> >+}
> >+
> >+static const struct mmio_range vgic_cpu_ranges[] = {
> >+ {
> >+ .base = GIC_CPU_CTRL,
> >+ .len = 12,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_ALIAS_BINPOINT,
> >+ .len = 4,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_ACTIVEPRIO,
> >+ .len = 16,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_IDENT,
> >+ .len = 4,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+};
> >+
> >+static int vgic_attr_regs_access(struct kvm_device *dev,
> >+ struct kvm_device_attr *attr,
> >+ u32 *reg, bool is_write)
> >+{
> >+ const struct mmio_range *r = NULL, *ranges;
> >+ phys_addr_t offset;
> >+ int ret, cpuid, c;
> >+ struct kvm_vcpu *vcpu, *tmp_vcpu;
> >+ struct vgic_dist *vgic;
> >+ struct kvm_exit_mmio mmio;
> >+
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
> >+ KVM_DEV_ARM_VGIC_CPUID_SHIFT;
> >+
> >+ mutex_lock(&dev->kvm->lock);
> >+
> >+ if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
> >+ ret = -EINVAL;
> >+ goto out;
> >+ }
> >+
> >+ vcpu = kvm_get_vcpu(dev->kvm, cpuid);
> >+ vgic = &dev->kvm->arch.vgic;
> >+
> >+ mmio.len = 4;
> >+ mmio.is_write = is_write;
> >+ if (is_write)
> >+ mmio_data_write(&mmio, ~0, *reg);
> >+ switch (attr->group) {
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ mmio.phys_addr = vgic->vgic_dist_base + offset;
> >+ ranges = vgic_dist_ranges;
> >+ break;
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> >+ mmio.phys_addr = vgic->vgic_cpu_base + offset;
> >+ ranges = vgic_cpu_ranges;
> >+ break;
> >+ default:
> >+ BUG();
> >+ }
> >+ r = find_matching_range(ranges, &mmio, offset);
> >+
> >+ if (unlikely(!r || !r->handle_mmio)) {
> >+ ret = -ENXIO;
> >+ goto out;
> >+ }
> >+
> >+
> >+ spin_lock(&vgic->lock);
> >+
> >+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
> >+ if (unlikely(tmp_vcpu->cpu != -1)) {
>
> What guarantees that the vcpu is not going to be restarted behind
> your back? I can't see anything that locks vcpu->mutex.
>
They can start the vcpu, but the vcpu will block on trying to get the
vgic->lock in kvm_vgic_sync_hwstate, which is all we care about here.
That deserves a comment.
> >+ spin_unlock(&vgic->lock);
> >+ ret = -EBUSY;
> >+ goto out;
> >+ }
> >+ }
> >+
> >+ offset -= r->base;
> >+ r->handle_mmio(vcpu, &mmio, offset);
> >+ spin_unlock(&vgic->lock);
> >+
> >+ if (!is_write)
> >+ *reg = mmio_data_read(&mmio, ~0);
> >+
> >+ ret = 0;
> >+out:
>
> How about moving the "spin_unlock(&vgic->lock);" here, and simplify
> the exit case of the loop above?
>
The out label is also used ealier in the function before the vgic
pointer is initialized, so it requires an extra label, which I didn't
think made it much simpler, but I can go with that if you prefer...
> >+ mutex_unlock(&dev->kvm->lock);
> >+ return ret;
> >+}
> >+
> > static int vgic_set_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> > {
> > int r;
> >@@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> > r = kvm_vgic_addr(dev->kvm, type, &addr, true);
> > return (r == -ENODEV) ? -ENXIO : r;
> > }
> >+
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> >+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> >+ u32 reg;
> >+
> >+ if (get_user(reg, uaddr))
> >+ return -EFAULT;
> >+
> >+ return vgic_attr_regs_access(dev, attr, ®, true);
> >+ }
> >+
> > }
> >
> > return -ENXIO;
> >@@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> >
> > if (copy_to_user(uaddr, &addr, sizeof(addr)))
> > return -EFAULT;
> >+ break;
> >+ }
> >+
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> >+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> >+ u32 reg = 0;
> >+
> >+ r = vgic_attr_regs_access(dev, attr, ®, false);
> >+ if (r)
> >+ return r;
> >+ r = put_user(reg, uaddr);
> >+ break;
> > }
> >+
> > }
> >
> > return r;
> > }
> >
> >+static int vgic_has_attr_regs(const struct mmio_range *ranges,
> >+ phys_addr_t offset)
> >+{
> >+ struct kvm_exit_mmio dev_attr_mmio;
> >+
> >+ dev_attr_mmio.len = 4;
> >+ if (find_matching_range(ranges, &dev_attr_mmio, offset))
> >+ return 0;
> >+ else
> >+ return -ENXIO;
> >+}
> >+
> > static int vgic_has_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> > {
> > phys_addr_t offset;
> >@@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> > return 0;
> > }
> > break;
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ return vgic_has_attr_regs(vgic_dist_ranges, offset);
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ return vgic_has_attr_regs(vgic_cpu_ranges, offset);
> > }
> > return -ENXIO;
> > }
>
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr
@ 2013-12-12 5:29 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Dec 09, 2013 at 04:02:13PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Add infrastructure to handle distributor and cpu interface register
> >accesses through the KVM_{GET/SET}_DEVICE_ATTR interface by adding
> >the
> >KVM_DEV_ARM_VGIC_GRP_DIST_REGS and KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> >groups
> >and defining the semantics of the attr field to be the MMIO offset as
> >specified in the GICv2 specs.
> >
> >Set the cpu field (physical CPU index) on the vcpu struct to -1 in
> >kvm_arch_vcpu_put so we have a method to check if the vcpu is
> >running or
> >not without having to grab the VCPU mutexes.
> >
> >Missing register accesses or other changes in individual register
> >access
> >functions to support save/restore of the VGIC state is added in
> >subsequent patches.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[v3]:
> > - Spelling and formatting nits
> > - Fill in the phys_addr in kvm_exit_mmio
> > - Put kvm_exit_mmio matching struct on stack
> > - Change if-else-if to switch statement in vgic_attr_regs_access
> > - Properly synchronize access to the VGIC with all VCPUs and the VM
> > structure, ensuring no VCPUs are running while user space is
> > accessing VGIC registers.
>
> I have some doubts about this last point, see below.
>
> >Changelog[v2]:
> > - Added implementation specific format for the GICC_APRn registers.
> >---
> > Documentation/virtual/kvm/devices/arm-vgic.txt | 52 ++++++++
> > arch/arm/kvm/arm.c | 1 +
> > virt/kvm/arm/vgic.c | 169
> >++++++++++++++++++++++++
> > 3 files changed, 222 insertions(+)
> >
> >diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >index c9febb2..7f4e91b 100644
> >--- a/Documentation/virtual/kvm/devices/arm-vgic.txt
> >+++ b/Documentation/virtual/kvm/devices/arm-vgic.txt
> >@@ -19,3 +19,55 @@ Groups:
> > KVM_VGIC_V2_ADDR_TYPE_CPU (rw, 64-bit)
> > Base address in the guest physical address space of the GIC
> >virtual cpu
> > interface register mappings.
> >+
> >+ KVM_DEV_ARM_VGIC_GRP_DIST_REGS
> >+ Attributes:
> >+ The attr field of kvm_device_attr encodes two values:
> >+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> >+ values: | reserved | cpu id | offset |
> >+
> >+ All distributor regs are (rw, 32-bit)
> >+
> >+ The offset is relative to the "Distributor base address" as
> >defined in the
> >+ GICv2 specs. Getting or setting such a register has the same
> >effect as
> >+ reading or writing the register on the actual hardware from
> >the cpu
> >+ specified with cpu id field. Note that most distributor
> >fields are not
> >+ banked, but return the same value regardless of the cpu id used
> >to access
> >+ the register.
> >+ Limitations:
> >+ - Priorities are not implemented, and registers are RAZ/WI
> >+ Errors:
> >+ -ENODEV: Getting or setting this register is not yet supported
> >+ -EBUSY: One or more VCPUs are running
> >+
> >+ KVM_DEV_ARM_VGIC_GRP_CPU_REGS
> >+ Attributes:
> >+ The attr field of kvm_device_attr encodes two values:
> >+ bits: | 63 .... 40 | 39 .. 32 | 31 .... 0 |
> >+ values: | reserved | cpu id | offset |
> >+
> >+ All CPU interface regs are (rw, 32-bit)
> >+
> >+ The offset specifies the offset from the "CPU interface base
> >address" as
> >+ defined in the GICv2 specs. Getting or setting such a
> >register has the
> >+ same effect as reading or writing the register on the actual
> >hardware.
> >+
> >+ The Active Priorities Registers APRn are implementation defined,
> >so we set a
> >+ fixed format for our implementation that fits with the model
> >of a "GICv2
> >+ implementation without the security extensions" which we
> >present to the
> >+ guest. This interface always exposes four register APR[0-3]
> >describing the
> >+ maximum possible 128 preemption levels. The semantics of the
> >register
> >+ indicate if any interrupts in a given preemption level are in
> >the active
> >+ state by setting the corresponding bit.
> >+
> >+ Thus, preemption level X has one or more active interrupts if
> >and only if:
> >+
> >+ APRn[X mod 32] == 0b1, where n = X / 32
> >+
> >+ Bits for undefined preemption levels are RAZ/WI.
> >+
> >+ Limitations:
> >+ - Priorities are not implemented, and registers are RAZ/WI
> >+ Errors:
> >+ -ENODEV: Getting or setting this register is not yet supported
> >+ -EBUSY: One or more VCPUs are running
> >diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
> >index a0bf0d8..35acac8 100644
> >--- a/arch/arm/kvm/arm.c
> >+++ b/arch/arm/kvm/arm.c
> >@@ -342,6 +342,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu,
> >int cpu)
> >
> > void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
> > {
> >+ vcpu->cpu = -1;
> > kvm_arm_set_running_vcpu(NULL);
> > }
>
> This guy deserves to be in a separate patch, with a proper comment
> and maybe a #define for the -1 value. Something like
> "KVM_VCPU_NOT_RUNNING"?
>
fair enough, separate patch it is.
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 9b9fa20..ecf6dcf 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -589,6 +589,20 @@ static bool handle_mmio_sgi_reg(struct
> >kvm_vcpu *vcpu,
> > return false;
> > }
> >
> >+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ return false;
> >+}
> >+
> >+static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ return false;
> >+}
> >+
> > /*
> > * I would have liked to use the kvm_bus_io_*() API instead, but it
> > * cannot cope with banked registers (only the VM pointer is passed
> >@@ -663,6 +677,16 @@ static const struct mmio_range
> >vgic_dist_ranges[] = {
> > .len = 4,
> > .handle_mmio = handle_mmio_sgi_reg,
> > },
> >+ {
> >+ .base = GIC_DIST_SGI_CLEAR,
> >+ .len = VGIC_NR_SGIS,
> >+ .handle_mmio = handle_mmio_sgi_clear,
> >+ },
> >+ {
> >+ .base = GIC_DIST_SGI_SET,
> >+ .len = VGIC_NR_SGIS,
> >+ .handle_mmio = handle_mmio_sgi_set,
> >+ },
> > {}
> > };
> >
> >@@ -1552,6 +1576,107 @@ int kvm_vgic_addr(struct kvm *kvm, unsigned
> >long type, u64 *addr, bool write)
> > return r;
> > }
> >
> >+static bool handle_cpu_mmio_misc(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio, phys_addr_t offset)
> >+{
> >+ return true;
> >+}
> >+
> >+static const struct mmio_range vgic_cpu_ranges[] = {
> >+ {
> >+ .base = GIC_CPU_CTRL,
> >+ .len = 12,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_ALIAS_BINPOINT,
> >+ .len = 4,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_ACTIVEPRIO,
> >+ .len = 16,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+ {
> >+ .base = GIC_CPU_IDENT,
> >+ .len = 4,
> >+ .handle_mmio = handle_cpu_mmio_misc,
> >+ },
> >+};
> >+
> >+static int vgic_attr_regs_access(struct kvm_device *dev,
> >+ struct kvm_device_attr *attr,
> >+ u32 *reg, bool is_write)
> >+{
> >+ const struct mmio_range *r = NULL, *ranges;
> >+ phys_addr_t offset;
> >+ int ret, cpuid, c;
> >+ struct kvm_vcpu *vcpu, *tmp_vcpu;
> >+ struct vgic_dist *vgic;
> >+ struct kvm_exit_mmio mmio;
> >+
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
> >+ KVM_DEV_ARM_VGIC_CPUID_SHIFT;
> >+
> >+ mutex_lock(&dev->kvm->lock);
> >+
> >+ if (cpuid >= atomic_read(&dev->kvm->online_vcpus)) {
> >+ ret = -EINVAL;
> >+ goto out;
> >+ }
> >+
> >+ vcpu = kvm_get_vcpu(dev->kvm, cpuid);
> >+ vgic = &dev->kvm->arch.vgic;
> >+
> >+ mmio.len = 4;
> >+ mmio.is_write = is_write;
> >+ if (is_write)
> >+ mmio_data_write(&mmio, ~0, *reg);
> >+ switch (attr->group) {
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ mmio.phys_addr = vgic->vgic_dist_base + offset;
> >+ ranges = vgic_dist_ranges;
> >+ break;
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> >+ mmio.phys_addr = vgic->vgic_cpu_base + offset;
> >+ ranges = vgic_cpu_ranges;
> >+ break;
> >+ default:
> >+ BUG();
> >+ }
> >+ r = find_matching_range(ranges, &mmio, offset);
> >+
> >+ if (unlikely(!r || !r->handle_mmio)) {
> >+ ret = -ENXIO;
> >+ goto out;
> >+ }
> >+
> >+
> >+ spin_lock(&vgic->lock);
> >+
> >+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm) {
> >+ if (unlikely(tmp_vcpu->cpu != -1)) {
>
> What guarantees that the vcpu is not going to be restarted behind
> your back? I can't see anything that locks vcpu->mutex.
>
They can start the vcpu, but the vcpu will block on trying to get the
vgic->lock in kvm_vgic_sync_hwstate, which is all we care about here.
That deserves a comment.
> >+ spin_unlock(&vgic->lock);
> >+ ret = -EBUSY;
> >+ goto out;
> >+ }
> >+ }
> >+
> >+ offset -= r->base;
> >+ r->handle_mmio(vcpu, &mmio, offset);
> >+ spin_unlock(&vgic->lock);
> >+
> >+ if (!is_write)
> >+ *reg = mmio_data_read(&mmio, ~0);
> >+
> >+ ret = 0;
> >+out:
>
> How about moving the "spin_unlock(&vgic->lock);" here, and simplify
> the exit case of the loop above?
>
The out label is also used ealier in the function before the vgic
pointer is initialized, so it requires an extra label, which I didn't
think made it much simpler, but I can go with that if you prefer...
> >+ mutex_unlock(&dev->kvm->lock);
> >+ return ret;
> >+}
> >+
> > static int vgic_set_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> > {
> > int r;
> >@@ -1568,6 +1693,18 @@ static int vgic_set_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> > r = kvm_vgic_addr(dev->kvm, type, &addr, true);
> > return (r == -ENODEV) ? -ENXIO : r;
> > }
> >+
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> >+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> >+ u32 reg;
> >+
> >+ if (get_user(reg, uaddr))
> >+ return -EFAULT;
> >+
> >+ return vgic_attr_regs_access(dev, attr, ®, true);
> >+ }
> >+
> > }
> >
> > return -ENXIO;
> >@@ -1589,12 +1726,38 @@ static int vgic_get_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> >
> > if (copy_to_user(uaddr, &addr, sizeof(addr)))
> > return -EFAULT;
> >+ break;
> >+ }
> >+
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS: {
> >+ u32 __user *uaddr = (u32 __user *)(long)attr->addr;
> >+ u32 reg = 0;
> >+
> >+ r = vgic_attr_regs_access(dev, attr, ®, false);
> >+ if (r)
> >+ return r;
> >+ r = put_user(reg, uaddr);
> >+ break;
> > }
> >+
> > }
> >
> > return r;
> > }
> >
> >+static int vgic_has_attr_regs(const struct mmio_range *ranges,
> >+ phys_addr_t offset)
> >+{
> >+ struct kvm_exit_mmio dev_attr_mmio;
> >+
> >+ dev_attr_mmio.len = 4;
> >+ if (find_matching_range(ranges, &dev_attr_mmio, offset))
> >+ return 0;
> >+ else
> >+ return -ENXIO;
> >+}
> >+
> > static int vgic_has_attr(struct kvm_device *dev, struct
> >kvm_device_attr *attr)
> > {
> > phys_addr_t offset;
> >@@ -1607,6 +1770,12 @@ static int vgic_has_attr(struct kvm_device
> >*dev, struct kvm_device_attr *attr)
> > return 0;
> > }
> > break;
> >+ case KVM_DEV_ARM_VGIC_GRP_DIST_REGS:
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ return vgic_has_attr_regs(vgic_dist_ranges, offset);
> >+ case KVM_DEV_ARM_VGIC_GRP_CPU_REGS:
> >+ offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
> >+ return vgic_has_attr_regs(vgic_cpu_ranges, offset);
> > }
> > return -ENXIO;
> > }
>
Thanks!
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
2013-12-09 16:38 ` Marc Zyngier
@ 2013-12-12 5:29 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On Mon, Dec 09, 2013 at 04:38:33PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >To properly access the VGIC state from user space it is very
> >unpractical
> >to have to loop through all the LRs in all register access functions.
> >Instead, support moving all pending state from LRs to the
> >distributor,
> >but leave active state LRs alone.
> >
> >Note that to accurately present the active and pending state to VCPUs
> >reading these distributor registers from a live VM, we would have to
> >stop all other VPUs than the calling VCPU and ask each CPU to unqueue
> >their LR state onto the distributor and add fields to track active
> >state
> >on the distributor side as well. We don't have any users of such
> >functionality yet and there are other inaccuracies of the GIC
> >emulation,
> >so don't provide accurate synchronized access to this state just yet.
> >However, when the time comes, having this function should help.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[3]:
> > - New patch in series
> >---
> > virt/kvm/arm/vgic.c | 80
> >+++++++++++++++++++++++++++++++++++++++++++++++----
> > 1 file changed, 75 insertions(+), 5 deletions(-)
> >
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index ecf6dcf..44c669b 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct
> >kvm_vcpu *vcpu,
> > return false;
> > }
> >
> >+#define LR_CPUID(lr) \
> >+ (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> >+#define LR_IRQID(lr) \
> >+ ((lr) & GICH_LR_VIRTUALID)
> >+
> >+static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu
> >*vgic_cpu)
> >+{
> >+ clear_bit(lr_nr, vgic_cpu->lr_used);
> >+ vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
> >+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> >+}
> >+
> >+/**
> >+ * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
> >+ * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
> >+ *
> >+ * Move any pending IRQs that have already been assigned to LRs
> >back to the
> >+ * emulated distributor state so that the complete emulated state
> >can be read
> >+ * from the main emulation structures without investigating the LRs.
> >+ *
> >+ * Note that IRQs in the active state in the LRs get their pending
> >state moved
> >+ * to the distributor but the active state stays in the LRs, because
> >we don't
> >+ * track the active state on the distributor side.
> >+ */
> >+static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
> >+{
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ int i, irq, source_cpu;
> >+ u32 *lr;
> >+
> >+ for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
> >+ lr = &vgic_cpu->vgic_lr[i];
> >+ irq = LR_IRQID(*lr);
> >+ source_cpu = LR_CPUID(*lr);
> >+
> >+ /*
> >+ * If the LR holds only an active interrupt (not pending) then
> >+ * just leave it alone.
> >+ */
> >+ if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
> >+ (unsigned long *)lr))
> >+ continue;
>
> Now you got me confused. The comment talks about the active bit, but
> you're actually clearing the pending bit. Surely that deserves a
> better explanation.
>
Fair enough, it's not obvious. There are three options for the LR state
at this point:
01: pending
10: active
11: pending and active
so if the pending bit is clear, it means the state is "active".
But I'll change to explicitly check the state and clear the pending bit
later.
> >+
> >+ /*
> >+ * If the interrupt was only pending (not "active" or "pending
> >+ * and active") then we have effectively cleared the LR and it
> >+ * should be marked as free for other use.
> >+ */
> >+ if (!(*lr & GICH_LR_STATE))
> >+ vgic_retire_lr(i, irq, vgic_cpu);
>
> Why not directly testing the active bit? It'd be more consistent
> with the above if you used a similar method.
>
Because then it's more obvious to everyone what's going on, which is
clearly a bad thing ;)
> >+ /*
> >+ * Finally, reestablish the pending state on the distributor
> >+ * and the CPU interface. It may have already been pending,
> >+ * but that is fine, then we are only setting a few bits that
> >+ * were already set.
> >+ */
> >+ vgic_dist_irq_set(vcpu, irq);
> >+ if (irq < VGIC_NR_SGIS)
> >+ dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
> >+ vgic_update_state(vcpu->kvm);
> >+ }
> >+}
> >+
> > static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> > struct kvm_exit_mmio *mmio,
> > phys_addr_t offset)
> >@@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
> > }
> > }
> >
> >-#define LR_CPUID(lr) \
> >- (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> > #define MK_LR_PEND(src, irq) \
> > (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) |
> >(irq))
> >
> >@@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct
> >kvm_vcpu *vcpu)
> > int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
> >
> > if (!vgic_irq_is_enabled(vcpu, irq)) {
> >- vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> >- clear_bit(lr, vgic_cpu->lr_used);
> >- vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
> >+ vgic_retire_lr(lr, irq, vgic_cpu);
> > if (vgic_irq_is_active(vcpu, irq))
> > vgic_irq_clear_active(vcpu, irq);
> > }
> >@@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct
> >kvm_device *dev,
> > }
> > }
> >
> >+ /*
> >+ * Move all pending IRQs from the LRs on all VCPUs so the pending
> >+ * state can be properly represented in the register state
> >accessible
> >+ * through this API.
> >+ */
> >+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
> >+ vgic_unqueue_irqs(tmp_vcpu);
> >+
> > offset -= r->base;
> > r->handle_mmio(vcpu, &mmio, offset);
> > spin_unlock(&vgic->lock);
>
Thanks!
--
Christoffer
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist
@ 2013-12-12 5:29 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Dec 09, 2013 at 04:38:33PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >To properly access the VGIC state from user space it is very
> >unpractical
> >to have to loop through all the LRs in all register access functions.
> >Instead, support moving all pending state from LRs to the
> >distributor,
> >but leave active state LRs alone.
> >
> >Note that to accurately present the active and pending state to VCPUs
> >reading these distributor registers from a live VM, we would have to
> >stop all other VPUs than the calling VCPU and ask each CPU to unqueue
> >their LR state onto the distributor and add fields to track active
> >state
> >on the distributor side as well. We don't have any users of such
> >functionality yet and there are other inaccuracies of the GIC
> >emulation,
> >so don't provide accurate synchronized access to this state just yet.
> >However, when the time comes, having this function should help.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
> >
> >Changelog[3]:
> > - New patch in series
> >---
> > virt/kvm/arm/vgic.c | 80
> >+++++++++++++++++++++++++++++++++++++++++++++++----
> > 1 file changed, 75 insertions(+), 5 deletions(-)
> >
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index ecf6dcf..44c669b 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -589,6 +589,72 @@ static bool handle_mmio_sgi_reg(struct
> >kvm_vcpu *vcpu,
> > return false;
> > }
> >
> >+#define LR_CPUID(lr) \
> >+ (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> >+#define LR_IRQID(lr) \
> >+ ((lr) & GICH_LR_VIRTUALID)
> >+
> >+static void vgic_retire_lr(int lr_nr, int irq, struct vgic_cpu
> >*vgic_cpu)
> >+{
> >+ clear_bit(lr_nr, vgic_cpu->lr_used);
> >+ vgic_cpu->vgic_lr[lr_nr] &= ~GICH_LR_STATE;
> >+ vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> >+}
> >+
> >+/**
> >+ * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
> >+ * @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
> >+ *
> >+ * Move any pending IRQs that have already been assigned to LRs
> >back to the
> >+ * emulated distributor state so that the complete emulated state
> >can be read
> >+ * from the main emulation structures without investigating the LRs.
> >+ *
> >+ * Note that IRQs in the active state in the LRs get their pending
> >state moved
> >+ * to the distributor but the active state stays in the LRs, because
> >we don't
> >+ * track the active state on the distributor side.
> >+ */
> >+static void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
> >+{
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ int i, irq, source_cpu;
> >+ u32 *lr;
> >+
> >+ for_each_set_bit(i, vgic_cpu->lr_used, vgic_cpu->nr_lr) {
> >+ lr = &vgic_cpu->vgic_lr[i];
> >+ irq = LR_IRQID(*lr);
> >+ source_cpu = LR_CPUID(*lr);
> >+
> >+ /*
> >+ * If the LR holds only an active interrupt (not pending) then
> >+ * just leave it alone.
> >+ */
> >+ if (!__test_and_clear_bit(__ffs(GICH_LR_PENDING_BIT),
> >+ (unsigned long *)lr))
> >+ continue;
>
> Now you got me confused. The comment talks about the active bit, but
> you're actually clearing the pending bit. Surely that deserves a
> better explanation.
>
Fair enough, it's not obvious. There are three options for the LR state
at this point:
01: pending
10: active
11: pending and active
so if the pending bit is clear, it means the state is "active".
But I'll change to explicitly check the state and clear the pending bit
later.
> >+
> >+ /*
> >+ * If the interrupt was only pending (not "active" or "pending
> >+ * and active") then we have effectively cleared the LR and it
> >+ * should be marked as free for other use.
> >+ */
> >+ if (!(*lr & GICH_LR_STATE))
> >+ vgic_retire_lr(i, irq, vgic_cpu);
>
> Why not directly testing the active bit? It'd be more consistent
> with the above if you used a similar method.
>
Because then it's more obvious to everyone what's going on, which is
clearly a bad thing ;)
> >+ /*
> >+ * Finally, reestablish the pending state on the distributor
> >+ * and the CPU interface. It may have already been pending,
> >+ * but that is fine, then we are only setting a few bits that
> >+ * were already set.
> >+ */
> >+ vgic_dist_irq_set(vcpu, irq);
> >+ if (irq < VGIC_NR_SGIS)
> >+ dist->irq_sgi_sources[vcpu_id][irq] |= 1 << source_cpu;
> >+ vgic_update_state(vcpu->kvm);
> >+ }
> >+}
> >+
> > static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> > struct kvm_exit_mmio *mmio,
> > phys_addr_t offset)
> >@@ -848,8 +914,6 @@ static void vgic_update_state(struct kvm *kvm)
> > }
> > }
> >
> >-#define LR_CPUID(lr) \
> >- (((lr) & GICH_LR_PHYSID_CPUID) >> GICH_LR_PHYSID_CPUID_SHIFT)
> > #define MK_LR_PEND(src, irq) \
> > (GICH_LR_PENDING_BIT | ((src) << GICH_LR_PHYSID_CPUID_SHIFT) |
> >(irq))
> >
> >@@ -871,9 +935,7 @@ static void vgic_retire_disabled_irqs(struct
> >kvm_vcpu *vcpu)
> > int irq = vgic_cpu->vgic_lr[lr] & GICH_LR_VIRTUALID;
> >
> > if (!vgic_irq_is_enabled(vcpu, irq)) {
> >- vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
> >- clear_bit(lr, vgic_cpu->lr_used);
> >- vgic_cpu->vgic_lr[lr] &= ~GICH_LR_STATE;
> >+ vgic_retire_lr(lr, irq, vgic_cpu);
> > if (vgic_irq_is_active(vcpu, irq))
> > vgic_irq_clear_active(vcpu, irq);
> > }
> >@@ -1664,6 +1726,14 @@ static int vgic_attr_regs_access(struct
> >kvm_device *dev,
> > }
> > }
> >
> >+ /*
> >+ * Move all pending IRQs from the LRs on all VCPUs so the pending
> >+ * state can be properly represented in the register state
> >accessible
> >+ * through this API.
> >+ */
> >+ kvm_for_each_vcpu(c, tmp_vcpu, dev->kvm)
> >+ vgic_unqueue_irqs(tmp_vcpu);
> >+
> > offset -= r->base;
> > r->handle_mmio(vcpu, &mmio, offset);
> > spin_unlock(&vgic->lock);
>
Thanks!
--
Christoffer
^ permalink raw reply [flat|nested] 48+ messages in thread
* Re: [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
2013-12-09 16:49 ` Marc Zyngier
@ 2013-12-12 5:29 ` Christoffer Dall
-1 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: Marc Zyngier; +Cc: kvmarm, kvm, linux-arm-kernel, patches
On Mon, Dec 09, 2013 at 04:49:45PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Handle MMIO accesses to the two registers which should support
> >both the
> >case where the VMs want to read/write either of these registers
> >and the
> >case where user space reads/writes these registers to do
> >save/restore of
> >the VGIC state.
> >
> >Note that the added complexity compared to simple set/clear enable
> >registers stems from the bookkeping of source cpu ids. It may be
> >possible to change the underlying data structure to simplify the
> >complexity, but since this is not in the critical path at all,
> >this will
> >do.
> >
> >Also note that reading this register from a live guest will not be
> >accurate compared to on hardware, because some state may be living on
> >the CPU LRs and the only way to give a consistent read would be to
> >force
> >stop all the VCPUs and request them to unqueu the LR state onto the
> >distributor. Until we have an actual user of live reading this
> >register, we can live with the difference.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Looks pretty good to me. Small note below, but otherwise:
>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>
> >
> >Changelog[v3]:
> > - Renamed read/write SGI set/clear functions
> > - Rely on unqueuing of interrupts from LRs instead of reading LRs
> > directly
> > - Deduplicate code
> >
> >Changelog[v2]:
> > - Use struct kvm_exit_mmio accessors for ->data field.
> >---
> > virt/kvm/arm/vgic.c | 70
> >++++++++++++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 66 insertions(+), 4 deletions(-)
> >
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 44c669b..16053eb 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct
> >kvm_vcpu *vcpu)
> > }
> > }
> >
> >-static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >- struct kvm_exit_mmio *mmio,
> >- phys_addr_t offset)
> >+/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
> >+static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> > {
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ int sgi;
> >+ int min_sgi = (offset & ~0x3) * 4;
> >+ int max_sgi = min_sgi + 3;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ u32 reg = 0;
> >+
> >+ /* Copy source SGIs from distributor side */
> >+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> >+ int shift = 8 * (sgi - min_sgi);
> >+ reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
> >+ }
> >+
> >+ mmio_data_write(mmio, ~0, reg);
> > return false;
> > }
> >
> >+static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset, bool set)
> >+{
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ int sgi;
> >+ int min_sgi = (offset & ~0x3) * 4;
> >+ int max_sgi = min_sgi + 3;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ u32 reg;
> >+ bool updated = false;
> >+
> >+ reg = mmio_data_read(mmio, ~0);
> >+
> >+ /* Clear pending SGIs on the distributor */
> >+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> >+ u8 mask = reg >> (8 * (sgi - min_sgi));
> >+ if (set) {
> >+ if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
> >+ updated = true;
> >+ dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
> >+ } else {
> >+ if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
> >+ updated = true;
> >+ dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
> >+ }
> >+ }
> >+
> >+ if (updated)
> >+ vgic_update_state(vcpu->kvm);
>
> So I realize we have that construct everywhere. Surely it'd be worth
> it moving the mmio calls to vgic_update_state into vgic_handle_mmio.
> Or have I missed something?
>
I have some very vague recollection that we once discussed if we wanted
to compare some of the register writes to their original values and only
update the vgic state in that case, and therefore have the functionality
locally (whereas now, if it's a write, we call update).
That being said, it should still work with only calling update whenever
we return true from those mmio handlers, so I guess it would work.
Are you ok with us fixing this after this code is merged, I'd really
like to get these patches in as they have been floating around for quite
a while.
Thanks,
-Christoffer
> >+ return updated;
> >+}
> >+
> > static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> > struct kvm_exit_mmio *mmio,
> > phys_addr_t offset)
> > {
> >- return false;
> >+ if (!mmio->is_write)
> >+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> >+ else
> >+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
> >+}
> >+
> >+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ if (!mmio->is_write)
> >+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> >+ else
> >+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
> > }
> >
> > /*
>
> --
> Fast, cheap, reliable. Pick two.
--
Christoffer
^ permalink raw reply [flat|nested] 48+ messages in thread
* [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers
@ 2013-12-12 5:29 ` Christoffer Dall
0 siblings, 0 replies; 48+ messages in thread
From: Christoffer Dall @ 2013-12-12 5:29 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Dec 09, 2013 at 04:49:45PM +0000, Marc Zyngier wrote:
> On 2013-11-17 04:30, Christoffer Dall wrote:
> >Handle MMIO accesses to the two registers which should support
> >both the
> >case where the VMs want to read/write either of these registers
> >and the
> >case where user space reads/writes these registers to do
> >save/restore of
> >the VGIC state.
> >
> >Note that the added complexity compared to simple set/clear enable
> >registers stems from the bookkeping of source cpu ids. It may be
> >possible to change the underlying data structure to simplify the
> >complexity, but since this is not in the critical path at all,
> >this will
> >do.
> >
> >Also note that reading this register from a live guest will not be
> >accurate compared to on hardware, because some state may be living on
> >the CPU LRs and the only way to give a consistent read would be to
> >force
> >stop all the VCPUs and request them to unqueu the LR state onto the
> >distributor. Until we have an actual user of live reading this
> >register, we can live with the difference.
> >
> >Signed-off-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Looks pretty good to me. Small note below, but otherwise:
>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>
> >
> >Changelog[v3]:
> > - Renamed read/write SGI set/clear functions
> > - Rely on unqueuing of interrupts from LRs instead of reading LRs
> > directly
> > - Deduplicate code
> >
> >Changelog[v2]:
> > - Use struct kvm_exit_mmio accessors for ->data field.
> >---
> > virt/kvm/arm/vgic.c | 70
> >++++++++++++++++++++++++++++++++++++++++++++++++---
> > 1 file changed, 66 insertions(+), 4 deletions(-)
> >
> >diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >index 44c669b..16053eb 100644
> >--- a/virt/kvm/arm/vgic.c
> >+++ b/virt/kvm/arm/vgic.c
> >@@ -655,18 +655,80 @@ static void vgic_unqueue_irqs(struct
> >kvm_vcpu *vcpu)
> > }
> > }
> >
> >-static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >- struct kvm_exit_mmio *mmio,
> >- phys_addr_t offset)
> >+/* Handle reads of GICD_CPENDSGIRn and GICD_SPENDSGIRn */
> >+static bool read_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> > {
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ int sgi;
> >+ int min_sgi = (offset & ~0x3) * 4;
> >+ int max_sgi = min_sgi + 3;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ u32 reg = 0;
> >+
> >+ /* Copy source SGIs from distributor side */
> >+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> >+ int shift = 8 * (sgi - min_sgi);
> >+ reg |= (u32)dist->irq_sgi_sources[vcpu_id][sgi] << shift;
> >+ }
> >+
> >+ mmio_data_write(mmio, ~0, reg);
> > return false;
> > }
> >
> >+static bool write_set_clear_sgi_pend_reg(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset, bool set)
> >+{
> >+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >+ int sgi;
> >+ int min_sgi = (offset & ~0x3) * 4;
> >+ int max_sgi = min_sgi + 3;
> >+ int vcpu_id = vcpu->vcpu_id;
> >+ u32 reg;
> >+ bool updated = false;
> >+
> >+ reg = mmio_data_read(mmio, ~0);
> >+
> >+ /* Clear pending SGIs on the distributor */
> >+ for (sgi = min_sgi; sgi <= max_sgi; sgi++) {
> >+ u8 mask = reg >> (8 * (sgi - min_sgi));
> >+ if (set) {
> >+ if ((dist->irq_sgi_sources[vcpu_id][sgi] & mask) != mask)
> >+ updated = true;
> >+ dist->irq_sgi_sources[vcpu_id][sgi] |= mask;
> >+ } else {
> >+ if (dist->irq_sgi_sources[vcpu_id][sgi] & mask)
> >+ updated = true;
> >+ dist->irq_sgi_sources[vcpu_id][sgi] &= ~mask;
> >+ }
> >+ }
> >+
> >+ if (updated)
> >+ vgic_update_state(vcpu->kvm);
>
> So I realize we have that construct everywhere. Surely it'd be worth
> it moving the mmio calls to vgic_update_state into vgic_handle_mmio.
> Or have I missed something?
>
I have some very vague recollection that we once discussed if we wanted
to compare some of the register writes to their original values and only
update the vgic state in that case, and therefore have the functionality
locally (whereas now, if it's a write, we call update).
That being said, it should still work with only calling update whenever
we return true from those mmio handlers, so I guess it would work.
Are you ok with us fixing this after this code is merged, I'd really
like to get these patches in as they have been floating around for quite
a while.
Thanks,
-Christoffer
> >+ return updated;
> >+}
> >+
> > static bool handle_mmio_sgi_set(struct kvm_vcpu *vcpu,
> > struct kvm_exit_mmio *mmio,
> > phys_addr_t offset)
> > {
> >- return false;
> >+ if (!mmio->is_write)
> >+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> >+ else
> >+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, true);
> >+}
> >+
> >+static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
> >+ struct kvm_exit_mmio *mmio,
> >+ phys_addr_t offset)
> >+{
> >+ if (!mmio->is_write)
> >+ return read_set_clear_sgi_pend_reg(vcpu, mmio, offset);
> >+ else
> >+ return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
> > }
> >
> > /*
>
> --
> Fast, cheap, reliable. Pick two.
--
Christoffer
^ permalink raw reply [flat|nested] 48+ messages in thread
end of thread, other threads:[~2013-12-12 5:29 UTC | newest]
Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-11-17 4:30 [PATCH v3 0/9] Support VGIC save/restore using device control API Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 1/9] ARM: KVM: Allow creating the VGIC after VCPUs Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 15:11 ` Marc Zyngier
2013-12-09 15:11 ` Marc Zyngier
2013-11-17 4:30 ` [PATCH v3 2/9] KVM: arm-vgic: Support KVM_CREATE_DEVICE for VGIC Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 15:19 ` Marc Zyngier
2013-12-09 15:19 ` Marc Zyngier
2013-12-12 5:29 ` Christoffer Dall
2013-12-12 5:29 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 3/9] KVM: arm-vgic: Set base addr through device API Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 15:38 ` Marc Zyngier
2013-12-09 15:38 ` Marc Zyngier
2013-12-12 5:29 ` Christoffer Dall
2013-12-12 5:29 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 4/9] irqchip: arm-gic: Define additional MMIO offsets and masks Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 15:40 ` Marc Zyngier
2013-12-09 15:40 ` Marc Zyngier
2013-11-17 4:30 ` [PATCH v3 5/9] KVM: arm-vgic: Make vgic mmio functions more generic Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 15:47 ` Marc Zyngier
2013-12-09 15:47 ` Marc Zyngier
2013-11-17 4:30 ` [PATCH v3 6/9] KVM: arm-vgic: Add vgic reg access from dev attr Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 16:02 ` Marc Zyngier
2013-12-09 16:02 ` Marc Zyngier
2013-12-12 5:29 ` Christoffer Dall
2013-12-12 5:29 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 7/9] KVM: arm-vgic: Support unqueueing of LRs to the dist Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 16:38 ` Marc Zyngier
2013-12-09 16:38 ` Marc Zyngier
2013-12-12 5:29 ` Christoffer Dall
2013-12-12 5:29 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 8/9] KVM: arm-vgic: Add GICD_SPENDSGIR and GICD_CPENDSGIR handlers Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 16:49 ` Marc Zyngier
2013-12-09 16:49 ` Marc Zyngier
2013-12-12 5:29 ` Christoffer Dall
2013-12-12 5:29 ` Christoffer Dall
2013-11-17 4:30 ` [PATCH v3 9/9] KVM: arm-vgic: Support CPU interface reg access Christoffer Dall
2013-11-17 4:30 ` Christoffer Dall
2013-12-09 17:13 ` Marc Zyngier
2013-12-09 17:13 ` Marc Zyngier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.