All of lore.kernel.org
 help / color / mirror / Atom feed
* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-01 17:43 ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

The ARM architecture defines the EL1 physical timer and the virtual timer,
and it is reasonable for an OS to expect to be able to access both.
However, the current KVM implementation does not provide the EL1 physical
timer to VMs but terminates VMs on access to the timer.

This patch series enables VMs to use the EL1 physical timer through
trap-and-emulate.  The KVM host emulates each EL1 physical timer register
access and sets up the background timer accordingly.  When the background
timer expires, the KVM host injects EL1 physical timer interrupts to the
VM.  Alternatively, it's also possible to allow VMs to access the EL1
physical timer without trapping.  However, this requires somehow using the
EL2 physical timer for the Linux host while running the VM instead of the
EL1 physical timer.  Right now I just implemented trap-and-emulate because
this was straightforward to do, and I leave it to future work to determine
if transferring the EL1 physical timer state to the EL2 timer provides any
performance benefit.

This feature will be useful for any OS that wishes to access the EL1
physical timer. Nested virtualization is one of those use cases. A nested
hypervisor running inside a VM would think it has full access to the
hardware and naturally tries to use the EL1 physical timer as Linux would
do. Other nested hypervisors may try to use the EL2 physical timer as Xen
would do, but supporting the EL2 physical timer to the VM is out of scope
of this patch series. This patch series will make it easy to add the EL2
timer support in the future, though.

Note that Linux VMs booting in EL1 will be unaffected by this patch series
and will continue to use only the virtual timer and this patch series will
therefore not introduce any performance degredation as a result of
trap-and-emulate.

v2 => v3:
 - Rebase on kvmarm/queue
 - Take kvm->lock to synchronize cntvoff across all vtimers
 - Remove unnecessary function parameters
 - Add comments

v1 => v2:
 - Rebase on kvm-arm-for-4.10-rc4
 - To make it simple, schedule the background timer for the EL1 physical timer
   emulation on every entry to the VM and cancel it on exit.
 - Change timer_context structure to have cntvoff and restore enable field back
   to arch_timer_cpu structure

Jintack Lim (10):
  KVM: arm/arm64: Abstract virtual timer context into separate structure
  KVM: arm/arm64: Move cntvoff to each timer context
  KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  KVM: arm/arm64: Add the EL1 physical timer context
  KVM: arm/arm64: Initialize the emulated EL1 physical timer
  KVM: arm/arm64: Update the physical timer interrupt level
  KVM: arm/arm64: Set a background timer to the earliest timer
    expiration
  KVM: arm/arm64: Set up a background timer for the physical timer
    emulation
  KVM: arm64: Add the EL1 physical timer access handler
  KVM: arm/arm64: Emulate the EL1 phys timer registers

 arch/arm/include/asm/kvm_host.h   |   3 -
 arch/arm/kvm/arm.c                |   4 +-
 arch/arm/kvm/reset.c              |   9 +-
 arch/arm64/include/asm/kvm_host.h |   3 -
 arch/arm64/kvm/reset.c            |   9 +-
 arch/arm64/kvm/sys_regs.c         |  65 +++++++++++++
 include/kvm/arm_arch_timer.h      |  39 ++++----
 virt/kvm/arm/arch_timer.c         | 193 ++++++++++++++++++++++++++------------
 virt/kvm/arm/hyp/timer-sr.c       |  13 +--
 9 files changed, 242 insertions(+), 96 deletions(-)

-- 
1.9.1

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-01 17:43 ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

The ARM architecture defines the EL1 physical timer and the virtual timer,
and it is reasonable for an OS to expect to be able to access both.
However, the current KVM implementation does not provide the EL1 physical
timer to VMs but terminates VMs on access to the timer.

This patch series enables VMs to use the EL1 physical timer through
trap-and-emulate.  The KVM host emulates each EL1 physical timer register
access and sets up the background timer accordingly.  When the background
timer expires, the KVM host injects EL1 physical timer interrupts to the
VM.  Alternatively, it's also possible to allow VMs to access the EL1
physical timer without trapping.  However, this requires somehow using the
EL2 physical timer for the Linux host while running the VM instead of the
EL1 physical timer.  Right now I just implemented trap-and-emulate because
this was straightforward to do, and I leave it to future work to determine
if transferring the EL1 physical timer state to the EL2 timer provides any
performance benefit.

This feature will be useful for any OS that wishes to access the EL1
physical timer. Nested virtualization is one of those use cases. A nested
hypervisor running inside a VM would think it has full access to the
hardware and naturally tries to use the EL1 physical timer as Linux would
do. Other nested hypervisors may try to use the EL2 physical timer as Xen
would do, but supporting the EL2 physical timer to the VM is out of scope
of this patch series. This patch series will make it easy to add the EL2
timer support in the future, though.

Note that Linux VMs booting in EL1 will be unaffected by this patch series
and will continue to use only the virtual timer and this patch series will
therefore not introduce any performance degredation as a result of
trap-and-emulate.

v2 => v3:
 - Rebase on kvmarm/queue
 - Take kvm->lock to synchronize cntvoff across all vtimers
 - Remove unnecessary function parameters
 - Add comments

v1 => v2:
 - Rebase on kvm-arm-for-4.10-rc4
 - To make it simple, schedule the background timer for the EL1 physical timer
   emulation on every entry to the VM and cancel it on exit.
 - Change timer_context structure to have cntvoff and restore enable field back
   to arch_timer_cpu structure

Jintack Lim (10):
  KVM: arm/arm64: Abstract virtual timer context into separate structure
  KVM: arm/arm64: Move cntvoff to each timer context
  KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  KVM: arm/arm64: Add the EL1 physical timer context
  KVM: arm/arm64: Initialize the emulated EL1 physical timer
  KVM: arm/arm64: Update the physical timer interrupt level
  KVM: arm/arm64: Set a background timer to the earliest timer
    expiration
  KVM: arm/arm64: Set up a background timer for the physical timer
    emulation
  KVM: arm64: Add the EL1 physical timer access handler
  KVM: arm/arm64: Emulate the EL1 phys timer registers

 arch/arm/include/asm/kvm_host.h   |   3 -
 arch/arm/kvm/arm.c                |   4 +-
 arch/arm/kvm/reset.c              |   9 +-
 arch/arm64/include/asm/kvm_host.h |   3 -
 arch/arm64/kvm/reset.c            |   9 +-
 arch/arm64/kvm/sys_regs.c         |  65 +++++++++++++
 include/kvm/arm_arch_timer.h      |  39 ++++----
 virt/kvm/arm/arch_timer.c         | 193 ++++++++++++++++++++++++++------------
 virt/kvm/arm/hyp/timer-sr.c       |  13 +--
 9 files changed, 242 insertions(+), 96 deletions(-)

-- 
1.9.1

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-01 17:43 ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

The ARM architecture defines the EL1 physical timer and the virtual timer,
and it is reasonable for an OS to expect to be able to access both.
However, the current KVM implementation does not provide the EL1 physical
timer to VMs but terminates VMs on access to the timer.

This patch series enables VMs to use the EL1 physical timer through
trap-and-emulate.  The KVM host emulates each EL1 physical timer register
access and sets up the background timer accordingly.  When the background
timer expires, the KVM host injects EL1 physical timer interrupts to the
VM.  Alternatively, it's also possible to allow VMs to access the EL1
physical timer without trapping.  However, this requires somehow using the
EL2 physical timer for the Linux host while running the VM instead of the
EL1 physical timer.  Right now I just implemented trap-and-emulate because
this was straightforward to do, and I leave it to future work to determine
if transferring the EL1 physical timer state to the EL2 timer provides any
performance benefit.

This feature will be useful for any OS that wishes to access the EL1
physical timer. Nested virtualization is one of those use cases. A nested
hypervisor running inside a VM would think it has full access to the
hardware and naturally tries to use the EL1 physical timer as Linux would
do. Other nested hypervisors may try to use the EL2 physical timer as Xen
would do, but supporting the EL2 physical timer to the VM is out of scope
of this patch series. This patch series will make it easy to add the EL2
timer support in the future, though.

Note that Linux VMs booting in EL1 will be unaffected by this patch series
and will continue to use only the virtual timer and this patch series will
therefore not introduce any performance degredation as a result of
trap-and-emulate.

v2 => v3:
 - Rebase on kvmarm/queue
 - Take kvm->lock to synchronize cntvoff across all vtimers
 - Remove unnecessary function parameters
 - Add comments

v1 => v2:
 - Rebase on kvm-arm-for-4.10-rc4
 - To make it simple, schedule the background timer for the EL1 physical timer
   emulation on every entry to the VM and cancel it on exit.
 - Change timer_context structure to have cntvoff and restore enable field back
   to arch_timer_cpu structure

Jintack Lim (10):
  KVM: arm/arm64: Abstract virtual timer context into separate structure
  KVM: arm/arm64: Move cntvoff to each timer context
  KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  KVM: arm/arm64: Add the EL1 physical timer context
  KVM: arm/arm64: Initialize the emulated EL1 physical timer
  KVM: arm/arm64: Update the physical timer interrupt level
  KVM: arm/arm64: Set a background timer to the earliest timer
    expiration
  KVM: arm/arm64: Set up a background timer for the physical timer
    emulation
  KVM: arm64: Add the EL1 physical timer access handler
  KVM: arm/arm64: Emulate the EL1 phys timer registers

 arch/arm/include/asm/kvm_host.h   |   3 -
 arch/arm/kvm/arm.c                |   4 +-
 arch/arm/kvm/reset.c              |   9 +-
 arch/arm64/include/asm/kvm_host.h |   3 -
 arch/arm64/kvm/reset.c            |   9 +-
 arch/arm64/kvm/sys_regs.c         |  65 +++++++++++++
 include/kvm/arm_arch_timer.h      |  39 ++++----
 virt/kvm/arm/arch_timer.c         | 193 ++++++++++++++++++++++++++------------
 virt/kvm/arm/hyp/timer-sr.c       |  13 +--
 9 files changed, 242 insertions(+), 96 deletions(-)

-- 
1.9.1

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 01/10] KVM: arm/arm64: Abstract virtual timer context into separate structure
  2017-02-01 17:43 ` Jintack Lim
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Abstract virtual timer context into a separate structure and change all
callers referring to timer registers, irq state and so on. No change in
functionality.

This is about to become very handy when adding the EL1 physical timer.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_arch_timer.h | 27 ++++++++---------
 virt/kvm/arm/arch_timer.c    | 69 +++++++++++++++++++++++---------------------
 virt/kvm/arm/hyp/timer-sr.c  | 10 ++++---
 3 files changed, 56 insertions(+), 50 deletions(-)

diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 5c970ce..daad3c1 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -28,15 +28,20 @@ struct arch_timer_kvm {
 	u64			cntvoff;
 };
 
-struct arch_timer_cpu {
+struct arch_timer_context {
 	/* Registers: control register, timer value */
-	u32				cntv_ctl;	/* Saved/restored */
-	u64				cntv_cval;	/* Saved/restored */
+	u32				cnt_ctl;
+	u64				cnt_cval;
+
+	/* Timer IRQ */
+	struct kvm_irq_level		irq;
+
+	/* Active IRQ state caching */
+	bool				active_cleared_last;
+};
 
-	/*
-	 * Anything that is not used directly from assembly code goes
-	 * here.
-	 */
+struct arch_timer_cpu {
+	struct arch_timer_context	vtimer;
 
 	/* Background timer used when the guest is not running */
 	struct hrtimer			timer;
@@ -47,12 +52,6 @@ struct arch_timer_cpu {
 	/* Background timer active */
 	bool				armed;
 
-	/* Timer IRQ */
-	struct kvm_irq_level		irq;
-
-	/* Active IRQ state caching */
-	bool				active_cleared_last;
-
 	/* Is the timer enabled */
 	bool			enabled;
 };
@@ -77,4 +76,6 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
 
 void kvm_timer_init_vhe(void);
+
+#define vcpu_vtimer(v)	(&(v)->arch.timer_cpu.vtimer)
 #endif
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 91ecf48..d3556b3 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -37,7 +37,7 @@
 
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
 {
-	vcpu->arch.timer_cpu.active_cleared_last = false;
+	vcpu_vtimer(vcpu)->active_cleared_last = false;
 }
 
 static u64 kvm_phys_timer_read(void)
@@ -102,7 +102,7 @@ static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
 {
 	u64 cval, now;
 
-	cval = vcpu->arch.timer_cpu.cntv_cval;
+	cval = vcpu_vtimer(vcpu)->cnt_cval;
 	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 
 	if (now < cval) {
@@ -144,21 +144,21 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 
 static bool kvm_timer_irq_can_fire(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	return !(timer->cntv_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(timer->cntv_ctl & ARCH_TIMER_CTRL_ENABLE);
+	return !(vtimer->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(vtimer->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
 }
 
 bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 cval, now;
 
 	if (!kvm_timer_irq_can_fire(vcpu))
 		return false;
 
-	cval = timer->cntv_cval;
+	cval = vtimer->cnt_cval;
 	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 
 	return cval <= now;
@@ -167,18 +167,18 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
 {
 	int ret;
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(!vgic_initialized(vcpu->kvm));
 
-	timer->active_cleared_last = false;
-	timer->irq.level = new_level;
-	trace_kvm_timer_update_irq(vcpu->vcpu_id, timer->irq.irq,
-				   timer->irq.level);
+	vtimer->active_cleared_last = false;
+	vtimer->irq.level = new_level;
+	trace_kvm_timer_update_irq(vcpu->vcpu_id, vtimer->irq.irq,
+				   vtimer->irq.level);
 
 	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
-					 timer->irq.irq,
-					 timer->irq.level);
+					 vtimer->irq.irq,
+					 vtimer->irq.level);
 	WARN_ON(ret);
 }
 
@@ -189,18 +189,19 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
 static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	/*
 	 * If userspace modified the timer registers via SET_ONE_REG before
-	 * the vgic was initialized, we mustn't set the timer->irq.level value
+	 * the vgic was initialized, we mustn't set the vtimer->irq.level value
 	 * because the guest would never see the interrupt.  Instead wait
 	 * until we call this function from kvm_timer_flush_hwstate.
 	 */
 	if (!vgic_initialized(vcpu->kvm) || !timer->enabled)
 		return -ENODEV;
 
-	if (kvm_timer_should_fire(vcpu) != timer->irq.level)
-		kvm_timer_update_irq(vcpu, !timer->irq.level);
+	if (kvm_timer_should_fire(vcpu) != vtimer->irq.level)
+		kvm_timer_update_irq(vcpu, !vtimer->irq.level);
 
 	return 0;
 }
@@ -250,7 +251,7 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
  */
 void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	bool phys_active;
 	int ret;
 
@@ -274,8 +275,8 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	* to ensure that hardware interrupts from the timer triggers a guest
 	* exit.
 	*/
-	phys_active = timer->irq.level ||
-			kvm_vgic_map_is_active(vcpu, timer->irq.irq);
+	phys_active = vtimer->irq.level ||
+			kvm_vgic_map_is_active(vcpu, vtimer->irq.irq);
 
 	/*
 	 * We want to avoid hitting the (re)distributor as much as
@@ -297,7 +298,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	 * - cached value is "active clear"
 	 * - value to be programmed is "active clear"
 	 */
-	if (timer->active_cleared_last && !phys_active)
+	if (vtimer->active_cleared_last && !phys_active)
 		return;
 
 	ret = irq_set_irqchip_state(host_vtimer_irq,
@@ -305,7 +306,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 				    phys_active);
 	WARN_ON(ret);
 
-	timer->active_cleared_last = !phys_active;
+	vtimer->active_cleared_last = !phys_active;
 }
 
 /**
@@ -331,7 +332,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 			 const struct kvm_irq_level *irq)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	/*
 	 * The vcpu timer irq number cannot be determined in
@@ -339,7 +340,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * kvm_vcpu_set_target(). To handle this, we determine
 	 * vcpu timer irq number when the vcpu is reset.
 	 */
-	timer->irq.irq = irq->irq;
+	vtimer->irq.irq = irq->irq;
 
 	/*
 	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
@@ -347,7 +348,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * resets the timer to be disabled and unmasked and is compliant with
 	 * the ARMv7 architecture.
 	 */
-	timer->cntv_ctl = 0;
+	vtimer->cnt_ctl = 0;
 	kvm_timer_update_state(vcpu);
 
 	return 0;
@@ -369,17 +370,17 @@ static void kvm_timer_init_interrupt(void *info)
 
 int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	switch (regid) {
 	case KVM_REG_ARM_TIMER_CTL:
-		timer->cntv_ctl = value;
+		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
 		vcpu->kvm->arch.timer.cntvoff = kvm_phys_timer_read() - value;
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
-		timer->cntv_cval = value;
+		vtimer->cnt_cval = value;
 		break;
 	default:
 		return -1;
@@ -391,15 +392,15 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 
 u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	switch (regid) {
 	case KVM_REG_ARM_TIMER_CTL:
-		return timer->cntv_ctl;
+		return vtimer->cnt_ctl;
 	case KVM_REG_ARM_TIMER_CNT:
 		return kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 	case KVM_REG_ARM_TIMER_CVAL:
-		return timer->cntv_cval;
+		return vtimer->cnt_cval;
 	}
 	return (u64)-1;
 }
@@ -463,14 +464,16 @@ int kvm_timer_hyp_init(void)
 void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	timer_disarm(timer);
-	kvm_vgic_unmap_phys_irq(vcpu, timer->irq.irq);
+	kvm_vgic_unmap_phys_irq(vcpu, vtimer->irq.irq);
 }
 
 int kvm_timer_enable(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	struct irq_desc *desc;
 	struct irq_data *data;
 	int phys_irq;
@@ -498,7 +501,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 	 * Tell the VGIC that the virtual interrupt is tied to a
 	 * physical interrupt. We do that once per VCPU.
 	 */
-	ret = kvm_vgic_map_phys_irq(vcpu, timer->irq.irq, phys_irq);
+	ret = kvm_vgic_map_phys_irq(vcpu, vtimer->irq.irq, phys_irq);
 	if (ret)
 		return ret;
 
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index 63e28dd..0cf0895 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -25,11 +25,12 @@
 void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
 
 	if (timer->enabled) {
-		timer->cntv_ctl = read_sysreg_el0(cntv_ctl);
-		timer->cntv_cval = read_sysreg_el0(cntv_cval);
+		vtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);
+		vtimer->cnt_cval = read_sysreg_el0(cntv_cval);
 	}
 
 	/* Disable the virtual timer */
@@ -54,6 +55,7 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 {
 	struct kvm *kvm = kern_hyp_va(vcpu->kvm);
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
 
 	/* Those bits are already configured at boot on VHE-system */
@@ -70,8 +72,8 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 
 	if (timer->enabled) {
 		write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
-		write_sysreg_el0(timer->cntv_cval, cntv_cval);
+		write_sysreg_el0(vtimer->cnt_cval, cntv_cval);
 		isb();
-		write_sysreg_el0(timer->cntv_ctl, cntv_ctl);
+		write_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);
 	}
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 01/10] KVM: arm/arm64: Abstract virtual timer context into separate structure
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Abstract virtual timer context into a separate structure and change all
callers referring to timer registers, irq state and so on. No change in
functionality.

This is about to become very handy when adding the EL1 physical timer.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
---
 include/kvm/arm_arch_timer.h | 27 ++++++++---------
 virt/kvm/arm/arch_timer.c    | 69 +++++++++++++++++++++++---------------------
 virt/kvm/arm/hyp/timer-sr.c  | 10 ++++---
 3 files changed, 56 insertions(+), 50 deletions(-)

diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 5c970ce..daad3c1 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -28,15 +28,20 @@ struct arch_timer_kvm {
 	u64			cntvoff;
 };
 
-struct arch_timer_cpu {
+struct arch_timer_context {
 	/* Registers: control register, timer value */
-	u32				cntv_ctl;	/* Saved/restored */
-	u64				cntv_cval;	/* Saved/restored */
+	u32				cnt_ctl;
+	u64				cnt_cval;
+
+	/* Timer IRQ */
+	struct kvm_irq_level		irq;
+
+	/* Active IRQ state caching */
+	bool				active_cleared_last;
+};
 
-	/*
-	 * Anything that is not used directly from assembly code goes
-	 * here.
-	 */
+struct arch_timer_cpu {
+	struct arch_timer_context	vtimer;
 
 	/* Background timer used when the guest is not running */
 	struct hrtimer			timer;
@@ -47,12 +52,6 @@ struct arch_timer_cpu {
 	/* Background timer active */
 	bool				armed;
 
-	/* Timer IRQ */
-	struct kvm_irq_level		irq;
-
-	/* Active IRQ state caching */
-	bool				active_cleared_last;
-
 	/* Is the timer enabled */
 	bool			enabled;
 };
@@ -77,4 +76,6 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
 
 void kvm_timer_init_vhe(void);
+
+#define vcpu_vtimer(v)	(&(v)->arch.timer_cpu.vtimer)
 #endif
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 91ecf48..d3556b3 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -37,7 +37,7 @@
 
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
 {
-	vcpu->arch.timer_cpu.active_cleared_last = false;
+	vcpu_vtimer(vcpu)->active_cleared_last = false;
 }
 
 static u64 kvm_phys_timer_read(void)
@@ -102,7 +102,7 @@ static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
 {
 	u64 cval, now;
 
-	cval = vcpu->arch.timer_cpu.cntv_cval;
+	cval = vcpu_vtimer(vcpu)->cnt_cval;
 	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 
 	if (now < cval) {
@@ -144,21 +144,21 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 
 static bool kvm_timer_irq_can_fire(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	return !(timer->cntv_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(timer->cntv_ctl & ARCH_TIMER_CTRL_ENABLE);
+	return !(vtimer->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(vtimer->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
 }
 
 bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 cval, now;
 
 	if (!kvm_timer_irq_can_fire(vcpu))
 		return false;
 
-	cval = timer->cntv_cval;
+	cval = vtimer->cnt_cval;
 	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 
 	return cval <= now;
@@ -167,18 +167,18 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
 {
 	int ret;
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(!vgic_initialized(vcpu->kvm));
 
-	timer->active_cleared_last = false;
-	timer->irq.level = new_level;
-	trace_kvm_timer_update_irq(vcpu->vcpu_id, timer->irq.irq,
-				   timer->irq.level);
+	vtimer->active_cleared_last = false;
+	vtimer->irq.level = new_level;
+	trace_kvm_timer_update_irq(vcpu->vcpu_id, vtimer->irq.irq,
+				   vtimer->irq.level);
 
 	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
-					 timer->irq.irq,
-					 timer->irq.level);
+					 vtimer->irq.irq,
+					 vtimer->irq.level);
 	WARN_ON(ret);
 }
 
@@ -189,18 +189,19 @@ static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
 static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	/*
 	 * If userspace modified the timer registers via SET_ONE_REG before
-	 * the vgic was initialized, we mustn't set the timer->irq.level value
+	 * the vgic was initialized, we mustn't set the vtimer->irq.level value
 	 * because the guest would never see the interrupt.  Instead wait
 	 * until we call this function from kvm_timer_flush_hwstate.
 	 */
 	if (!vgic_initialized(vcpu->kvm) || !timer->enabled)
 		return -ENODEV;
 
-	if (kvm_timer_should_fire(vcpu) != timer->irq.level)
-		kvm_timer_update_irq(vcpu, !timer->irq.level);
+	if (kvm_timer_should_fire(vcpu) != vtimer->irq.level)
+		kvm_timer_update_irq(vcpu, !vtimer->irq.level);
 
 	return 0;
 }
@@ -250,7 +251,7 @@ void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
  */
 void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	bool phys_active;
 	int ret;
 
@@ -274,8 +275,8 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	* to ensure that hardware interrupts from the timer triggers a guest
 	* exit.
 	*/
-	phys_active = timer->irq.level ||
-			kvm_vgic_map_is_active(vcpu, timer->irq.irq);
+	phys_active = vtimer->irq.level ||
+			kvm_vgic_map_is_active(vcpu, vtimer->irq.irq);
 
 	/*
 	 * We want to avoid hitting the (re)distributor as much as
@@ -297,7 +298,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	 * - cached value is "active clear"
 	 * - value to be programmed is "active clear"
 	 */
-	if (timer->active_cleared_last && !phys_active)
+	if (vtimer->active_cleared_last && !phys_active)
 		return;
 
 	ret = irq_set_irqchip_state(host_vtimer_irq,
@@ -305,7 +306,7 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 				    phys_active);
 	WARN_ON(ret);
 
-	timer->active_cleared_last = !phys_active;
+	vtimer->active_cleared_last = !phys_active;
 }
 
 /**
@@ -331,7 +332,7 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 			 const struct kvm_irq_level *irq)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	/*
 	 * The vcpu timer irq number cannot be determined in
@@ -339,7 +340,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * kvm_vcpu_set_target(). To handle this, we determine
 	 * vcpu timer irq number when the vcpu is reset.
 	 */
-	timer->irq.irq = irq->irq;
+	vtimer->irq.irq = irq->irq;
 
 	/*
 	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
@@ -347,7 +348,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * resets the timer to be disabled and unmasked and is compliant with
 	 * the ARMv7 architecture.
 	 */
-	timer->cntv_ctl = 0;
+	vtimer->cnt_ctl = 0;
 	kvm_timer_update_state(vcpu);
 
 	return 0;
@@ -369,17 +370,17 @@ static void kvm_timer_init_interrupt(void *info)
 
 int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	switch (regid) {
 	case KVM_REG_ARM_TIMER_CTL:
-		timer->cntv_ctl = value;
+		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
 		vcpu->kvm->arch.timer.cntvoff = kvm_phys_timer_read() - value;
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
-		timer->cntv_cval = value;
+		vtimer->cnt_cval = value;
 		break;
 	default:
 		return -1;
@@ -391,15 +392,15 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 
 u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
 {
-	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	switch (regid) {
 	case KVM_REG_ARM_TIMER_CTL:
-		return timer->cntv_ctl;
+		return vtimer->cnt_ctl;
 	case KVM_REG_ARM_TIMER_CNT:
 		return kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
 	case KVM_REG_ARM_TIMER_CVAL:
-		return timer->cntv_cval;
+		return vtimer->cnt_cval;
 	}
 	return (u64)-1;
 }
@@ -463,14 +464,16 @@ int kvm_timer_hyp_init(void)
 void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	timer_disarm(timer);
-	kvm_vgic_unmap_phys_irq(vcpu, timer->irq.irq);
+	kvm_vgic_unmap_phys_irq(vcpu, vtimer->irq.irq);
 }
 
 int kvm_timer_enable(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	struct irq_desc *desc;
 	struct irq_data *data;
 	int phys_irq;
@@ -498,7 +501,7 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 	 * Tell the VGIC that the virtual interrupt is tied to a
 	 * physical interrupt. We do that once per VCPU.
 	 */
-	ret = kvm_vgic_map_phys_irq(vcpu, timer->irq.irq, phys_irq);
+	ret = kvm_vgic_map_phys_irq(vcpu, vtimer->irq.irq, phys_irq);
 	if (ret)
 		return ret;
 
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index 63e28dd..0cf0895 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -25,11 +25,12 @@
 void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
 
 	if (timer->enabled) {
-		timer->cntv_ctl = read_sysreg_el0(cntv_ctl);
-		timer->cntv_cval = read_sysreg_el0(cntv_cval);
+		vtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);
+		vtimer->cnt_cval = read_sysreg_el0(cntv_cval);
 	}
 
 	/* Disable the virtual timer */
@@ -54,6 +55,7 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 {
 	struct kvm *kvm = kern_hyp_va(vcpu->kvm);
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
 
 	/* Those bits are already configured at boot on VHE-system */
@@ -70,8 +72,8 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 
 	if (timer->enabled) {
 		write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
-		write_sysreg_el0(timer->cntv_cval, cntv_cval);
+		write_sysreg_el0(vtimer->cnt_cval, cntv_cval);
 		isb();
-		write_sysreg_el0(timer->cntv_ctl, cntv_ctl);
+		write_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);
 	}
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Make cntvoff per each timer context. This is helpful to abstract kvm
timer functions to work with timer context without considering timer
types (e.g. physical timer or virtual timer).

This also would pave the way for ever doing adjustments of the cntvoff
on a per-CPU basis if that should ever make sense.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/include/asm/kvm_host.h   |  3 ---
 arch/arm/kvm/arm.c                |  1 -
 arch/arm64/include/asm/kvm_host.h |  3 ---
 include/kvm/arm_arch_timer.h      |  9 +++------
 virt/kvm/arm/arch_timer.c         | 31 +++++++++++++++++++++----------
 virt/kvm/arm/hyp/timer-sr.c       |  3 +--
 6 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index d5423ab..cc495d79 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -60,9 +60,6 @@ struct kvm_arch {
 	/* The last vcpu id that ran on each physical CPU */
 	int __percpu *last_vcpu_ran;
 
-	/* Timer */
-	struct arch_timer_kvm	timer;
-
 	/*
 	 * Anything that is not used directly from assembly code goes
 	 * here.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 9d74464..f93f2171 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -135,7 +135,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		goto out_free_stage2_pgd;
 
 	kvm_vgic_early_init(kvm);
-	kvm_timer_init(kvm);
 
 	/* Mark the initial VMID generation invalid */
 	kvm->arch.vmid_gen = 0;
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e505038..4a758cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -70,9 +70,6 @@ struct kvm_arch {
 
 	/* Interrupt controller */
 	struct vgic_dist	vgic;
-
-	/* Timer */
-	struct arch_timer_kvm	timer;
 };
 
 #define KVM_NR_MEM_OBJS     40
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index daad3c1..2c8560b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -23,11 +23,6 @@
 #include <linux/hrtimer.h>
 #include <linux/workqueue.h>
 
-struct arch_timer_kvm {
-	/* Virtual offset */
-	u64			cntvoff;
-};
-
 struct arch_timer_context {
 	/* Registers: control register, timer value */
 	u32				cnt_ctl;
@@ -38,6 +33,9 @@ struct arch_timer_context {
 
 	/* Active IRQ state caching */
 	bool				active_cleared_last;
+
+	/* Virtual offset */
+	u64			cntvoff;
 };
 
 struct arch_timer_cpu {
@@ -58,7 +56,6 @@ struct arch_timer_cpu {
 
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
-void kvm_timer_init(struct kvm *kvm);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 			 const struct kvm_irq_level *irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index d3556b3..19fbaaf 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -101,9 +101,10 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
 {
 	u64 cval, now;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vcpu_vtimer(vcpu)->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	cval = vtimer->cnt_cval;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -159,7 +160,7 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 		return false;
 
 	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	return cval <= now;
 }
@@ -354,10 +355,25 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+/* Make the updates of cntvoff for all vtimer contexts atomic */
+static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+{
+	int i;
+	struct kvm_vcpu *vcpu;
+
+	mutex_lock(&kvm->lock);
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	mutex_unlock(&kvm->lock);
+}
+
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
+	/* Synchronize cntvoff across all vtimers of a VM. */
+	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
 	timer->timer.function = kvm_timer_expire;
@@ -377,7 +393,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		vcpu->kvm->arch.timer.cntvoff = kvm_phys_timer_read() - value;
+		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;
@@ -398,7 +414,7 @@ u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
 	case KVM_REG_ARM_TIMER_CTL:
 		return vtimer->cnt_ctl;
 	case KVM_REG_ARM_TIMER_CNT:
-		return kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+		return kvm_phys_timer_read() - vtimer->cntvoff;
 	case KVM_REG_ARM_TIMER_CVAL:
 		return vtimer->cnt_cval;
 	}
@@ -510,11 +526,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-void kvm_timer_init(struct kvm *kvm)
-{
-	kvm->arch.timer.cntvoff = kvm_phys_timer_read();
-}
-
 /*
  * On VHE system, we only need to configure trap on physical timer and counter
  * accesses in EL0 and EL1 once, not for every world switch.
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index 0cf0895..4734915 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -53,7 +53,6 @@ void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
 
 void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 {
-	struct kvm *kvm = kern_hyp_va(vcpu->kvm);
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
@@ -71,7 +70,7 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 	}
 
 	if (timer->enabled) {
-		write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
+		write_sysreg(vtimer->cntvoff, cntvoff_el2);
 		write_sysreg_el0(vtimer->cnt_cval, cntv_cval);
 		isb();
 		write_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Make cntvoff per each timer context. This is helpful to abstract kvm
timer functions to work with timer context without considering timer
types (e.g. physical timer or virtual timer).

This also would pave the way for ever doing adjustments of the cntvoff
on a per-CPU basis if that should ever make sense.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/include/asm/kvm_host.h   |  3 ---
 arch/arm/kvm/arm.c                |  1 -
 arch/arm64/include/asm/kvm_host.h |  3 ---
 include/kvm/arm_arch_timer.h      |  9 +++------
 virt/kvm/arm/arch_timer.c         | 31 +++++++++++++++++++++----------
 virt/kvm/arm/hyp/timer-sr.c       |  3 +--
 6 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index d5423ab..cc495d79 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -60,9 +60,6 @@ struct kvm_arch {
 	/* The last vcpu id that ran on each physical CPU */
 	int __percpu *last_vcpu_ran;
 
-	/* Timer */
-	struct arch_timer_kvm	timer;
-
 	/*
 	 * Anything that is not used directly from assembly code goes
 	 * here.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 9d74464..f93f2171 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -135,7 +135,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		goto out_free_stage2_pgd;
 
 	kvm_vgic_early_init(kvm);
-	kvm_timer_init(kvm);
 
 	/* Mark the initial VMID generation invalid */
 	kvm->arch.vmid_gen = 0;
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e505038..4a758cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -70,9 +70,6 @@ struct kvm_arch {
 
 	/* Interrupt controller */
 	struct vgic_dist	vgic;
-
-	/* Timer */
-	struct arch_timer_kvm	timer;
 };
 
 #define KVM_NR_MEM_OBJS     40
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index daad3c1..2c8560b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -23,11 +23,6 @@
 #include <linux/hrtimer.h>
 #include <linux/workqueue.h>
 
-struct arch_timer_kvm {
-	/* Virtual offset */
-	u64			cntvoff;
-};
-
 struct arch_timer_context {
 	/* Registers: control register, timer value */
 	u32				cnt_ctl;
@@ -38,6 +33,9 @@ struct arch_timer_context {
 
 	/* Active IRQ state caching */
 	bool				active_cleared_last;
+
+	/* Virtual offset */
+	u64			cntvoff;
 };
 
 struct arch_timer_cpu {
@@ -58,7 +56,6 @@ struct arch_timer_cpu {
 
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
-void kvm_timer_init(struct kvm *kvm);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 			 const struct kvm_irq_level *irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index d3556b3..19fbaaf 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -101,9 +101,10 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
 {
 	u64 cval, now;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vcpu_vtimer(vcpu)->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	cval = vtimer->cnt_cval;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -159,7 +160,7 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 		return false;
 
 	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	return cval <= now;
 }
@@ -354,10 +355,25 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+/* Make the updates of cntvoff for all vtimer contexts atomic */
+static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+{
+	int i;
+	struct kvm_vcpu *vcpu;
+
+	mutex_lock(&kvm->lock);
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	mutex_unlock(&kvm->lock);
+}
+
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
+	/* Synchronize cntvoff across all vtimers of a VM. */
+	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
 	timer->timer.function = kvm_timer_expire;
@@ -377,7 +393,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		vcpu->kvm->arch.timer.cntvoff = kvm_phys_timer_read() - value;
+		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;
@@ -398,7 +414,7 @@ u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
 	case KVM_REG_ARM_TIMER_CTL:
 		return vtimer->cnt_ctl;
 	case KVM_REG_ARM_TIMER_CNT:
-		return kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+		return kvm_phys_timer_read() - vtimer->cntvoff;
 	case KVM_REG_ARM_TIMER_CVAL:
 		return vtimer->cnt_cval;
 	}
@@ -510,11 +526,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-void kvm_timer_init(struct kvm *kvm)
-{
-	kvm->arch.timer.cntvoff = kvm_phys_timer_read();
-}
-
 /*
  * On VHE system, we only need to configure trap on physical timer and counter
  * accesses in EL0 and EL1 once, not for every world switch.
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index 0cf0895..4734915 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -53,7 +53,6 @@ void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
 
 void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 {
-	struct kvm *kvm = kern_hyp_va(vcpu->kvm);
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
@@ -71,7 +70,7 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 	}
 
 	if (timer->enabled) {
-		write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
+		write_sysreg(vtimer->cntvoff, cntvoff_el2);
 		write_sysreg_el0(vtimer->cnt_cval, cntv_cval);
 		isb();
 		write_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Make cntvoff per each timer context. This is helpful to abstract kvm
timer functions to work with timer context without considering timer
types (e.g. physical timer or virtual timer).

This also would pave the way for ever doing adjustments of the cntvoff
on a per-CPU basis if that should ever make sense.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/include/asm/kvm_host.h   |  3 ---
 arch/arm/kvm/arm.c                |  1 -
 arch/arm64/include/asm/kvm_host.h |  3 ---
 include/kvm/arm_arch_timer.h      |  9 +++------
 virt/kvm/arm/arch_timer.c         | 31 +++++++++++++++++++++----------
 virt/kvm/arm/hyp/timer-sr.c       |  3 +--
 6 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index d5423ab..cc495d79 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -60,9 +60,6 @@ struct kvm_arch {
 	/* The last vcpu id that ran on each physical CPU */
 	int __percpu *last_vcpu_ran;
 
-	/* Timer */
-	struct arch_timer_kvm	timer;
-
 	/*
 	 * Anything that is not used directly from assembly code goes
 	 * here.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 9d74464..f93f2171 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -135,7 +135,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		goto out_free_stage2_pgd;
 
 	kvm_vgic_early_init(kvm);
-	kvm_timer_init(kvm);
 
 	/* Mark the initial VMID generation invalid */
 	kvm->arch.vmid_gen = 0;
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e505038..4a758cb 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -70,9 +70,6 @@ struct kvm_arch {
 
 	/* Interrupt controller */
 	struct vgic_dist	vgic;
-
-	/* Timer */
-	struct arch_timer_kvm	timer;
 };
 
 #define KVM_NR_MEM_OBJS     40
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index daad3c1..2c8560b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -23,11 +23,6 @@
 #include <linux/hrtimer.h>
 #include <linux/workqueue.h>
 
-struct arch_timer_kvm {
-	/* Virtual offset */
-	u64			cntvoff;
-};
-
 struct arch_timer_context {
 	/* Registers: control register, timer value */
 	u32				cnt_ctl;
@@ -38,6 +33,9 @@ struct arch_timer_context {
 
 	/* Active IRQ state caching */
 	bool				active_cleared_last;
+
+	/* Virtual offset */
+	u64			cntvoff;
 };
 
 struct arch_timer_cpu {
@@ -58,7 +56,6 @@ struct arch_timer_cpu {
 
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
-void kvm_timer_init(struct kvm *kvm);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 			 const struct kvm_irq_level *irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index d3556b3..19fbaaf 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -101,9 +101,10 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
 {
 	u64 cval, now;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vcpu_vtimer(vcpu)->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	cval = vtimer->cnt_cval;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -159,7 +160,7 @@ bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
 		return false;
 
 	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+	now = kvm_phys_timer_read() - vtimer->cntvoff;
 
 	return cval <= now;
 }
@@ -354,10 +355,25 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
+/* Make the updates of cntvoff for all vtimer contexts atomic */
+static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+{
+	int i;
+	struct kvm_vcpu *vcpu;
+
+	mutex_lock(&kvm->lock);
+	kvm_for_each_vcpu(i, vcpu, kvm)
+		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	mutex_unlock(&kvm->lock);
+}
+
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
+	/* Synchronize cntvoff across all vtimers of a VM. */
+	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
 	timer->timer.function = kvm_timer_expire;
@@ -377,7 +393,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		vcpu->kvm->arch.timer.cntvoff = kvm_phys_timer_read() - value;
+		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;
@@ -398,7 +414,7 @@ u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
 	case KVM_REG_ARM_TIMER_CTL:
 		return vtimer->cnt_ctl;
 	case KVM_REG_ARM_TIMER_CNT:
-		return kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+		return kvm_phys_timer_read() - vtimer->cntvoff;
 	case KVM_REG_ARM_TIMER_CVAL:
 		return vtimer->cnt_cval;
 	}
@@ -510,11 +526,6 @@ int kvm_timer_enable(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-void kvm_timer_init(struct kvm *kvm)
-{
-	kvm->arch.timer.cntvoff = kvm_phys_timer_read();
-}
-
 /*
  * On VHE system, we only need to configure trap on physical timer and counter
  * accesses in EL0 and EL1 once, not for every world switch.
diff --git a/virt/kvm/arm/hyp/timer-sr.c b/virt/kvm/arm/hyp/timer-sr.c
index 0cf0895..4734915 100644
--- a/virt/kvm/arm/hyp/timer-sr.c
+++ b/virt/kvm/arm/hyp/timer-sr.c
@@ -53,7 +53,6 @@ void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
 
 void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 {
-	struct kvm *kvm = kern_hyp_va(vcpu->kvm);
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 val;
@@ -71,7 +70,7 @@ void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
 	}
 
 	if (timer->enabled) {
-		write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
+		write_sysreg(vtimer->cntvoff, cntvoff_el2);
 		write_sysreg_el0(vtimer->cnt_cval, cntv_cval);
 		isb();
 		write_sysreg_el0(vtimer->cnt_ctl, cntv_ctl);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Now that we have a separate structure for timer context, make functions
generic so that they can work with any timer context, not just the
virtual timer context.  This does not change the virtual timer
functionality.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/arm.c           |  2 +-
 include/kvm/arm_arch_timer.h |  2 +-
 virt/kvm/arm/arch_timer.c    | 54 ++++++++++++++++++++------------------------
 3 files changed, 27 insertions(+), 31 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index f93f2171..0ecd6cf 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu);
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 2c8560b..f46fa3b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -66,7 +66,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 u64 kvm_arm_timer_get_reg(struct kvm_vcpu *, u64 regid);
 int kvm_arm_timer_set_reg(struct kvm_vcpu *, u64 regid, u64 value);
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu);
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx);
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 19fbaaf..c42bca5 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -98,13 +98,12 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 	kvm_vcpu_kick(vcpu);
 }
 
-static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
+static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -133,7 +132,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu);
+	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -143,43 +142,39 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct kvm_vcpu *vcpu)
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-
-	return !(vtimer->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(vtimer->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
 }
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 cval, now;
 
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(timer_ctx))
 		return false;
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	return cval <= now;
 }
 
-static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
+static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
+				 struct arch_timer_context *timer_ctx)
 {
 	int ret;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(!vgic_initialized(vcpu->kvm));
 
-	vtimer->active_cleared_last = false;
-	vtimer->irq.level = new_level;
-	trace_kvm_timer_update_irq(vcpu->vcpu_id, vtimer->irq.irq,
-				   vtimer->irq.level);
+	timer_ctx->active_cleared_last = false;
+	timer_ctx->irq.level = new_level;
+	trace_kvm_timer_update_irq(vcpu->vcpu_id, timer_ctx->irq.irq,
+				   timer_ctx->irq.level);
 
-	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
-					 vtimer->irq.irq,
-					 vtimer->irq.level);
+	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id, timer_ctx->irq.irq,
+				  timer_ctx->irq.level);
 	WARN_ON(ret);
 }
 
@@ -201,8 +196,8 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (!vgic_initialized(vcpu->kvm) || !timer->enabled)
 		return -ENODEV;
 
-	if (kvm_timer_should_fire(vcpu) != vtimer->irq.level)
-		kvm_timer_update_irq(vcpu, !vtimer->irq.level);
+	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
+		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
 	return 0;
 }
@@ -215,6 +210,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
@@ -223,18 +219,18 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vcpu))
+	if (kvm_timer_should_fire(vtimer))
 		return;
 
 	/*
 	 * If the timer is not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(vtimer))
 		return;
 
 	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vcpu));
+	timer_arm(timer, kvm_timer_compute_delta(vtimer));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Now that we have a separate structure for timer context, make functions
generic so that they can work with any timer context, not just the
virtual timer context.  This does not change the virtual timer
functionality.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/arm.c           |  2 +-
 include/kvm/arm_arch_timer.h |  2 +-
 virt/kvm/arm/arch_timer.c    | 54 ++++++++++++++++++++------------------------
 3 files changed, 27 insertions(+), 31 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index f93f2171..0ecd6cf 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu);
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 2c8560b..f46fa3b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -66,7 +66,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 u64 kvm_arm_timer_get_reg(struct kvm_vcpu *, u64 regid);
 int kvm_arm_timer_set_reg(struct kvm_vcpu *, u64 regid, u64 value);
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu);
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx);
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 19fbaaf..c42bca5 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -98,13 +98,12 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 	kvm_vcpu_kick(vcpu);
 }
 
-static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
+static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -133,7 +132,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu);
+	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -143,43 +142,39 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct kvm_vcpu *vcpu)
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-
-	return !(vtimer->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(vtimer->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
 }
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 cval, now;
 
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(timer_ctx))
 		return false;
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	return cval <= now;
 }
 
-static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
+static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
+				 struct arch_timer_context *timer_ctx)
 {
 	int ret;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(!vgic_initialized(vcpu->kvm));
 
-	vtimer->active_cleared_last = false;
-	vtimer->irq.level = new_level;
-	trace_kvm_timer_update_irq(vcpu->vcpu_id, vtimer->irq.irq,
-				   vtimer->irq.level);
+	timer_ctx->active_cleared_last = false;
+	timer_ctx->irq.level = new_level;
+	trace_kvm_timer_update_irq(vcpu->vcpu_id, timer_ctx->irq.irq,
+				   timer_ctx->irq.level);
 
-	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
-					 vtimer->irq.irq,
-					 vtimer->irq.level);
+	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id, timer_ctx->irq.irq,
+				  timer_ctx->irq.level);
 	WARN_ON(ret);
 }
 
@@ -201,8 +196,8 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (!vgic_initialized(vcpu->kvm) || !timer->enabled)
 		return -ENODEV;
 
-	if (kvm_timer_should_fire(vcpu) != vtimer->irq.level)
-		kvm_timer_update_irq(vcpu, !vtimer->irq.level);
+	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
+		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
 	return 0;
 }
@@ -215,6 +210,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
@@ -223,18 +219,18 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vcpu))
+	if (kvm_timer_should_fire(vtimer))
 		return;
 
 	/*
 	 * If the timer is not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(vtimer))
 		return;
 
 	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vcpu));
+	timer_arm(timer, kvm_timer_compute_delta(vtimer));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Now that we have a separate structure for timer context, make functions
generic so that they can work with any timer context, not just the
virtual timer context.  This does not change the virtual timer
functionality.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
---
 arch/arm/kvm/arm.c           |  2 +-
 include/kvm/arm_arch_timer.h |  2 +-
 virt/kvm/arm/arch_timer.c    | 54 ++++++++++++++++++++------------------------
 3 files changed, 27 insertions(+), 31 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index f93f2171..0ecd6cf 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,7 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu);
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 2c8560b..f46fa3b 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -66,7 +66,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 u64 kvm_arm_timer_get_reg(struct kvm_vcpu *, u64 regid);
 int kvm_arm_timer_set_reg(struct kvm_vcpu *, u64 regid, u64 value);
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu);
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx);
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 19fbaaf..c42bca5 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -98,13 +98,12 @@ static void kvm_timer_inject_irq_work(struct work_struct *work)
 	kvm_vcpu_kick(vcpu);
 }
 
-static u64 kvm_timer_compute_delta(struct kvm_vcpu *vcpu)
+static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	if (now < cval) {
 		u64 ns;
@@ -133,7 +132,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu);
+	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -143,43 +142,39 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct kvm_vcpu *vcpu)
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
-
-	return !(vtimer->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(vtimer->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
 }
 
-bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
+bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 	u64 cval, now;
 
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(timer_ctx))
 		return false;
 
-	cval = vtimer->cnt_cval;
-	now = kvm_phys_timer_read() - vtimer->cntvoff;
+	cval = timer_ctx->cnt_cval;
+	now = kvm_phys_timer_read() - timer_ctx->cntvoff;
 
 	return cval <= now;
 }
 
-static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level)
+static void kvm_timer_update_irq(struct kvm_vcpu *vcpu, bool new_level,
+				 struct arch_timer_context *timer_ctx)
 {
 	int ret;
-	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(!vgic_initialized(vcpu->kvm));
 
-	vtimer->active_cleared_last = false;
-	vtimer->irq.level = new_level;
-	trace_kvm_timer_update_irq(vcpu->vcpu_id, vtimer->irq.irq,
-				   vtimer->irq.level);
+	timer_ctx->active_cleared_last = false;
+	timer_ctx->irq.level = new_level;
+	trace_kvm_timer_update_irq(vcpu->vcpu_id, timer_ctx->irq.irq,
+				   timer_ctx->irq.level);
 
-	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id,
-					 vtimer->irq.irq,
-					 vtimer->irq.level);
+	ret = kvm_vgic_inject_irq(vcpu->kvm, vcpu->vcpu_id, timer_ctx->irq.irq,
+				  timer_ctx->irq.level);
 	WARN_ON(ret);
 }
 
@@ -201,8 +196,8 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (!vgic_initialized(vcpu->kvm) || !timer->enabled)
 		return -ENODEV;
 
-	if (kvm_timer_should_fire(vcpu) != vtimer->irq.level)
-		kvm_timer_update_irq(vcpu, !vtimer->irq.level);
+	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
+		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
 	return 0;
 }
@@ -215,6 +210,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
@@ -223,18 +219,18 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vcpu))
+	if (kvm_timer_should_fire(vtimer))
 		return;
 
 	/*
 	 * If the timer is not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vcpu))
+	if (!kvm_timer_irq_can_fire(vtimer))
 		return;
 
 	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vcpu));
+	timer_arm(timer, kvm_timer_compute_delta(vtimer));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Add the EL1 physical timer context.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 include/kvm/arm_arch_timer.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f46fa3b..6445a3d 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -40,6 +40,7 @@ struct arch_timer_context {
 
 struct arch_timer_cpu {
 	struct arch_timer_context	vtimer;
+	struct arch_timer_context	ptimer;
 
 	/* Background timer used when the guest is not running */
 	struct hrtimer			timer;
@@ -75,4 +76,5 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_init_vhe(void);
 
 #define vcpu_vtimer(v)	(&(v)->arch.timer_cpu.vtimer)
+#define vcpu_ptimer(v)	(&(v)->arch.timer_cpu.ptimer)
 #endif
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Add the EL1 physical timer context.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 include/kvm/arm_arch_timer.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f46fa3b..6445a3d 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -40,6 +40,7 @@ struct arch_timer_context {
 
 struct arch_timer_cpu {
 	struct arch_timer_context	vtimer;
+	struct arch_timer_context	ptimer;
 
 	/* Background timer used when the guest is not running */
 	struct hrtimer			timer;
@@ -75,4 +76,5 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_init_vhe(void);
 
 #define vcpu_vtimer(v)	(&(v)->arch.timer_cpu.vtimer)
+#define vcpu_ptimer(v)	(&(v)->arch.timer_cpu.ptimer)
 #endif
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Add the EL1 physical timer context.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 include/kvm/arm_arch_timer.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f46fa3b..6445a3d 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -40,6 +40,7 @@ struct arch_timer_context {
 
 struct arch_timer_cpu {
 	struct arch_timer_context	vtimer;
+	struct arch_timer_context	ptimer;
 
 	/* Background timer used when the guest is not running */
 	struct hrtimer			timer;
@@ -75,4 +76,5 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_init_vhe(void);
 
 #define vcpu_vtimer(v)	(&(v)->arch.timer_cpu.vtimer)
+#define vcpu_ptimer(v)	(&(v)->arch.timer_cpu.ptimer)
 #endif
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Initialize the emulated EL1 physical timer with the default irq number.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/reset.c         | 9 ++++++++-
 arch/arm64/kvm/reset.c       | 9 ++++++++-
 include/kvm/arm_arch_timer.h | 3 ++-
 virt/kvm/arm/arch_timer.c    | 9 +++++++--
 4 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
index 4b5e802..1da8b2d 100644
--- a/arch/arm/kvm/reset.c
+++ b/arch/arm/kvm/reset.c
@@ -37,6 +37,11 @@
 	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
 };
 
+static const struct kvm_irq_level cortexa_ptimer_irq = {
+	{ .irq = 30 },
+	.level = 1,
+};
+
 static const struct kvm_irq_level cortexa_vtimer_irq = {
 	{ .irq = 27 },
 	.level = 1,
@@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct kvm_regs *reset_regs;
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 
 	switch (vcpu->arch.target) {
 	case KVM_ARM_TARGET_CORTEX_A7:
@@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		reset_regs = &cortexa_regs_reset;
 		vcpu->arch.midr = read_cpuid_id();
 		cpu_vtimer_irq = &cortexa_vtimer_irq;
+		cpu_ptimer_irq = &cortexa_ptimer_irq;
 		break;
 	default:
 		return -ENODEV;
@@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_reset_coprocs(vcpu);
 
 	/* Reset arch_timer context */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index e95d4f6..d9e9697 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -46,6 +46,11 @@
 			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
 };
 
+static const struct kvm_irq_level default_ptimer_irq = {
+	.irq	= 30,
+	.level	= 1,
+};
+
 static const struct kvm_irq_level default_vtimer_irq = {
 	.irq	= 27,
 	.level	= 1,
@@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
 int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 	const struct kvm_regs *cpu_reset;
 
 	switch (vcpu->arch.target) {
@@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		}
 
 		cpu_vtimer_irq = &default_vtimer_irq;
+		cpu_ptimer_irq = &default_ptimer_irq;
 		break;
 	}
 
@@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_pmu_vcpu_reset(vcpu);
 
 	/* Reset timer */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 6445a3d..f1d2fba0 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -58,7 +58,8 @@ struct arch_timer_cpu {
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq);
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
 void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
 void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index c42bca5..ae38703 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 }
 
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq)
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq)
 {
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * The vcpu timer irq number cannot be determined in
@@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * kvm_vcpu_set_target(). To handle this, we determine
 	 * vcpu timer irq number when the vcpu is reset.
 	 */
-	vtimer->irq.irq = irq->irq;
+	vtimer->irq.irq = virt_irq->irq;
+	ptimer->irq.irq = phys_irq->irq;
 
 	/*
 	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
@@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * the ARMv7 architecture.
 	 */
 	vtimer->cnt_ctl = 0;
+	ptimer->cnt_ctl = 0;
 	kvm_timer_update_state(vcpu);
 
 	return 0;
@@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
 	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Initialize the emulated EL1 physical timer with the default irq number.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/reset.c         | 9 ++++++++-
 arch/arm64/kvm/reset.c       | 9 ++++++++-
 include/kvm/arm_arch_timer.h | 3 ++-
 virt/kvm/arm/arch_timer.c    | 9 +++++++--
 4 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
index 4b5e802..1da8b2d 100644
--- a/arch/arm/kvm/reset.c
+++ b/arch/arm/kvm/reset.c
@@ -37,6 +37,11 @@
 	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
 };
 
+static const struct kvm_irq_level cortexa_ptimer_irq = {
+	{ .irq = 30 },
+	.level = 1,
+};
+
 static const struct kvm_irq_level cortexa_vtimer_irq = {
 	{ .irq = 27 },
 	.level = 1,
@@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct kvm_regs *reset_regs;
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 
 	switch (vcpu->arch.target) {
 	case KVM_ARM_TARGET_CORTEX_A7:
@@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		reset_regs = &cortexa_regs_reset;
 		vcpu->arch.midr = read_cpuid_id();
 		cpu_vtimer_irq = &cortexa_vtimer_irq;
+		cpu_ptimer_irq = &cortexa_ptimer_irq;
 		break;
 	default:
 		return -ENODEV;
@@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_reset_coprocs(vcpu);
 
 	/* Reset arch_timer context */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index e95d4f6..d9e9697 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -46,6 +46,11 @@
 			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
 };
 
+static const struct kvm_irq_level default_ptimer_irq = {
+	.irq	= 30,
+	.level	= 1,
+};
+
 static const struct kvm_irq_level default_vtimer_irq = {
 	.irq	= 27,
 	.level	= 1,
@@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
 int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 	const struct kvm_regs *cpu_reset;
 
 	switch (vcpu->arch.target) {
@@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		}
 
 		cpu_vtimer_irq = &default_vtimer_irq;
+		cpu_ptimer_irq = &default_ptimer_irq;
 		break;
 	}
 
@@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_pmu_vcpu_reset(vcpu);
 
 	/* Reset timer */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 6445a3d..f1d2fba0 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -58,7 +58,8 @@ struct arch_timer_cpu {
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq);
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
 void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
 void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index c42bca5..ae38703 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 }
 
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq)
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq)
 {
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * The vcpu timer irq number cannot be determined in
@@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * kvm_vcpu_set_target(). To handle this, we determine
 	 * vcpu timer irq number when the vcpu is reset.
 	 */
-	vtimer->irq.irq = irq->irq;
+	vtimer->irq.irq = virt_irq->irq;
+	ptimer->irq.irq = phys_irq->irq;
 
 	/*
 	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
@@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * the ARMv7 architecture.
 	 */
 	vtimer->cnt_ctl = 0;
+	ptimer->cnt_ctl = 0;
 	kvm_timer_update_state(vcpu);
 
 	return 0;
@@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
 	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Initialize the emulated EL1 physical timer with the default irq number.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/reset.c         | 9 ++++++++-
 arch/arm64/kvm/reset.c       | 9 ++++++++-
 include/kvm/arm_arch_timer.h | 3 ++-
 virt/kvm/arm/arch_timer.c    | 9 +++++++--
 4 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
index 4b5e802..1da8b2d 100644
--- a/arch/arm/kvm/reset.c
+++ b/arch/arm/kvm/reset.c
@@ -37,6 +37,11 @@
 	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
 };
 
+static const struct kvm_irq_level cortexa_ptimer_irq = {
+	{ .irq = 30 },
+	.level = 1,
+};
+
 static const struct kvm_irq_level cortexa_vtimer_irq = {
 	{ .irq = 27 },
 	.level = 1,
@@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	struct kvm_regs *reset_regs;
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 
 	switch (vcpu->arch.target) {
 	case KVM_ARM_TARGET_CORTEX_A7:
@@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		reset_regs = &cortexa_regs_reset;
 		vcpu->arch.midr = read_cpuid_id();
 		cpu_vtimer_irq = &cortexa_vtimer_irq;
+		cpu_ptimer_irq = &cortexa_ptimer_irq;
 		break;
 	default:
 		return -ENODEV;
@@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_reset_coprocs(vcpu);
 
 	/* Reset arch_timer context */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
index e95d4f6..d9e9697 100644
--- a/arch/arm64/kvm/reset.c
+++ b/arch/arm64/kvm/reset.c
@@ -46,6 +46,11 @@
 			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
 };
 
+static const struct kvm_irq_level default_ptimer_irq = {
+	.irq	= 30,
+	.level	= 1,
+};
+
 static const struct kvm_irq_level default_vtimer_irq = {
 	.irq	= 27,
 	.level	= 1,
@@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
 int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 {
 	const struct kvm_irq_level *cpu_vtimer_irq;
+	const struct kvm_irq_level *cpu_ptimer_irq;
 	const struct kvm_regs *cpu_reset;
 
 	switch (vcpu->arch.target) {
@@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 		}
 
 		cpu_vtimer_irq = &default_vtimer_irq;
+		cpu_ptimer_irq = &default_ptimer_irq;
 		break;
 	}
 
@@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
 	kvm_pmu_vcpu_reset(vcpu);
 
 	/* Reset timer */
-	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
+	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
 }
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index 6445a3d..f1d2fba0 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -58,7 +58,8 @@ struct arch_timer_cpu {
 int kvm_timer_hyp_init(void);
 int kvm_timer_enable(struct kvm_vcpu *vcpu);
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq);
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq);
 void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
 void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
 void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index c42bca5..ae38703 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 }
 
 int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
-			 const struct kvm_irq_level *irq)
+			 const struct kvm_irq_level *virt_irq,
+			 const struct kvm_irq_level *phys_irq)
 {
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * The vcpu timer irq number cannot be determined in
@@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * kvm_vcpu_set_target(). To handle this, we determine
 	 * vcpu timer irq number when the vcpu is reset.
 	 */
-	vtimer->irq.irq = irq->irq;
+	vtimer->irq.irq = virt_irq->irq;
+	ptimer->irq.irq = phys_irq->irq;
 
 	/*
 	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
@@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 	 * the ARMv7 architecture.
 	 */
 	vtimer->cnt_ctl = 0;
+	ptimer->cnt_ctl = 0;
 	kvm_timer_update_state(vcpu);
 
 	return 0;
@@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
 	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
 	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Now that we maintain the EL1 physical timer register states of VMs,
update the physical timer interrupt level along with the virtual one.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index ae38703..1b086fd6 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -186,6 +186,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * If userspace modified the timer registers via SET_ONE_REG before
@@ -199,6 +200,9 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
 		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
+	if (kvm_timer_should_fire(ptimer) != ptimer->irq.level)
+		kvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);
+
 	return 0;
 }
 
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Now that we maintain the EL1 physical timer register states of VMs,
update the physical timer interrupt level along with the virtual one.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index ae38703..1b086fd6 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -186,6 +186,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * If userspace modified the timer registers via SET_ONE_REG before
@@ -199,6 +200,9 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
 		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
+	if (kvm_timer_should_fire(ptimer) != ptimer->irq.level)
+		kvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);
+
 	return 0;
 }
 
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Now that we maintain the EL1 physical timer register states of VMs,
update the physical timer interrupt level along with the virtual one.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index ae38703..1b086fd6 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -186,6 +186,7 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	/*
 	 * If userspace modified the timer registers via SET_ONE_REG before
@@ -199,6 +200,9 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	if (kvm_timer_should_fire(vtimer) != vtimer->irq.level)
 		kvm_timer_update_irq(vcpu, !vtimer->irq.level, vtimer);
 
+	if (kvm_timer_should_fire(ptimer) != ptimer->irq.level)
+		kvm_timer_update_irq(vcpu, !ptimer->irq.level, ptimer);
+
 	return 0;
 }
 
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

When scheduling a background timer, consider both of the virtual and
physical timer and pick the earliest expiration time.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/arm.c        |  3 ++-
 virt/kvm/arm/arch_timer.c | 53 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 0ecd6cf..21c493a 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,8 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu)) ||
+	       kvm_timer_should_fire(vcpu_ptimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1b086fd6..89bdb79 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -118,6 +118,35 @@ static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 	return 0;
 }
 
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
+{
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+}
+
+/*
+ * Returns the earliest expiration time in ns among guest timers.
+ * Note that it will return 0 if none of timers can fire.
+ */
+static u64 kvm_timer_earliest_exp(struct kvm_vcpu *vcpu)
+{
+	u64 min_virt = ULLONG_MAX, min_phys = ULLONG_MAX;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (kvm_timer_irq_can_fire(vtimer))
+		min_virt = kvm_timer_compute_delta(vtimer);
+
+	if (kvm_timer_irq_can_fire(ptimer))
+		min_phys = kvm_timer_compute_delta(ptimer);
+
+	/* If none of timers can fire, then return 0 */
+	if ((min_virt == ULLONG_MAX) && (min_phys == ULLONG_MAX))
+		return 0;
+
+	return min(min_virt, min_phys);
+}
+
 static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 {
 	struct arch_timer_cpu *timer;
@@ -132,7 +161,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
+	ns = kvm_timer_earliest_exp(vcpu);
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -142,12 +171,6 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
-{
-	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
-}
-
 bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
@@ -215,26 +238,30 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
 	/*
-	 * No need to schedule a background timer if the guest timer has
+	 * No need to schedule a background timer if any guest timer has
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vtimer))
+	if (kvm_timer_should_fire(vtimer) || kvm_timer_should_fire(ptimer))
 		return;
 
 	/*
-	 * If the timer is not capable of raising interrupts (disabled or
+	 * If both timers are not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vtimer))
+	if (!kvm_timer_irq_can_fire(vtimer) && !kvm_timer_irq_can_fire(ptimer))
 		return;
 
-	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vtimer));
+	/*
+	 * The guest timers have not yet expired, schedule a background timer.
+	 * Set the earliest expiration time among the guest timers.
+	 */
+	timer_arm(timer, kvm_timer_earliest_exp(vcpu));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

When scheduling a background timer, consider both of the virtual and
physical timer and pick the earliest expiration time.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/arm.c        |  3 ++-
 virt/kvm/arm/arch_timer.c | 53 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 0ecd6cf..21c493a 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,8 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu)) ||
+	       kvm_timer_should_fire(vcpu_ptimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1b086fd6..89bdb79 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -118,6 +118,35 @@ static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 	return 0;
 }
 
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
+{
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+}
+
+/*
+ * Returns the earliest expiration time in ns among guest timers.
+ * Note that it will return 0 if none of timers can fire.
+ */
+static u64 kvm_timer_earliest_exp(struct kvm_vcpu *vcpu)
+{
+	u64 min_virt = ULLONG_MAX, min_phys = ULLONG_MAX;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (kvm_timer_irq_can_fire(vtimer))
+		min_virt = kvm_timer_compute_delta(vtimer);
+
+	if (kvm_timer_irq_can_fire(ptimer))
+		min_phys = kvm_timer_compute_delta(ptimer);
+
+	/* If none of timers can fire, then return 0 */
+	if ((min_virt == ULLONG_MAX) && (min_phys == ULLONG_MAX))
+		return 0;
+
+	return min(min_virt, min_phys);
+}
+
 static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 {
 	struct arch_timer_cpu *timer;
@@ -132,7 +161,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
+	ns = kvm_timer_earliest_exp(vcpu);
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -142,12 +171,6 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
-{
-	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
-}
-
 bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
@@ -215,26 +238,30 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
 	/*
-	 * No need to schedule a background timer if the guest timer has
+	 * No need to schedule a background timer if any guest timer has
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vtimer))
+	if (kvm_timer_should_fire(vtimer) || kvm_timer_should_fire(ptimer))
 		return;
 
 	/*
-	 * If the timer is not capable of raising interrupts (disabled or
+	 * If both timers are not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vtimer))
+	if (!kvm_timer_irq_can_fire(vtimer) && !kvm_timer_irq_can_fire(ptimer))
 		return;
 
-	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vtimer));
+	/*
+	 * The guest timers have not yet expired, schedule a background timer.
+	 * Set the earliest expiration time among the guest timers.
+	 */
+	timer_arm(timer, kvm_timer_earliest_exp(vcpu));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

When scheduling a background timer, consider both of the virtual and
physical timer and pick the earliest expiration time.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm/kvm/arm.c        |  3 ++-
 virt/kvm/arm/arch_timer.c | 53 +++++++++++++++++++++++++++++++++++------------
 2 files changed, 42 insertions(+), 14 deletions(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 0ecd6cf..21c493a 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -300,7 +300,8 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu)
 
 int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 {
-	return kvm_timer_should_fire(vcpu_vtimer(vcpu));
+	return kvm_timer_should_fire(vcpu_vtimer(vcpu)) ||
+	       kvm_timer_should_fire(vcpu_ptimer(vcpu));
 }
 
 void kvm_arch_vcpu_blocking(struct kvm_vcpu *vcpu)
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1b086fd6..89bdb79 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -118,6 +118,35 @@ static u64 kvm_timer_compute_delta(struct arch_timer_context *timer_ctx)
 	return 0;
 }
 
+static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
+{
+	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
+		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
+}
+
+/*
+ * Returns the earliest expiration time in ns among guest timers.
+ * Note that it will return 0 if none of timers can fire.
+ */
+static u64 kvm_timer_earliest_exp(struct kvm_vcpu *vcpu)
+{
+	u64 min_virt = ULLONG_MAX, min_phys = ULLONG_MAX;
+	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (kvm_timer_irq_can_fire(vtimer))
+		min_virt = kvm_timer_compute_delta(vtimer);
+
+	if (kvm_timer_irq_can_fire(ptimer))
+		min_phys = kvm_timer_compute_delta(ptimer);
+
+	/* If none of timers can fire, then return 0 */
+	if ((min_virt == ULLONG_MAX) && (min_phys == ULLONG_MAX))
+		return 0;
+
+	return min(min_virt, min_phys);
+}
+
 static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 {
 	struct arch_timer_cpu *timer;
@@ -132,7 +161,7 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	 * PoV (NTP on the host may have forced it to expire
 	 * early). If we should have slept longer, restart it.
 	 */
-	ns = kvm_timer_compute_delta(vcpu_vtimer(vcpu));
+	ns = kvm_timer_earliest_exp(vcpu);
 	if (unlikely(ns)) {
 		hrtimer_forward_now(hrt, ns_to_ktime(ns));
 		return HRTIMER_RESTART;
@@ -142,12 +171,6 @@ static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
 	return HRTIMER_NORESTART;
 }
 
-static bool kvm_timer_irq_can_fire(struct arch_timer_context *timer_ctx)
-{
-	return !(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_IT_MASK) &&
-		(timer_ctx->cnt_ctl & ARCH_TIMER_CTRL_ENABLE);
-}
-
 bool kvm_timer_should_fire(struct arch_timer_context *timer_ctx)
 {
 	u64 cval, now;
@@ -215,26 +238,30 @@ void kvm_timer_schedule(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
 
 	BUG_ON(timer_is_armed(timer));
 
 	/*
-	 * No need to schedule a background timer if the guest timer has
+	 * No need to schedule a background timer if any guest timer has
 	 * already expired, because kvm_vcpu_block will return before putting
 	 * the thread to sleep.
 	 */
-	if (kvm_timer_should_fire(vtimer))
+	if (kvm_timer_should_fire(vtimer) || kvm_timer_should_fire(ptimer))
 		return;
 
 	/*
-	 * If the timer is not capable of raising interrupts (disabled or
+	 * If both timers are not capable of raising interrupts (disabled or
 	 * masked), then there's no more work for us to do.
 	 */
-	if (!kvm_timer_irq_can_fire(vtimer))
+	if (!kvm_timer_irq_can_fire(vtimer) && !kvm_timer_irq_can_fire(ptimer))
 		return;
 
-	/*  The timer has not yet expired, schedule a background timer */
-	timer_arm(timer, kvm_timer_compute_delta(vtimer));
+	/*
+	 * The guest timers have not yet expired, schedule a background timer.
+	 * Set the earliest expiration time among the guest timers.
+	 */
+	timer_arm(timer, kvm_timer_earliest_exp(vcpu));
 }
 
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu)
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Set a background timer for the EL1 physical timer emulation while VMs
are running, so that VMs get the physical timer interrupts in a timely
manner.

Schedule the background timer on entry to the VM and cancel it on exit.
This would not have any performance impact to the guest OSes that
currently use the virtual timer since the physical timer is always not
enabled.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 89bdb79..1806e5e 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -229,6 +229,22 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+/* Schedule the background timer for the emulated timer. */
+static void kvm_timer_emulate(struct kvm_vcpu *vcpu,
+			      struct arch_timer_context *timer_ctx)
+{
+	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+
+	if (kvm_timer_should_fire(timer_ctx))
+		return;
+
+	if (!kvm_timer_irq_can_fire(timer_ctx))
+		return;
+
+	/*  The timer has not yet expired, schedule a background timer */
+	timer_arm(timer, kvm_timer_compute_delta(timer_ctx));
+}
+
 /*
  * Schedule the background timer before calling kvm_vcpu_block, so that this
  * thread is removed from its waitqueue and made runnable when there's a timer
@@ -286,6 +302,9 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	if (kvm_timer_update_state(vcpu))
 		return;
 
+	/* Set the background timer for the physical timer emulation. */
+	kvm_timer_emulate(vcpu, vcpu_ptimer(vcpu));
+
 	/*
 	* If we enter the guest with the virtual input level to the VGIC
 	* asserted, then we have already told the VGIC what we need to, and
@@ -348,7 +367,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
-	BUG_ON(timer_is_armed(timer));
+	/*
+	 * This is to cancel the background timer for the physical timer
+	 * emulation if it is set.
+	 */
+	timer_disarm(timer);
 
 	/*
 	 * The guest could have modified the timer registers or the timer
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Set a background timer for the EL1 physical timer emulation while VMs
are running, so that VMs get the physical timer interrupts in a timely
manner.

Schedule the background timer on entry to the VM and cancel it on exit.
This would not have any performance impact to the guest OSes that
currently use the virtual timer since the physical timer is always not
enabled.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 89bdb79..1806e5e 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -229,6 +229,22 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+/* Schedule the background timer for the emulated timer. */
+static void kvm_timer_emulate(struct kvm_vcpu *vcpu,
+			      struct arch_timer_context *timer_ctx)
+{
+	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+
+	if (kvm_timer_should_fire(timer_ctx))
+		return;
+
+	if (!kvm_timer_irq_can_fire(timer_ctx))
+		return;
+
+	/*  The timer has not yet expired, schedule a background timer */
+	timer_arm(timer, kvm_timer_compute_delta(timer_ctx));
+}
+
 /*
  * Schedule the background timer before calling kvm_vcpu_block, so that this
  * thread is removed from its waitqueue and made runnable when there's a timer
@@ -286,6 +302,9 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	if (kvm_timer_update_state(vcpu))
 		return;
 
+	/* Set the background timer for the physical timer emulation. */
+	kvm_timer_emulate(vcpu, vcpu_ptimer(vcpu));
+
 	/*
 	* If we enter the guest with the virtual input level to the VGIC
 	* asserted, then we have already told the VGIC what we need to, and
@@ -348,7 +367,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
-	BUG_ON(timer_is_armed(timer));
+	/*
+	 * This is to cancel the background timer for the physical timer
+	 * emulation if it is set.
+	 */
+	timer_disarm(timer);
 
 	/*
 	 * The guest could have modified the timer registers or the timer
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Set a background timer for the EL1 physical timer emulation while VMs
are running, so that VMs get the physical timer interrupts in a timely
manner.

Schedule the background timer on entry to the VM and cancel it on exit.
This would not have any performance impact to the guest OSes that
currently use the virtual timer since the physical timer is always not
enabled.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 virt/kvm/arm/arch_timer.c | 25 ++++++++++++++++++++++++-
 1 file changed, 24 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 89bdb79..1806e5e 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -229,6 +229,22 @@ static int kvm_timer_update_state(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+/* Schedule the background timer for the emulated timer. */
+static void kvm_timer_emulate(struct kvm_vcpu *vcpu,
+			      struct arch_timer_context *timer_ctx)
+{
+	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+
+	if (kvm_timer_should_fire(timer_ctx))
+		return;
+
+	if (!kvm_timer_irq_can_fire(timer_ctx))
+		return;
+
+	/*  The timer has not yet expired, schedule a background timer */
+	timer_arm(timer, kvm_timer_compute_delta(timer_ctx));
+}
+
 /*
  * Schedule the background timer before calling kvm_vcpu_block, so that this
  * thread is removed from its waitqueue and made runnable when there's a timer
@@ -286,6 +302,9 @@ void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu)
 	if (kvm_timer_update_state(vcpu))
 		return;
 
+	/* Set the background timer for the physical timer emulation. */
+	kvm_timer_emulate(vcpu, vcpu_ptimer(vcpu));
+
 	/*
 	* If we enter the guest with the virtual input level to the VGIC
 	* asserted, then we have already told the VGIC what we need to, and
@@ -348,7 +367,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
 {
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
-	BUG_ON(timer_is_armed(timer));
+	/*
+	 * This is to cancel the background timer for the physical timer
+	 * emulation if it is set.
+	 */
+	timer_disarm(timer);
 
 	/*
 	 * The guest could have modified the timer registers or the timer
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
those traps. This results in terminating VMs. Instead, set a handler for
the EL1 phys timer access, and inject an undefined exception as an
intermediate step.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index caa47ce..1cd3464 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -820,6 +820,30 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	  CRm((0b1100 | (((n) >> 3) & 0x3))), Op2(((n) & 0x7)),		\
 	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+static bool access_cntp_tval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_cval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
 /*
  * Architected system registers.
  * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1029,6 +1053,16 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	{ Op0(0b11), Op1(0b011), CRn(0b1101), CRm(0b0000), Op2(0b011),
 	  NULL, reset_unknown, TPIDRRO_EL0 },
 
+	/* CNTP_TVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b000),
+	  access_cntp_tval },
+	/* CNTP_CTL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b001),
+	  access_cntp_ctl },
+	/* CNTP_CVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b010),
+	  access_cntp_cval },
+
 	/* PMEVCNTRn_EL0 */
 	PMU_PMEVCNTR_EL0(0),
 	PMU_PMEVCNTR_EL0(1),
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
those traps. This results in terminating VMs. Instead, set a handler for
the EL1 phys timer access, and inject an undefined exception as an
intermediate step.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index caa47ce..1cd3464 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -820,6 +820,30 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	  CRm((0b1100 | (((n) >> 3) & 0x3))), Op2(((n) & 0x7)),		\
 	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+static bool access_cntp_tval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_cval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
 /*
  * Architected system registers.
  * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1029,6 +1053,16 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	{ Op0(0b11), Op1(0b011), CRn(0b1101), CRm(0b0000), Op2(0b011),
 	  NULL, reset_unknown, TPIDRRO_EL0 },
 
+	/* CNTP_TVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b000),
+	  access_cntp_tval },
+	/* CNTP_CTL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b001),
+	  access_cntp_ctl },
+	/* CNTP_CVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b010),
+	  access_cntp_cval },
+
 	/* PMEVCNTRn_EL0 */
 	PMU_PMEVCNTR_EL0(0),
 	PMU_PMEVCNTR_EL0(1),
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
those traps. This results in terminating VMs. Instead, set a handler for
the EL1 phys timer access, and inject an undefined exception as an
intermediate step.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index caa47ce..1cd3464 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -820,6 +820,30 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	  CRm((0b1100 | (((n) >> 3) & 0x3))), Op2(((n) & 0x7)),		\
 	  access_pmu_evtyper, reset_unknown, (PMEVTYPER0_EL0 + n), }
 
+static bool access_cntp_tval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
+static bool access_cntp_cval(struct kvm_vcpu *vcpu,
+		struct sys_reg_params *p,
+		const struct sys_reg_desc *r)
+{
+	kvm_inject_undefined(vcpu);
+	return true;
+}
+
 /*
  * Architected system registers.
  * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -1029,6 +1053,16 @@ static bool access_pmuserenr(struct kvm_vcpu *vcpu, struct sys_reg_params *p,
 	{ Op0(0b11), Op1(0b011), CRn(0b1101), CRm(0b0000), Op2(0b011),
 	  NULL, reset_unknown, TPIDRRO_EL0 },
 
+	/* CNTP_TVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b000),
+	  access_cntp_tval },
+	/* CNTP_CTL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b001),
+	  access_cntp_ctl },
+	/* CNTP_CVAL_EL0 */
+	{ Op0(0b11), Op1(0b011), CRn(0b1110), CRm(0b0010), Op2(0b010),
+	  access_cntp_cval },
+
 	/* PMEVCNTRn_EL0 */
 	PMU_PMEVCNTR_EL0(0),
 	PMU_PMEVCNTR_EL0(1),
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-01 17:43   ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel
  Cc: jintack

Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
Now VMs are able to use the EL1 physical timer.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c    | 37 ++++++++++++++++++++++++++++++++++---
 include/kvm/arm_arch_timer.h |  2 ++
 virt/kvm/arm/arch_timer.c    |  2 +-
 3 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1cd3464..0e26f8c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -824,7 +824,14 @@ static bool access_cntp_tval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+	u64 now = kvm_phys_timer_read();
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval + now;
+	else
+		p->regval = ptimer->cnt_cval - now;
+
 	return true;
 }
 
@@ -832,7 +839,25 @@ static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write) {
+		/* ISTATUS bit is read-only */
+		ptimer->cnt_ctl = p->regval & ~ARCH_TIMER_CTRL_IT_STAT;
+	} else {
+		u64 now = kvm_phys_timer_read();
+
+		p->regval = ptimer->cnt_ctl;
+		/*
+		 * Set ISTATUS bit if it's expired.
+		 * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is
+		 * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit
+		 * regardless of ENABLE bit for our implementation convenience.
+		 */
+		if (ptimer->cnt_cval <= now)
+			p->regval |= ARCH_TIMER_CTRL_IT_STAT;
+	}
+
 	return true;
 }
 
@@ -840,7 +865,13 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval;
+	else
+		p->regval = ptimer->cnt_cval;
+
 	return true;
 }
 
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f1d2fba0..fe797d6 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -72,6 +72,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
+u64 kvm_phys_timer_read(void);
+
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
 
 void kvm_timer_init_vhe(void);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1806e5e..93c811c 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -40,7 +40,7 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
 	vcpu_vtimer(vcpu)->active_cleared_last = false;
 }
 
-static u64 kvm_phys_timer_read(void)
+u64 kvm_phys_timer_read(void)
 {
 	return timecounter->cc->read(timecounter->cc);
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
Now VMs are able to use the EL1 physical timer.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c    | 37 ++++++++++++++++++++++++++++++++++---
 include/kvm/arm_arch_timer.h |  2 ++
 virt/kvm/arm/arch_timer.c    |  2 +-
 3 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1cd3464..0e26f8c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -824,7 +824,14 @@ static bool access_cntp_tval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+	u64 now = kvm_phys_timer_read();
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval + now;
+	else
+		p->regval = ptimer->cnt_cval - now;
+
 	return true;
 }
 
@@ -832,7 +839,25 @@ static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write) {
+		/* ISTATUS bit is read-only */
+		ptimer->cnt_ctl = p->regval & ~ARCH_TIMER_CTRL_IT_STAT;
+	} else {
+		u64 now = kvm_phys_timer_read();
+
+		p->regval = ptimer->cnt_ctl;
+		/*
+		 * Set ISTATUS bit if it's expired.
+		 * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is
+		 * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit
+		 * regardless of ENABLE bit for our implementation convenience.
+		 */
+		if (ptimer->cnt_cval <= now)
+			p->regval |= ARCH_TIMER_CTRL_IT_STAT;
+	}
+
 	return true;
 }
 
@@ -840,7 +865,13 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval;
+	else
+		p->regval = ptimer->cnt_cval;
+
 	return true;
 }
 
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f1d2fba0..fe797d6 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -72,6 +72,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
+u64 kvm_phys_timer_read(void);
+
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
 
 void kvm_timer_init_vhe(void);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1806e5e..93c811c 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -40,7 +40,7 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
 	vcpu_vtimer(vcpu)->active_cleared_last = false;
 }
 
-static u64 kvm_phys_timer_read(void)
+u64 kvm_phys_timer_read(void)
 {
 	return timecounter->cc->read(timecounter->cc);
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
@ 2017-02-01 17:43   ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-01 17:43 UTC (permalink / raw)
  To: linux-arm-kernel

Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
Now VMs are able to use the EL1 physical timer.

Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
---
 arch/arm64/kvm/sys_regs.c    | 37 ++++++++++++++++++++++++++++++++++---
 include/kvm/arm_arch_timer.h |  2 ++
 virt/kvm/arm/arch_timer.c    |  2 +-
 3 files changed, 37 insertions(+), 4 deletions(-)

diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 1cd3464..0e26f8c 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -824,7 +824,14 @@ static bool access_cntp_tval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+	u64 now = kvm_phys_timer_read();
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval + now;
+	else
+		p->regval = ptimer->cnt_cval - now;
+
 	return true;
 }
 
@@ -832,7 +839,25 @@ static bool access_cntp_ctl(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write) {
+		/* ISTATUS bit is read-only */
+		ptimer->cnt_ctl = p->regval & ~ARCH_TIMER_CTRL_IT_STAT;
+	} else {
+		u64 now = kvm_phys_timer_read();
+
+		p->regval = ptimer->cnt_ctl;
+		/*
+		 * Set ISTATUS bit if it's expired.
+		 * Note that according to ARMv8 ARM Issue A.k, ISTATUS bit is
+		 * UNKNOWN when ENABLE bit is 0, so we chose to set ISTATUS bit
+		 * regardless of ENABLE bit for our implementation convenience.
+		 */
+		if (ptimer->cnt_cval <= now)
+			p->regval |= ARCH_TIMER_CTRL_IT_STAT;
+	}
+
 	return true;
 }
 
@@ -840,7 +865,13 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
 		struct sys_reg_params *p,
 		const struct sys_reg_desc *r)
 {
-	kvm_inject_undefined(vcpu);
+	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
+
+	if (p->is_write)
+		ptimer->cnt_cval = p->regval;
+	else
+		p->regval = ptimer->cnt_cval;
+
 	return true;
 }
 
diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
index f1d2fba0..fe797d6 100644
--- a/include/kvm/arm_arch_timer.h
+++ b/include/kvm/arm_arch_timer.h
@@ -72,6 +72,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 void kvm_timer_schedule(struct kvm_vcpu *vcpu);
 void kvm_timer_unschedule(struct kvm_vcpu *vcpu);
 
+u64 kvm_phys_timer_read(void);
+
 void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu);
 
 void kvm_timer_init_vhe(void);
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 1806e5e..93c811c 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -40,7 +40,7 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu)
 	vcpu_vtimer(vcpu)->active_cleared_last = false;
 }
 
-static u64 kvm_phys_timer_read(void)
+u64 kvm_phys_timer_read(void)
 {
 	return timecounter->cc->read(timecounter->cc);
 }
-- 
1.9.1

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* Re: [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context
  2017-02-01 17:43   ` Jintack Lim
@ 2017-02-02 10:03     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:03 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:02PM -0500, Jintack Lim wrote:
> Make cntvoff per each timer context. This is helpful to abstract kvm
> timer functions to work with timer context without considering timer
> types (e.g. physical timer or virtual timer).
> 
> This also would pave the way for ever doing adjustments of the cntvoff
> on a per-CPU basis if that should ever make sense.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context
@ 2017-02-02 10:03     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:03 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:02PM -0500, Jintack Lim wrote:
> Make cntvoff per each timer context. This is helpful to abstract kvm
> timer functions to work with timer context without considering timer
> types (e.g. physical timer or virtual timer).
> 
> This also would pave the way for ever doing adjustments of the cntvoff
> on a per-CPU basis if that should ever make sense.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer
  2017-02-01 17:43   ` Jintack Lim
@ 2017-02-02 10:04     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:04 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:03PM -0500, Jintack Lim wrote:
> Now that we have a separate structure for timer context, make functions
> generic so that they can work with any timer context, not just the
> virtual timer context.  This does not change the virtual timer
> functionality.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer
@ 2017-02-02 10:04     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:04 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:03PM -0500, Jintack Lim wrote:
> Now that we have a separate structure for timer context, make functions
> generic so that they can work with any timer context, not just the
> virtual timer context.  This does not change the virtual timer
> functionality.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context
  2017-02-01 17:43   ` Jintack Lim
@ 2017-02-02 10:15     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:04PM -0500, Jintack Lim wrote:
> Add the EL1 physical timer context.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context
@ 2017-02-02 10:15     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:04PM -0500, Jintack Lim wrote:
> Add the EL1 physical timer context.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:15     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:05PM -0500, Jintack Lim wrote:
> Initialize the emulated EL1 physical timer with the default irq number.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

We really need to do something about that IRQ number thing.  I'll add
this to my todo list.

Since this is no worse than what we already do and doesn't place any
restrictions on future ABI to let userspace decide on the IRQ numbers:

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> ---
>  arch/arm/kvm/reset.c         | 9 ++++++++-
>  arch/arm64/kvm/reset.c       | 9 ++++++++-
>  include/kvm/arm_arch_timer.h | 3 ++-
>  virt/kvm/arm/arch_timer.c    | 9 +++++++--
>  4 files changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
> index 4b5e802..1da8b2d 100644
> --- a/arch/arm/kvm/reset.c
> +++ b/arch/arm/kvm/reset.c
> @@ -37,6 +37,11 @@
>  	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
>  };
>  
> +static const struct kvm_irq_level cortexa_ptimer_irq = {
> +	{ .irq = 30 },
> +	.level = 1,
> +};
> +
>  static const struct kvm_irq_level cortexa_vtimer_irq = {
>  	{ .irq = 27 },
>  	.level = 1,
> @@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	struct kvm_regs *reset_regs;
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  
>  	switch (vcpu->arch.target) {
>  	case KVM_ARM_TARGET_CORTEX_A7:
> @@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		reset_regs = &cortexa_regs_reset;
>  		vcpu->arch.midr = read_cpuid_id();
>  		cpu_vtimer_irq = &cortexa_vtimer_irq;
> +		cpu_ptimer_irq = &cortexa_ptimer_irq;
>  		break;
>  	default:
>  		return -ENODEV;
> @@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_reset_coprocs(vcpu);
>  
>  	/* Reset arch_timer context */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index e95d4f6..d9e9697 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -46,6 +46,11 @@
>  			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
>  };
>  
> +static const struct kvm_irq_level default_ptimer_irq = {
> +	.irq	= 30,
> +	.level	= 1,
> +};
> +
>  static const struct kvm_irq_level default_vtimer_irq = {
>  	.irq	= 27,
>  	.level	= 1,
> @@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
>  int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  	const struct kvm_regs *cpu_reset;
>  
>  	switch (vcpu->arch.target) {
> @@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		}
>  
>  		cpu_vtimer_irq = &default_vtimer_irq;
> +		cpu_ptimer_irq = &default_ptimer_irq;
>  		break;
>  	}
>  
> @@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_pmu_vcpu_reset(vcpu);
>  
>  	/* Reset timer */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
> index 6445a3d..f1d2fba0 100644
> --- a/include/kvm/arm_arch_timer.h
> +++ b/include/kvm/arm_arch_timer.h
> @@ -58,7 +58,8 @@ struct arch_timer_cpu {
>  int kvm_timer_hyp_init(void);
>  int kvm_timer_enable(struct kvm_vcpu *vcpu);
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq);
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq);
>  void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
>  void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index c42bca5..ae38703 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
>  }
>  
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq)
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq)
>  {
>  	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
> +	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
>  
>  	/*
>  	 * The vcpu timer irq number cannot be determined in
> @@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * kvm_vcpu_set_target(). To handle this, we determine
>  	 * vcpu timer irq number when the vcpu is reset.
>  	 */
> -	vtimer->irq.irq = irq->irq;
> +	vtimer->irq.irq = virt_irq->irq;
> +	ptimer->irq.irq = phys_irq->irq;
>  
>  	/*
>  	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
> @@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * the ARMv7 architecture.
>  	 */
>  	vtimer->cnt_ctl = 0;
> +	ptimer->cnt_ctl = 0;
>  	kvm_timer_update_state(vcpu);
>  
>  	return 0;
> @@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>  
>  	/* Synchronize cntvoff across all vtimers of a VM. */
>  	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +	vcpu_ptimer(vcpu)->cntvoff = 0;
>  
>  	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
>  	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> -- 
> 1.9.1
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
@ 2017-02-02 10:15     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:05PM -0500, Jintack Lim wrote:
> Initialize the emulated EL1 physical timer with the default irq number.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

We really need to do something about that IRQ number thing.  I'll add
this to my todo list.

Since this is no worse than what we already do and doesn't place any
restrictions on future ABI to let userspace decide on the IRQ numbers:

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> ---
>  arch/arm/kvm/reset.c         | 9 ++++++++-
>  arch/arm64/kvm/reset.c       | 9 ++++++++-
>  include/kvm/arm_arch_timer.h | 3 ++-
>  virt/kvm/arm/arch_timer.c    | 9 +++++++--
>  4 files changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
> index 4b5e802..1da8b2d 100644
> --- a/arch/arm/kvm/reset.c
> +++ b/arch/arm/kvm/reset.c
> @@ -37,6 +37,11 @@
>  	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
>  };
>  
> +static const struct kvm_irq_level cortexa_ptimer_irq = {
> +	{ .irq = 30 },
> +	.level = 1,
> +};
> +
>  static const struct kvm_irq_level cortexa_vtimer_irq = {
>  	{ .irq = 27 },
>  	.level = 1,
> @@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	struct kvm_regs *reset_regs;
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  
>  	switch (vcpu->arch.target) {
>  	case KVM_ARM_TARGET_CORTEX_A7:
> @@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		reset_regs = &cortexa_regs_reset;
>  		vcpu->arch.midr = read_cpuid_id();
>  		cpu_vtimer_irq = &cortexa_vtimer_irq;
> +		cpu_ptimer_irq = &cortexa_ptimer_irq;
>  		break;
>  	default:
>  		return -ENODEV;
> @@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_reset_coprocs(vcpu);
>  
>  	/* Reset arch_timer context */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index e95d4f6..d9e9697 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -46,6 +46,11 @@
>  			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
>  };
>  
> +static const struct kvm_irq_level default_ptimer_irq = {
> +	.irq	= 30,
> +	.level	= 1,
> +};
> +
>  static const struct kvm_irq_level default_vtimer_irq = {
>  	.irq	= 27,
>  	.level	= 1,
> @@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
>  int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  	const struct kvm_regs *cpu_reset;
>  
>  	switch (vcpu->arch.target) {
> @@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		}
>  
>  		cpu_vtimer_irq = &default_vtimer_irq;
> +		cpu_ptimer_irq = &default_ptimer_irq;
>  		break;
>  	}
>  
> @@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_pmu_vcpu_reset(vcpu);
>  
>  	/* Reset timer */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
> index 6445a3d..f1d2fba0 100644
> --- a/include/kvm/arm_arch_timer.h
> +++ b/include/kvm/arm_arch_timer.h
> @@ -58,7 +58,8 @@ struct arch_timer_cpu {
>  int kvm_timer_hyp_init(void);
>  int kvm_timer_enable(struct kvm_vcpu *vcpu);
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq);
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq);
>  void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
>  void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index c42bca5..ae38703 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
>  }
>  
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq)
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq)
>  {
>  	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
> +	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
>  
>  	/*
>  	 * The vcpu timer irq number cannot be determined in
> @@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * kvm_vcpu_set_target(). To handle this, we determine
>  	 * vcpu timer irq number when the vcpu is reset.
>  	 */
> -	vtimer->irq.irq = irq->irq;
> +	vtimer->irq.irq = virt_irq->irq;
> +	ptimer->irq.irq = phys_irq->irq;
>  
>  	/*
>  	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
> @@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * the ARMv7 architecture.
>  	 */
>  	vtimer->cnt_ctl = 0;
> +	ptimer->cnt_ctl = 0;
>  	kvm_timer_update_state(vcpu);
>  
>  	return 0;
> @@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>  
>  	/* Synchronize cntvoff across all vtimers of a VM. */
>  	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +	vcpu_ptimer(vcpu)->cntvoff = 0;
>  
>  	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
>  	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> -- 
> 1.9.1
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer
@ 2017-02-02 10:15     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:05PM -0500, Jintack Lim wrote:
> Initialize the emulated EL1 physical timer with the default irq number.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

We really need to do something about that IRQ number thing.  I'll add
this to my todo list.

Since this is no worse than what we already do and doesn't place any
restrictions on future ABI to let userspace decide on the IRQ numbers:

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

> ---
>  arch/arm/kvm/reset.c         | 9 ++++++++-
>  arch/arm64/kvm/reset.c       | 9 ++++++++-
>  include/kvm/arm_arch_timer.h | 3 ++-
>  virt/kvm/arm/arch_timer.c    | 9 +++++++--
>  4 files changed, 25 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm/kvm/reset.c b/arch/arm/kvm/reset.c
> index 4b5e802..1da8b2d 100644
> --- a/arch/arm/kvm/reset.c
> +++ b/arch/arm/kvm/reset.c
> @@ -37,6 +37,11 @@
>  	.usr_regs.ARM_cpsr = SVC_MODE | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT,
>  };
>  
> +static const struct kvm_irq_level cortexa_ptimer_irq = {
> +	{ .irq = 30 },
> +	.level = 1,
> +};
> +
>  static const struct kvm_irq_level cortexa_vtimer_irq = {
>  	{ .irq = 27 },
>  	.level = 1,
> @@ -58,6 +63,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	struct kvm_regs *reset_regs;
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  
>  	switch (vcpu->arch.target) {
>  	case KVM_ARM_TARGET_CORTEX_A7:
> @@ -65,6 +71,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		reset_regs = &cortexa_regs_reset;
>  		vcpu->arch.midr = read_cpuid_id();
>  		cpu_vtimer_irq = &cortexa_vtimer_irq;
> +		cpu_ptimer_irq = &cortexa_ptimer_irq;
>  		break;
>  	default:
>  		return -ENODEV;
> @@ -77,5 +84,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_reset_coprocs(vcpu);
>  
>  	/* Reset arch_timer context */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
> index e95d4f6..d9e9697 100644
> --- a/arch/arm64/kvm/reset.c
> +++ b/arch/arm64/kvm/reset.c
> @@ -46,6 +46,11 @@
>  			COMPAT_PSR_I_BIT | COMPAT_PSR_F_BIT),
>  };
>  
> +static const struct kvm_irq_level default_ptimer_irq = {
> +	.irq	= 30,
> +	.level	= 1,
> +};
> +
>  static const struct kvm_irq_level default_vtimer_irq = {
>  	.irq	= 27,
>  	.level	= 1,
> @@ -104,6 +109,7 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
>  int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  {
>  	const struct kvm_irq_level *cpu_vtimer_irq;
> +	const struct kvm_irq_level *cpu_ptimer_irq;
>  	const struct kvm_regs *cpu_reset;
>  
>  	switch (vcpu->arch.target) {
> @@ -117,6 +123,7 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  		}
>  
>  		cpu_vtimer_irq = &default_vtimer_irq;
> +		cpu_ptimer_irq = &default_ptimer_irq;
>  		break;
>  	}
>  
> @@ -130,5 +137,5 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu)
>  	kvm_pmu_vcpu_reset(vcpu);
>  
>  	/* Reset timer */
> -	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq);
> +	return kvm_timer_vcpu_reset(vcpu, cpu_vtimer_irq, cpu_ptimer_irq);
>  }
> diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h
> index 6445a3d..f1d2fba0 100644
> --- a/include/kvm/arm_arch_timer.h
> +++ b/include/kvm/arm_arch_timer.h
> @@ -58,7 +58,8 @@ struct arch_timer_cpu {
>  int kvm_timer_hyp_init(void);
>  int kvm_timer_enable(struct kvm_vcpu *vcpu);
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq);
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq);
>  void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu);
>  void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu);
>  void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu);
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index c42bca5..ae38703 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -327,9 +327,11 @@ void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu)
>  }
>  
>  int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
> -			 const struct kvm_irq_level *irq)
> +			 const struct kvm_irq_level *virt_irq,
> +			 const struct kvm_irq_level *phys_irq)
>  {
>  	struct arch_timer_context *vtimer = vcpu_vtimer(vcpu);
> +	struct arch_timer_context *ptimer = vcpu_ptimer(vcpu);
>  
>  	/*
>  	 * The vcpu timer irq number cannot be determined in
> @@ -337,7 +339,8 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * kvm_vcpu_set_target(). To handle this, we determine
>  	 * vcpu timer irq number when the vcpu is reset.
>  	 */
> -	vtimer->irq.irq = irq->irq;
> +	vtimer->irq.irq = virt_irq->irq;
> +	ptimer->irq.irq = phys_irq->irq;
>  
>  	/*
>  	 * The bits in CNTV_CTL are architecturally reset to UNKNOWN for ARMv8
> @@ -346,6 +349,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  	 * the ARMv7 architecture.
>  	 */
>  	vtimer->cnt_ctl = 0;
> +	ptimer->cnt_ctl = 0;
>  	kvm_timer_update_state(vcpu);
>  
>  	return 0;
> @@ -369,6 +373,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>  
>  	/* Synchronize cntvoff across all vtimers of a VM. */
>  	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +	vcpu_ptimer(vcpu)->cntvoff = 0;
>  
>  	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
>  	hrtimer_init(&timer->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> -- 
> 1.9.1
> 
> 

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:15     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:06PM -0500, Jintack Lim wrote:
> Now that we maintain the EL1 physical timer register states of VMs,
> update the physical timer interrupt level along with the virtual one.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
@ 2017-02-02 10:15     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:06PM -0500, Jintack Lim wrote:
> Now that we maintain the EL1 physical timer register states of VMs,
> update the physical timer interrupt level along with the virtual one.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level
@ 2017-02-02 10:15     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:15 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:06PM -0500, Jintack Lim wrote:
> Now that we maintain the EL1 physical timer register states of VMs,
> update the physical timer interrupt level along with the virtual one.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:30     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:07PM -0500, Jintack Lim wrote:
> When scheduling a background timer, consider both of the virtual and
> physical timer and pick the earliest expiration time.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
@ 2017-02-02 10:30     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:07PM -0500, Jintack Lim wrote:
> When scheduling a background timer, consider both of the virtual and
> physical timer and pick the earliest expiration time.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration
@ 2017-02-02 10:30     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:07PM -0500, Jintack Lim wrote:
> When scheduling a background timer, consider both of the virtual and
> physical timer and pick the earliest expiration time.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:30     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:08PM -0500, Jintack Lim wrote:
> Set a background timer for the EL1 physical timer emulation while VMs
> are running, so that VMs get the physical timer interrupts in a timely
> manner.
> 
> Schedule the background timer on entry to the VM and cancel it on exit.
> This would not have any performance impact to the guest OSes that
> currently use the virtual timer since the physical timer is always not
> enabled.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
@ 2017-02-02 10:30     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:08PM -0500, Jintack Lim wrote:
> Set a background timer for the EL1 physical timer emulation while VMs
> are running, so that VMs get the physical timer interrupts in a timely
> manner.
> 
> Schedule the background timer on entry to the VM and cancel it on exit.
> This would not have any performance impact to the guest OSes that
> currently use the virtual timer since the physical timer is always not
> enabled.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation
@ 2017-02-02 10:30     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:08PM -0500, Jintack Lim wrote:
> Set a background timer for the EL1 physical timer emulation while VMs
> are running, so that VMs get the physical timer interrupts in a timely
> manner.
> 
> Schedule the background timer on entry to the VM and cancel it on exit.
> This would not have any performance impact to the guest OSes that
> currently use the virtual timer since the physical timer is always not
> enabled.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:31     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:09PM -0500, Jintack Lim wrote:
> KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
> those traps. This results in terminating VMs. Instead, set a handler for
> the EL1 phys timer access, and inject an undefined exception as an
> intermediate step.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
@ 2017-02-02 10:31     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:09PM -0500, Jintack Lim wrote:
> KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
> those traps. This results in terminating VMs. Instead, set a handler for
> the EL1 phys timer access, and inject an undefined exception as an
> intermediate step.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler
@ 2017-02-02 10:31     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:09PM -0500, Jintack Lim wrote:
> KVM traps on the EL1 phys timer accesses from VMs, but it doesn't handle
> those traps. This results in terminating VMs. Instead, set a handler for
> the EL1 phys timer access, and inject an undefined exception as an
> intermediate step.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
  2017-02-01 17:43   ` Jintack Lim
  (?)
@ 2017-02-02 10:31     ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

On Wed, Feb 01, 2017 at 12:43:10PM -0500, Jintack Lim wrote:
> Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
> Now VMs are able to use the EL1 physical timer.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
@ 2017-02-02 10:31     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

On Wed, Feb 01, 2017 at 12:43:10PM -0500, Jintack Lim wrote:
> Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
> Now VMs are able to use the EL1 physical timer.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers
@ 2017-02-02 10:31     ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 10:31 UTC (permalink / raw)
  To: linux-arm-kernel

On Wed, Feb 01, 2017 at 12:43:10PM -0500, Jintack Lim wrote:
> Emulate read and write operations to CNTP_TVAL, CNTP_CVAL and CNTP_CTL.
> Now VMs are able to use the EL1 physical timer.
> 
> Signed-off-by: Jintack Lim <jintack@cs.columbia.edu>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-01 17:43 ` Jintack Lim
  (?)
@ 2017-02-02 12:31   ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 12:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: pbonzini, rkrcmar, christoffer.dall, marc.zyngier, linux,
	catalin.marinas, will.deacon, andre.przywara, kvm,
	linux-arm-kernel, kvmarm, linux-kernel

Hi Jintack,

On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> The ARM architecture defines the EL1 physical timer and the virtual timer,
> and it is reasonable for an OS to expect to be able to access both.
> However, the current KVM implementation does not provide the EL1 physical
> timer to VMs but terminates VMs on access to the timer.
> 
> This patch series enables VMs to use the EL1 physical timer through
> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> access and sets up the background timer accordingly.  When the background
> timer expires, the KVM host injects EL1 physical timer interrupts to the
> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> physical timer without trapping.  However, this requires somehow using the
> EL2 physical timer for the Linux host while running the VM instead of the
> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> this was straightforward to do, and I leave it to future work to determine
> if transferring the EL1 physical timer state to the EL2 timer provides any
> performance benefit.
> 
> This feature will be useful for any OS that wishes to access the EL1
> physical timer. Nested virtualization is one of those use cases. A nested
> hypervisor running inside a VM would think it has full access to the
> hardware and naturally tries to use the EL1 physical timer as Linux would
> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> would do, but supporting the EL2 physical timer to the VM is out of scope
> of this patch series. This patch series will make it easy to add the EL2
> timer support in the future, though.
> 
> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> and will continue to use only the virtual timer and this patch series will
> therefore not introduce any performance degredation as a result of
> trap-and-emulate.
> 
> v2 => v3:
>  - Rebase on kvmarm/queue
>  - Take kvm->lock to synchronize cntvoff across all vtimers
>  - Remove unnecessary function parameters
>  - Add comments

I just gave v3 a test run on my TC2 (32-bit platform) and my guest
quickly locks up trying to run cyclictest or when booting the machine it
stalls with RCU timeouts.

Could you have a look?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 12:31   ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 12:31 UTC (permalink / raw)
  To: Jintack Lim
  Cc: kvm, marc.zyngier, catalin.marinas, will.deacon, linux,
	linux-kernel, linux-arm-kernel, andre.przywara, pbonzini, kvmarm

Hi Jintack,

On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> The ARM architecture defines the EL1 physical timer and the virtual timer,
> and it is reasonable for an OS to expect to be able to access both.
> However, the current KVM implementation does not provide the EL1 physical
> timer to VMs but terminates VMs on access to the timer.
> 
> This patch series enables VMs to use the EL1 physical timer through
> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> access and sets up the background timer accordingly.  When the background
> timer expires, the KVM host injects EL1 physical timer interrupts to the
> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> physical timer without trapping.  However, this requires somehow using the
> EL2 physical timer for the Linux host while running the VM instead of the
> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> this was straightforward to do, and I leave it to future work to determine
> if transferring the EL1 physical timer state to the EL2 timer provides any
> performance benefit.
> 
> This feature will be useful for any OS that wishes to access the EL1
> physical timer. Nested virtualization is one of those use cases. A nested
> hypervisor running inside a VM would think it has full access to the
> hardware and naturally tries to use the EL1 physical timer as Linux would
> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> would do, but supporting the EL2 physical timer to the VM is out of scope
> of this patch series. This patch series will make it easy to add the EL2
> timer support in the future, though.
> 
> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> and will continue to use only the virtual timer and this patch series will
> therefore not introduce any performance degredation as a result of
> trap-and-emulate.
> 
> v2 => v3:
>  - Rebase on kvmarm/queue
>  - Take kvm->lock to synchronize cntvoff across all vtimers
>  - Remove unnecessary function parameters
>  - Add comments

I just gave v3 a test run on my TC2 (32-bit platform) and my guest
quickly locks up trying to run cyclictest or when booting the machine it
stalls with RCU timeouts.

Could you have a look?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 12:31   ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 12:31 UTC (permalink / raw)
  To: linux-arm-kernel

Hi Jintack,

On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> The ARM architecture defines the EL1 physical timer and the virtual timer,
> and it is reasonable for an OS to expect to be able to access both.
> However, the current KVM implementation does not provide the EL1 physical
> timer to VMs but terminates VMs on access to the timer.
> 
> This patch series enables VMs to use the EL1 physical timer through
> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> access and sets up the background timer accordingly.  When the background
> timer expires, the KVM host injects EL1 physical timer interrupts to the
> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> physical timer without trapping.  However, this requires somehow using the
> EL2 physical timer for the Linux host while running the VM instead of the
> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> this was straightforward to do, and I leave it to future work to determine
> if transferring the EL1 physical timer state to the EL2 timer provides any
> performance benefit.
> 
> This feature will be useful for any OS that wishes to access the EL1
> physical timer. Nested virtualization is one of those use cases. A nested
> hypervisor running inside a VM would think it has full access to the
> hardware and naturally tries to use the EL1 physical timer as Linux would
> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> would do, but supporting the EL2 physical timer to the VM is out of scope
> of this patch series. This patch series will make it easy to add the EL2
> timer support in the future, though.
> 
> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> and will continue to use only the virtual timer and this patch series will
> therefore not introduce any performance degredation as a result of
> trap-and-emulate.
> 
> v2 => v3:
>  - Rebase on kvmarm/queue
>  - Take kvm->lock to synchronize cntvoff across all vtimers
>  - Remove unnecessary function parameters
>  - Add comments

I just gave v3 a test run on my TC2 (32-bit platform) and my guest
quickly locks up trying to run cyclictest or when booting the machine it
stalls with RCU timeouts.

Could you have a look?

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-02 12:31   ` Christoffer Dall
  (?)
@ 2017-02-02 14:51     ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-02 14:51 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: Paolo Bonzini, Radim Krčmář,
	Christoffer Dall, Marc Zyngier, linux, Catalin Marinas,
	Will Deacon, Andre Przywara, KVM General, arm-mail-list, kvmarm,
	lkml - Kernel Mailing List

On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> Hi Jintack,
>
> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> and it is reasonable for an OS to expect to be able to access both.
>> However, the current KVM implementation does not provide the EL1 physical
>> timer to VMs but terminates VMs on access to the timer.
>>
>> This patch series enables VMs to use the EL1 physical timer through
>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> access and sets up the background timer accordingly.  When the background
>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> physical timer without trapping.  However, this requires somehow using the
>> EL2 physical timer for the Linux host while running the VM instead of the
>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> this was straightforward to do, and I leave it to future work to determine
>> if transferring the EL1 physical timer state to the EL2 timer provides any
>> performance benefit.
>>
>> This feature will be useful for any OS that wishes to access the EL1
>> physical timer. Nested virtualization is one of those use cases. A nested
>> hypervisor running inside a VM would think it has full access to the
>> hardware and naturally tries to use the EL1 physical timer as Linux would
>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> would do, but supporting the EL2 physical timer to the VM is out of scope
>> of this patch series. This patch series will make it easy to add the EL2
>> timer support in the future, though.
>>
>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> and will continue to use only the virtual timer and this patch series will
>> therefore not introduce any performance degredation as a result of
>> trap-and-emulate.
>>
>> v2 => v3:
>>  - Rebase on kvmarm/queue
>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>  - Remove unnecessary function parameters
>>  - Add comments
>
> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> quickly locks up trying to run cyclictest or when booting the machine it
> stalls with RCU timeouts.

Ok. It's my fault not to specify that the emulated physical timer is
supported/tested on arm64.
On 32-bit platform, it is supposed to show the same behavior as
before, but I haven't tested.
Were you using the physical timer or the virtual timer for the guest?

>
> Could you have a look?

Sure, I'll have a look. I don't have access to my Cubietruck today,
but I can work on that tomorrow.

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 14:51     ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-02 14:51 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: KVM General, Marc Zyngier, Catalin Marinas, Will Deacon, linux,
	lkml - Kernel Mailing List, arm-mail-list, Andre Przywara,
	Paolo Bonzini, kvmarm

On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> Hi Jintack,
>
> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> and it is reasonable for an OS to expect to be able to access both.
>> However, the current KVM implementation does not provide the EL1 physical
>> timer to VMs but terminates VMs on access to the timer.
>>
>> This patch series enables VMs to use the EL1 physical timer through
>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> access and sets up the background timer accordingly.  When the background
>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> physical timer without trapping.  However, this requires somehow using the
>> EL2 physical timer for the Linux host while running the VM instead of the
>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> this was straightforward to do, and I leave it to future work to determine
>> if transferring the EL1 physical timer state to the EL2 timer provides any
>> performance benefit.
>>
>> This feature will be useful for any OS that wishes to access the EL1
>> physical timer. Nested virtualization is one of those use cases. A nested
>> hypervisor running inside a VM would think it has full access to the
>> hardware and naturally tries to use the EL1 physical timer as Linux would
>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> would do, but supporting the EL2 physical timer to the VM is out of scope
>> of this patch series. This patch series will make it easy to add the EL2
>> timer support in the future, though.
>>
>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> and will continue to use only the virtual timer and this patch series will
>> therefore not introduce any performance degredation as a result of
>> trap-and-emulate.
>>
>> v2 => v3:
>>  - Rebase on kvmarm/queue
>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>  - Remove unnecessary function parameters
>>  - Add comments
>
> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> quickly locks up trying to run cyclictest or when booting the machine it
> stalls with RCU timeouts.

Ok. It's my fault not to specify that the emulated physical timer is
supported/tested on arm64.
On 32-bit platform, it is supposed to show the same behavior as
before, but I haven't tested.
Were you using the physical timer or the virtual timer for the guest?

>
> Could you have a look?

Sure, I'll have a look. I don't have access to my Cubietruck today,
but I can work on that tomorrow.

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 14:51     ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-02 14:51 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> Hi Jintack,
>
> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> and it is reasonable for an OS to expect to be able to access both.
>> However, the current KVM implementation does not provide the EL1 physical
>> timer to VMs but terminates VMs on access to the timer.
>>
>> This patch series enables VMs to use the EL1 physical timer through
>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> access and sets up the background timer accordingly.  When the background
>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> physical timer without trapping.  However, this requires somehow using the
>> EL2 physical timer for the Linux host while running the VM instead of the
>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> this was straightforward to do, and I leave it to future work to determine
>> if transferring the EL1 physical timer state to the EL2 timer provides any
>> performance benefit.
>>
>> This feature will be useful for any OS that wishes to access the EL1
>> physical timer. Nested virtualization is one of those use cases. A nested
>> hypervisor running inside a VM would think it has full access to the
>> hardware and naturally tries to use the EL1 physical timer as Linux would
>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> would do, but supporting the EL2 physical timer to the VM is out of scope
>> of this patch series. This patch series will make it easy to add the EL2
>> timer support in the future, though.
>>
>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> and will continue to use only the virtual timer and this patch series will
>> therefore not introduce any performance degredation as a result of
>> trap-and-emulate.
>>
>> v2 => v3:
>>  - Rebase on kvmarm/queue
>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>  - Remove unnecessary function parameters
>>  - Add comments
>
> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> quickly locks up trying to run cyclictest or when booting the machine it
> stalls with RCU timeouts.

Ok. It's my fault not to specify that the emulated physical timer is
supported/tested on arm64.
On 32-bit platform, it is supposed to show the same behavior as
before, but I haven't tested.
Were you using the physical timer or the virtual timer for the guest?

>
> Could you have a look?

Sure, I'll have a look. I don't have access to my Cubietruck today,
but I can work on that tomorrow.

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-02 14:51     ` Jintack Lim
  (?)
@ 2017-02-02 15:08       ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 15:08 UTC (permalink / raw)
  To: Jintack Lim
  Cc: Paolo Bonzini, Radim Krčmář,
	Marc Zyngier, Russell King, Catalin Marinas, Will Deacon,
	Andre Przywara, KVM General, arm-mail-list, kvmarm,
	lkml - Kernel Mailing List

On Thu, Feb 2, 2017 at 3:51 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> Hi Jintack,
>>
>> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> and it is reasonable for an OS to expect to be able to access both.
>>> However, the current KVM implementation does not provide the EL1 physical
>>> timer to VMs but terminates VMs on access to the timer.
>>>
>>> This patch series enables VMs to use the EL1 physical timer through
>>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> access and sets up the background timer accordingly.  When the background
>>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> physical timer without trapping.  However, this requires somehow using the
>>> EL2 physical timer for the Linux host while running the VM instead of the
>>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> this was straightforward to do, and I leave it to future work to determine
>>> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> performance benefit.
>>>
>>> This feature will be useful for any OS that wishes to access the EL1
>>> physical timer. Nested virtualization is one of those use cases. A nested
>>> hypervisor running inside a VM would think it has full access to the
>>> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> of this patch series. This patch series will make it easy to add the EL2
>>> timer support in the future, though.
>>>
>>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> and will continue to use only the virtual timer and this patch series will
>>> therefore not introduce any performance degredation as a result of
>>> trap-and-emulate.
>>>
>>> v2 => v3:
>>>  - Rebase on kvmarm/queue
>>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>>  - Remove unnecessary function parameters
>>>  - Add comments
>>
>> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> quickly locks up trying to run cyclictest or when booting the machine it
>> stalls with RCU timeouts.
>
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
>

I used the same guest and QEMU that I always test with so I expect it
to only use the virtual timer.

I wonder if we can somehow manage to not reset the timer properly on
the 32-bit side and end up in a form of endless interrupt loop?

>>
>> Could you have a look?
>
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
>
ok, thanks.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 15:08       ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 15:08 UTC (permalink / raw)
  To: Jintack Lim
  Cc: KVM General, Marc Zyngier, Catalin Marinas, Will Deacon,
	Russell King, lkml - Kernel Mailing List, Andre Przywara,
	Paolo Bonzini, kvmarm, arm-mail-list

On Thu, Feb 2, 2017 at 3:51 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> Hi Jintack,
>>
>> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> and it is reasonable for an OS to expect to be able to access both.
>>> However, the current KVM implementation does not provide the EL1 physical
>>> timer to VMs but terminates VMs on access to the timer.
>>>
>>> This patch series enables VMs to use the EL1 physical timer through
>>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> access and sets up the background timer accordingly.  When the background
>>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> physical timer without trapping.  However, this requires somehow using the
>>> EL2 physical timer for the Linux host while running the VM instead of the
>>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> this was straightforward to do, and I leave it to future work to determine
>>> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> performance benefit.
>>>
>>> This feature will be useful for any OS that wishes to access the EL1
>>> physical timer. Nested virtualization is one of those use cases. A nested
>>> hypervisor running inside a VM would think it has full access to the
>>> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> of this patch series. This patch series will make it easy to add the EL2
>>> timer support in the future, though.
>>>
>>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> and will continue to use only the virtual timer and this patch series will
>>> therefore not introduce any performance degredation as a result of
>>> trap-and-emulate.
>>>
>>> v2 => v3:
>>>  - Rebase on kvmarm/queue
>>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>>  - Remove unnecessary function parameters
>>>  - Add comments
>>
>> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> quickly locks up trying to run cyclictest or when booting the machine it
>> stalls with RCU timeouts.
>
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
>

I used the same guest and QEMU that I always test with so I expect it
to only use the virtual timer.

I wonder if we can somehow manage to not reset the timer properly on
the 32-bit side and end up in a form of endless interrupt loop?

>>
>> Could you have a look?
>
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
>
ok, thanks.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-02 15:08       ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-02 15:08 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 2, 2017 at 3:51 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> Hi Jintack,
>>
>> On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> and it is reasonable for an OS to expect to be able to access both.
>>> However, the current KVM implementation does not provide the EL1 physical
>>> timer to VMs but terminates VMs on access to the timer.
>>>
>>> This patch series enables VMs to use the EL1 physical timer through
>>> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> access and sets up the background timer accordingly.  When the background
>>> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> physical timer without trapping.  However, this requires somehow using the
>>> EL2 physical timer for the Linux host while running the VM instead of the
>>> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> this was straightforward to do, and I leave it to future work to determine
>>> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> performance benefit.
>>>
>>> This feature will be useful for any OS that wishes to access the EL1
>>> physical timer. Nested virtualization is one of those use cases. A nested
>>> hypervisor running inside a VM would think it has full access to the
>>> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> of this patch series. This patch series will make it easy to add the EL2
>>> timer support in the future, though.
>>>
>>> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> and will continue to use only the virtual timer and this patch series will
>>> therefore not introduce any performance degredation as a result of
>>> trap-and-emulate.
>>>
>>> v2 => v3:
>>>  - Rebase on kvmarm/queue
>>>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>>  - Remove unnecessary function parameters
>>>  - Add comments
>>
>> I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> quickly locks up trying to run cyclictest or when booting the machine it
>> stalls with RCU timeouts.
>
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
>

I used the same guest and QEMU that I always test with so I expect it
to only use the virtual timer.

I wonder if we can somehow manage to not reset the timer properly on
the 32-bit side and end up in a form of endless interrupt loop?

>>
>> Could you have a look?
>
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
>
ok, thanks.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-02 14:51     ` Jintack Lim
  (?)
@ 2017-02-03 12:33       ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 12:33 UTC (permalink / raw)
  To: Jintack Lim
  Cc: Paolo Bonzini, Radim Krčmář,
	Christoffer Dall, Marc Zyngier, linux, Catalin Marinas,
	Will Deacon, Andre Przywara, KVM General, arm-mail-list, kvmarm,
	lkml - Kernel Mailing List

On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> > Hi Jintack,
> >
> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
> >> and it is reasonable for an OS to expect to be able to access both.
> >> However, the current KVM implementation does not provide the EL1 physical
> >> timer to VMs but terminates VMs on access to the timer.
> >>
> >> This patch series enables VMs to use the EL1 physical timer through
> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> >> access and sets up the background timer accordingly.  When the background
> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> >> physical timer without trapping.  However, this requires somehow using the
> >> EL2 physical timer for the Linux host while running the VM instead of the
> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> >> this was straightforward to do, and I leave it to future work to determine
> >> if transferring the EL1 physical timer state to the EL2 timer provides any
> >> performance benefit.
> >>
> >> This feature will be useful for any OS that wishes to access the EL1
> >> physical timer. Nested virtualization is one of those use cases. A nested
> >> hypervisor running inside a VM would think it has full access to the
> >> hardware and naturally tries to use the EL1 physical timer as Linux would
> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> >> would do, but supporting the EL2 physical timer to the VM is out of scope
> >> of this patch series. This patch series will make it easy to add the EL2
> >> timer support in the future, though.
> >>
> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> >> and will continue to use only the virtual timer and this patch series will
> >> therefore not introduce any performance degredation as a result of
> >> trap-and-emulate.
> >>
> >> v2 => v3:
> >>  - Rebase on kvmarm/queue
> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
> >>  - Remove unnecessary function parameters
> >>  - Add comments
> >
> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> > quickly locks up trying to run cyclictest or when booting the machine it
> > stalls with RCU timeouts.
> 
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
> 
> >
> > Could you have a look?
> 
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
> 

Don't bother, I've figured this out for you.

You need the following fixup to your patch:

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 93c811c..35d7100 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 }
 
 /* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
 {
 	int i;
-	struct kvm_vcpu *vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_vcpu *tmp;
 
 	mutex_lock(&kvm->lock);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	kvm_for_each_vcpu(i, tmp, kvm)
+		vcpu_vtimer(tmp)->cntvoff = cntvoff;
+
+	/*
+	 * When called from the vcpu create path, the CPU being created is not
+	 * included in the loop above, so we just set it here as well.
+	 */
+	vcpu_vtimer(vcpu)->cntvoff = cntvoff;
 	mutex_unlock(&kvm->lock);
 }
 
@@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
-	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
 	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
@@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
+		update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;

This is an amuzing one.

Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 12:33       ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 12:33 UTC (permalink / raw)
  To: Jintack Lim
  Cc: KVM General, Marc Zyngier, Catalin Marinas, Will Deacon, linux,
	lkml - Kernel Mailing List, arm-mail-list, Andre Przywara,
	Paolo Bonzini, kvmarm

On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> > Hi Jintack,
> >
> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
> >> and it is reasonable for an OS to expect to be able to access both.
> >> However, the current KVM implementation does not provide the EL1 physical
> >> timer to VMs but terminates VMs on access to the timer.
> >>
> >> This patch series enables VMs to use the EL1 physical timer through
> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> >> access and sets up the background timer accordingly.  When the background
> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> >> physical timer without trapping.  However, this requires somehow using the
> >> EL2 physical timer for the Linux host while running the VM instead of the
> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> >> this was straightforward to do, and I leave it to future work to determine
> >> if transferring the EL1 physical timer state to the EL2 timer provides any
> >> performance benefit.
> >>
> >> This feature will be useful for any OS that wishes to access the EL1
> >> physical timer. Nested virtualization is one of those use cases. A nested
> >> hypervisor running inside a VM would think it has full access to the
> >> hardware and naturally tries to use the EL1 physical timer as Linux would
> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> >> would do, but supporting the EL2 physical timer to the VM is out of scope
> >> of this patch series. This patch series will make it easy to add the EL2
> >> timer support in the future, though.
> >>
> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> >> and will continue to use only the virtual timer and this patch series will
> >> therefore not introduce any performance degredation as a result of
> >> trap-and-emulate.
> >>
> >> v2 => v3:
> >>  - Rebase on kvmarm/queue
> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
> >>  - Remove unnecessary function parameters
> >>  - Add comments
> >
> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> > quickly locks up trying to run cyclictest or when booting the machine it
> > stalls with RCU timeouts.
> 
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
> 
> >
> > Could you have a look?
> 
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
> 

Don't bother, I've figured this out for you.

You need the following fixup to your patch:

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 93c811c..35d7100 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 }
 
 /* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
 {
 	int i;
-	struct kvm_vcpu *vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_vcpu *tmp;
 
 	mutex_lock(&kvm->lock);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	kvm_for_each_vcpu(i, tmp, kvm)
+		vcpu_vtimer(tmp)->cntvoff = cntvoff;
+
+	/*
+	 * When called from the vcpu create path, the CPU being created is not
+	 * included in the loop above, so we just set it here as well.
+	 */
+	vcpu_vtimer(vcpu)->cntvoff = cntvoff;
 	mutex_unlock(&kvm->lock);
 }
 
@@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
-	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
 	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
@@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
+		update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;

This is an amuzing one.

Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 12:33       ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 12:33 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
> > Hi Jintack,
> >
> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
> >> and it is reasonable for an OS to expect to be able to access both.
> >> However, the current KVM implementation does not provide the EL1 physical
> >> timer to VMs but terminates VMs on access to the timer.
> >>
> >> This patch series enables VMs to use the EL1 physical timer through
> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
> >> access and sets up the background timer accordingly.  When the background
> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
> >> physical timer without trapping.  However, this requires somehow using the
> >> EL2 physical timer for the Linux host while running the VM instead of the
> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
> >> this was straightforward to do, and I leave it to future work to determine
> >> if transferring the EL1 physical timer state to the EL2 timer provides any
> >> performance benefit.
> >>
> >> This feature will be useful for any OS that wishes to access the EL1
> >> physical timer. Nested virtualization is one of those use cases. A nested
> >> hypervisor running inside a VM would think it has full access to the
> >> hardware and naturally tries to use the EL1 physical timer as Linux would
> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
> >> would do, but supporting the EL2 physical timer to the VM is out of scope
> >> of this patch series. This patch series will make it easy to add the EL2
> >> timer support in the future, though.
> >>
> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
> >> and will continue to use only the virtual timer and this patch series will
> >> therefore not introduce any performance degredation as a result of
> >> trap-and-emulate.
> >>
> >> v2 => v3:
> >>  - Rebase on kvmarm/queue
> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
> >>  - Remove unnecessary function parameters
> >>  - Add comments
> >
> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
> > quickly locks up trying to run cyclictest or when booting the machine it
> > stalls with RCU timeouts.
> 
> Ok. It's my fault not to specify that the emulated physical timer is
> supported/tested on arm64.
> On 32-bit platform, it is supposed to show the same behavior as
> before, but I haven't tested.
> Were you using the physical timer or the virtual timer for the guest?
> 
> >
> > Could you have a look?
> 
> Sure, I'll have a look. I don't have access to my Cubietruck today,
> but I can work on that tomorrow.
> 

Don't bother, I've figured this out for you.

You need the following fixup to your patch:

diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 93c811c..35d7100 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
 }
 
 /* Make the updates of cntvoff for all vtimer contexts atomic */
-static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
+static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
 {
 	int i;
-	struct kvm_vcpu *vcpu;
+	struct kvm *kvm = vcpu->kvm;
+	struct kvm_vcpu *tmp;
 
 	mutex_lock(&kvm->lock);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu_vtimer(vcpu)->cntvoff = cntvoff;
+	kvm_for_each_vcpu(i, tmp, kvm)
+		vcpu_vtimer(tmp)->cntvoff = cntvoff;
+
+	/*
+	 * When called from the vcpu create path, the CPU being created is not
+	 * included in the loop above, so we just set it here as well.
+	 */
+	vcpu_vtimer(vcpu)->cntvoff = cntvoff;
 	mutex_unlock(&kvm->lock);
 }
 
@@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
 	struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
 
 	/* Synchronize cntvoff across all vtimers of a VM. */
-	update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
+	update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
 	vcpu_ptimer(vcpu)->cntvoff = 0;
 
 	INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
@@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
 		vtimer->cnt_ctl = value;
 		break;
 	case KVM_REG_ARM_TIMER_CNT:
-		update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
+		update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
 		break;
 	case KVM_REG_ARM_TIMER_CVAL:
 		vtimer->cnt_cval = value;

This is an amuzing one.

Thanks,
-Christoffer

^ permalink raw reply related	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-03 12:33       ` Christoffer Dall
  (?)
@ 2017-02-03 13:14         ` Jintack Lim
  -1 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-03 13:14 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: Paolo Bonzini, Radim Krčmář,
	Christoffer Dall, Marc Zyngier, linux, Catalin Marinas,
	Will Deacon, Andre Przywara, KVM General, arm-mail-list, kvmarm,
	lkml - Kernel Mailing List

On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> > Hi Jintack,
>> >
>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> >> and it is reasonable for an OS to expect to be able to access both.
>> >> However, the current KVM implementation does not provide the EL1 physical
>> >> timer to VMs but terminates VMs on access to the timer.
>> >>
>> >> This patch series enables VMs to use the EL1 physical timer through
>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> >> access and sets up the background timer accordingly.  When the background
>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> >> physical timer without trapping.  However, this requires somehow using the
>> >> EL2 physical timer for the Linux host while running the VM instead of the
>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> >> this was straightforward to do, and I leave it to future work to determine
>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>> >> performance benefit.
>> >>
>> >> This feature will be useful for any OS that wishes to access the EL1
>> >> physical timer. Nested virtualization is one of those use cases. A nested
>> >> hypervisor running inside a VM would think it has full access to the
>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>> >> of this patch series. This patch series will make it easy to add the EL2
>> >> timer support in the future, though.
>> >>
>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> >> and will continue to use only the virtual timer and this patch series will
>> >> therefore not introduce any performance degredation as a result of
>> >> trap-and-emulate.
>> >>
>> >> v2 => v3:
>> >>  - Rebase on kvmarm/queue
>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>> >>  - Remove unnecessary function parameters
>> >>  - Add comments
>> >
>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> > quickly locks up trying to run cyclictest or when booting the machine it
>> > stalls with RCU timeouts.
>>
>> Ok. It's my fault not to specify that the emulated physical timer is
>> supported/tested on arm64.
>> On 32-bit platform, it is supposed to show the same behavior as
>> before, but I haven't tested.
>> Were you using the physical timer or the virtual timer for the guest?
>>
>> >
>> > Could you have a look?
>>
>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>> but I can work on that tomorrow.
>>
>
> Don't bother, I've figured this out for you.

Thanks a lot.

>
> You need the following fixup to your patch:

Ok. I'll post v4 soon.
You've already do "acked-by" for this commit. Do I need to change it
to "signed-off-by"?

>
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index 93c811c..35d7100 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  }
>
>  /* Make the updates of cntvoff for all vtimer contexts atomic */
> -static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
> +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
>  {
>         int i;
> -       struct kvm_vcpu *vcpu;
> +       struct kvm *kvm = vcpu->kvm;
> +       struct kvm_vcpu *tmp;
>
>         mutex_lock(&kvm->lock);
> -       kvm_for_each_vcpu(i, vcpu, kvm)
> -               vcpu_vtimer(vcpu)->cntvoff = cntvoff;
> +       kvm_for_each_vcpu(i, tmp, kvm)
> +               vcpu_vtimer(tmp)->cntvoff = cntvoff;
> +
> +       /*
> +        * When called from the vcpu create path, the CPU being created is not
> +        * included in the loop above, so we just set it here as well.
> +        */
> +       vcpu_vtimer(vcpu)->cntvoff = cntvoff;
>         mutex_unlock(&kvm->lock);
>  }
>
> @@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>         struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
>
>         /* Synchronize cntvoff across all vtimers of a VM. */
> -       update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +       update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
>         vcpu_ptimer(vcpu)->cntvoff = 0;
>
>         INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
> @@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
>                 vtimer->cnt_ctl = value;
>                 break;
>         case KVM_REG_ARM_TIMER_CNT:
> -               update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
> +               update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
>                 break;
>         case KVM_REG_ARM_TIMER_CVAL:
>                 vtimer->cnt_cval = value;
>
> This is an amuzing one.

nice catch!

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 13:14         ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-03 13:14 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: KVM General, Marc Zyngier, Catalin Marinas, Will Deacon, linux,
	lkml - Kernel Mailing List, arm-mail-list, Andre Przywara,
	Paolo Bonzini, kvmarm

On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> > Hi Jintack,
>> >
>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> >> and it is reasonable for an OS to expect to be able to access both.
>> >> However, the current KVM implementation does not provide the EL1 physical
>> >> timer to VMs but terminates VMs on access to the timer.
>> >>
>> >> This patch series enables VMs to use the EL1 physical timer through
>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> >> access and sets up the background timer accordingly.  When the background
>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> >> physical timer without trapping.  However, this requires somehow using the
>> >> EL2 physical timer for the Linux host while running the VM instead of the
>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> >> this was straightforward to do, and I leave it to future work to determine
>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>> >> performance benefit.
>> >>
>> >> This feature will be useful for any OS that wishes to access the EL1
>> >> physical timer. Nested virtualization is one of those use cases. A nested
>> >> hypervisor running inside a VM would think it has full access to the
>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>> >> of this patch series. This patch series will make it easy to add the EL2
>> >> timer support in the future, though.
>> >>
>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> >> and will continue to use only the virtual timer and this patch series will
>> >> therefore not introduce any performance degredation as a result of
>> >> trap-and-emulate.
>> >>
>> >> v2 => v3:
>> >>  - Rebase on kvmarm/queue
>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>> >>  - Remove unnecessary function parameters
>> >>  - Add comments
>> >
>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> > quickly locks up trying to run cyclictest or when booting the machine it
>> > stalls with RCU timeouts.
>>
>> Ok. It's my fault not to specify that the emulated physical timer is
>> supported/tested on arm64.
>> On 32-bit platform, it is supposed to show the same behavior as
>> before, but I haven't tested.
>> Were you using the physical timer or the virtual timer for the guest?
>>
>> >
>> > Could you have a look?
>>
>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>> but I can work on that tomorrow.
>>
>
> Don't bother, I've figured this out for you.

Thanks a lot.

>
> You need the following fixup to your patch:

Ok. I'll post v4 soon.
You've already do "acked-by" for this commit. Do I need to change it
to "signed-off-by"?

>
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index 93c811c..35d7100 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  }
>
>  /* Make the updates of cntvoff for all vtimer contexts atomic */
> -static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
> +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
>  {
>         int i;
> -       struct kvm_vcpu *vcpu;
> +       struct kvm *kvm = vcpu->kvm;
> +       struct kvm_vcpu *tmp;
>
>         mutex_lock(&kvm->lock);
> -       kvm_for_each_vcpu(i, vcpu, kvm)
> -               vcpu_vtimer(vcpu)->cntvoff = cntvoff;
> +       kvm_for_each_vcpu(i, tmp, kvm)
> +               vcpu_vtimer(tmp)->cntvoff = cntvoff;
> +
> +       /*
> +        * When called from the vcpu create path, the CPU being created is not
> +        * included in the loop above, so we just set it here as well.
> +        */
> +       vcpu_vtimer(vcpu)->cntvoff = cntvoff;
>         mutex_unlock(&kvm->lock);
>  }
>
> @@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>         struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
>
>         /* Synchronize cntvoff across all vtimers of a VM. */
> -       update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +       update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
>         vcpu_ptimer(vcpu)->cntvoff = 0;
>
>         INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
> @@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
>                 vtimer->cnt_ctl = value;
>                 break;
>         case KVM_REG_ARM_TIMER_CNT:
> -               update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
> +               update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
>                 break;
>         case KVM_REG_ARM_TIMER_CVAL:
>                 vtimer->cnt_cval = value;
>
> This is an amuzing one.

nice catch!

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 13:14         ` Jintack Lim
  0 siblings, 0 replies; 74+ messages in thread
From: Jintack Lim @ 2017-02-03 13:14 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> > Hi Jintack,
>> >
>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>> >> and it is reasonable for an OS to expect to be able to access both.
>> >> However, the current KVM implementation does not provide the EL1 physical
>> >> timer to VMs but terminates VMs on access to the timer.
>> >>
>> >> This patch series enables VMs to use the EL1 physical timer through
>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>> >> access and sets up the background timer accordingly.  When the background
>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>> >> physical timer without trapping.  However, this requires somehow using the
>> >> EL2 physical timer for the Linux host while running the VM instead of the
>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>> >> this was straightforward to do, and I leave it to future work to determine
>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>> >> performance benefit.
>> >>
>> >> This feature will be useful for any OS that wishes to access the EL1
>> >> physical timer. Nested virtualization is one of those use cases. A nested
>> >> hypervisor running inside a VM would think it has full access to the
>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>> >> of this patch series. This patch series will make it easy to add the EL2
>> >> timer support in the future, though.
>> >>
>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>> >> and will continue to use only the virtual timer and this patch series will
>> >> therefore not introduce any performance degredation as a result of
>> >> trap-and-emulate.
>> >>
>> >> v2 => v3:
>> >>  - Rebase on kvmarm/queue
>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>> >>  - Remove unnecessary function parameters
>> >>  - Add comments
>> >
>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>> > quickly locks up trying to run cyclictest or when booting the machine it
>> > stalls with RCU timeouts.
>>
>> Ok. It's my fault not to specify that the emulated physical timer is
>> supported/tested on arm64.
>> On 32-bit platform, it is supposed to show the same behavior as
>> before, but I haven't tested.
>> Were you using the physical timer or the virtual timer for the guest?
>>
>> >
>> > Could you have a look?
>>
>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>> but I can work on that tomorrow.
>>
>
> Don't bother, I've figured this out for you.

Thanks a lot.

>
> You need the following fixup to your patch:

Ok. I'll post v4 soon.
You've already do "acked-by" for this commit. Do I need to change it
to "signed-off-by"?

>
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index 93c811c..35d7100 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -410,14 +410,21 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
>  }
>
>  /* Make the updates of cntvoff for all vtimer contexts atomic */
> -static void update_vtimer_cntvoff(struct kvm *kvm, u64 cntvoff)
> +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff)
>  {
>         int i;
> -       struct kvm_vcpu *vcpu;
> +       struct kvm *kvm = vcpu->kvm;
> +       struct kvm_vcpu *tmp;
>
>         mutex_lock(&kvm->lock);
> -       kvm_for_each_vcpu(i, vcpu, kvm)
> -               vcpu_vtimer(vcpu)->cntvoff = cntvoff;
> +       kvm_for_each_vcpu(i, tmp, kvm)
> +               vcpu_vtimer(tmp)->cntvoff = cntvoff;
> +
> +       /*
> +        * When called from the vcpu create path, the CPU being created is not
> +        * included in the loop above, so we just set it here as well.
> +        */
> +       vcpu_vtimer(vcpu)->cntvoff = cntvoff;
>         mutex_unlock(&kvm->lock);
>  }
>
> @@ -426,7 +433,7 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>         struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
>
>         /* Synchronize cntvoff across all vtimers of a VM. */
> -       update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read());
> +       update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
>         vcpu_ptimer(vcpu)->cntvoff = 0;
>
>         INIT_WORK(&timer->expired, kvm_timer_inject_irq_work);
> @@ -448,7 +455,7 @@ int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
>                 vtimer->cnt_ctl = value;
>                 break;
>         case KVM_REG_ARM_TIMER_CNT:
> -               update_vtimer_cntvoff(vcpu->kvm, kvm_phys_timer_read() - value);
> +               update_vtimer_cntvoff(vcpu, kvm_phys_timer_read() - value);
>                 break;
>         case KVM_REG_ARM_TIMER_CVAL:
>                 vtimer->cnt_cval = value;
>
> This is an amuzing one.

nice catch!

>
> Thanks,
> -Christoffer
>

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
  2017-02-03 13:14         ` Jintack Lim
  (?)
@ 2017-02-03 13:34           ` Christoffer Dall
  -1 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 13:34 UTC (permalink / raw)
  To: Jintack Lim
  Cc: Paolo Bonzini, Radim Krčmář,
	Marc Zyngier, Russell King, Catalin Marinas, Will Deacon,
	Andre Przywara, KVM General, arm-mail-list, kvmarm,
	lkml - Kernel Mailing List

On Fri, Feb 3, 2017 at 2:14 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>>> > Hi Jintack,
>>> >
>>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> >> and it is reasonable for an OS to expect to be able to access both.
>>> >> However, the current KVM implementation does not provide the EL1 physical
>>> >> timer to VMs but terminates VMs on access to the timer.
>>> >>
>>> >> This patch series enables VMs to use the EL1 physical timer through
>>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> >> access and sets up the background timer accordingly.  When the background
>>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> >> physical timer without trapping.  However, this requires somehow using the
>>> >> EL2 physical timer for the Linux host while running the VM instead of the
>>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> >> this was straightforward to do, and I leave it to future work to determine
>>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> >> performance benefit.
>>> >>
>>> >> This feature will be useful for any OS that wishes to access the EL1
>>> >> physical timer. Nested virtualization is one of those use cases. A nested
>>> >> hypervisor running inside a VM would think it has full access to the
>>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> >> of this patch series. This patch series will make it easy to add the EL2
>>> >> timer support in the future, though.
>>> >>
>>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> >> and will continue to use only the virtual timer and this patch series will
>>> >> therefore not introduce any performance degredation as a result of
>>> >> trap-and-emulate.
>>> >>
>>> >> v2 => v3:
>>> >>  - Rebase on kvmarm/queue
>>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>> >>  - Remove unnecessary function parameters
>>> >>  - Add comments
>>> >
>>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>>> > quickly locks up trying to run cyclictest or when booting the machine it
>>> > stalls with RCU timeouts.
>>>
>>> Ok. It's my fault not to specify that the emulated physical timer is
>>> supported/tested on arm64.
>>> On 32-bit platform, it is supposed to show the same behavior as
>>> before, but I haven't tested.
>>> Were you using the physical timer or the virtual timer for the guest?
>>>
>>> >
>>> > Could you have a look?
>>>
>>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>>> but I can work on that tomorrow.
>>>
>>
>> Don't bother, I've figured this out for you.
>
> Thanks a lot.
>
>>
>> You need the following fixup to your patch:
>
> Ok. I'll post v4 soon.
> You've already do "acked-by" for this commit. Do I need to change it
> to "signed-off-by"?
>

I guess so, technically.  I don't care deeply though.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* Re: [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 13:34           ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 13:34 UTC (permalink / raw)
  To: Jintack Lim
  Cc: KVM General, Marc Zyngier, Catalin Marinas, Will Deacon,
	Russell King, lkml - Kernel Mailing List, Andre Przywara,
	Paolo Bonzini, kvmarm, arm-mail-list

On Fri, Feb 3, 2017 at 2:14 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>>> > Hi Jintack,
>>> >
>>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> >> and it is reasonable for an OS to expect to be able to access both.
>>> >> However, the current KVM implementation does not provide the EL1 physical
>>> >> timer to VMs but terminates VMs on access to the timer.
>>> >>
>>> >> This patch series enables VMs to use the EL1 physical timer through
>>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> >> access and sets up the background timer accordingly.  When the background
>>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> >> physical timer without trapping.  However, this requires somehow using the
>>> >> EL2 physical timer for the Linux host while running the VM instead of the
>>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> >> this was straightforward to do, and I leave it to future work to determine
>>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> >> performance benefit.
>>> >>
>>> >> This feature will be useful for any OS that wishes to access the EL1
>>> >> physical timer. Nested virtualization is one of those use cases. A nested
>>> >> hypervisor running inside a VM would think it has full access to the
>>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> >> of this patch series. This patch series will make it easy to add the EL2
>>> >> timer support in the future, though.
>>> >>
>>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> >> and will continue to use only the virtual timer and this patch series will
>>> >> therefore not introduce any performance degredation as a result of
>>> >> trap-and-emulate.
>>> >>
>>> >> v2 => v3:
>>> >>  - Rebase on kvmarm/queue
>>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>> >>  - Remove unnecessary function parameters
>>> >>  - Add comments
>>> >
>>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>>> > quickly locks up trying to run cyclictest or when booting the machine it
>>> > stalls with RCU timeouts.
>>>
>>> Ok. It's my fault not to specify that the emulated physical timer is
>>> supported/tested on arm64.
>>> On 32-bit platform, it is supposed to show the same behavior as
>>> before, but I haven't tested.
>>> Were you using the physical timer or the virtual timer for the guest?
>>>
>>> >
>>> > Could you have a look?
>>>
>>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>>> but I can work on that tomorrow.
>>>
>>
>> Don't bother, I've figured this out for you.
>
> Thanks a lot.
>
>>
>> You need the following fixup to your patch:
>
> Ok. I'll post v4 soon.
> You've already do "acked-by" for this commit. Do I need to change it
> to "signed-off-by"?
>

I guess so, technically.  I don't care deeply though.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

* [RFC v3 00/10] Provide the EL1 physical timer to the VM
@ 2017-02-03 13:34           ` Christoffer Dall
  0 siblings, 0 replies; 74+ messages in thread
From: Christoffer Dall @ 2017-02-03 13:34 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Feb 3, 2017 at 2:14 PM, Jintack Lim <jintack@cs.columbia.edu> wrote:
> On Fri, Feb 3, 2017 at 7:33 AM, Christoffer Dall <cdall@linaro.org> wrote:
>> On Thu, Feb 02, 2017 at 09:51:13AM -0500, Jintack Lim wrote:
>>> On Thu, Feb 2, 2017 at 7:31 AM, Christoffer Dall <cdall@linaro.org> wrote:
>>> > Hi Jintack,
>>> >
>>> > On Wed, Feb 01, 2017 at 12:43:00PM -0500, Jintack Lim wrote:
>>> >> The ARM architecture defines the EL1 physical timer and the virtual timer,
>>> >> and it is reasonable for an OS to expect to be able to access both.
>>> >> However, the current KVM implementation does not provide the EL1 physical
>>> >> timer to VMs but terminates VMs on access to the timer.
>>> >>
>>> >> This patch series enables VMs to use the EL1 physical timer through
>>> >> trap-and-emulate.  The KVM host emulates each EL1 physical timer register
>>> >> access and sets up the background timer accordingly.  When the background
>>> >> timer expires, the KVM host injects EL1 physical timer interrupts to the
>>> >> VM.  Alternatively, it's also possible to allow VMs to access the EL1
>>> >> physical timer without trapping.  However, this requires somehow using the
>>> >> EL2 physical timer for the Linux host while running the VM instead of the
>>> >> EL1 physical timer.  Right now I just implemented trap-and-emulate because
>>> >> this was straightforward to do, and I leave it to future work to determine
>>> >> if transferring the EL1 physical timer state to the EL2 timer provides any
>>> >> performance benefit.
>>> >>
>>> >> This feature will be useful for any OS that wishes to access the EL1
>>> >> physical timer. Nested virtualization is one of those use cases. A nested
>>> >> hypervisor running inside a VM would think it has full access to the
>>> >> hardware and naturally tries to use the EL1 physical timer as Linux would
>>> >> do. Other nested hypervisors may try to use the EL2 physical timer as Xen
>>> >> would do, but supporting the EL2 physical timer to the VM is out of scope
>>> >> of this patch series. This patch series will make it easy to add the EL2
>>> >> timer support in the future, though.
>>> >>
>>> >> Note that Linux VMs booting in EL1 will be unaffected by this patch series
>>> >> and will continue to use only the virtual timer and this patch series will
>>> >> therefore not introduce any performance degredation as a result of
>>> >> trap-and-emulate.
>>> >>
>>> >> v2 => v3:
>>> >>  - Rebase on kvmarm/queue
>>> >>  - Take kvm->lock to synchronize cntvoff across all vtimers
>>> >>  - Remove unnecessary function parameters
>>> >>  - Add comments
>>> >
>>> > I just gave v3 a test run on my TC2 (32-bit platform) and my guest
>>> > quickly locks up trying to run cyclictest or when booting the machine it
>>> > stalls with RCU timeouts.
>>>
>>> Ok. It's my fault not to specify that the emulated physical timer is
>>> supported/tested on arm64.
>>> On 32-bit platform, it is supposed to show the same behavior as
>>> before, but I haven't tested.
>>> Were you using the physical timer or the virtual timer for the guest?
>>>
>>> >
>>> > Could you have a look?
>>>
>>> Sure, I'll have a look. I don't have access to my Cubietruck today,
>>> but I can work on that tomorrow.
>>>
>>
>> Don't bother, I've figured this out for you.
>
> Thanks a lot.
>
>>
>> You need the following fixup to your patch:
>
> Ok. I'll post v4 soon.
> You've already do "acked-by" for this commit. Do I need to change it
> to "signed-off-by"?
>

I guess so, technically.  I don't care deeply though.

-Christoffer

^ permalink raw reply	[flat|nested] 74+ messages in thread

end of thread, other threads:[~2017-02-03 13:34 UTC | newest]

Thread overview: 74+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-01 17:43 [RFC v3 00/10] Provide the EL1 physical timer to the VM Jintack Lim
2017-02-01 17:43 ` Jintack Lim
2017-02-01 17:43 ` Jintack Lim
2017-02-01 17:43 ` [RFC v3 01/10] KVM: arm/arm64: Abstract virtual timer context into separate structure Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43 ` [RFC v3 02/10] KVM: arm/arm64: Move cntvoff to each timer context Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:03   ` Christoffer Dall
2017-02-02 10:03     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 03/10] KVM: arm/arm64: Decouple kvm timer functions from virtual timer Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:04   ` Christoffer Dall
2017-02-02 10:04     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 04/10] KVM: arm/arm64: Add the EL1 physical timer context Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:15   ` Christoffer Dall
2017-02-02 10:15     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 05/10] KVM: arm/arm64: Initialize the emulated EL1 physical timer Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:15   ` Christoffer Dall
2017-02-02 10:15     ` Christoffer Dall
2017-02-02 10:15     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 06/10] KVM: arm/arm64: Update the physical timer interrupt level Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:15   ` Christoffer Dall
2017-02-02 10:15     ` Christoffer Dall
2017-02-02 10:15     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 07/10] KVM: arm/arm64: Set a background timer to the earliest timer expiration Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:30   ` Christoffer Dall
2017-02-02 10:30     ` Christoffer Dall
2017-02-02 10:30     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 08/10] KVM: arm/arm64: Set up a background timer for the physical timer emulation Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:30   ` Christoffer Dall
2017-02-02 10:30     ` Christoffer Dall
2017-02-02 10:30     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 09/10] KVM: arm64: Add the EL1 physical timer access handler Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:31   ` Christoffer Dall
2017-02-02 10:31     ` Christoffer Dall
2017-02-02 10:31     ` Christoffer Dall
2017-02-01 17:43 ` [RFC v3 10/10] KVM: arm/arm64: Emulate the EL1 phys timer registers Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-01 17:43   ` Jintack Lim
2017-02-02 10:31   ` Christoffer Dall
2017-02-02 10:31     ` Christoffer Dall
2017-02-02 10:31     ` Christoffer Dall
2017-02-02 12:31 ` [RFC v3 00/10] Provide the EL1 physical timer to the VM Christoffer Dall
2017-02-02 12:31   ` Christoffer Dall
2017-02-02 12:31   ` Christoffer Dall
2017-02-02 14:51   ` Jintack Lim
2017-02-02 14:51     ` Jintack Lim
2017-02-02 14:51     ` Jintack Lim
2017-02-02 15:08     ` Christoffer Dall
2017-02-02 15:08       ` Christoffer Dall
2017-02-02 15:08       ` Christoffer Dall
2017-02-03 12:33     ` Christoffer Dall
2017-02-03 12:33       ` Christoffer Dall
2017-02-03 12:33       ` Christoffer Dall
2017-02-03 13:14       ` Jintack Lim
2017-02-03 13:14         ` Jintack Lim
2017-02-03 13:14         ` Jintack Lim
2017-02-03 13:34         ` Christoffer Dall
2017-02-03 13:34           ` Christoffer Dall
2017-02-03 13:34           ` Christoffer Dall

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.