Re: KVM Arm64 and Linux-RT issues

[Julien Grall <julien.grall@arm.com>]

Hi Sebastian,

On 8/13/19 1:58 PM, bigeasy@linutronix.de wrote:
> On 2019-07-27 14:37:11 [+0100], Julien Grall wrote:
>>>> 8<------------
>>>> --- a/virt/kvm/arm/arch_timer.c
>>>> +++ b/virt/kvm/arm/arch_timer.c
>>>> @@ -80,7 +80,7 @@ static inline bool userspace_irqchip(str
>>>>    static void soft_timer_start(struct hrtimer *hrt, u64 ns)
>>>>    {
>>>>    	hrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),
>>>> -		      HRTIMER_MODE_ABS);
>>>> +		      HRTIMER_MODE_ABS_HARD);
>>>>    }
>>>
>>> That's pretty neat, and matches the patch you already have for
>>> x86. Feel free to add my
>>>
>>> Acked-by: Marc Zyngier <maz@kernel.org>
>>
>> I can confirm the warning now disappeared. Feel free to added my tested-by:
>>
>> Tested-by: Julien Grall <julien.grall@arm.com>
>>
> 
> |kvm_hrtimer_expire()
> | kvm_timer_update_irq()
> |   kvm_vgic_inject_irq()
> |     vgic_lazy_init()
> |                if (unlikely(!vgic_initialized(kvm))) {
> |                 if (kvm->arch.vgic.vgic_model != KVM_DEV_TYPE_ARM_VGIC_V2)
> |                         return -EBUSY;
> |
> |                 mutex_lock(&kvm->lock);
> 
> Is this possible path of any concern? This should throw a warning also
> for !RT so probably not…

Hmmm, theoretically yes. In practice, it looks like the hrtimer will not 
be started before kvm_vcpu_first_run_init() is called on the first run.

The function will call kvm_vgic_map_resources() which will initialize 
the vgic if not already done.

Looking around, I think this is here to cater the case where 
KVM_IRQ_LINE is called before running.

I am not yet familiar with the vgic, so I may have missed something.

> 
> I prepared the patch below. This one could go straight to tglx's timer tree
> since he has the _HARD bits there. I *think* it requires to set the bits
> _HARD during _init() and _start() otherwise there is (or was) a warning…
> 
> Sebastian
> 8<------------
> 
> From: Thomas Gleixner <tglx@linutronix.de>
> Date: Tue, 13 Aug 2019 14:29:41 +0200
> Subject: [PATCH] KVM: arm/arm64: Let the timer expire in hardirq context on RT
> 
> The timers are canceled from an preempt-notifier which is invoked with
> disabled preemption which is not allowed on PREEMPT_RT.
> The timer callback is short so in could be invoked in hard-IRQ context
> on -RT.
> 
> Let the timer expire on hard-IRQ context even on -RT.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Acked-by: Marc Zyngier <maz@kernel.org>
> Tested-by: Julien Grall <julien.grall@arm.com>
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
>   virt/kvm/arm/arch_timer.c | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index 1be486d5d7cb4..0bfa7c5b5c890 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -80,7 +80,7 @@ static inline bool userspace_irqchip(struct kvm *kvm)
>   static void soft_timer_start(struct hrtimer *hrt, u64 ns)
>   {
>   	hrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),
> -		      HRTIMER_MODE_ABS);
> +		      HRTIMER_MODE_ABS_HARD);
>   }
>   
>   static void soft_timer_cancel(struct hrtimer *hrt)
> @@ -697,11 +697,11 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
>   	update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
>   	ptimer->cntvoff = 0;
>   
> -	hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> +	hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
>   	timer->bg_timer.function = kvm_bg_timer_expire;
>   
> -	hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> -	hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> +	hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
> +	hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
>   	vtimer->hrtimer.function = kvm_hrtimer_expire;
>   	ptimer->hrtimer.function = kvm_hrtimer_expire;
>   
> 

-- 
Julien Grall
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm