Re: KVM Arm64 and Linux-RT issues

[Marc Zyngier <maz@kernel.org>]

On Tue, 13 Aug 2019 16:44:21 +0100,
Julien Grall <julien.grall@arm.com> wrote:
> 
> Hi Sebastian,
> 
> On 8/13/19 1:58 PM, bigeasy@linutronix.de wrote:
> > On 2019-07-27 14:37:11 [+0100], Julien Grall wrote:
> >>>> 8<------------
> >>>> --- a/virt/kvm/arm/arch_timer.c
> >>>> +++ b/virt/kvm/arm/arch_timer.c
> >>>> @@ -80,7 +80,7 @@ static inline bool userspace_irqchip(str
> >>>>    static void soft_timer_start(struct hrtimer *hrt, u64 ns)
> >>>>    {
> >>>>    	hrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),
> >>>> -		      HRTIMER_MODE_ABS);
> >>>> +		      HRTIMER_MODE_ABS_HARD);
> >>>>    }
> >>> 
> >>> That's pretty neat, and matches the patch you already have for
> >>> x86. Feel free to add my
> >>> 
> >>> Acked-by: Marc Zyngier <maz@kernel.org>
> >> 
> >> I can confirm the warning now disappeared. Feel free to added my tested-by:
> >> 
> >> Tested-by: Julien Grall <julien.grall@arm.com>
> >> 
> > 
> > |kvm_hrtimer_expire()
> > | kvm_timer_update_irq()
> > |   kvm_vgic_inject_irq()
> > |     vgic_lazy_init()
> > |                if (unlikely(!vgic_initialized(kvm))) {
> > |                 if (kvm->arch.vgic.vgic_model != KVM_DEV_TYPE_ARM_VGIC_V2)
> > |                         return -EBUSY;
> > |
> > |                 mutex_lock(&kvm->lock);
> > 
> > Is this possible path of any concern? This should throw a warning also
> > for !RT so probably not…
> 
> Hmmm, theoretically yes. In practice, it looks like the hrtimer will
> not be started before kvm_vcpu_first_run_init() is called on the first
> run.

Exactly. Even if you restore the timer in a "firing" configuration,
you'll have to execute the vgic init before any background timer gets
programmed, let alone expired.

Yes, the interface is terrible.

> The function will call kvm_vgic_map_resources() which will initialize
> the vgic if not already done.
> 
> Looking around, I think this is here to cater the case where
> KVM_IRQ_LINE is called before running.
> 
> I am not yet familiar with the vgic, so I may have missed something.
> 
> > 
> > I prepared the patch below. This one could go straight to tglx's timer tree
> > since he has the _HARD bits there. I *think* it requires to set the bits
> > _HARD during _init() and _start() otherwise there is (or was) a warning…
> > 
> > Sebastian
> > 8<------------
> > 
> > From: Thomas Gleixner <tglx@linutronix.de>
> > Date: Tue, 13 Aug 2019 14:29:41 +0200
> > Subject: [PATCH] KVM: arm/arm64: Let the timer expire in hardirq context on RT
> > 
> > The timers are canceled from an preempt-notifier which is invoked with
> > disabled preemption which is not allowed on PREEMPT_RT.
> > The timer callback is short so in could be invoked in hard-IRQ context
> > on -RT.
> > 
> > Let the timer expire on hard-IRQ context even on -RT.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> > Acked-by: Marc Zyngier <maz@kernel.org>
> > Tested-by: Julien Grall <julien.grall@arm.com>
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> > ---
> >   virt/kvm/arm/arch_timer.c | 8 ++++----
> >   1 file changed, 4 insertions(+), 4 deletions(-)
> > 
> > diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> > index 1be486d5d7cb4..0bfa7c5b5c890 100644
> > --- a/virt/kvm/arm/arch_timer.c
> > +++ b/virt/kvm/arm/arch_timer.c
> > @@ -80,7 +80,7 @@ static inline bool userspace_irqchip(struct kvm *kvm)
> >   static void soft_timer_start(struct hrtimer *hrt, u64 ns)
> >   {
> >   	hrtimer_start(hrt, ktime_add_ns(ktime_get(), ns),
> > -		      HRTIMER_MODE_ABS);
> > +		      HRTIMER_MODE_ABS_HARD);
> >   }
> >     static void soft_timer_cancel(struct hrtimer *hrt)
> > @@ -697,11 +697,11 @@ void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu)
> >   	update_vtimer_cntvoff(vcpu, kvm_phys_timer_read());
> >   	ptimer->cntvoff = 0;
> >   -	hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC,
> > HRTIMER_MODE_ABS);
> > +	hrtimer_init(&timer->bg_timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
> >   	timer->bg_timer.function = kvm_bg_timer_expire;
> >   -	hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC,
> > HRTIMER_MODE_ABS);
> > -	hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS);
> > +	hrtimer_init(&vtimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
> > +	hrtimer_init(&ptimer->hrtimer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_HARD);
> >   	vtimer->hrtimer.function = kvm_hrtimer_expire;
> >   	ptimer->hrtimer.function = kvm_hrtimer_expire;
> >   

Patch looks fine, please add it to the pile of RT stuff! ;-)

Thanks,

	M.

-- 
Jazz is not dead, it just smells funny.