[PATCH] KVM: Disable irq while unregistering user notifier

[PATCH] KVM: Disable irq while unregistering user notifier

[Ignacio Alvarado]

Function user_notifier_unregister should be called only once for each
registered user notifier.

Function kvm_arch_hardware_disable can be executed from an IPI context
which could cause a race condition with a VCPU returning to user mode
and attempting to unregister the notifier.

Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
---
 arch/x86/kvm/x86.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e375235..51f87f0 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -210,7 +210,15 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
 	struct kvm_shared_msrs *locals
 		= container_of(urn, struct kvm_shared_msrs, urn);
 	struct kvm_shared_msr_values *values;
+	unsigned long flags;
 
+	/*
+	 * Disabling irqs at this point since the following code could be
+	 * interrupted and executed through kvm_arch_hardware_disable()
+	 */
+	local_irq_save(flags);
+	if (!locals->registered)
+		goto out;
 	for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
 		values = &locals->values[slot];
 		if (values->host != values->curr) {
@@ -220,6 +228,8 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
 	}
 	locals->registered = false;
 	user_return_notifier_unregister(urn);
+out:
+	local_irq_restore(flags);
 }
 
 static void shared_msr_update(unsigned slot, u32 msr)
-- 
2.8.0.rc3.226.g39d4020

Re: [PATCH] KVM: Disable irq while unregistering user notifier

[Paolo Bonzini]

On 22/10/2016 00:08, Ignacio Alvarado wrote:
> Function user_notifier_unregister should be called only once for each
> registered user notifier.
> 
> Function kvm_arch_hardware_disable can be executed from an IPI context
> which could cause a race condition with a VCPU returning to user mode
> and attempting to unregister the notifier.
> 
> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>

Should drop_user_return_notifiers instead be moved to kvm_arch_exit?

Thanks,

Paolo

> ---
>  arch/x86/kvm/x86.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e375235..51f87f0 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -210,7 +210,15 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>  	struct kvm_shared_msrs *locals
>  		= container_of(urn, struct kvm_shared_msrs, urn);
>  	struct kvm_shared_msr_values *values;
> +	unsigned long flags;
>  
> +	/*
> +	 * Disabling irqs at this point since the following code could be
> +	 * interrupted and executed through kvm_arch_hardware_disable()
> +	 */
> +	local_irq_save(flags);
> +	if (!locals->registered)
> +		goto out;
>  	for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
>  		values = &locals->values[slot];
>  		if (values->host != values->curr) {
> @@ -220,6 +228,8 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>  	}
>  	locals->registered = false;
>  	user_return_notifier_unregister(urn);
> +out:
> +	local_irq_restore(flags);
>  }
>  
>  static void shared_msr_update(unsigned slot, u32 msr)
> 

Re: [PATCH] KVM: Disable irq while unregistering user notifier

[Ignacio Alvarado]

The risk would still be present since fire_user_return_notifiers could
potentially run at any time and kvm_on_user return could be called
from drop_user_return_notifiers either from the kvm_put_kvm path or
the kvm_arch_exit path.

Regards

On Wed, Nov 2, 2016 at 10:25 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 22/10/2016 00:08, Ignacio Alvarado wrote:
>> Function user_notifier_unregister should be called only once for each
>> registered user notifier.
>>
>> Function kvm_arch_hardware_disable can be executed from an IPI context
>> which could cause a race condition with a VCPU returning to user mode
>> and attempting to unregister the notifier.
>>
>> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
>
> Should drop_user_return_notifiers instead be moved to kvm_arch_exit?
>
> Thanks,
>
> Paolo
>
>> ---
>>  arch/x86/kvm/x86.c | 10 ++++++++++
>>  1 file changed, 10 insertions(+)
>>
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> index e375235..51f87f0 100644
>> --- a/arch/x86/kvm/x86.c
>> +++ b/arch/x86/kvm/x86.c
>> @@ -210,7 +210,15 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>       struct kvm_shared_msrs *locals
>>               = container_of(urn, struct kvm_shared_msrs, urn);
>>       struct kvm_shared_msr_values *values;
>> +     unsigned long flags;
>>
>> +     /*
>> +      * Disabling irqs at this point since the following code could be
>> +      * interrupted and executed through kvm_arch_hardware_disable()
>> +      */
>> +     local_irq_save(flags);
>> +     if (!locals->registered)
>> +             goto out;
>>       for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
>>               values = &locals->values[slot];
>>               if (values->host != values->curr) {
>> @@ -220,6 +228,8 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>       }
>>       locals->registered = false;
>>       user_return_notifier_unregister(urn);
>> +out:
>> +     local_irq_restore(flags);
>>  }
>>
>>  static void shared_msr_update(unsigned slot, u32 msr)
>>



-- 
Ignacio Alvarado

Re: [PATCH] KVM: Disable irq while unregistering user notifier

[Paolo Bonzini]

On 03/11/2016 23:26, Ignacio Alvarado wrote:
> The risk would still be present since fire_user_return_notifiers could
> potentially run at any time and kvm_on_user return could be called
> from drop_user_return_notifiers either from the kvm_put_kvm path or
> the kvm_arch_exit path.

So, do the unregistration

        locals->registered = false;
        user_return_notifier_unregister(urn);

first in kvm_on_user_return?  Nothing else needs to run with IRQs
disabled, and doing it first prevents reentering the code from
kvm_arch_hardware_disable.

Paolo

> Regards
> 
> On Wed, Nov 2, 2016 at 10:25 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>>
>> On 22/10/2016 00:08, Ignacio Alvarado wrote:
>>> Function user_notifier_unregister should be called only once for each
>>> registered user notifier.
>>>
>>> Function kvm_arch_hardware_disable can be executed from an IPI context
>>> which could cause a race condition with a VCPU returning to user mode
>>> and attempting to unregister the notifier.
>>>
>>> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
>>
>> Should drop_user_return_notifiers instead be moved to kvm_arch_exit?
>>
>> Thanks,
>>
>> Paolo
>>
>>> ---
>>>  arch/x86/kvm/x86.c | 10 ++++++++++
>>>  1 file changed, 10 insertions(+)
>>>
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index e375235..51f87f0 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -210,7 +210,15 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>>       struct kvm_shared_msrs *locals
>>>               = container_of(urn, struct kvm_shared_msrs, urn);
>>>       struct kvm_shared_msr_values *values;
>>> +     unsigned long flags;
>>>
>>> +     /*
>>> +      * Disabling irqs at this point since the following code could be
>>> +      * interrupted and executed through kvm_arch_hardware_disable()
>>> +      */
>>> +     local_irq_save(flags);
>>> +     if (!locals->registered)
>>> +             goto out;
>>>       for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
>>>               values = &locals->values[slot];
>>>               if (values->host != values->curr) {
>>> @@ -220,6 +228,8 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>>       }
>>>       locals->registered = false;
>>>       user_return_notifier_unregister(urn);
>>> +out:
>>> +     local_irq_restore(flags);
>>>  }
>>>
>>>  static void shared_msr_update(unsigned slot, u32 msr)
>>>
> 
> 
> 

Re: [PATCH] KVM: Disable irq while unregistering user notifier

[Ignacio Alvarado]

On Fri, Nov 4, 2016 at 2:48 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
>
> On 03/11/2016 23:26, Ignacio Alvarado wrote:
>> The risk would still be present since fire_user_return_notifiers could
>> potentially run at any time and kvm_on_user return could be called
>> from drop_user_return_notifiers either from the kvm_put_kvm path or
>> the kvm_arch_exit path.
>
> So, do the unregistration
>
>         locals->registered = false;
>         user_return_notifier_unregister(urn);
>
> first in kvm_on_user_return?  Nothing else needs to run with IRQs
> disabled, and doing it first prevents reentering the code from
> kvm_arch_hardware_disable.
>

Will do.

> Paolo
>
>> Regards
>>
>> On Wed, Nov 2, 2016 at 10:25 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>>
>>>
>>> On 22/10/2016 00:08, Ignacio Alvarado wrote:
>>>> Function user_notifier_unregister should be called only once for each
>>>> registered user notifier.
>>>>
>>>> Function kvm_arch_hardware_disable can be executed from an IPI context
>>>> which could cause a race condition with a VCPU returning to user mode
>>>> and attempting to unregister the notifier.
>>>>
>>>> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
>>>
>>> Should drop_user_return_notifiers instead be moved to kvm_arch_exit?
>>>
>>> Thanks,
>>>
>>> Paolo
>>>
>>>> ---
>>>>  arch/x86/kvm/x86.c | 10 ++++++++++
>>>>  1 file changed, 10 insertions(+)
>>>>
>>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>>> index e375235..51f87f0 100644
>>>> --- a/arch/x86/kvm/x86.c
>>>> +++ b/arch/x86/kvm/x86.c
>>>> @@ -210,7 +210,15 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>>>       struct kvm_shared_msrs *locals
>>>>               = container_of(urn, struct kvm_shared_msrs, urn);
>>>>       struct kvm_shared_msr_values *values;
>>>> +     unsigned long flags;
>>>>
>>>> +     /*
>>>> +      * Disabling irqs at this point since the following code could be
>>>> +      * interrupted and executed through kvm_arch_hardware_disable()
>>>> +      */
>>>> +     local_irq_save(flags);
>>>> +     if (!locals->registered)
>>>> +             goto out;
>>>>       for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
>>>>               values = &locals->values[slot];
>>>>               if (values->host != values->curr) {
>>>> @@ -220,6 +228,8 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>>>>       }
>>>>       locals->registered = false;
>>>>       user_return_notifier_unregister(urn);
>>>> +out:
>>>> +     local_irq_restore(flags);
>>>>  }
>>>>
>>>>  static void shared_msr_update(unsigned slot, u32 msr)
>>>>
>>
>>
>>



-- 
Ignacio Alvarado

[PATCH v2] KVM: Disable irq while unregistering user notifier

[Ignacio Alvarado]

Function user_notifier_unregister should be called only once for each
registered user notifier.

Function kvm_arch_hardware_disable can be executed from an IPI context
which could cause a race condition with a VCPU returning to user mode
and attempting to unregister the notifier.

Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
---
Changelog since v1:
- Move unregistration to the beginning of kvm_on_user_return

 arch/x86/kvm/x86.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index e375235..952e19d 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -210,7 +210,18 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
 	struct kvm_shared_msrs *locals
 		= container_of(urn, struct kvm_shared_msrs, urn);
 	struct kvm_shared_msr_values *values;
+	unsigned long flags;
 
+	/*
+	 * Disabling irqs at this point since the following code could be
+	 * interrupted and executed through kvm_arch_hardware_disable()
+	 */
+	local_irq_save(flags);
+	if (locals->registered) {
+		locals->registered = false;
+		user_return_notifier_unregister(urn);
+	}
+	local_irq_restore(flags);
 	for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
 		values = &locals->values[slot];
 		if (values->host != values->curr) {
@@ -218,8 +229,6 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
 			values->curr = values->host;
 		}
 	}
-	locals->registered = false;
-	user_return_notifier_unregister(urn);
 }
 
 static void shared_msr_update(unsigned slot, u32 msr)
-- 
2.8.0.rc3.226.g39d4020

Re: [PATCH v2] KVM: Disable irq while unregistering user notifier

[Paolo Bonzini]

On 04/11/2016 20:15, Ignacio Alvarado wrote:
> Function user_notifier_unregister should be called only once for each
> registered user notifier.
> 
> Function kvm_arch_hardware_disable can be executed from an IPI context
> which could cause a race condition with a VCPU returning to user mode
> and attempting to unregister the notifier.
> 
> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>

Cc: stable@vger.kernel.org
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

> ---
> Changelog since v1:
> - Move unregistration to the beginning of kvm_on_user_return
> 
>  arch/x86/kvm/x86.c | 13 +++++++++++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index e375235..952e19d 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -210,7 +210,18 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>  	struct kvm_shared_msrs *locals
>  		= container_of(urn, struct kvm_shared_msrs, urn);
>  	struct kvm_shared_msr_values *values;
> +	unsigned long flags;
>  
> +	/*
> +	 * Disabling irqs at this point since the following code could be
> +	 * interrupted and executed through kvm_arch_hardware_disable()
> +	 */
> +	local_irq_save(flags);
> +	if (locals->registered) {
> +		locals->registered = false;
> +		user_return_notifier_unregister(urn);
> +	}
> +	local_irq_restore(flags);
>  	for (slot = 0; slot < shared_msrs_global.nr; ++slot) {
>  		values = &locals->values[slot];
>  		if (values->host != values->curr) {
> @@ -218,8 +229,6 @@ static void kvm_on_user_return(struct user_return_notifier *urn)
>  			values->curr = values->host;
>  		}
>  	}
> -	locals->registered = false;
> -	user_return_notifier_unregister(urn);
>  }
>  
>  static void shared_msr_update(unsigned slot, u32 msr)
> 

Re: [PATCH v2] KVM: Disable irq while unregistering user notifier

[Radim Krčmář]

2016-11-17 13:42+0100, Paolo Bonzini:
> On 04/11/2016 20:15, Ignacio Alvarado wrote:
>> Function user_notifier_unregister should be called only once for each
>> registered user notifier.
>> 
>> Function kvm_arch_hardware_disable can be executed from an IPI context
>> which could cause a race condition with a VCPU returning to user mode
>> and attempting to unregister the notifier.
>> 
>> Signed-off-by: Ignacio Alvarado <ikalvarado@google.com>
> 
> Cc: stable@vger.kernel.org
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Applied to kvm/master, thanks.