[PATCH] KVM: SVM: Fix x2apic logical cluster mode decoding and sanity check

[PATCH] KVM: SVM: Fix x2apic logical cluster mode decoding and sanity check

[Suravee Suthikulpanit]

When sending IPI in the X2APIC logical cluster mode, the destination
APIC ID is encoded as:

  * Cluster ID = ICRH[31:16]
  * Logical ID = ICRH[15:0]

Current logic incorrectly decode the ICRH, which causes VM running
with x2AVIC support to fail to boot. Therefore, fix the decoding logic.

The commit 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
also added a check for multiple logical destinations before using
the fast-path. However, the same logic is already existed prior to
the commit. Therefore, remove redundant checking logic.

Fixes: 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
Cc: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
---
 arch/x86/kvm/svm/avic.c | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
index 6919dee69f18..45ab49d1f0b8 100644
--- a/arch/x86/kvm/svm/avic.c
+++ b/arch/x86/kvm/svm/avic.c
@@ -378,8 +378,8 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 
 		if (apic_x2apic_mode(source)) {
 			/* 16 bit dest mask, 16 bit cluster id */
-			bitmap = dest & 0xFFFF0000;
-			cluster = (dest >> 16) << 4;
+			bitmap = dest & 0xffff;
+			cluster = (dest & 0xffff0000) >> 16;
 		} else if (kvm_lapic_get_reg(source, APIC_DFR) == APIC_DFR_FLAT) {
 			/* 8 bit dest mask*/
 			bitmap = dest;
@@ -387,7 +387,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 		} else {
 			/* 4 bit desk mask, 4 bit cluster id */
 			bitmap = dest & 0xF;
-			cluster = (dest >> 4) << 2;
+			cluster = (dest & 0xf0) >> 4;
 		}
 
 		if (unlikely(!bitmap))
@@ -420,18 +420,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
 			 * For x2APIC logical mode, cannot leverage the index.
 			 * Instead, calculate physical ID from logical ID in ICRH.
 			 */
-			int cluster = (icrh & 0xffff0000) >> 16;
-			int apic = ffs(icrh & 0xffff) - 1;
-
-			/*
-			 * If the x2APIC logical ID sub-field (i.e. icrh[15:0])
-			 * contains anything but a single bit, we cannot use the
-			 * fast path, because it is limited to a single vCPU.
-			 */
-			if (apic < 0 || icrh != (1 << apic))
-				return -EINVAL;
-
-			l1_physical_id = (cluster << 4) + apic;
+			l1_physical_id = (cluster << 4) + (ffs(bitmap) - 1);
 		}
 	}
 
-- 
2.34.1

Re: [PATCH] KVM: SVM: Fix x2apic logical cluster mode decoding and sanity check

[Sean Christopherson]

On Mon, Sep 12, 2022, Suravee Suthikulpanit wrote:
> When sending IPI in the X2APIC logical cluster mode, the destination
> APIC ID is encoded as:
> 
>   * Cluster ID = ICRH[31:16]
>   * Logical ID = ICRH[15:0]
> 
> Current logic incorrectly decode the ICRH, which causes VM running
> with x2AVIC support to fail to boot. Therefore, fix the decoding logic.

There are already patches pending[1][2] for the x2AVIC bugs.  

[1] https://lore.kernel.org/all/20220903002254.2411750-9-seanjc@google.com
[2] https://lore.kernel.org/all/20220903002254.2411750-10-seanjc@google.com

> 
> The commit 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
> also added a check for multiple logical destinations before using
> the fast-path. However, the same logic is already existed prior to
> the commit. Therefore, remove redundant checking logic.
> 
> Fixes: 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> ---
>  arch/x86/kvm/svm/avic.c | 19 ++++---------------
>  1 file changed, 4 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
> index 6919dee69f18..45ab49d1f0b8 100644
> --- a/arch/x86/kvm/svm/avic.c
> +++ b/arch/x86/kvm/svm/avic.c
> @@ -378,8 +378,8 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
>  
>  		if (apic_x2apic_mode(source)) {
>  			/* 16 bit dest mask, 16 bit cluster id */
> -			bitmap = dest & 0xFFFF0000;
> -			cluster = (dest >> 16) << 4;
> +			bitmap = dest & 0xffff;
> +			cluster = (dest & 0xffff0000) >> 16;
>  		} else if (kvm_lapic_get_reg(source, APIC_DFR) == APIC_DFR_FLAT) {
>  			/* 8 bit dest mask*/
>  			bitmap = dest;
> @@ -387,7 +387,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
>  		} else {
>  			/* 4 bit desk mask, 4 bit cluster id */
>  			bitmap = dest & 0xF;
> -			cluster = (dest >> 4) << 2;
> +			cluster = (dest & 0xf0) >> 4;

This is wrong and unrelated.  The cluster needs to be shifted back by 2, i.e. multiplied
by 4, to leap over each cluster of 4 IDs.

>  		}
>  
>  		if (unlikely(!bitmap))
> @@ -420,18 +420,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
>  			 * For x2APIC logical mode, cannot leverage the index.
>  			 * Instead, calculate physical ID from logical ID in ICRH.
>  			 */
> -			int cluster = (icrh & 0xffff0000) >> 16;
> -			int apic = ffs(icrh & 0xffff) - 1;
> -
> -			/*
> -			 * If the x2APIC logical ID sub-field (i.e. icrh[15:0])
> -			 * contains anything but a single bit, we cannot use the
> -			 * fast path, because it is limited to a single vCPU.
> -			 */
> -			if (apic < 0 || icrh != (1 << apic))
> -				return -EINVAL;
> -
> -			l1_physical_id = (cluster << 4) + apic;
> +			l1_physical_id = (cluster << 4) + (ffs(bitmap) - 1);
>  		}
>  	}
>  
> -- 
> 2.34.1
> 

Re: [PATCH] KVM: SVM: Fix x2apic logical cluster mode decoding and sanity check

[Suthikulpanit, Suravee]

On 9/13/2022 12:29 AM, Sean Christopherson wrote:
> On Mon, Sep 12, 2022, Suravee Suthikulpanit wrote:
>> When sending IPI in the X2APIC logical cluster mode, the destination
>> APIC ID is encoded as:
>>
>>    * Cluster ID = ICRH[31:16]
>>    * Logical ID = ICRH[15:0]
>>
>> Current logic incorrectly decode the ICRH, which causes VM running
>> with x2AVIC support to fail to boot. Therefore, fix the decoding logic.
> 
> There are already patches pending[1][2] for the x2AVIC bugs.
> 
> [1] https://lore.kernel.org/all/20220903002254.2411750-9-seanjc@xxxxxxxxxx
> [2] https://lore.kernel.org/all/20220903002254.2411750-10-seanjc@xxxxxxxxxx

Ok. Thanks for the pointer.

>>
>> The commit 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
>> also added a check for multiple logical destinations before using
>> the fast-path. However, the same logic is already existed prior to
>> the commit. Therefore, remove redundant checking logic.
>>
>> Fixes: 603ccef42ce9 ("KVM: x86: SVM: fix avic_kick_target_vcpus_fast")
>> Cc: Maxim Levitsky <mlevitsk@redhat.com>
>> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>> ---
>>   arch/x86/kvm/svm/avic.c | 19 ++++---------------
>>   1 file changed, 4 insertions(+), 15 deletions(-)
>>
>> diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
>> index 6919dee69f18..45ab49d1f0b8 100644
>> --- a/arch/x86/kvm/svm/avic.c
>> +++ b/arch/x86/kvm/svm/avic.c
>> @@ -378,8 +378,8 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
>>   
>>   		if (apic_x2apic_mode(source)) {
>>   			/* 16 bit dest mask, 16 bit cluster id */
>> -			bitmap = dest & 0xFFFF0000;
>> -			cluster = (dest >> 16) << 4;
>> +			bitmap = dest & 0xffff;
>> +			cluster = (dest & 0xffff0000) >> 16;
>>   		} else if (kvm_lapic_get_reg(source, APIC_DFR) == APIC_DFR_FLAT) {
>>   			/* 8 bit dest mask*/
>>   			bitmap = dest;
>> @@ -387,7 +387,7 @@ static int avic_kick_target_vcpus_fast(struct kvm *kvm, struct kvm_lapic *source
>>   		} else {
>>   			/* 4 bit desk mask, 4 bit cluster id */
>>   			bitmap = dest & 0xF;
>> -			cluster = (dest >> 4) << 2;
>> +			cluster = (dest & 0xf0) >> 4;
> 
> This is wrong and unrelated.  The cluster needs to be shifted back by 2, i.e. multiplied
> by 4, to leap over each cluster of 4 IDs.

Ah, I missed the logic that was using the _cluster_ value below.

Thanks,
Suravee