[PATCH 1/1] KVM: x86: to track if L1 is running L2 VM

[PATCH 1/1] KVM: x86: to track if L1 is running L2 VM

[Dongli Zhang]

The new per-cpu stat 'nested_run' is introduced in order to track if L1 VM
is running or used to run L2 VM.

An example of the usage of 'nested_run' is to help the host administrator
to easily track if any L1 VM is used to run L2 VM. Suppose there is issue
that may happen with nested virtualization, the administrator will be able
to easily narrow down and confirm if the issue is due to nested
virtualization via 'nested_run'. For example, whether the fix like
commit 88dddc11a8d6 ("KVM: nVMX: do not use dangling shadow VMCS after
guest reset") is required.

Cc: Joe Jin <joe.jin@oracle.com>
Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/svm/nested.c       | 2 ++
 arch/x86/kvm/vmx/nested.c       | 2 ++
 arch/x86/kvm/x86.c              | 1 +
 4 files changed, 6 insertions(+)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 877a4025d8da..7669215426ac 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1125,6 +1125,7 @@ struct kvm_vcpu_stat {
 	u64 req_event;
 	u64 halt_poll_success_ns;
 	u64 halt_poll_fail_ns;
+	u64 nested_run;
 };
 
 struct x86_instruction_info;
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 35891d9a1099..18c02e958a09 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -494,6 +494,8 @@ int nested_svm_vmrun(struct vcpu_svm *svm)
 	struct kvm_host_map map;
 	u64 vmcb12_gpa;
 
+	++svm->vcpu.stat.nested_run;
+
 	if (is_smm(&svm->vcpu)) {
 		kvm_queue_exception(&svm->vcpu, UD_VECTOR);
 		return 1;
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index bcca0b80e0d0..bd1343a0896e 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -3453,6 +3453,8 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)
 	u32 interrupt_shadow = vmx_get_interrupt_shadow(vcpu);
 	enum nested_evmptrld_status evmptrld_status;
 
+	++vcpu->stat.nested_run;
+
 	if (!nested_vmx_check_permission(vcpu))
 		return 1;
 
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 2a20ce60152e..f296febb0485 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -245,6 +245,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
 	VCPU_STAT("l1d_flush", l1d_flush),
 	VCPU_STAT("halt_poll_success_ns", halt_poll_success_ns),
 	VCPU_STAT("halt_poll_fail_ns", halt_poll_fail_ns),
+	VCPU_STAT("nested_run", nested_run),
 	VM_STAT("mmu_shadow_zapped", mmu_shadow_zapped),
 	VM_STAT("mmu_pte_write", mmu_pte_write),
 	VM_STAT("mmu_pde_zapped", mmu_pde_zapped),
-- 
2.17.1

Re: [PATCH 1/1] KVM: x86: to track if L1 is running L2 VM

[Paolo Bonzini]

On 05/03/21 23:57, Dongli Zhang wrote:
> The new per-cpu stat 'nested_run' is introduced in order to track if L1 VM
> is running or used to run L2 VM.
> 
> An example of the usage of 'nested_run' is to help the host administrator
> to easily track if any L1 VM is used to run L2 VM. Suppose there is issue
> that may happen with nested virtualization, the administrator will be able
> to easily narrow down and confirm if the issue is due to nested
> virtualization via 'nested_run'. For example, whether the fix like
> commit 88dddc11a8d6 ("KVM: nVMX: do not use dangling shadow VMCS after
> guest reset") is required.
> 
> Cc: Joe Jin <joe.jin@oracle.com>
> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
> ---
>   arch/x86/include/asm/kvm_host.h | 1 +
>   arch/x86/kvm/svm/nested.c       | 2 ++
>   arch/x86/kvm/vmx/nested.c       | 2 ++
>   arch/x86/kvm/x86.c              | 1 +
>   4 files changed, 6 insertions(+)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 877a4025d8da..7669215426ac 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1125,6 +1125,7 @@ struct kvm_vcpu_stat {
>   	u64 req_event;
>   	u64 halt_poll_success_ns;
>   	u64 halt_poll_fail_ns;
> +	u64 nested_run;
>   };
>   
>   struct x86_instruction_info;
> diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
> index 35891d9a1099..18c02e958a09 100644
> --- a/arch/x86/kvm/svm/nested.c
> +++ b/arch/x86/kvm/svm/nested.c
> @@ -494,6 +494,8 @@ int nested_svm_vmrun(struct vcpu_svm *svm)
>   	struct kvm_host_map map;
>   	u64 vmcb12_gpa;
>   
> +	++svm->vcpu.stat.nested_run;
> +
>   	if (is_smm(&svm->vcpu)) {
>   		kvm_queue_exception(&svm->vcpu, UD_VECTOR);
>   		return 1;
> diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
> index bcca0b80e0d0..bd1343a0896e 100644
> --- a/arch/x86/kvm/vmx/nested.c
> +++ b/arch/x86/kvm/vmx/nested.c
> @@ -3453,6 +3453,8 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch)
>   	u32 interrupt_shadow = vmx_get_interrupt_shadow(vcpu);
>   	enum nested_evmptrld_status evmptrld_status;
>   
> +	++vcpu->stat.nested_run;
> +
>   	if (!nested_vmx_check_permission(vcpu))
>   		return 1;
>   
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 2a20ce60152e..f296febb0485 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -245,6 +245,7 @@ struct kvm_stats_debugfs_item debugfs_entries[] = {
>   	VCPU_STAT("l1d_flush", l1d_flush),
>   	VCPU_STAT("halt_poll_success_ns", halt_poll_success_ns),
>   	VCPU_STAT("halt_poll_fail_ns", halt_poll_fail_ns),
> +	VCPU_STAT("nested_run", nested_run),
>   	VM_STAT("mmu_shadow_zapped", mmu_shadow_zapped),
>   	VM_STAT("mmu_pte_write", mmu_pte_write),
>   	VM_STAT("mmu_pde_zapped", mmu_pde_zapped),
> 

Queued, thanks.

Paolo

Re: [PATCH 1/1] KVM: x86: to track if L1 is running L2 VM

[Dongli Zhang]

Hi Paolo,

On 3/6/21 5:56 AM, Paolo Bonzini wrote:
> On 05/03/21 23:57, Dongli Zhang wrote:
>> The new per-cpu stat 'nested_run' is introduced in order to track if L1 VM
>> is running or used to run L2 VM.
>>
>> An example of the usage of 'nested_run' is to help the host administrator
>> to easily track if any L1 VM is used to run L2 VM. Suppose there is issue
>> that may happen with nested virtualization, the administrator will be able
>> to easily narrow down and confirm if the issue is due to nested
>> virtualization via 'nested_run'. For example, whether the fix like
>> commit 88dddc11a8d6 ("KVM: nVMX: do not use dangling shadow VMCS after
>> guest reset") is required.
>>
>> Cc: Joe Jin <joe.jin@oracle.com>
>> Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
...
> 
> Queued, thanks.
> 
> Paolo
>

While testing the most recent kvm tree, I did not find this patch queued
(debugfs entry for nested_run not available). Would you mind help confirm?

Thank you very much!

Dongli Zhang