Re: [PATCH v5 01/13] KVM: x86: Cache total page count to avoid traversing the memslot array

From: "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
	Vitaly Kuznetsov <vkuznets@redhat.com>,
	Wanpeng Li <wanpengli@tencent.com>,
	Jim Mattson <jmattson@google.com>,
	Igor Mammedov <imammedo@redhat.com>,
	Marc Zyngier <maz@kernel.org>, James Morse <james.morse@arm.com>,
	Julien Thierry <julien.thierry.kdev@gmail.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Huacai Chen <chenhuacai@kernel.org>,
	Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>,
	Paul Mackerras <paulus@ozlabs.org>,
	Christian Borntraeger <borntraeger@de.ibm.com>,
	Janosch Frank <frankja@linux.ibm.com>,
	David Hildenbrand <david@redhat.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Claudio Imbrenda <imbrenda@linux.ibm.com>,
	Joerg Roedel <joro@8bytes.org>,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 01/13] KVM: x86: Cache total page count to avoid traversing the memslot array
Date: Wed, 20 Oct 2021 20:40:19 +0200	[thread overview]
Message-ID: <59217787-416c-fc04-f69f-61801477b2ea@maciej.szmigiero.name> (raw)
In-Reply-To: <YW9HL3FOkOk56I5g@google.com>

On 20.10.2021 00:31, Sean Christopherson wrote:
> On Tue, Oct 19, 2021, Sean Christopherson wrote:
>> On Mon, Sep 20, 2021, Maciej S. Szmigiero wrote:
>>> From: "Maciej S. Szmigiero" <maciej.szmigiero@oracle.com>
>>>
>>> There is no point in recalculating from scratch the total number of pages
>>> in all memslots each time a memslot is created or deleted.
>>>
>>> Just cache the value and update it accordingly on each such operation so
>>> the code doesn't need to traverse the whole memslot array each time.
>>>
>>> Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
>>> ---
>>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>>> index 28ef14155726..65fdf27b9423 100644
>>> --- a/arch/x86/kvm/x86.c
>>> +++ b/arch/x86/kvm/x86.c
>>> @@ -11609,9 +11609,23 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>>>   				const struct kvm_memory_slot *new,
>>>   				enum kvm_mr_change change)
>>>   {
>>> -	if (!kvm->arch.n_requested_mmu_pages)
>>> -		kvm_mmu_change_mmu_pages(kvm,
>>> -				kvm_mmu_calculate_default_mmu_pages(kvm));
>>> +	if (change == KVM_MR_CREATE)
>>> +		kvm->arch.n_memslots_pages += new->npages;
>>> +	else if (change == KVM_MR_DELETE) {
>>> +		WARN_ON(kvm->arch.n_memslots_pages < old->npages);
>>> +		kvm->arch.n_memslots_pages -= old->npages;
>>> +	}
>>> +
>>> +	if (!kvm->arch.n_requested_mmu_pages) {
>>
>> Hmm, once n_requested_mmu_pages is set it can't be unset.  That means this can be
>> further optimized to skip avoid taking mmu_lock on flags-only changes (and
>> memslot movement).  E.g.
>>
>> 	if (!kvm->arch.n_requested_mmu_pages &&
>> 	    (change == KVM_MR_CREATE || change == KVM_MR_DELETE)) {
>>
>> 	}
>>
>> It's a little risky, but kvm_vm_ioctl_set_nr_mmu_pages() would need to be modified
>> to allow clearing n_requested_mmu_pages and it already takes slots_lock, so IMO
>> it's ok to force kvm_vm_ioctl_set_nr_mmu_pages() to recalculate pages if it wants
>> to allow reverting back to the default.
> 
> Doh, and then I read patch 2...
> 
> I would swap the order of patch 2 and patch 1, that way the optimization patch is
> super simple, and you don't end up reworking a bunch of code that was added in the
> immediately preceding patch.  E.g. as a first patch
> 
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 28ef14155726..f3b1aed08566 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -11609,7 +11609,8 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
>                                  const struct kvm_memory_slot *new,
>                                  enum kvm_mr_change change)
>   {
> -       if (!kvm->arch.n_requested_mmu_pages)
> +       if (!kvm->arch.n_requested_mmu_pages &&
> +           (change == KVM_MR_CREATE || change == KVM_MR_DELETE))
>                  kvm_mmu_change_mmu_pages(kvm,
>                                  kvm_mmu_calculate_default_mmu_pages(kvm));
> 
> 
> 

Will do.

Thanks,
Maciej