* [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush
2021-10-22 1:00 [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Sean Christopherson
@ 2021-10-22 1:00 ` Sean Christopherson
2021-10-22 2:58 ` Lai Jiangshan
2021-10-22 15:39 ` Maxim Levitsky
2021-10-22 1:00 ` [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range() Sean Christopherson
` (2 subsequent siblings)
3 siblings, 2 replies; 11+ messages in thread
From: Sean Christopherson @ 2021-10-22 1:00 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Sean Christopherson, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, linux-kernel, Maxim Levitsky,
Maciej S . Szmigiero, Ben Gardon
A recent commit to fix the calls to kvm_flush_remote_tlbs_with_address()
in kvm_zap_gfn_range() inadvertantly added yet another flush instead of
fixing the existing flush. Drop the redundant flush, and fix the params
for the existing flush.
Fixes: 2822da446640 ("KVM: x86/mmu: fix parameters to kvm_flush_remote_tlbs_with_address")
Cc: Maxim Levitsky <mlevitsk@redhat.com>
Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/mmu/mmu.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index c6ddb042b281..f82b192bba0b 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5709,13 +5709,11 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start,
gfn_end, flush);
- if (flush)
- kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
- gfn_end - gfn_start);
}
if (flush)
- kvm_flush_remote_tlbs_with_address(kvm, gfn_start, gfn_end);
+ kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
+ gfn_end - gfn_start);
kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush
2021-10-22 1:00 ` [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush Sean Christopherson
@ 2021-10-22 2:58 ` Lai Jiangshan
2021-10-22 3:03 ` Lai Jiangshan
2021-10-22 15:39 ` Maxim Levitsky
1 sibling, 1 reply; 11+ messages in thread
From: Lai Jiangshan @ 2021-10-22 2:58 UTC (permalink / raw)
To: Sean Christopherson
Cc: Paolo Bonzini, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, LKML, Maxim Levitsky, Maciej S . Szmigiero,
Ben Gardon
On Fri, Oct 22, 2021 at 9:01 AM Sean Christopherson <seanjc@google.com> wrote:
>
> A recent commit to fix the calls to kvm_flush_remote_tlbs_with_address()
> in kvm_zap_gfn_range() inadvertantly added yet another flush instead of
> fixing the existing flush. Drop the redundant flush, and fix the params
> for the existing flush.
>
> Fixes: 2822da446640 ("KVM: x86/mmu: fix parameters to kvm_flush_remote_tlbs_with_address")
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/kvm/mmu/mmu.c | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index c6ddb042b281..f82b192bba0b 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -5709,13 +5709,11 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
> for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
> flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start,
> gfn_end, flush);
> - if (flush)
> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> - gfn_end - gfn_start);
In the recent queue branch of kvm tree, there is the same "if (flush)" in
the previous "if (kvm_memslots_have_rmaps(kvm))" branch. The "if (flush)"
branch needs to be removed too.
Reviewed-by: Lai Jiangshan <jiangshanlai@gmail.com>
> }
>
> if (flush)
> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start, gfn_end);
> + kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> + gfn_end - gfn_start);
>
> kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
>
> --
> 2.33.0.1079.g6e70778dc9-goog
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush
2021-10-22 2:58 ` Lai Jiangshan
@ 2021-10-22 3:03 ` Lai Jiangshan
0 siblings, 0 replies; 11+ messages in thread
From: Lai Jiangshan @ 2021-10-22 3:03 UTC (permalink / raw)
To: Sean Christopherson
Cc: Paolo Bonzini, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, LKML, Maxim Levitsky, Maciej S . Szmigiero,
Ben Gardon
On Fri, Oct 22, 2021 at 10:58 AM Lai Jiangshan
<jiangshanlai+lkml@gmail.com> wrote:
>
> On Fri, Oct 22, 2021 at 9:01 AM Sean Christopherson <seanjc@google.com> wrote:
> >
> > A recent commit to fix the calls to kvm_flush_remote_tlbs_with_address()
> > in kvm_zap_gfn_range() inadvertantly added yet another flush instead of
> > fixing the existing flush. Drop the redundant flush, and fix the params
> > for the existing flush.
> >
> > Fixes: 2822da446640 ("KVM: x86/mmu: fix parameters to kvm_flush_remote_tlbs_with_address")
> > Cc: Maxim Levitsky <mlevitsk@redhat.com>
> > Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> > arch/x86/kvm/mmu/mmu.c | 6 ++----
> > 1 file changed, 2 insertions(+), 4 deletions(-)
> >
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index c6ddb042b281..f82b192bba0b 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -5709,13 +5709,11 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
> > for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
> > flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start,
> > gfn_end, flush);
> > - if (flush)
> > - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> > - gfn_end - gfn_start);
>
> In the recent queue branch of kvm tree, there is the same "if (flush)" in
> the previous "if (kvm_memslots_have_rmaps(kvm))" branch. The "if (flush)"
> branch needs to be removed too.
Oh, it is in the patch 2. For patch 1 and 2:
Reviewed-by: Lai Jiangshan <jiangshanlai@gmail.com>
>
> Reviewed-by: Lai Jiangshan <jiangshanlai@gmail.com>
>
> > }
> >
> > if (flush)
> > - kvm_flush_remote_tlbs_with_address(kvm, gfn_start, gfn_end);
> > + kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> > + gfn_end - gfn_start);
> >
> > kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
> >
> > --
> > 2.33.0.1079.g6e70778dc9-goog
> >
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush
2021-10-22 1:00 ` [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush Sean Christopherson
2021-10-22 2:58 ` Lai Jiangshan
@ 2021-10-22 15:39 ` Maxim Levitsky
1 sibling, 0 replies; 11+ messages in thread
From: Maxim Levitsky @ 2021-10-22 15:39 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini
Cc: Vitaly Kuznetsov, Wanpeng Li, Jim Mattson, Joerg Roedel, kvm,
linux-kernel, Maciej S . Szmigiero, Ben Gardon
On Thu, 2021-10-21 at 18:00 -0700, Sean Christopherson wrote:
> A recent commit to fix the calls to kvm_flush_remote_tlbs_with_address()
> in kvm_zap_gfn_range() inadvertantly added yet another flush instead of
> fixing the existing flush. Drop the redundant flush, and fix the params
> for the existing flush.
>
> Fixes: 2822da446640 ("KVM: x86/mmu: fix parameters to kvm_flush_remote_tlbs_with_address")
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/kvm/mmu/mmu.c | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index c6ddb042b281..f82b192bba0b 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -5709,13 +5709,11 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
> for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
> flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start,
> gfn_end, flush);
> - if (flush)
> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> - gfn_end - gfn_start);
> }
>
> if (flush)
> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start, gfn_end);
> + kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> + gfn_end - gfn_start);
>
> kvm_dec_notifier_count(kvm, gfn_start, gfn_end);
>
Opps, I didn't notice that the revert added back another flush. I probablyhaven't had
the revert in place when I wrote the patch that fixed parameters to
kvm_flush_remote_tlbs_with_address.
Best regards,
Maxim Levitsky
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range()
2021-10-22 1:00 [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Sean Christopherson
2021-10-22 1:00 ` [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush Sean Christopherson
@ 2021-10-22 1:00 ` Sean Christopherson
2021-10-22 11:44 ` Maciej S. Szmigiero
2021-10-22 1:00 ` [PATCH 3/3] KVM: x86/mmu: Extract zapping of rmaps for gfn range to separate helper Sean Christopherson
2021-10-22 15:11 ` [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Paolo Bonzini
3 siblings, 1 reply; 11+ messages in thread
From: Sean Christopherson @ 2021-10-22 1:00 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Sean Christopherson, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, linux-kernel, Maxim Levitsky,
Maciej S . Szmigiero, Ben Gardon
Remove an unnecessary remote TLB flush in kvm_zap_gfn_range() now that
said function holds mmu_lock for write for its entire duration. The
flush was added by the now-reverted commit to allow TDP MMU to flush while
holding mmu_lock for read, as the transition from write=>read required
dropping the lock and thus a pending flush needed to be serviced.
Fixes: 5a324c24b638 ("Revert "KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock"")
Cc: Maxim Levitsky <mlevitsk@redhat.com>
Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Cc: Ben Gardon <bgardon@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/mmu/mmu.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index f82b192bba0b..e8b8a665e2e9 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5700,9 +5700,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
end - 1, true, flush);
}
}
- if (flush)
- kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
- gfn_end - gfn_start);
}
if (is_tdp_mmu_enabled(kvm)) {
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range()
2021-10-22 1:00 ` [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range() Sean Christopherson
@ 2021-10-22 11:44 ` Maciej S. Szmigiero
2021-10-25 15:39 ` Sean Christopherson
0 siblings, 1 reply; 11+ messages in thread
From: Maciej S. Szmigiero @ 2021-10-22 11:44 UTC (permalink / raw)
To: Sean Christopherson, Paolo Bonzini
Cc: Vitaly Kuznetsov, Wanpeng Li, Jim Mattson, Joerg Roedel, kvm,
linux-kernel, Maxim Levitsky, Ben Gardon
On 22.10.2021 03:00, Sean Christopherson wrote:
> Remove an unnecessary remote TLB flush in kvm_zap_gfn_range() now that
> said function holds mmu_lock for write for its entire duration. The
> flush was added by the now-reverted commit to allow TDP MMU to flush while
> holding mmu_lock for read, as the transition from write=>read required
> dropping the lock and thus a pending flush needed to be serviced.
>
> Fixes: 5a324c24b638 ("Revert "KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock"")
> Cc: Maxim Levitsky <mlevitsk@redhat.com>
> Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
> Cc: Ben Gardon <bgardon@google.com>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/kvm/mmu/mmu.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index f82b192bba0b..e8b8a665e2e9 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -5700,9 +5700,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
> end - 1, true, flush);
> }
> }
> - if (flush)
> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> - gfn_end - gfn_start);
> }
>
> if (is_tdp_mmu_enabled(kvm)) {
>
Unfortunately, it seems that a pending flush from __kvm_zap_rmaps()
can be reset back to false by the following line:
> flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start, gfn_end, flush);
kvm_tdp_mmu_zap_gfn_range() calls __kvm_tdp_mmu_zap_gfn_range with
"can_yield" set to true, which passes it to zap_gfn_range, which has
this code:
> if (can_yield &&
> tdp_mmu_iter_cond_resched(kvm, &iter, flush, shared)) {
> flush = false;
> continue;
> }
Thanks,
Maciej
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range()
2021-10-22 11:44 ` Maciej S. Szmigiero
@ 2021-10-25 15:39 ` Sean Christopherson
2021-10-26 14:22 ` Maciej S. Szmigiero
0 siblings, 1 reply; 11+ messages in thread
From: Sean Christopherson @ 2021-10-25 15:39 UTC (permalink / raw)
To: Maciej S. Szmigiero
Cc: Paolo Bonzini, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, linux-kernel, Maxim Levitsky, Ben Gardon
On Fri, Oct 22, 2021, Maciej S. Szmigiero wrote:
> On 22.10.2021 03:00, Sean Christopherson wrote:
> > Remove an unnecessary remote TLB flush in kvm_zap_gfn_range() now that
> > said function holds mmu_lock for write for its entire duration. The
> > flush was added by the now-reverted commit to allow TDP MMU to flush while
> > holding mmu_lock for read, as the transition from write=>read required
> > dropping the lock and thus a pending flush needed to be serviced.
> >
> > Fixes: 5a324c24b638 ("Revert "KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock"")
> > Cc: Maxim Levitsky <mlevitsk@redhat.com>
> > Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
> > Cc: Ben Gardon <bgardon@google.com>
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> > arch/x86/kvm/mmu/mmu.c | 3 ---
> > 1 file changed, 3 deletions(-)
> >
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index f82b192bba0b..e8b8a665e2e9 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -5700,9 +5700,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
> > end - 1, true, flush);
> > }
> > }
> > - if (flush)
> > - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
> > - gfn_end - gfn_start);
> > }
> > if (is_tdp_mmu_enabled(kvm)) {
> >
>
> Unfortunately, it seems that a pending flush from __kvm_zap_rmaps()
> can be reset back to false by the following line:
> > flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start, gfn_end, flush);
>
> kvm_tdp_mmu_zap_gfn_range() calls __kvm_tdp_mmu_zap_gfn_range with
> "can_yield" set to true, which passes it to zap_gfn_range, which has
> this code:
> > if (can_yield &&
> > tdp_mmu_iter_cond_resched(kvm, &iter, flush, shared)) {
> > flush = false;
> > continue;
> > }
That's working by design. If the MMU (legacy or TDP) yields during zap, it _must_
flush before dropping mmu_lock so that any SPTE modifications are guaranteed to be
observed by all vCPUs. Clearing "flush" is deliberate/correct as another is flush
is needed if and only if additional SPTE modifications are made.
static inline bool tdp_mmu_iter_cond_resched(struct kvm *kvm,
struct tdp_iter *iter, bool flush,
bool shared)
{
/* Ensure forward progress has been made before yielding. */
if (iter->next_last_level_gfn == iter->yielded_gfn)
return false;
if (need_resched() || rwlock_needbreak(&kvm->mmu_lock)) {
rcu_read_unlock();
if (flush)
kvm_flush_remote_tlbs(kvm); <------- ****** HERE ******
if (shared)
cond_resched_rwlock_read(&kvm->mmu_lock);
else
cond_resched_rwlock_write(&kvm->mmu_lock);
rcu_read_lock();
WARN_ON(iter->gfn > iter->next_last_level_gfn);
tdp_iter_restart(iter);
return true;
}
return false;
}
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range()
2021-10-25 15:39 ` Sean Christopherson
@ 2021-10-26 14:22 ` Maciej S. Szmigiero
0 siblings, 0 replies; 11+ messages in thread
From: Maciej S. Szmigiero @ 2021-10-26 14:22 UTC (permalink / raw)
To: Sean Christopherson
Cc: Paolo Bonzini, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, linux-kernel, Maxim Levitsky, Ben Gardon
On 25.10.2021 17:39, Sean Christopherson wrote:
> On Fri, Oct 22, 2021, Maciej S. Szmigiero wrote:
>> On 22.10.2021 03:00, Sean Christopherson wrote:
>>> Remove an unnecessary remote TLB flush in kvm_zap_gfn_range() now that
>>> said function holds mmu_lock for write for its entire duration. The
>>> flush was added by the now-reverted commit to allow TDP MMU to flush while
>>> holding mmu_lock for read, as the transition from write=>read required
>>> dropping the lock and thus a pending flush needed to be serviced.
>>>
>>> Fixes: 5a324c24b638 ("Revert "KVM: x86/mmu: Allow zap gfn range to operate under the mmu read lock"")
>>> Cc: Maxim Levitsky <mlevitsk@redhat.com>
>>> Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
>>> Cc: Ben Gardon <bgardon@google.com>
>>> Signed-off-by: Sean Christopherson <seanjc@google.com>
>>> ---
>>> arch/x86/kvm/mmu/mmu.c | 3 ---
>>> 1 file changed, 3 deletions(-)
>>>
>>> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
>>> index f82b192bba0b..e8b8a665e2e9 100644
>>> --- a/arch/x86/kvm/mmu/mmu.c
>>> +++ b/arch/x86/kvm/mmu/mmu.c
>>> @@ -5700,9 +5700,6 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
>>> end - 1, true, flush);
>>> }
>>> }
>>> - if (flush)
>>> - kvm_flush_remote_tlbs_with_address(kvm, gfn_start,
>>> - gfn_end - gfn_start);
>>> }
>>> if (is_tdp_mmu_enabled(kvm)) {
>>>
>>
>> Unfortunately, it seems that a pending flush from __kvm_zap_rmaps()
>> can be reset back to false by the following line:
>>> flush = kvm_tdp_mmu_zap_gfn_range(kvm, i, gfn_start, gfn_end, flush);
>>
>> kvm_tdp_mmu_zap_gfn_range() calls __kvm_tdp_mmu_zap_gfn_range with
>> "can_yield" set to true, which passes it to zap_gfn_range, which has
>> this code:
>>> if (can_yield &&
>>> tdp_mmu_iter_cond_resched(kvm, &iter, flush, shared)) {
>>> flush = false;
>>> continue;
>>> }
>
> That's working by design. If the MMU (legacy or TDP) yields during zap, it _must_
> flush before dropping mmu_lock so that any SPTE modifications are guaranteed to be
> observed by all vCPUs. Clearing "flush" is deliberate/correct as another is flush
> is needed if and only if additional SPTE modifications are made.
>
Got it, thanks.
Maciej
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH 3/3] KVM: x86/mmu: Extract zapping of rmaps for gfn range to separate helper
2021-10-22 1:00 [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Sean Christopherson
2021-10-22 1:00 ` [PATCH 1/3] KVM: x86/mmu: Drop a redundant, broken remote TLB flush Sean Christopherson
2021-10-22 1:00 ` [PATCH 2/3] KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range() Sean Christopherson
@ 2021-10-22 1:00 ` Sean Christopherson
2021-10-22 15:11 ` [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Paolo Bonzini
3 siblings, 0 replies; 11+ messages in thread
From: Sean Christopherson @ 2021-10-22 1:00 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Sean Christopherson, Vitaly Kuznetsov, Wanpeng Li, Jim Mattson,
Joerg Roedel, kvm, linux-kernel, Maxim Levitsky,
Maciej S . Szmigiero, Ben Gardon
Extract the zapping of rmaps, a.k.a. legacy MMU, for a gfn range to a
separate helper to clean up the unholy mess that kvm_zap_gfn_range() has
become. In addition to deep nesting, the rmaps zapping spreads out the
declaration of several variables and is generally a mess. Clean up the
mess now so that future work to improve the memslots implementation
doesn't need to deal with it.
Cc: Maciej S. Szmigiero <maciej.szmigiero@oracle.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kvm/mmu/mmu.c | 52 ++++++++++++++++++++++++------------------
1 file changed, 30 insertions(+), 22 deletions(-)
diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index e8b8a665e2e9..182d35a216d4 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -5667,40 +5667,48 @@ void kvm_mmu_uninit_vm(struct kvm *kvm)
kvm_mmu_uninit_tdp_mmu(kvm);
}
+static bool __kvm_zap_rmaps(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
+{
+ const struct kvm_memory_slot *memslot;
+ struct kvm_memslots *slots;
+ bool flush = false;
+ gfn_t start, end;
+ int i;
+
+ if (!kvm_memslots_have_rmaps(kvm))
+ return flush;
+
+ for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
+ slots = __kvm_memslots(kvm, i);
+ kvm_for_each_memslot(memslot, slots) {
+ start = max(gfn_start, memslot->base_gfn);
+ end = min(gfn_end, memslot->base_gfn + memslot->npages);
+ if (start >= end)
+ continue;
+
+ flush = slot_handle_level_range(kvm, memslot, kvm_zap_rmapp,
+ PG_LEVEL_4K, KVM_MAX_HUGEPAGE_LEVEL,
+ start, end - 1, true, flush);
+ }
+ }
+
+ return flush;
+}
+
/*
* Invalidate (zap) SPTEs that cover GFNs from gfn_start and up to gfn_end
* (not including it)
*/
void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end)
{
- struct kvm_memslots *slots;
- struct kvm_memory_slot *memslot;
+ bool flush;
int i;
- bool flush = false;
write_lock(&kvm->mmu_lock);
kvm_inc_notifier_count(kvm, gfn_start, gfn_end);
- if (kvm_memslots_have_rmaps(kvm)) {
- for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++) {
- slots = __kvm_memslots(kvm, i);
- kvm_for_each_memslot(memslot, slots) {
- gfn_t start, end;
-
- start = max(gfn_start, memslot->base_gfn);
- end = min(gfn_end, memslot->base_gfn + memslot->npages);
- if (start >= end)
- continue;
-
- flush = slot_handle_level_range(kvm,
- (const struct kvm_memory_slot *) memslot,
- kvm_zap_rmapp, PG_LEVEL_4K,
- KVM_MAX_HUGEPAGE_LEVEL, start,
- end - 1, true, flush);
- }
- }
- }
+ flush = __kvm_zap_rmaps(kvm, gfn_start, gfn_end);
if (is_tdp_mmu_enabled(kvm)) {
for (i = 0; i < KVM_ADDRESS_SPACE_NUM; i++)
--
2.33.0.1079.g6e70778dc9-goog
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range()
2021-10-22 1:00 [PATCH 0/3] KVM: x86/mmu: Clean up kvm_zap_gfn_range() Sean Christopherson
` (2 preceding siblings ...)
2021-10-22 1:00 ` [PATCH 3/3] KVM: x86/mmu: Extract zapping of rmaps for gfn range to separate helper Sean Christopherson
@ 2021-10-22 15:11 ` Paolo Bonzini
3 siblings, 0 replies; 11+ messages in thread
From: Paolo Bonzini @ 2021-10-22 15:11 UTC (permalink / raw)
To: Sean Christopherson
Cc: Vitaly Kuznetsov, Wanpeng Li, Jim Mattson, Joerg Roedel, kvm,
linux-kernel, Maxim Levitsky, Maciej S . Szmigiero, Ben Gardon
On 22/10/21 03:00, Sean Christopherson wrote:
> Fix overzealous flushing in kvm_zap_gfn_range(), and clean up the mess
> that it's become by extracting the legacy MMU logic to a separate
> helper.
>
> Sean Christopherson (3):
> KVM: x86/mmu: Drop a redundant, broken remote TLB flush
> KVM: x86/mmu: Drop a redundant remote TLB flush in kvm_zap_gfn_range()
> KVM: x86/mmu: Extract zapping of rmaps for gfn range to separate
> helper
Queued, with Cc: stable for patch 1. (The other two patches depend on
it, so I don't feel like including it in 5.15-rc).
Paolo
^ permalink raw reply [flat|nested] 11+ messages in thread