From: David Matlack <dmatlack@google.com>
To: "wangyanan (Y)" <wangyanan55@huawei.com>
Cc: Sean Christopherson <seanjc@google.com>,
Paolo Bonzini <pbonzini@redhat.com>,
Wanpeng Li <wanpengli@tencent.com>, kvm <kvm@vger.kernel.org>,
David Hildenbrand <david@redhat.com>,
"Kernel Mailing List, Linux" <linux-kernel@vger.kernel.org>,
Paul Mackerras <paulus@ozlabs.org>,
Claudio Imbrenda <imbrenda@linux.ibm.com>,
KVM ARM <kvmarm@lists.cs.columbia.edu>,
Janosch Frank <frankja@linux.ibm.com>,
Marc Zyngier <maz@kernel.org>, Joerg Roedel <joro@8bytes.org>,
Huacai Chen <chenhuacai@kernel.org>,
Christian Borntraeger <borntraeger@de.ibm.com>,
Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>,
Jon Cargille <jcargill@google.com>,
kvm-ppc <kvm-ppc@vger.kernel.org>,
linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
Jim Mattson <jmattson@google.com>,
Cornelia Huck <cohuck@redhat.com>,
"open list:MIPS" <linux-mips@vger.kernel.org>,
Vitaly Kuznetsov <vkuznets@redhat.com>
Subject: Re: disabling halt polling broken? (was Re: [PATCH 00/14] KVM: Halt-polling fixes, cleanups and a new stat)
Date: Wed, 16 Nov 2022 09:19:28 -0800 [thread overview]
Message-ID: <Y3UboELxugwDJkIG@google.com> (raw)
In-Reply-To: <32810c89-44c6-6780-9d05-e49f6b897b6e@huawei.com>
On Tue, Nov 15, 2022 at 11:28:56AM +0800, wangyanan (Y) wrote:
> Hi Sean, Paolo,
>
> I recently also notice the behavior change of param halt_poll_ns.
> Now it loses the ability to:
> 1) dynamically disable halt polling for all the running VMs
> by `echo 0 > /sys`
> 2) dynamically adjust the halt polling interval for all the
> running VMs by `echo * > /sys`
>
> While in our cases, we usually use above two abilities, and
> KVM_CAP_HALT_POLL is not used yet.
I think the right path forward is to make KVM_CAP_HALT_POLL a pure
override of halt_poll_ns, and restore the pre-existing behavior of
halt_poll_ns whenever KVM_CAP_HALT_POLL is not used. e.g. see the patch
below.
That will fix issues (1) and (2) above for any VM not using
KVM_CAP_HALT_POLL. If a VM is using KVM_CAP_HALT_POLL, it will ignore
all changes to halt_poll_ns. If we truly need a mechanism for admins to
disable halt-polling on VMs using KVM_CAP_HALT_POLL, we can introduce a
separate module parameter for that. But IMO, any setup that is
sophisticated enough to use KVM_CAP_HALT_POLL should also be able to use
KVM_CAP_HALT_POLL to disable halt polling.
If everyone is happy with this approach I can test and send a real patch
to the mailing list.
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index e6e66c5e56f2..253ad055b6ad 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -788,6 +788,7 @@ struct kvm {
struct srcu_struct srcu;
struct srcu_struct irq_srcu;
pid_t userspace_pid;
+ bool override_halt_poll_ns;
unsigned int max_halt_poll_ns;
u32 dirty_ring_size;
bool vm_bugged;
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 43bbe4fde078..479d0d0da0b5 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1198,8 +1198,6 @@ static struct kvm *kvm_create_vm(unsigned long type, const char *fdname)
goto out_err_no_arch_destroy_vm;
}
- kvm->max_halt_poll_ns = halt_poll_ns;
-
r = kvm_arch_init_vm(kvm, type);
if (r)
goto out_err_no_arch_destroy_vm;
@@ -3371,7 +3369,7 @@ void kvm_sigset_deactivate(struct kvm_vcpu *vcpu)
sigemptyset(¤t->real_blocked);
}
-static void grow_halt_poll_ns(struct kvm_vcpu *vcpu)
+static void grow_halt_poll_ns(struct kvm_vcpu *vcpu, unsigned int max)
{
unsigned int old, val, grow, grow_start;
@@ -3385,8 +3383,8 @@ static void grow_halt_poll_ns(struct kvm_vcpu *vcpu)
if (val < grow_start)
val = grow_start;
- if (val > vcpu->kvm->max_halt_poll_ns)
- val = vcpu->kvm->max_halt_poll_ns;
+ if (val > max)
+ val = max;
vcpu->halt_poll_ns = val;
out:
@@ -3501,10 +3499,17 @@ void kvm_vcpu_halt(struct kvm_vcpu *vcpu)
{
bool halt_poll_allowed = !kvm_arch_no_poll(vcpu);
bool do_halt_poll = halt_poll_allowed && vcpu->halt_poll_ns;
+ unsigned int max_halt_poll_ns;
ktime_t start, cur, poll_end;
+ struct kvm *kvm = vcpu->kvm;
bool waited = false;
u64 halt_ns;
+ if (kvm->override_halt_poll_ns)
+ max_halt_poll_ns = kvm->max_halt_poll_ns;
+ else
+ max_halt_poll_ns = READ_ONCE(halt_poll_ns);
+
start = cur = poll_end = ktime_get();
if (do_halt_poll) {
ktime_t stop = ktime_add_ns(start, vcpu->halt_poll_ns);
@@ -3545,17 +3550,16 @@ void kvm_vcpu_halt(struct kvm_vcpu *vcpu)
if (halt_poll_allowed) {
if (!vcpu_valid_wakeup(vcpu)) {
shrink_halt_poll_ns(vcpu);
- } else if (vcpu->kvm->max_halt_poll_ns) {
+ } else if (max_halt_poll_ns) {
if (halt_ns <= vcpu->halt_poll_ns)
;
/* we had a long block, shrink polling */
- else if (vcpu->halt_poll_ns &&
- halt_ns > vcpu->kvm->max_halt_poll_ns)
+ else if (vcpu->halt_poll_ns && halt_ns > max_halt_poll_ns)
shrink_halt_poll_ns(vcpu);
/* we had a short halt and our poll time is too small */
- else if (vcpu->halt_poll_ns < vcpu->kvm->max_halt_poll_ns &&
- halt_ns < vcpu->kvm->max_halt_poll_ns)
- grow_halt_poll_ns(vcpu);
+ else if (vcpu->halt_poll_ns < max_halt_poll_ns &&
+ halt_ns < max_halt_poll_ns)
+ grow_halt_poll_ns(vcpu, max_halt_poll_ns);
} else {
vcpu->halt_poll_ns = 0;
}
@@ -4588,6 +4592,7 @@ static int kvm_vm_ioctl_enable_cap_generic(struct kvm *kvm,
if (cap->flags || cap->args[0] != (unsigned int)cap->args[0])
return -EINVAL;
+ kvm->override_halt_poll_ns = true;
kvm->max_halt_poll_ns = cap->args[0];
return 0;
}
>
> On 2021/9/28 1:33, Sean Christopherson wrote:
> > On Mon, Sep 27, 2021, Paolo Bonzini wrote:
> > > On Mon, Sep 27, 2021 at 5:17 PM Christian Borntraeger
> > > <borntraeger@de.ibm.com> wrote:
> > > > > So I think there are two possibilities that makes sense:
> > > > >
> > > > > * track what is using KVM_CAP_HALT_POLL, and make writes to halt_poll_ns follow that
> > > > what about using halt_poll_ns for those VMs that did not uses KVM_CAP_HALT_POLL and the private number for those that did.
> > > Yes, that's what I meant. David pointed out that doesn't allow you to
> > > disable halt polling altogether, but for that you can always ask each
> > > VM's userspace one by one, or just not use KVM_CAP_HALT_POLL. (Also, I
> > > don't know about Google's usecase, but mine was actually more about
> > > using KVM_CAP_HALT_POLL to *disable* halt polling on some VMs!).
> > I kinda like the idea if special-casing halt_poll_ns=0, e.g. for testing or
> > in-the-field mitigation if halt-polling is broken. It'd be trivial to support, e.g.
> Do we have any plan to repost the diff as a fix?
> I would be very nice that this issue can be solved.
>
> Besides, I think we may need some Doc for users to describe
> how halt_poll_ns works with KVM_CAP_HALT_POLL, like
> "Documentation/virt/guest-halt-polling.rst".
> > @@ -3304,19 +3304,23 @@ void kvm_vcpu_halt(struct kvm_vcpu *vcpu)
> > update_halt_poll_stats(vcpu, start, poll_end, !waited);
> >
> > if (halt_poll_allowed) {
> > + max_halt_poll_ns = vcpu->kvm->max_halt_poll_ns;
> > + if (!max_halt_poll_ns || !halt_poll_ns) <------ squish the max if halt_poll_ns==0
> > + max_halt_poll_ns = halt_poll_ns;
> > +
> Does this mean that KVM_CAP_HALT_POLL will not be able to
> disable halt polling for a VM individually when halt_poll_ns !=0?
> > if (!vcpu_valid_wakeup(vcpu)) {
> > shrink_halt_poll_ns(vcpu);
> > - } else if (vcpu->kvm->max_halt_poll_ns) {
> > + } else if (max_halt_poll_ns) {
> > if (halt_ns <= vcpu->halt_poll_ns)
> > ;
> > /* we had a long block, shrink polling */
> > else if (vcpu->halt_poll_ns &&
> > - halt_ns > vcpu->kvm->max_halt_poll_ns)
> > + halt_ns > max_halt_poll_ns)
> > shrink_halt_poll_ns(vcpu);
> > /* we had a short halt and our poll time is too small */
> > - else if (vcpu->halt_poll_ns < vcpu->kvm->max_halt_poll_ns &&
> > - halt_ns < vcpu->kvm->max_halt_poll_ns)
> > - grow_halt_poll_ns(vcpu);
> > + else if (vcpu->halt_poll_ns < max_halt_poll_ns &&
> > + halt_ns < max_halt_poll_ns)
> > + grow_halt_poll_ns(vcpu, max_halt_poll_ns);
> > } else {
> > vcpu->halt_poll_ns = 0;
> > }
> > _______________________________________________
> > kvmarm mailing list
> > kvmarm@lists.cs.columbia.edu
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> > .
> Thanks,
> Yanan
next prev parent reply other threads:[~2022-11-16 17:19 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-25 0:55 [PATCH 00/14] KVM: Halt-polling fixes, cleanups and a new stat Sean Christopherson
2021-09-25 0:55 ` [PATCH 01/14] KVM: s390: Ensure kvm_arch_no_poll() is read once when blocking vCPU Sean Christopherson
2021-09-27 6:54 ` Christian Borntraeger
2021-09-25 0:55 ` [PATCH 02/14] KVM: Update halt-polling stats if and only if halt-polling was attempted Sean Christopherson
2021-09-28 18:57 ` David Matlack
2021-09-25 0:55 ` [PATCH 03/14] KVM: Refactor and document halt-polling stats update helper Sean Christopherson
2021-09-28 19:01 ` David Matlack
2021-09-25 0:55 ` [PATCH 04/14] KVM: Reconcile discrepancies in halt-polling stats Sean Christopherson
2021-09-28 21:26 ` David Matlack
2021-09-25 0:55 ` [PATCH 05/14] KVM: s390: Clear valid_wakeup in kvm_s390_handle_wait(), not in arch hook Sean Christopherson
2021-09-27 6:58 ` Christian Borntraeger
2021-09-25 0:55 ` [PATCH 06/14] KVM: Drop obsolete kvm_arch_vcpu_block_finish() Sean Christopherson
2021-09-27 6:58 ` Christian Borntraeger
2021-09-28 21:28 ` David Matlack
2021-09-25 0:55 ` [PATCH 07/14] KVM: Don't block+unblock when halt-polling is successful Sean Christopherson
2021-09-25 9:50 ` Marc Zyngier
2021-09-26 6:27 ` Paolo Bonzini
2021-09-26 9:02 ` Marc Zyngier
2021-09-27 17:28 ` Sean Christopherson
2021-09-28 9:24 ` Marc Zyngier
2021-09-28 16:21 ` Sean Christopherson
2021-09-30 9:36 ` Marc Zyngier
2021-09-25 0:55 ` [PATCH 08/14] KVM: x86: Tweak halt emulation helper names to free up kvm_vcpu_halt() Sean Christopherson
2021-09-28 21:59 ` David Matlack
2021-09-25 0:55 ` [PATCH 09/14] KVM: Rename kvm_vcpu_block() => kvm_vcpu_halt() Sean Christopherson
2021-09-27 7:06 ` Christian Borntraeger
2021-09-28 22:01 ` David Matlack
2021-09-25 0:55 ` [PATCH 10/14] KVM: Split out a kvm_vcpu_block() helper from kvm_vcpu_halt() Sean Christopherson
2021-09-27 7:41 ` Christian Borntraeger
2021-09-28 22:03 ` David Matlack
2021-09-25 0:55 ` [PATCH 11/14] KVM: stats: Add stat to detect if vcpu is currently blocking Sean Christopherson
2021-09-28 22:04 ` David Matlack
2021-09-25 0:55 ` [PATCH 12/14] KVM: Don't redo ktime_get() when calculating halt-polling stop/deadline Sean Christopherson
2021-09-28 22:08 ` David Matlack
2021-09-25 0:55 ` [PATCH 13/14] KVM: x86: Directly block (instead of "halting") UNINITIALIZED vCPUs Sean Christopherson
2021-09-28 22:12 ` David Matlack
2021-09-25 0:55 ` [PATCH 14/14] KVM: x86: Invoke kvm_vcpu_block() directly for non-HALTED wait states Sean Christopherson
2021-09-28 22:14 ` David Matlack
2021-09-27 7:22 ` disabling halt polling broken? (was Re: [PATCH 00/14] KVM: Halt-polling fixes, cleanups and a new stat) Christian Borntraeger
2021-09-27 14:59 ` Sean Christopherson
2021-09-27 15:03 ` Paolo Bonzini
2021-09-27 15:15 ` Sean Christopherson
2021-09-27 15:16 ` Christian Borntraeger
2021-09-27 16:58 ` David Matlack
2021-09-29 6:56 ` Christian Borntraeger
2021-09-27 17:24 ` Paolo Bonzini
2021-09-27 17:33 ` Sean Christopherson
2022-11-15 3:28 ` wangyanan (Y)
2022-11-16 17:19 ` David Matlack [this message]
2022-11-18 2:29 ` wangyanan (Y)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y3UboELxugwDJkIG@google.com \
--to=dmatlack@google.com \
--cc=aleksandar.qemu.devel@gmail.com \
--cc=borntraeger@de.ibm.com \
--cc=chenhuacai@kernel.org \
--cc=cohuck@redhat.com \
--cc=david@redhat.com \
--cc=frankja@linux.ibm.com \
--cc=imbrenda@linux.ibm.com \
--cc=jcargill@google.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=kvm-ppc@vger.kernel.org \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=maz@kernel.org \
--cc=paulus@ozlabs.org \
--cc=pbonzini@redhat.com \
--cc=seanjc@google.com \
--cc=vkuznets@redhat.com \
--cc=wangyanan55@huawei.com \
--cc=wanpengli@tencent.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).