linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
@ 2019-08-01  7:06 Wanpeng Li
  2019-08-01 16:51 ` Rafael J. Wysocki
  0 siblings, 1 reply; 19+ messages in thread
From: Wanpeng Li @ 2019-08-01  7:06 UTC (permalink / raw)
  To: linux-kernel, kvm
  Cc: Paolo Bonzini, Radim Krčmář,
	Rafael J . Wysocki, Marcelo Tosatti

From: Wanpeng Li <wanpengli@tencent.com>

The downside of guest side polling is that polling is performed even 
with other runnable tasks in the host. However, even if poll in kvm 
can aware whether or not other runnable tasks in the same pCPU, it 
can still incur extra overhead in over-subscribe scenario. Now we can 
just enable guest polling when dedicated pCPUs are available.

Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
 drivers/cpuidle/cpuidle-haltpoll.c   | 3 ++-
 drivers/cpuidle/governors/haltpoll.c | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/cpuidle/cpuidle-haltpoll.c b/drivers/cpuidle/cpuidle-haltpoll.c
index 9ac093d..7aee38a 100644
--- a/drivers/cpuidle/cpuidle-haltpoll.c
+++ b/drivers/cpuidle/cpuidle-haltpoll.c
@@ -53,7 +53,8 @@ static int __init haltpoll_init(void)
 
 	cpuidle_poll_state_init(drv);
 
-	if (!kvm_para_available())
+	if (!kvm_para_available() ||
+		!kvm_para_has_hint(KVM_HINTS_REALTIME))
 		return 0;
 
 	ret = cpuidle_register(&haltpoll_driver, NULL);
diff --git a/drivers/cpuidle/governors/haltpoll.c b/drivers/cpuidle/governors/haltpoll.c
index 797477b..685c7007 100644
--- a/drivers/cpuidle/governors/haltpoll.c
+++ b/drivers/cpuidle/governors/haltpoll.c
@@ -141,7 +141,7 @@ static struct cpuidle_governor haltpoll_governor = {
 
 static int __init init_haltpoll(void)
 {
-	if (kvm_para_available())
+	if (kvm_para_available() && kvm_para_has_hint(KVM_HINTS_REALTIME))
 		return cpuidle_register_governor(&haltpoll_governor);
 
 	return 0;
-- 
2.7.4


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-01  7:06 [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available Wanpeng Li
@ 2019-08-01 16:51 ` Rafael J. Wysocki
  2019-08-01 16:54   ` Paolo Bonzini
  0 siblings, 1 reply; 19+ messages in thread
From: Rafael J. Wysocki @ 2019-08-01 16:51 UTC (permalink / raw)
  To: Wanpeng Li, Paolo Bonzini, Marcelo Tosatti
  Cc: linux-kernel, kvm, Radim Krčmář, Linux PM

On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> From: Wanpeng Li <wanpengli@tencent.com>
>
> The downside of guest side polling is that polling is performed even
> with other runnable tasks in the host. However, even if poll in kvm
> can aware whether or not other runnable tasks in the same pCPU, it
> can still incur extra overhead in over-subscribe scenario. Now we can
> just enable guest polling when dedicated pCPUs are available.
>
> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>

Paolo, Marcelo, any comments?


> ---
>   drivers/cpuidle/cpuidle-haltpoll.c   | 3 ++-
>   drivers/cpuidle/governors/haltpoll.c | 2 +-
>   2 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/cpuidle/cpuidle-haltpoll.c b/drivers/cpuidle/cpuidle-haltpoll.c
> index 9ac093d..7aee38a 100644
> --- a/drivers/cpuidle/cpuidle-haltpoll.c
> +++ b/drivers/cpuidle/cpuidle-haltpoll.c
> @@ -53,7 +53,8 @@ static int __init haltpoll_init(void)
>   
>   	cpuidle_poll_state_init(drv);
>   
> -	if (!kvm_para_available())
> +	if (!kvm_para_available() ||
> +		!kvm_para_has_hint(KVM_HINTS_REALTIME))
>   		return 0;
>   
>   	ret = cpuidle_register(&haltpoll_driver, NULL);
> diff --git a/drivers/cpuidle/governors/haltpoll.c b/drivers/cpuidle/governors/haltpoll.c
> index 797477b..685c7007 100644
> --- a/drivers/cpuidle/governors/haltpoll.c
> +++ b/drivers/cpuidle/governors/haltpoll.c
> @@ -141,7 +141,7 @@ static struct cpuidle_governor haltpoll_governor = {
>   
>   static int __init init_haltpoll(void)
>   {
> -	if (kvm_para_available())
> +	if (kvm_para_available() && kvm_para_has_hint(KVM_HINTS_REALTIME))
>   		return cpuidle_register_governor(&haltpoll_governor);
>   
>   	return 0;



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-01 16:51 ` Rafael J. Wysocki
@ 2019-08-01 16:54   ` Paolo Bonzini
  2019-08-03 20:21     ` Marcelo Tosatti
  0 siblings, 1 reply; 19+ messages in thread
From: Paolo Bonzini @ 2019-08-01 16:54 UTC (permalink / raw)
  To: Rafael J. Wysocki, Wanpeng Li, Marcelo Tosatti
  Cc: linux-kernel, kvm, Radim Krčmář, Linux PM

On 01/08/19 18:51, Rafael J. Wysocki wrote:
> On 8/1/2019 9:06 AM, Wanpeng Li wrote:
>> From: Wanpeng Li <wanpengli@tencent.com>
>>
>> The downside of guest side polling is that polling is performed even
>> with other runnable tasks in the host. However, even if poll in kvm
>> can aware whether or not other runnable tasks in the same pCPU, it
>> can still incur extra overhead in over-subscribe scenario. Now we can
>> just enable guest polling when dedicated pCPUs are available.
>>
>> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Radim Krčmář <rkrcmar@redhat.com>
>> Cc: Marcelo Tosatti <mtosatti@redhat.com>
>> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> 
> Paolo, Marcelo, any comments?

Yes, it's a good idea.

Acked-by: Paolo Bonzini <pbonzini@redhat.com>

Paolo

> 
>> ---
>>   drivers/cpuidle/cpuidle-haltpoll.c   | 3 ++-
>>   drivers/cpuidle/governors/haltpoll.c | 2 +-
>>   2 files changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/cpuidle/cpuidle-haltpoll.c
>> b/drivers/cpuidle/cpuidle-haltpoll.c
>> index 9ac093d..7aee38a 100644
>> --- a/drivers/cpuidle/cpuidle-haltpoll.c
>> +++ b/drivers/cpuidle/cpuidle-haltpoll.c
>> @@ -53,7 +53,8 @@ static int __init haltpoll_init(void)
>>         cpuidle_poll_state_init(drv);
>>   -    if (!kvm_para_available())
>> +    if (!kvm_para_available() ||
>> +        !kvm_para_has_hint(KVM_HINTS_REALTIME))
>>           return 0;
>>         ret = cpuidle_register(&haltpoll_driver, NULL);
>> diff --git a/drivers/cpuidle/governors/haltpoll.c
>> b/drivers/cpuidle/governors/haltpoll.c
>> index 797477b..685c7007 100644
>> --- a/drivers/cpuidle/governors/haltpoll.c
>> +++ b/drivers/cpuidle/governors/haltpoll.c
>> @@ -141,7 +141,7 @@ static struct cpuidle_governor haltpoll_governor = {
>>     static int __init init_haltpoll(void)
>>   {
>> -    if (kvm_para_available())
>> +    if (kvm_para_available() && kvm_para_has_hint(KVM_HINTS_REALTIME))
>>           return cpuidle_register_governor(&haltpoll_governor);
>>         return 0;
> 
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-01 16:54   ` Paolo Bonzini
@ 2019-08-03 20:21     ` Marcelo Tosatti
  2019-08-13  0:55       ` Wanpeng Li
  0 siblings, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-03 20:21 UTC (permalink / raw)
  To: Paolo Bonzini
  Cc: Rafael J. Wysocki, Wanpeng Li, linux-kernel, kvm,
	Radim Krčmář,
	Linux PM

On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> >> From: Wanpeng Li <wanpengli@tencent.com>
> >>
> >> The downside of guest side polling is that polling is performed even
> >> with other runnable tasks in the host. However, even if poll in kvm
> >> can aware whether or not other runnable tasks in the same pCPU, it
> >> can still incur extra overhead in over-subscribe scenario. Now we can
> >> just enable guest polling when dedicated pCPUs are available.
> >>
> >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > 
> > Paolo, Marcelo, any comments?
> 
> Yes, it's a good idea.
> 
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> Paolo

I think KVM_HINTS_REALTIME is being abused somewhat.
It has no clear meaning and used in different locations 
for different purposes.

For example, i think that using pv queued spinlocks and 
haltpoll is a desired scenario, which the patch below disallows.

Wanpeng Li, currently the driver does not autoload. So polling in 
the guest has to be enabled manually. Isnt that sufficient?



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-03 20:21     ` Marcelo Tosatti
@ 2019-08-13  0:55       ` Wanpeng Li
  2019-08-13  7:55         ` Paolo Bonzini
  2019-08-26 20:40         ` Marcelo Tosatti
  0 siblings, 2 replies; 19+ messages in thread
From: Wanpeng Li @ 2019-08-13  0:55 UTC (permalink / raw)
  To: Marcelo Tosatti
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM

On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>
> On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > >> From: Wanpeng Li <wanpengli@tencent.com>
> > >>
> > >> The downside of guest side polling is that polling is performed even
> > >> with other runnable tasks in the host. However, even if poll in kvm
> > >> can aware whether or not other runnable tasks in the same pCPU, it
> > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > >> just enable guest polling when dedicated pCPUs are available.
> > >>
> > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > >
> > > Paolo, Marcelo, any comments?
> >
> > Yes, it's a good idea.
> >
> > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> >
> > Paolo
>

Hi Marcelo,

Sorry for the late response.

> I think KVM_HINTS_REALTIME is being abused somewhat.
> It has no clear meaning and used in different locations
> for different purposes.

================== ============ =================================
KVM_HINTS_REALTIME 0                      guest checks this feature bit to

determine that vCPUs are never

preempted for an unlimited time

allowing optimizations
================== ============ =================================

Now it disables pv queued spinlock, pv tlb shootdown, pv sched yield
which are not expected present in vCPUs are never preempted for an
unlimited time scenario.

>
> For example, i think that using pv queued spinlocks and
> haltpoll is a desired scenario, which the patch below disallows.

So even if dedicated pCPU is available, pv queued spinlocks should
still be chose if something like vhost-kthreads are used instead of
DPDK/vhost-user. kvm adaptive halt-polling will compete with
vhost-kthreads, however, poll in guest unaware other runnable tasks in
the host which will defeat vhost-kthreads.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-13  0:55       ` Wanpeng Li
@ 2019-08-13  7:55         ` Paolo Bonzini
  2019-08-26 20:40         ` Marcelo Tosatti
  1 sibling, 0 replies; 19+ messages in thread
From: Paolo Bonzini @ 2019-08-13  7:55 UTC (permalink / raw)
  To: Wanpeng Li, Marcelo Tosatti
  Cc: Rafael J. Wysocki, LKML, kvm, Radim Krčmář, Linux PM

On 13/08/19 02:55, Wanpeng Li wrote:
>> I think KVM_HINTS_REALTIME is being abused somewhat.
>> It has no clear meaning and used in different locations
>> for different purposes.
> 
> Now it disables pv queued spinlock, pv tlb shootdown, pv sched yield
> which are not expected present in vCPUs are never preempted for an
> unlimited time scenario.

Guest side polling definitely matches the purpose of KVM_HINTS_REALTIME.
 While host-side polling is conditional on single_task_running, this is
obviously not true of guest-side polling.

The alternative would be to enable it only if KVM_FEATURE_POLL_CONTROL
is available, but I prefer Wanpeng's patch.

Paolo

>> For example, i think that using pv queued spinlocks and
>> haltpoll is a desired scenario, which the patch below disallows.
> 
> So even if dedicated pCPU is available, pv queued spinlocks should
> still be chose if something like vhost-kthreads are used instead of
> DPDK/vhost-user. kvm adaptive halt-polling will compete with
> vhost-kthreads, however, poll in guest unaware other runnable tasks in
> the host which will defeat vhost-kthreads.
> 
> Regards,
> Wanpeng Li
> 


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-13  0:55       ` Wanpeng Li
  2019-08-13  7:55         ` Paolo Bonzini
@ 2019-08-26 20:40         ` Marcelo Tosatti
  2019-08-27  0:43           ` Wanpeng Li
  1 sibling, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-26 20:40 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM

On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> >
> > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > >>
> > > >> The downside of guest side polling is that polling is performed even
> > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > >> just enable guest polling when dedicated pCPUs are available.
> > > >>
> > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > >
> > > > Paolo, Marcelo, any comments?
> > >
> > > Yes, it's a good idea.
> > >
> > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> > >
> > > Paolo
> >
> 
> Hi Marcelo,
> 
> Sorry for the late response.
> 
> > I think KVM_HINTS_REALTIME is being abused somewhat.
> > It has no clear meaning and used in different locations
> > for different purposes.
> 
> ================== ============ =================================
> KVM_HINTS_REALTIME 0                      guest checks this feature bit to
> 
> determine that vCPUs are never
> 
> preempted for an unlimited time

Unlimited time means infinite time, or unlimited time means 
10s ? 1s ?

The previous definition was much better IMO: HINTS_DEDICATED.


> allowing optimizations
> ================== ============ =================================
> 
> Now it disables pv queued spinlock, 

OK. 

> pv tlb shootdown, 

OK.

> pv sched yield

"The idea is from Xen, when sending a call-function IPI-many to vCPUs,
yield if any of the IPI target vCPUs was preempted. 17% performance
increasement of ebizzy benchmark can be observed in an over-subscribe
environment. (w/ kvm-pv-tlb disabled, testing TLB flush call-function
IPI-many since call-function is not easy to be trigged by userspace
workload)."

This can probably hurt if vcpus are rarely preempted. 

> which are not expected present in vCPUs are never preempted for an
> unlimited time scenario.
> 
> >
> > For example, i think that using pv queued spinlocks and
> > haltpoll is a desired scenario, which the patch below disallows.
> 
> So even if dedicated pCPU is available, pv queued spinlocks should
> still be chose if something like vhost-kthreads are used instead of
> DPDK/vhost-user. 

Can't you enable the individual features you need for optimizing 
the overcommitted case? This is how things have been done historically:
If a new feature is available, you enable it to get the desired
performance. x2apic, invariant-tsc, cpuidle haltpoll...

So in your case: enable pv schedyield, enable pv tlb shootdown.

> kvm adaptive halt-polling will compete with
> vhost-kthreads, however, poll in guest unaware other runnable tasks in
> the host which will defeat vhost-kthreads.

It depends on how much work vhost-kthreads needs to do, how successful 
halt-poll in the guest is, and what improvement halt-polling brings.
The amount of polling will be reduced to zero if polling 
is not successful.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-26 20:40         ` Marcelo Tosatti
@ 2019-08-27  0:43           ` Wanpeng Li
  2019-08-28  8:35             ` Wanpeng Li
  2019-08-28 14:48             ` Marcelo Tosatti
  0 siblings, 2 replies; 19+ messages in thread
From: Wanpeng Li @ 2019-08-27  0:43 UTC (permalink / raw)
  To: Marcelo Tosatti
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

Cc Michael S. Tsirkin,
On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>
> On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > >
> > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > >>
> > > > >> The downside of guest side polling is that polling is performed even
> > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > >>
> > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > >
> > > > > Paolo, Marcelo, any comments?
> > > >
> > > > Yes, it's a good idea.
> > > >
> > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> > > >
> > > > Paolo
> > >
> >
> > Hi Marcelo,
> >
> > Sorry for the late response.
> >
> > > I think KVM_HINTS_REALTIME is being abused somewhat.
> > > It has no clear meaning and used in different locations
> > > for different purposes.
> >
> > ================== ============ =================================
> > KVM_HINTS_REALTIME 0                      guest checks this feature bit to
> >
> > determine that vCPUs are never
> >
> > preempted for an unlimited time
>
> Unlimited time means infinite time, or unlimited time means
> 10s ? 1s ?

The former one I think. There is a discussion here
https://lkml.org/lkml/2018/5/17/612

>
> The previous definition was much better IMO: HINTS_DEDICATED.
>
>
> > allowing optimizations
> > ================== ============ =================================
> >
> > Now it disables pv queued spinlock,
>
> OK.
>
> > pv tlb shootdown,
>
> OK.
>
> > pv sched yield
>
> "The idea is from Xen, when sending a call-function IPI-many to vCPUs,
> yield if any of the IPI target vCPUs was preempted. 17% performance
> increasement of ebizzy benchmark can be observed in an over-subscribe
> environment. (w/ kvm-pv-tlb disabled, testing TLB flush call-function
> IPI-many since call-function is not easy to be trigged by userspace
> workload)."
>
> This can probably hurt if vcpus are rarely preempted.

That's why I add the KVM_HINTS_REALTIME checking here.

>
> > which are not expected present in vCPUs are never preempted for an
> > unlimited time scenario.
> >
> > >
> > > For example, i think that using pv queued spinlocks and
> > > haltpoll is a desired scenario, which the patch below disallows.
> >
> > So even if dedicated pCPU is available, pv queued spinlocks should
> > still be chose if something like vhost-kthreads are used instead of
> > DPDK/vhost-user.
>
> Can't you enable the individual features you need for optimizing
> the overcommitted case? This is how things have been done historically:
> If a new feature is available, you enable it to get the desired
> performance. x2apic, invariant-tsc, cpuidle haltpoll...
>
> So in your case: enable pv schedyield, enable pv tlb shootdown.

Both of them are used to optimize function-call IPIs. pv sched yield
for call function interrupts, and pv tlb shootdown for tlb
invalidation. So still different here. Our latest testing against an
80 pCPUs host, and three 80 vCPUs VMs, the number is more better than
64 pCPUs host which I used when posting patches:

ebizzy -M
              vanilla    optimized     boost
1VM            31234       34489        10%
2VM             5380       26664       396%
3VM             2967       23140       679%

>
> > kvm adaptive halt-polling will compete with
> > vhost-kthreads, however, poll in guest unaware other runnable tasks in
> > the host which will defeat vhost-kthreads.
>
> It depends on how much work vhost-kthreads needs to do, how successful
> halt-poll in the guest is, and what improvement halt-polling brings.
> The amount of polling will be reduced to zero if polling
> is not successful.

We observe vhost-kthreads compete with vCPUs adaptive halt-polling in
kvm, it hurt performance in over-subscribe product environment,
polling in guest can make it worse.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-27  0:43           ` Wanpeng Li
@ 2019-08-28  8:35             ` Wanpeng Li
  2019-08-28  8:45               ` Rafael J. Wysocki
  2019-08-28 14:48             ` Marcelo Tosatti
  1 sibling, 1 reply; 19+ messages in thread
From: Wanpeng Li @ 2019-08-28  8:35 UTC (permalink / raw)
  To: Marcelo Tosatti
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
>
> Cc Michael S. Tsirkin,
> On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> >
> > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > >
> > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > >>
> > > > > >> The downside of guest side polling is that polling is performed even
> > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > >>
> > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > >
> > > > > > Paolo, Marcelo, any comments?
> > > > >
> > > > > Yes, it's a good idea.
> > > > >
> > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>

Hi Marcelo,

If you don't have more concern, I guess Rafael can apply this patch
now since the merge window is not too far.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28  8:35             ` Wanpeng Li
@ 2019-08-28  8:45               ` Rafael J. Wysocki
  2019-08-28  8:48                 ` Wanpeng Li
  2019-08-28 14:39                 ` Marcelo Tosatti
  0 siblings, 2 replies; 19+ messages in thread
From: Rafael J. Wysocki @ 2019-08-28  8:45 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Marcelo Tosatti, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
>
> On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> >
> > Cc Michael S. Tsirkin,
> > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > >
> > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > >
> > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > >>
> > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > >>
> > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > >
> > > > > > > Paolo, Marcelo, any comments?
> > > > > >
> > > > > > Yes, it's a good idea.
> > > > > >
> > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
>
> Hi Marcelo,
>
> If you don't have more concern, I guess Rafael can apply this patch
> now since the merge window is not too far.

I will likely queue it up later today and it will go to linux-next
early next week.

Thanks!

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28  8:45               ` Rafael J. Wysocki
@ 2019-08-28  8:48                 ` Wanpeng Li
  2019-08-28 14:39                 ` Marcelo Tosatti
  1 sibling, 0 replies; 19+ messages in thread
From: Wanpeng Li @ 2019-08-28  8:48 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Marcelo Tosatti, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Wed, 28 Aug 2019 at 16:45, Rafael J. Wysocki <rafael@kernel.org> wrote:
>
> On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
> >
> > On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> > >
> > > Cc Michael S. Tsirkin,
> > > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > >
> > > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > >>
> > > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > > >>
> > > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > >
> > > > > > > > Paolo, Marcelo, any comments?
> > > > > > >
> > > > > > > Yes, it's a good idea.
> > > > > > >
> > > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> >
> > Hi Marcelo,
> >
> > If you don't have more concern, I guess Rafael can apply this patch
> > now since the merge window is not too far.
>
> I will likely queue it up later today and it will go to linux-next
> early next week.

Thank you, Rafael.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28  8:45               ` Rafael J. Wysocki
  2019-08-28  8:48                 ` Wanpeng Li
@ 2019-08-28 14:39                 ` Marcelo Tosatti
  2019-08-28 23:37                   ` Rafael J. Wysocki
  1 sibling, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-28 14:39 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Wanpeng Li, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Wed, Aug 28, 2019 at 10:45:44AM +0200, Rafael J. Wysocki wrote:
> On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
> >
> > On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> > >
> > > Cc Michael S. Tsirkin,
> > > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > >
> > > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > >>
> > > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > > >>
> > > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > >
> > > > > > > > Paolo, Marcelo, any comments?
> > > > > > >
> > > > > > > Yes, it's a good idea.
> > > > > > >
> > > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> >
> > Hi Marcelo,
> >
> > If you don't have more concern, I guess Rafael can apply this patch
> > now since the merge window is not too far.
> 
> I will likely queue it up later today and it will go to linux-next
> early next week.
> 
> Thanks!

NACK patch.

Just don't load the haltpoll driver.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-27  0:43           ` Wanpeng Li
  2019-08-28  8:35             ` Wanpeng Li
@ 2019-08-28 14:48             ` Marcelo Tosatti
  2019-08-28 14:52               ` Marcelo Tosatti
  1 sibling, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-28 14:48 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Tue, Aug 27, 2019 at 08:43:13AM +0800, Wanpeng Li wrote:
> > > kvm adaptive halt-polling will compete with
> > > vhost-kthreads, however, poll in guest unaware other runnable tasks in
> > > the host which will defeat vhost-kthreads.
> >
> > It depends on how much work vhost-kthreads needs to do, how successful
> > halt-poll in the guest is, and what improvement halt-polling brings.
> > The amount of polling will be reduced to zero if polling
> > is not successful.
> 
> We observe vhost-kthreads compete with vCPUs adaptive halt-polling in
> kvm, it hurt performance in over-subscribe product environment,
> polling in guest can make it worse.
> 
> Regards,
> Wanpeng Li

Wanpeng,

Polling should not be performed if there is other work to do. For
example, halt-polling could check a host/guest shared memory 
region indicating whether there are other runnable tasks in the host.

Disabling polling means you will not achieve the improvement 
even in the transitional periods where the system is not
overcommitted (which should be frequent given that idling 
is common).

Again, about your patch: it brings no benefit to anyone. 

Guest halt polling should be already disabled by default
(the driver has to be loaded for guest polling to take place).


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28 14:48             ` Marcelo Tosatti
@ 2019-08-28 14:52               ` Marcelo Tosatti
  0 siblings, 0 replies; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-28 14:52 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Wed, Aug 28, 2019 at 11:48:58AM -0300, Marcelo Tosatti wrote:
> On Tue, Aug 27, 2019 at 08:43:13AM +0800, Wanpeng Li wrote:
> > > > kvm adaptive halt-polling will compete with
> > > > vhost-kthreads, however, poll in guest unaware other runnable tasks in
> > > > the host which will defeat vhost-kthreads.
> > >
> > > It depends on how much work vhost-kthreads needs to do, how successful
> > > halt-poll in the guest is, and what improvement halt-polling brings.
> > > The amount of polling will be reduced to zero if polling
> > > is not successful.
> > 
> > We observe vhost-kthreads compete with vCPUs adaptive halt-polling in
> > kvm, it hurt performance in over-subscribe product environment,
> > polling in guest can make it worse.
> > 
> > Regards,
> > Wanpeng Li
> 
> Wanpeng,
> 
> Polling should not be performed if there is other work to do. For
> example, halt-polling could check a host/guest shared memory 
> region indicating whether there are other runnable tasks in the host.
> 
> Disabling polling means you will not achieve the improvement 
> even in the transitional periods where the system is not
> overcommitted (which should be frequent given that idling 
> is common).
> 
> Again, about your patch: it brings no benefit to anyone. 
> 
> Guest halt polling should be already disabled by default
> (the driver has to be loaded for guest polling to take place).

The most efficient solution would be to mwait on a memory 
region that both host and guest would write to.

No cpu cycles burned, full efficiency.

However both host and guest would have to write to this region, which
brings security concerns.



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28 14:39                 ` Marcelo Tosatti
@ 2019-08-28 23:37                   ` Rafael J. Wysocki
  2019-08-29 12:04                     ` Marcelo Tosatti
  0 siblings, 1 reply; 19+ messages in thread
From: Rafael J. Wysocki @ 2019-08-28 23:37 UTC (permalink / raw)
  To: Marcelo Tosatti
  Cc: Rafael J. Wysocki, Wanpeng Li, Paolo Bonzini, Rafael J. Wysocki,
	LKML, kvm, Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Wed, Aug 28, 2019 at 4:39 PM Marcelo Tosatti <mtosatti@redhat.com> wrote:
>
> On Wed, Aug 28, 2019 at 10:45:44AM +0200, Rafael J. Wysocki wrote:
> > On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
> > >
> > > On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> > > >
> > > > Cc Michael S. Tsirkin,
> > > > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > >
> > > > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > > >
> > > > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > >>
> > > > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > > > >>
> > > > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > >
> > > > > > > > > Paolo, Marcelo, any comments?
> > > > > > > >
> > > > > > > > Yes, it's a good idea.
> > > > > > > >
> > > > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> > >
> > > Hi Marcelo,
> > >
> > > If you don't have more concern, I guess Rafael can apply this patch
> > > now since the merge window is not too far.
> >
> > I will likely queue it up later today and it will go to linux-next
> > early next week.
> >
> > Thanks!
>
> NACK patch.

I got an ACK from Paolo on it, though.  Convince Paolo to withdraw his
ACK if you want it to not be applied.

> Just don't load the haltpoll driver.

And why would that be better?

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-28 23:37                   ` Rafael J. Wysocki
@ 2019-08-29 12:04                     ` Marcelo Tosatti
  2019-08-29 12:16                       ` Wanpeng Li
  0 siblings, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-29 12:04 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Wanpeng Li, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Thu, Aug 29, 2019 at 01:37:35AM +0200, Rafael J. Wysocki wrote:
> On Wed, Aug 28, 2019 at 4:39 PM Marcelo Tosatti <mtosatti@redhat.com> wrote:
> >
> > On Wed, Aug 28, 2019 at 10:45:44AM +0200, Rafael J. Wysocki wrote:
> > > On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
> > > >
> > > > On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> > > > >
> > > > > Cc Michael S. Tsirkin,
> > > > > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > >
> > > > > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > > > >
> > > > > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > > >>
> > > > > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > > > > >>
> > > > > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > > >
> > > > > > > > > > Paolo, Marcelo, any comments?
> > > > > > > > >
> > > > > > > > > Yes, it's a good idea.
> > > > > > > > >
> > > > > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> > > >
> > > > Hi Marcelo,
> > > >
> > > > If you don't have more concern, I guess Rafael can apply this patch
> > > > now since the merge window is not too far.
> > >
> > > I will likely queue it up later today and it will go to linux-next
> > > early next week.
> > >
> > > Thanks!
> >
> > NACK patch.
> 
> I got an ACK from Paolo on it, though.  Convince Paolo to withdraw his
> ACK if you want it to not be applied.
> 
> > Just don't load the haltpoll driver.
> 
> And why would that be better?

Split the group of all kvm users in two: overcommit group and non-overcommit
group.

Current situation regarding haltpoll driver is:

overcommit group: haltpoll driver is not loaded by default, they are
happy.

non overcommit group: boots without "realtime hints" flag, loads haltpoll driver, 
happy.

Situation with patch above:

overcommit group: haltpoll driver is not loaded by default, they are
happy.

non overcommit group: boots without "realtime hints" flag, haltpoll driver
cannot be loaded.





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-29 12:04                     ` Marcelo Tosatti
@ 2019-08-29 12:16                       ` Wanpeng Li
  2019-08-29 12:53                         ` Marcelo Tosatti
  0 siblings, 1 reply; 19+ messages in thread
From: Wanpeng Li @ 2019-08-29 12:16 UTC (permalink / raw)
  To: Marcelo Tosatti
  Cc: Rafael J. Wysocki, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Thu, 29 Aug 2019 at 20:04, Marcelo Tosatti <mtosatti@redhat.com> wrote:
>
> On Thu, Aug 29, 2019 at 01:37:35AM +0200, Rafael J. Wysocki wrote:
> > On Wed, Aug 28, 2019 at 4:39 PM Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > >
> > > On Wed, Aug 28, 2019 at 10:45:44AM +0200, Rafael J. Wysocki wrote:
> > > > On Wed, Aug 28, 2019 at 10:34 AM Wanpeng Li <kernellwp@gmail.com> wrote:
> > > > >
> > > > > On Tue, 27 Aug 2019 at 08:43, Wanpeng Li <kernellwp@gmail.com> wrote:
> > > > > >
> > > > > > Cc Michael S. Tsirkin,
> > > > > > On Tue, 27 Aug 2019 at 04:42, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > > >
> > > > > > > On Tue, Aug 13, 2019 at 08:55:29AM +0800, Wanpeng Li wrote:
> > > > > > > > On Sun, 4 Aug 2019 at 04:21, Marcelo Tosatti <mtosatti@redhat.com> wrote:
> > > > > > > > >
> > > > > > > > > On Thu, Aug 01, 2019 at 06:54:49PM +0200, Paolo Bonzini wrote:
> > > > > > > > > > On 01/08/19 18:51, Rafael J. Wysocki wrote:
> > > > > > > > > > > On 8/1/2019 9:06 AM, Wanpeng Li wrote:
> > > > > > > > > > >> From: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > > > >>
> > > > > > > > > > >> The downside of guest side polling is that polling is performed even
> > > > > > > > > > >> with other runnable tasks in the host. However, even if poll in kvm
> > > > > > > > > > >> can aware whether or not other runnable tasks in the same pCPU, it
> > > > > > > > > > >> can still incur extra overhead in over-subscribe scenario. Now we can
> > > > > > > > > > >> just enable guest polling when dedicated pCPUs are available.
> > > > > > > > > > >>
> > > > > > > > > > >> Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> > > > > > > > > > >> Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > > > > > > > > >> Cc: Radim Krčmář <rkrcmar@redhat.com>
> > > > > > > > > > >> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > > > > > > > > > >> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > > > > > > > > > >
> > > > > > > > > > > Paolo, Marcelo, any comments?
> > > > > > > > > >
> > > > > > > > > > Yes, it's a good idea.
> > > > > > > > > >
> > > > > > > > > > Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> > > > >
> > > > > Hi Marcelo,
> > > > >
> > > > > If you don't have more concern, I guess Rafael can apply this patch
> > > > > now since the merge window is not too far.
> > > >
> > > > I will likely queue it up later today and it will go to linux-next
> > > > early next week.
> > > >
> > > > Thanks!
> > >
> > > NACK patch.
> >
> > I got an ACK from Paolo on it, though.  Convince Paolo to withdraw his
> > ACK if you want it to not be applied.
> >
> > > Just don't load the haltpoll driver.
> >
> > And why would that be better?
>
> Split the group of all kvm users in two: overcommit group and non-overcommit
> group.
>
> Current situation regarding haltpoll driver is:
>
> overcommit group: haltpoll driver is not loaded by default, they are
> happy.
>
> non overcommit group: boots without "realtime hints" flag, loads haltpoll driver,
> happy.
>
> Situation with patch above:
>
> overcommit group: haltpoll driver is not loaded by default, they are
> happy.
>
> non overcommit group: boots without "realtime hints" flag, haltpoll driver
> cannot be loaded.

non overcommit group, if they don't care latency/performance, they
don't need to enable haltpoll, "realtime hints" etc. Otherwise, they
should better tune.

Regards,
Wanpeng Li

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-29 12:16                       ` Wanpeng Li
@ 2019-08-29 12:53                         ` Marcelo Tosatti
  2019-08-29 14:39                           ` Marcelo Tosatti
  0 siblings, 1 reply; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-29 12:53 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Rafael J. Wysocki, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Thu, Aug 29, 2019 at 08:16:41PM +0800, Wanpeng Li wrote:
> > Current situation regarding haltpoll driver is:
> >
> > overcommit group: haltpoll driver is not loaded by default, they are
> > happy.
> >
> > non overcommit group: boots without "realtime hints" flag, loads haltpoll driver,
> > happy.
> >
> > Situation with patch above:
> >
> > overcommit group: haltpoll driver is not loaded by default, they are
> > happy.
> >
> > non overcommit group: boots without "realtime hints" flag, haltpoll driver
> > cannot be loaded.
> 
> non overcommit group, if they don't care latency/performance, they
> don't need to enable haltpoll, "realtime hints" etc. Otherwise, they
> should better tune.

As mentioned before, "being overcommitted" is a property which is transitional.

A static true/false scheme reflects this poorly.

Therefore the OS should detect it and act accordingly.


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available
  2019-08-29 12:53                         ` Marcelo Tosatti
@ 2019-08-29 14:39                           ` Marcelo Tosatti
  0 siblings, 0 replies; 19+ messages in thread
From: Marcelo Tosatti @ 2019-08-29 14:39 UTC (permalink / raw)
  To: Wanpeng Li
  Cc: Rafael J. Wysocki, Paolo Bonzini, Rafael J. Wysocki, LKML, kvm,
	Radim Krčmář,
	Linux PM, Michael S. Tsirkin

On Thu, Aug 29, 2019 at 09:53:04AM -0300, Marcelo Tosatti wrote:
> On Thu, Aug 29, 2019 at 08:16:41PM +0800, Wanpeng Li wrote:
> > > Current situation regarding haltpoll driver is:
> > >
> > > overcommit group: haltpoll driver is not loaded by default, they are
> > > happy.
> > >
> > > non overcommit group: boots without "realtime hints" flag, loads haltpoll driver,
> > > happy.
> > >
> > > Situation with patch above:
> > >
> > > overcommit group: haltpoll driver is not loaded by default, they are
> > > happy.
> > >
> > > non overcommit group: boots without "realtime hints" flag, haltpoll driver
> > > cannot be loaded.
> > 
> > non overcommit group, if they don't care latency/performance, they
> > don't need to enable haltpoll, "realtime hints" etc. Otherwise, they
> > should better tune.
> 
> As mentioned before, "being overcommitted" is a property which is transitional.
> 
> A static true/false scheme reflects this poorly.
> 
> Therefore the OS should detect it and act accordingly.

Hi Wanpeng Li,

One suggestion for a dynamic "is overcommited" scheme:

If the amount of stolen time, in the past record_steal_time window, 
is more than 20% of the time in that window, then mark system
as overcommitted. Otherwise, clear it.

Make that 20% configurable by as kvm module parameter.

Use that info to enable/disable overcommit features.

That should work, right?





^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2019-08-29 14:39 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-01  7:06 [PATCH] cpuidle-haltpoll: Enable kvm guest polling when dedicated physical CPUs are available Wanpeng Li
2019-08-01 16:51 ` Rafael J. Wysocki
2019-08-01 16:54   ` Paolo Bonzini
2019-08-03 20:21     ` Marcelo Tosatti
2019-08-13  0:55       ` Wanpeng Li
2019-08-13  7:55         ` Paolo Bonzini
2019-08-26 20:40         ` Marcelo Tosatti
2019-08-27  0:43           ` Wanpeng Li
2019-08-28  8:35             ` Wanpeng Li
2019-08-28  8:45               ` Rafael J. Wysocki
2019-08-28  8:48                 ` Wanpeng Li
2019-08-28 14:39                 ` Marcelo Tosatti
2019-08-28 23:37                   ` Rafael J. Wysocki
2019-08-29 12:04                     ` Marcelo Tosatti
2019-08-29 12:16                       ` Wanpeng Li
2019-08-29 12:53                         ` Marcelo Tosatti
2019-08-29 14:39                           ` Marcelo Tosatti
2019-08-28 14:48             ` Marcelo Tosatti
2019-08-28 14:52               ` Marcelo Tosatti

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).