Re: [PATCH] xen/arm: introduce vwfi parameter

[Stefano Stabellini <sstabellini@kernel.org>]

CC'ing xen-devel, I forgot on the original patch

On Fri, 17 Feb 2017, Julien Grall wrote:
> Hi Stefano,
> 
> On 02/16/2017 11:04 PM, Stefano Stabellini wrote:
> > Introduce new Xen command line parameter called "vwfi", which stands for
> > virtual wfi. The default is "sleep": on guest wfi, Xen calls vcpu_block
> > on the guest vcpu. The behavior can be changed setting vwfi to "idle",
> > in that case Xen calls vcpu_yield.
> > 
> > The result is strong reduction in irq latency (8050ns -> 3500ns) at the
> > cost of idle_loop being called less often, leading to higher power
> > consumption.
> 
> Please explain in which context this will be beneficial. My gut feeling is
> only will make performance worst if a multiple vCPU of the same guest is
> running on vCPU

I am not a scheduler expert, but I don't think so. Let me explain the
difference:

- vcpu_block blocks a vcpu until an event occurs, for example until it
  receives an interrupt

- vcpu_yield stops the vcpu from running until the next scheduler slot

In both cases the vcpus is not run until the next slot, so I don't think
it should make the performance worse in multi-vcpus scenarios. But I can
do some tests to double check.


> > Signed-off-by: Stefano Stabellini <sstabellini@kernel.org>
> > CC: dario.faggioli@citrix.com
> > ---
> >  docs/misc/xen-command-line.markdown | 11 +++++++++++
> >  xen/arch/arm/traps.c                | 17 +++++++++++++++--
> >  2 files changed, 26 insertions(+), 2 deletions(-)
> > 
> > diff --git a/docs/misc/xen-command-line.markdown
> > b/docs/misc/xen-command-line.markdown
> > index a11fdf9..5d003e4 100644
> > --- a/docs/misc/xen-command-line.markdown
> > +++ b/docs/misc/xen-command-line.markdown
> > @@ -1632,6 +1632,17 @@ Note that if **watchdog** option is also specified
> > vpmu will be turned off.
> >  As the virtualisation is not 100% safe, don't use the vpmu flag on
> >  production systems (see http://xenbits.xen.org/xsa/advisory-163.html)!
> > 
> > +### vwfi
> > +> `= sleep | idle
> > +
> > +> Default: `sleep`
> > +
> > +WFI is the ARM instruction to "wait for interrupt". This option, which
> > +is ARM specific, changes the way guest WFI is implemented in Xen. By
> > +default, Xen blocks the guest vcpu, putting it to sleep. When setting
> > +vwfi to `idle`, Xen idles the guest vcpu instead, resulting in lower
> > +interrupt latency, but higher power consumption.
> 
> The main point of using wfi is for power saving. With this change, you will
> end up in a busy loop and as you said consume more power.

That's not true: the vcpu is still descheduled until the next slot.
There is no busy loop (that would be indeed very bad).


> I don't think this is acceptable even to get a better interrupt latency. Some
> workload will care about interrupt latency and power.
> 
> I think a better approach would be to check whether the scheduler has another
> vCPU to run. If not wait for an interrupt in the trap.
> 
> This would save the context switch to the idle vCPU if we are still on the
> time slice of the vCPU.

From my limited understanding of how schedulers work, I think this
cannot work reliably. It is the scheduler that needs to tell the
arch-specific code to put a pcpu to sleep, not the other way around. I
would appreciate if Dario could confirm this though.


> Likely this may not fit everyone, so I would add some knowledge to change the
> behavior of WFI depending on how many vCPU are scheduled on the current pCPU.
> But this could be done as a second step.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel