Re: [RFC PATCH 0/4] Gang scheduling in CFS

[Nikunj A Dadhania <nikunj@linux.vnet.ibm.com>]

On Sun, 25 Dec 2011 12:58:15 +0200, Avi Kivity <avi@redhat.com> wrote:
> On 12/23/2011 12:36 PM, Ingo Molnar wrote:
> > * Nikunj A Dadhania <nikunj@linux.vnet.ibm.com> wrote:
> >
[...]
> > > 
> > > I see the main difference between both the reports is:
> > > native_flush_tlb_others.
> >
> > So it would be important to figure out why ebizzy gets into so 
> > many TLB flushes and why gang scheduling makes it go away.
> 
> The second part is easy - a remote tlb flush involves IPIs to many other
> vcpus (possible waking them up and scheduling them), then busy-waiting
> until they acknowledge the flush.  Gang scheduling is really good here
> since it shortens the busy wait, would be even better if we schedule
> halted vcpus (see the yield_on_hlt module parameter, set to 0). 
I will check this.

> Directed yield on PLE should provide intermediate results between doing
> nothing and gang sched.
>
Yes, thats true, I have pasted the results from my first mail to
highlight this:

    +-------------+---------------------------+-------------------------+
    |             |            V1 (%)         |             V2 (%)      |
    + Benchmarks  +-------------+-------------+-------------------------+
    |             | GangVsBase  |   GangVsPin |  GangVsBase | GangVsPin |
    +-------------+-------------+-------------+-------------------------+
    | ebizzy  2vm |        0    |        3    |        2    |        5  |
    | ebizzy  4vm |        1    |        0    |        4    |        3  |
    | ebizzy  8vm |        0    |        1    |       23    |       26  |
    +-------------+-------------+-------------+-------------------------+
 
> 
> btw you can get an additional speedup by enabling x2apic, for
> default_send_IPI_mask_logical().
> 
In the host?

Nikunj