Re: Load increase after memory upgrade (part2)

[Konrad Rzeszutek Wilk <konrad@darnok.org>]

On Sat, Nov 26, 2011 at 10:14:08AM +0100, Carsten Schiers wrote:
> To add (read from some munin statistics I made over the time):
> 
>   - with load I mean the %CPU of xentop
>   - there is no change in CPU usage of the DomU or Dom0

Uhh, which matrix are using for that? CPU usage...? This is if you
change the DomU or the amount of memory the guest has? This is not
the load number (xentop value)?

>   - xenpm shows the core dedicated to that DomU is doing more work
> 
> Also I need to say that reduction to 4GB was performed by Xen parameter.
> 
> Carsten.
> 
> 
> -----Urspr?ngliche Nachricht-----
> Von: Konrad Rzeszutek Wilk [mailto:konrad@darnok.org] 
> Gesendet: Freitag, 25. November 2011 19:43
> An: Carsten Schiers
> Cc: konrad.wilk; xen-devel
> Betreff: Re: [Xen-devel] Load increase after memory upgrade (part2)
> 
> On Thu, Nov 24, 2011 at 01:28:44PM +0100, Carsten Schiers wrote:
> > Hello again, I would like to come back to that thing...sorry that I did not have the time up to now.
> > 
> > ??
> > We (now) speak about
> > 
> > ??
> > *	Xen 4.1.2
> > *	Dom0 is Jeremy's 2.6.32.46 64 bit
> > *	DomU in question is now 3.1.2 64 bit
> > *	Same thing if DomU is also 2.6.32.46
> > *	DomU owns two PCI cards (DVB-C) that o DMA
> > *	Machine has 8GB, Dom0 pinned at 512MB
> > 
> > ??
> > As compared to 2.6.34 Kernel with backported patches, the load on the DomU is at least twice as high. It
> > 
> > will be "close to normal" if I reduce the memory used to 4GB.
> 
> That is in the dom0 or just in general on the machine?
> > 
> > ??
> > As you can see from the attachment, you once had an idea. So should we try to find something...?
> 
> I think that was to instrument swiotlb to give an idea of how
> often it is called and basically have a matrix of its load. And
> from there figure out if the issue is that:
> 
>  1). The drivers allocoate/bounce/deallocate buffers on every interrupt
>     (bad, driver should be using some form of dma pool and most of the
>     ivtv do that)
> 
>  2). The buffers allocated to the drivers are above the 4GB and we end
>     up bouncing it needlessly. That can happen if the dom0 has most of
>     the precious memory under 4GB. However, that is usually not the case
>     as the domain isusually allocated from the top of the memory. The
>     fix for that was to set dom0_mem=max:XX. .. but with Dom0 kernels
>     before 3.1, the parameter would be ignored, so you had to use
>     'mem=XX' on the Linux command line as well.
> 
>  3). Where did you get the load values? Was it dom0? or domU?
> 
> 
> 
> > 
> > ??
> > Carsten.
> > ??
> > -----Urspr??ngliche Nachricht-----
> > An:konrad.wilk <konrad.wilk@oracle.com>; 
> > CC:linux <linux@eikelenboom.it>; xen-devel <xen-devel@lists.xensource.com>; 
> > Von:Carsten Schiers <carsten@schiers.de>
> > Gesendet:Mi 29.06.2011 23:17
> > Betreff:AW: Re: Re: Re: AW: Re: [Xen-devel] AW: Load increase after memory upgrade?
> > > Lets first do the c) experiment as that will likely explain your load average increase.
> > ...
> > > >c). If you want to see if the fault here lies in the bounce buffer 
> > > being used more
> > > >often in the DomU b/c you have 8GB of memory now and you end up using 
> > > more pages
> > > >past 4GB (in DomU), I can cook up a patch to figure this out. But an 
> > > easier way is
> > > >to just do (on the Xen hypervisor line): mem=4G and that will make 
> > > think you only have
> > > >4GB of physical RAM. ??If the load comes back to the normal "amount" 
> > > then the likely
> > > >culprit is that and we can think on how to fix this.
> > 
> > You are on the right track. Load was going down to "normal" 10% when reducing
> > Xen to 4GB by the parameter. Load seems to be still a little, little bit lower
> > with Xenified Kernel (8-9%), but this is drastically lower than the 20% we had
> > before.
> 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
>