Perf regression of netperf compared Xen 4.4 to origin/master

Perf regression of netperf compared Xen 4.4 to origin/master

[Konrad Rzeszutek Wilk]

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]

The test is pretty silly.

I've 14 guests, and I make each guest run netperf against the other.
I start with 1 pair, then 2 pairs, then up to 7. Each guest
is 768 MB, runs in a RAMdisk and just has one vif. The box is
a SuperMicro X8DB8+ with two E5520 and 12GB of RAM.
The netperf runs for a minute. The guests are PVHVM and I am
using qemu-upstream.  The guest configs are:

builder='hvm'
disk = [ 'file:/mnt/lab/bootstrap-x86_64/root_image.iso,hdc:cdrom,r']
memory=768
boot='d'
vcpus=1
serial='file:/var/log/xen/console-g-vm0.log'
vnclisten='0.0.0.0'
name='g-vm0'
on_crash='preserve'
# IP is 192.168.102.90
vif = [ 'mac=00:0f:4b:01:01:00, bridge=switch' ]
.. and naturally the ip, name and MAC change for each guest.

The kernel for both the guest and domain 0 is the same - no
configuration change. The Xen is built without debug=y.

When doing this test under Xen 4.4:

 0,  8823.14,  8823.14,  8823.14,  8823.14
 1,  9645.67,  4822.84,  4085.88,  5559.79
 2,  11725.3,  3908.44,  2652.09,  4698.15
 3,  12199.3,  3049.83,  1473.95,  3631.93
 4,  12223.2,  2444.64,  1023.48,  2887.83
 5,  11733.4,  1955.57,   909.66,  3057.11
 6,  11927.7,  1703.95,  1187.29,  1933.76
 7,  11623.8,  1452.97,   538.11,  2228.21

And when doing it under 'master' branch:


 0,  8478.57,  8478.57,  8478.57,  8478.57
 1,  7663.38,  3831.69,  3427.70,  4235.68
 2,  8381.77,  2793.92,  1977.75,  3242.46
 3,  8165.55,  2041.39,  1373.43,  2294.48
 4,  7742.64,  1548.53,   763.08,  2065.43
 5,  7691.87,  1281.98,   626.85,  1784.72
 6,  8337.06,  1191.01,   626.98,  1847.55
 7,   8074.8,  1009.35,   384.84,  1380.00

I am going to start a bit of git bisection to figure this out
but if somebody has some ideas on where/when to concentrate I
would appreciate the help.


[-- Attachment #2: network.png --]
[-- Type: image/png, Size: 10317 bytes --]

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Perf regression of netperf compared Xen 4.4 to origin/master

[Andrew Cooper]

On 02/07/14 17:52, Konrad Rzeszutek Wilk wrote:
> The test is pretty silly.
>
> I've 14 guests, and I make each guest run netperf against the other.
> I start with 1 pair, then 2 pairs, then up to 7. Each guest
> is 768 MB, runs in a RAMdisk and just has one vif. The box is
> a SuperMicro X8DB8+ with two E5520 and 12GB of RAM.
> The netperf runs for a minute. The guests are PVHVM and I am
> using qemu-upstream.  The guest configs are:
>
> builder='hvm'
> disk = [ 'file:/mnt/lab/bootstrap-x86_64/root_image.iso,hdc:cdrom,r']
> memory=768
> boot='d'
> vcpus=1
> serial='file:/var/log/xen/console-g-vm0.log'
> vnclisten='0.0.0.0'
> name='g-vm0'
> on_crash='preserve'
> # IP is 192.168.102.90
> vif = [ 'mac=00:0f:4b:01:01:00, bridge=switch' ]
> .. and naturally the ip, name and MAC change for each guest.
>
> The kernel for both the guest and domain 0 is the same - no
> configuration change. The Xen is built without debug=y.

What do you mean by this?  master has debugging enabled by default so
unless you build with debug=n, you will have a debug build.

~Andrew

Re: Perf regression of netperf compared Xen 4.4 to origin/master

[George Dunlap]

On Wed, Jul 2, 2014 at 5:55 PM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 02/07/14 17:52, Konrad Rzeszutek Wilk wrote:
>> The test is pretty silly.
>>
>> I've 14 guests, and I make each guest run netperf against the other.
>> I start with 1 pair, then 2 pairs, then up to 7. Each guest
>> is 768 MB, runs in a RAMdisk and just has one vif. The box is
>> a SuperMicro X8DB8+ with two E5520 and 12GB of RAM.
>> The netperf runs for a minute. The guests are PVHVM and I am
>> using qemu-upstream.  The guest configs are:
>>
>> builder='hvm'
>> disk = [ 'file:/mnt/lab/bootstrap-x86_64/root_image.iso,hdc:cdrom,r']
>> memory=768
>> boot='d'
>> vcpus=1
>> serial='file:/var/log/xen/console-g-vm0.log'
>> vnclisten='0.0.0.0'
>> name='g-vm0'
>> on_crash='preserve'
>> # IP is 192.168.102.90
>> vif = [ 'mac=00:0f:4b:01:01:00, bridge=switch' ]
>> .. and naturally the ip, name and MAC change for each guest.
>>
>> The kernel for both the guest and domain 0 is the same - no
>> configuration change. The Xen is built without debug=y.
>
> What do you mean by this?  master has debugging enabled by default so
> unless you build with debug=n, you will have a debug build.

You don't know how many times I've tripped over this...

 -George

Re: Perf regression of netperf compared Xen 4.4 to origin/master

[Konrad Rzeszutek Wilk]

On Thu, Jul 03, 2014 at 11:46:30AM +0100, George Dunlap wrote:
> On Wed, Jul 2, 2014 at 5:55 PM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> > On 02/07/14 17:52, Konrad Rzeszutek Wilk wrote:
> >> The test is pretty silly.
> >>
> >> I've 14 guests, and I make each guest run netperf against the other.
> >> I start with 1 pair, then 2 pairs, then up to 7. Each guest
> >> is 768 MB, runs in a RAMdisk and just has one vif. The box is
> >> a SuperMicro X8DB8+ with two E5520 and 12GB of RAM.
> >> The netperf runs for a minute. The guests are PVHVM and I am
> >> using qemu-upstream.  The guest configs are:
> >>
> >> builder='hvm'
> >> disk = [ 'file:/mnt/lab/bootstrap-x86_64/root_image.iso,hdc:cdrom,r']
> >> memory=768
> >> boot='d'
> >> vcpus=1
> >> serial='file:/var/log/xen/console-g-vm0.log'
> >> vnclisten='0.0.0.0'
> >> name='g-vm0'
> >> on_crash='preserve'
> >> # IP is 192.168.102.90
> >> vif = [ 'mac=00:0f:4b:01:01:00, bridge=switch' ]
> >> .. and naturally the ip, name and MAC change for each guest.
> >>
> >> The kernel for both the guest and domain 0 is the same - no
> >> configuration change. The Xen is built without debug=y.
> >
> > What do you mean by this?  master has debugging enabled by default so
> > unless you build with debug=n, you will have a debug build.
> 
> You don't know how many times I've tripped over this...

<end flushes>

Whew. It was indeed that, and next time I have to make that part of
my checklist before sending to xen-devel!

> 
>  -George