From: Avi Kivity <avi@redhat.com>
To: Erik Jacobson <erikj@sgi.com>
Cc: kvm@vger.kernel.org
Subject: Re: slow guest performance with build load, looking for ideas
Date: Sun, 28 Jun 2009 17:17:48 +0300 [thread overview]
Message-ID: <4A477B8C.2010502@redhat.com> (raw)
In-Reply-To: <20090618230744.GA19307@sgi.com>
On 06/19/2009 02:07 AM, Erik Jacobson wrote:
> Hello. I'll top-post since the quoted text is just for reference.
>
> Sorry the follow-up testing took so long. We're very low on 5500/Nehalem
> resources at the moment and I had to track down lots of stuff before
> getting to the test.
>
> I ran some tests on a 2-socket, 8-core system. I wasn't pleased with the
> results for a couple reasons. One, the issue of it being twice as slow
> as the host with no guest was still present.
>
> However, in trying to make use of this system using Fedora 11, I ran in to
> several issues not directly related to virtualization. So these test runs
> have that grain of salt. Example issues...
>
>
<snip>
> * In some of the timing runs on this system, the "real time" reported by
> the time command was off by 10 to 11 times. Issues were found in
> the messages file that seemed to relate to this including HUGE time
> adjustments by NTP and kernel hrtimer 'interrupt too slow' messages.
> This specific problem seems to be intermittent.
>
This is on the host? It can easily ruin your day.
> So those are the grains of salt. I've found that, when doing the timing by
> hand instead of using the time command, the build time seems to be around
> 10 to 12 minutes. I'm not sure how trustworthy the output from the time
> command are in these trials. In any event, that's still more than double
> for host alone with no guests.
>
> System:
> SGI XE270, 8-core, Xeon X5570 (Nehalem), Hyperthreading turned off
>
Shoot, was about to blame hyperthreading.
> Test, as before, was simply this for a kernel build. The .config file has
> plenty of modules configured.
> time (make -j12&& make -j12 modules)
>
>
>
> host only, no guest, baseline
> -----------------------------
> trial 1:
> real 5m44.823s
> user 28m45.725s
> sys 5m46.633s
>
> trial 2:
> real 5m34.438s
> user 28m14.347s
> sys 5m41.597s
>
>
> guest, 8 vcpu, 4096 mem, virtio, no cache param, disk device supplied in full
> -----------------------------------------------------------------------------
> trial 1:
> real 125m5.995s
> user 31m23.790s
> sys 9m17.602s
>
>
> trial 2 (changed to 7168 mb memory for the guest):
> real 120m48.431s
> user 14m38.967s
> sys 6m12.437s
>
>
> That's real strange... The 'time' command is showing whacked out results.
>
> I then watched a run by hand and counted it at about 10 minutes. However,
> this third run had the proper time! So whatever the weirdness is, it doesn't
> happen every time:
>
> real 9m49.802s
> user 24m46.009s
> sys 8m10.349s
>
> I decided this could be related to ntp running as I saw this in messages:
> Jun 18 16:34:23 localhost ntpd[1916]: time reset -0.229209 s
> Jun 18 16:34:23 localhost ntpd[1916]: kernel time sync status change 0001
> Jun 18 16:40:17 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2
>
> and earlier:
>
> Jun 18 16:19:09 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2
> Jun 18 16:19:09 localhost ntpd[1916]: time reset +6609.851122 s
> Jun 18 16:23:39 localhost ntpd[1916]: synchronized to 128.162.244.1, stratum 2
> Jun 18 16:24:04 localhost kernel: hrtimer: interrupt too slow, forcing clock min delta to 62725995 ns
>
>
> I then installed all F11 updates in the guest and tried again (host had
> updates all along). I got these strange results, strange because of the
> timing difference. I didn't "watch a non-computer clock" for these.
>
kvm guests should have an accurate clock without ntp in the guest
(/sys/.../current_clocksource should say 'kvmclock').
Can you post kvm_stat output during the run?
--
error compiling committee.c: too many arguments to function
next prev parent reply other threads:[~2009-06-28 14:16 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-06-12 21:04 slow guest performance with build load, looking for ideas Erik Jacobson
2009-06-14 9:33 ` Avi Kivity
2009-06-15 14:15 ` Erik Jacobson
2009-06-15 14:24 ` Avi Kivity
2009-06-15 15:25 ` Michael Tokarev
2009-06-15 15:27 ` Avi Kivity
2009-06-16 7:03 ` Michael Tokarev
2009-06-16 8:07 ` Avi Kivity
2009-06-18 23:07 ` Erik Jacobson
2009-06-28 14:17 ` Avi Kivity [this message]
2009-06-28 19:05 ` Erik Jacobson
2009-06-28 21:28 ` Avi Kivity
2009-07-01 21:41 ` Erik Jacobson
2009-07-02 5:48 ` Avi Kivity
2009-07-02 9:41 ` Avi Kivity
2009-07-03 15:43 ` Mark McLoughlin
2009-07-03 16:28 ` Erik Jacobson
2009-07-09 2:36 ` Erik Jacobson
2009-07-09 7:48 ` Avi Kivity
2009-07-09 18:01 ` Erik Jacobson
2009-07-12 8:38 ` Avi Kivity
2009-07-16 12:20 ` Jes Sorensen
2009-07-25 14:33 ` Avi Kivity
2009-07-03 10:41 ` Matty
2009-07-05 8:07 ` Avi Kivity
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4A477B8C.2010502@redhat.com \
--to=avi@redhat.com \
--cc=erikj@sgi.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).