From: "Pallipadi, Venkatesh" <venkatesh.pallipadi@intel.com>
To: Chris Mason <chris.mason@oracle.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
Andrew Morton <akpm@linux-foundation.org>,
"Styner, Douglas W" <douglas.w.styner@intel.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Tripathi, Sharad C" <sharad.c.tripathi@intel.com>,
"arjan@linux.intel.com" <arjan@linux.intel.com>,
"Wilcox, Matthew R" <matthew.r.wilcox@intel.com>,
"Kleen, Andi" <andi.kleen@intel.com>,
"Siddha, Suresh B" <suresh.b.siddha@intel.com>,
"Ma, Chinang" <chinang.ma@intel.com>,
"Wang, Peter Xihong" <peter.xihong.wang@intel.com>,
"Nueckel, Hubert" <hubert.nueckel@intel.com>,
"Recalde, Luis F" <luis.f.recalde@intel.com>,
"Nelson, Doug" <doug.nelson@intel.com>,
"Cheng, Wu-sun" <wu-sun.cheng@intel.com>,
"Prickett, Terry O" <terry.o.prickett@intel.com>,
"Shunmuganathan,
Rajalakshmi" <rajalakshmi.shunmuganathan@intel.com>,
"Garg, Anil K" <anil.k.garg@intel.com>,
"Chilukuri, Harita" <harita.chilukuri@intel.com>,
Ingo Molnar <mingo@elte.hu>
Subject: Re: Mainline kernel OLTP performance update
Date: Wed, 29 Apr 2009 11:06:53 -0700 [thread overview]
Message-ID: <1241028413.4529.9247.camel@localhost.localdomain> (raw)
In-Reply-To: <1241027196.20099.42.camel@think.oraclecorp.com>
On Wed, 2009-04-29 at 10:46 -0700, Chris Mason wrote:
> On Wed, 2009-04-29 at 18:25 +0200, Peter Zijlstra wrote:
> > On Wed, 2009-04-29 at 09:07 -0700, Andrew Morton wrote:
> > > On Wed, 29 Apr 2009 08:48:19 -0700 "Styner, Douglas W" <douglas.w.styner@intel.com> wrote:
> > >
> > > > >-----Original Message-----
> > > > >From: Andrew Morton [mailto:akpm@linux-foundation.org]
> > > > >Sent: Wednesday, April 29, 2009 12:30 AM
> > > > >To: Styner, Douglas W
> > > > >Cc: linux-kernel@vger.kernel.org; Tripathi, Sharad C;
> > > > >arjan@linux.intel.com; Wilcox, Matthew R; Kleen, Andi; Siddha, Suresh B;
> > > > >Ma, Chinang; Wang, Peter Xihong; Nueckel, Hubert; Recalde, Luis F; Nelson,
> > > > >Doug; Cheng, Wu-sun; Prickett, Terry O; Shunmuganathan, Rajalakshmi; Garg,
> > > > >Anil K; Chilukuri, Harita; chris.mason@oracle.com
> > > > >Subject: Re: Mainline kernel OLTP performance update
> > > > >
> > > > >On Tue, 28 Apr 2009 10:08:22 -0700 "Styner, Douglas W"
> > > > ><douglas.w.styner@intel.com> wrote:
> > > > >
> > > > >> Summary: Measured the mainline kernel from kernel.org (2.6.30-rc3).
> > > > >>
> > > > >> The regression for 2.6.30-rc3 against the baseline, 2.6.24.2 is 1.91%.
> > > > >Oprofile reports 71.1626% user, 28.8295% system.
> > > > >>
> > > > >> Linux OLTP Performance summary
> > > > >> Kernel# Speedup(x) Intr/s CtxSw/s us% sys% idle%
> > > > >iowait%
> > > > >> 2.6.24.2 1.000 22106 43709 75 24 0 0
> > > > >> 2.6.30-rc3 0.981 30645 43027 75 25 0 0
> > > > >
> > > > >The main difference there is the interrupt frequency. Do we know which
> > > > >interrupt source(s) caused this?
> > > >
> > > > Our analysis of the interrupts shows that rescheduling interrupts are
> > > > up 2.2x from 2.6.24.2 --> 2.6.30-rc3. Qla2xxx interrupts are roughly
> > > > the same.
> > >
> > > (top-posting repaired)
> > >
> > > OK, thanks. Seems odd that the rescheduling interrupt rate increased
> > > while the context-switch rate actually fell a couple of percent.
> > >
> > > This came up a few weeks ago and iirc Peter was mainly involved, and I
> > > don't believe that anything conclusive ended up happening. Peter,
> > > could you please remind us of (and summarise) the story here?
> >
> > I've had several reports about the resched-ipi going in overdrive, but
> > nobody bothered to bisect it, nor have I yet done so -- no clear ideas
> > on why it is doing so.
> >
> > I'll put it somewhere higher on the todo list.
> >
>
> One cause of them in the past was the ondemand cpufreq module. It got
> fixed up for my laptop workload at least starting w/2.6.29, but it might
> make sense to try without ondemand if you're running it.
>
Output of
# grep . /sys/devices/system/cpu/cpu0/cpufreq/*
can tell us whether P-state software coordination is the reason behind
excessive resched IPIs. Look for ondemand being the current_governor and
affected_cpus containing more than one CPU in it.
Thanks,
Venki
next prev parent reply other threads:[~2009-04-29 18:10 UTC|newest]
Thread overview: 118+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-04-28 17:08 Mainline kernel OLTP performance update Styner, Douglas W
2009-04-29 7:29 ` Andrew Morton
2009-04-29 8:28 ` Andi Kleen
2009-04-29 16:00 ` Styner, Douglas W
2009-04-29 16:06 ` Wilcox, Matthew R
2009-04-29 16:19 ` Andi Kleen
2009-04-29 15:48 ` Styner, Douglas W
2009-04-29 16:07 ` Andrew Morton
2009-04-29 16:25 ` Peter Zijlstra
2009-04-29 17:46 ` Chris Mason
2009-04-29 18:06 ` Pallipadi, Venkatesh [this message]
2009-04-29 18:25 ` Styner, Douglas W
2009-04-29 17:52 ` Styner, Douglas W
-- strict thread matches above, loose matches on Subject: below --
2010-01-25 18:26 Ma, Chinang
2009-05-04 15:54 Styner, Douglas W
2009-05-06 6:29 ` Anirban Chakraborty
2009-05-06 15:53 ` Wilcox, Matthew R
2009-05-06 18:05 ` Styner, Douglas W
2009-05-06 18:12 ` Wilcox, Matthew R
2009-05-06 18:24 ` Anirban Chakraborty
2009-05-06 19:25 ` Wilcox, Matthew R
2009-05-06 18:19 ` Styner, Douglas W
2009-04-28 17:22 Styner, Douglas W
2009-04-23 16:49 Styner, Douglas W
2009-04-27 7:02 ` Andi Kleen
2009-04-28 16:57 ` Chuck Ebbert
2009-04-28 17:15 ` James Bottomley
2009-04-28 17:17 ` Styner, Douglas W
2009-01-13 21:10 Ma, Chinang
2009-01-13 22:44 ` Wilcox, Matthew R
2009-01-15 0:35 ` Andrew Morton
2009-01-15 1:21 ` Matthew Wilcox
2009-01-15 2:04 ` Andrew Morton
2009-01-15 2:27 ` Steven Rostedt
2009-01-15 7:11 ` Ma, Chinang
2009-01-19 18:04 ` Chris Mason
2009-01-19 18:37 ` Steven Rostedt
2009-01-19 18:55 ` Chris Mason
2009-01-19 19:07 ` Steven Rostedt
2009-01-19 23:40 ` Ingo Molnar
2009-01-15 2:39 ` Andi Kleen
2009-01-15 2:47 ` Matthew Wilcox
2009-01-15 3:36 ` Andi Kleen
2009-01-20 13:27 ` Jens Axboe
[not found] ` <588992150B702C48B3312184F1B810AD03A497632C@azsmsx501.amr.corp.intel.com>
2009-01-22 11:29 ` Jens Axboe
[not found] ` <588992150B702C48B3312184F1B810AD03A4F59632@azsmsx501.amr.corp.intel.com>
2009-01-27 8:28 ` Jens Axboe
2009-01-15 7:24 ` Nick Piggin
2009-01-15 9:46 ` Pekka Enberg
2009-01-15 13:52 ` Matthew Wilcox
2009-01-15 14:42 ` Pekka Enberg
2009-01-16 10:16 ` Pekka Enberg
2009-01-16 10:21 ` Nick Piggin
2009-01-16 10:31 ` Pekka Enberg
2009-01-16 10:42 ` Nick Piggin
2009-01-16 10:55 ` Pekka Enberg
2009-01-19 7:13 ` Nick Piggin
2009-01-19 8:05 ` Pekka Enberg
2009-01-19 8:33 ` Nick Piggin
2009-01-19 8:42 ` Nick Piggin
2009-01-19 8:47 ` Pekka Enberg
2009-01-19 8:57 ` Nick Piggin
2009-01-19 9:48 ` Pekka Enberg
2009-01-19 10:03 ` Nick Piggin
2009-01-16 20:59 ` Christoph Lameter
2009-01-16 0:27 ` Andrew Morton
2009-01-16 4:03 ` Nick Piggin
2009-01-16 4:12 ` Andrew Morton
2009-01-16 6:46 ` Nick Piggin
2009-01-16 6:55 ` Matthew Wilcox
2009-01-16 7:06 ` Nick Piggin
2009-01-16 7:53 ` Zhang, Yanmin
2009-01-16 10:20 ` Andi Kleen
2009-01-20 5:16 ` Zhang, Yanmin
2009-01-21 23:58 ` Christoph Lameter
2009-01-22 8:36 ` Zhang, Yanmin
2009-01-22 9:15 ` Pekka Enberg
2009-01-22 9:28 ` Zhang, Yanmin
2009-01-22 9:47 ` Pekka Enberg
2009-01-23 3:02 ` Zhang, Yanmin
2009-01-23 6:52 ` Pekka Enberg
2009-01-23 8:06 ` Pekka Enberg
2009-01-23 8:30 ` Zhang, Yanmin
2009-01-23 8:40 ` Pekka Enberg
2009-01-23 9:46 ` Pekka Enberg
2009-01-23 15:22 ` Christoph Lameter
2009-01-23 15:31 ` Pekka Enberg
2009-01-23 15:55 ` Christoph Lameter
2009-01-23 16:01 ` Pekka Enberg
2009-01-24 2:55 ` Zhang, Yanmin
2009-01-24 7:36 ` Pekka Enberg
2009-02-12 5:22 ` Zhang, Yanmin
2009-02-12 5:47 ` Zhang, Yanmin
2009-02-12 15:25 ` Christoph Lameter
2009-02-12 16:07 ` Pekka Enberg
2009-02-12 16:03 ` Pekka Enberg
2009-01-26 17:36 ` Christoph Lameter
2009-02-01 2:52 ` Zhang, Yanmin
2009-01-23 8:33 ` Nick Piggin
2009-01-23 9:02 ` Zhang, Yanmin
2009-01-16 7:00 ` Andrew Morton
2009-01-16 7:25 ` Nick Piggin
2009-01-16 8:59 ` Nick Piggin
2009-01-16 18:11 ` Rick Jones
2009-01-19 7:43 ` Nick Piggin
2009-01-19 22:19 ` Rick Jones
2009-01-15 14:12 ` James Bottomley
2009-01-15 17:44 ` Andrew Morton
2009-01-15 18:00 ` Matthew Wilcox
2009-01-15 18:14 ` Steven Rostedt
2009-01-15 18:44 ` Gregory Haskins
2009-01-15 18:46 ` Wilcox, Matthew R
2009-01-15 19:44 ` Ma, Chinang
2009-01-16 18:14 ` Gregory Haskins
2009-01-16 19:09 ` Steven Rostedt
2009-01-20 12:45 ` Gregory Haskins
2009-01-15 19:28 ` Ma, Chinang
2009-01-15 16:48 ` Ma, Chinang
2009-01-12 18:30 Ma, Chinang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1241028413.4529.9247.camel@localhost.localdomain \
--to=venkatesh.pallipadi@intel.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=andi.kleen@intel.com \
--cc=anil.k.garg@intel.com \
--cc=arjan@linux.intel.com \
--cc=chinang.ma@intel.com \
--cc=chris.mason@oracle.com \
--cc=doug.nelson@intel.com \
--cc=douglas.w.styner@intel.com \
--cc=harita.chilukuri@intel.com \
--cc=hubert.nueckel@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luis.f.recalde@intel.com \
--cc=matthew.r.wilcox@intel.com \
--cc=mingo@elte.hu \
--cc=peter.xihong.wang@intel.com \
--cc=rajalakshmi.shunmuganathan@intel.com \
--cc=sharad.c.tripathi@intel.com \
--cc=suresh.b.siddha@intel.com \
--cc=terry.o.prickett@intel.com \
--cc=wu-sun.cheng@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).