From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1761630AbZAOONQ (ORCPT ); Thu, 15 Jan 2009 09:13:16 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1758966AbZAOOM4 (ORCPT ); Thu, 15 Jan 2009 09:12:56 -0500 Received: from accolon.hansenpartnership.com ([76.243.235.52]:40331 "EHLO accolon.hansenpartnership.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755526AbZAOOMy (ORCPT ); Thu, 15 Jan 2009 09:12:54 -0500 Subject: Re: Mainline kernel OLTP performance update From: James Bottomley To: Andrew Morton Cc: Matthew Wilcox , "Wilcox, Matthew R" , chinang.ma@intel.com, linux-kernel@vger.kernel.org, sharad.c.tripathi@intel.com, arjan@linux.intel.com, andi.kleen@intel.com, suresh.b.siddha@intel.com, harita.chilukuri@intel.com, douglas.w.styner@intel.com, peter.xihong.wang@intel.com, hubert.nueckel@intel.com, chris.mason@oracle.com, srostedt@redhat.com, linux-scsi@vger.kernel.org, Andrew Vasquez , Anirban Chakraborty In-Reply-To: <20090114180431.f4a96543.akpm@linux-foundation.org> References: <20090114163557.11e097f2.akpm@linux-foundation.org> <20090115012147.GW29283@parisc-linux.org> <20090114180431.f4a96543.akpm@linux-foundation.org> Content-Type: text/plain Date: Thu, 15 Jan 2009 09:12:46 -0500 Message-Id: <1232028766.5966.10.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.22.3.1 (2.22.3.1-1.fc9) Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2009-01-14 at 18:04 -0800, Andrew Morton wrote: > On Wed, 14 Jan 2009 18:21:47 -0700 Matthew Wilcox wrote: > > On Wed, Jan 14, 2009 at 04:35:57PM -0800, Andrew Morton wrote: > > > > > Linux OLTP Performance summary > > > > > Kernel# Speedup(x) Intr/s CtxSw/s us% sys% idle% iowait% > > > > > 2.6.24.2 1.000 21969 43425 76 24 0 0 > > > > > 2.6.27.2 0.973 30402 43523 74 25 0 1 > > > > > 2.6.29-rc1 0.965 30331 41970 74 26 0 0 > > > > > But the interrupt rate went through the roof. > > > > Yes. I forget why that was; I'll have to dig through my archives for > > that. > > Oh. I'd have thought that this alone could account for 3.5%. Me too. Anecdotally, I haven't noticed this in my lab machines, but what I have noticed is on someone else's laptop (a hyperthreaded atom) that I was trying to demo powertop on was that IPI reschedule interrupts seem to be out of control ... they were ticking over at a really high rate and preventing the CPU from spending much time in the low C and P states. To me this implicates some scheduler problem since that's the primary producer of IPI reschedules ... I think it wouldn't be a significant extrapolation to predict that the scheduler might be the cause of the above problem as well. James