From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754553AbZEZH07 (ORCPT ); Tue, 26 May 2009 03:26:59 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752579AbZEZH0t (ORCPT ); Tue, 26 May 2009 03:26:49 -0400 Received: from sh.osrg.net ([192.16.179.4]:56358 "EHLO sh.osrg.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751481AbZEZH0s (ORCPT ); Tue, 26 May 2009 03:26:48 -0400 Date: Tue, 26 May 2009 16:25:27 +0900 To: jens.axboe@oracle.com, James.Bottomley@HansenPartnership.com Cc: fujita.tomonori@lab.ntt.co.jp, rdreier@cisco.com, bharrosh@panasas.com, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, chris.mason@oracle.com, david@fromorbit.com, hch@infradead.org, akpm@linux-foundation.org, jack@suse.cz, yanmin_zhang@linux.intel.com, linux-scsi@vger.kernel.org Subject: Re: [PATCH 03/13] scsi: unify allocation of scsi command and sense buffer From: FUJITA Tomonori In-Reply-To: <20090526062952.GB11363@kernel.dk> References: <20090526132914W.fujita.tomonori@lab.ntt.co.jp> <20090526062952.GB11363@kernel.dk> Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20090526162545U.fujita.tomonori@lab.ntt.co.jp> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (sh.osrg.net [192.16.179.4]); Tue, 26 May 2009 16:25:29 +0900 (JST) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 26 May 2009 08:29:53 +0200 Jens Axboe wrote: > On Tue, May 26 2009, FUJITA Tomonori wrote: > > On Mon, 25 May 2009 18:45:25 -0700 > > Roland Dreier wrote: > > > > > > Ideally there should be a MACRO that is defined to WORD_SIZE on cache-coherent > > > > ARCHs and to SMP_CACHE_BYTES on none-cache-coherent systems and use that size > > > > at the __align() attribute. (So only stupid ARCHES get hurt) > > > > > > this seems to come up repeatedly -- I had a proposal a _long_ time ago > > > that never quite got merged, cf http://lwn.net/Articles/2265/ and > > > http://lwn.net/Articles/2269/ -- from 2002 (!?). The idea is to go a > > > > Yeah, I think that Benjamin did last time: > > > > http://www.mail-archive.com/linux-scsi@vger.kernel.org/msg12632.html > > > > IIRC, James didn't like it so I wrote the current code. I didn't see > > any big performance difference with scsi_debug: > > > > http://marc.info/?l=linux-scsi&m=120038907123706&w=2 > > > > Jens, you see the performance difference due to this unification? > > Yes, it's definitely a worth while optimization. The problem isn't as > such this specific allocation, it's the total number of allocations we > do for a piece of IO. This sense buffer one is just one of many, I'm > continually working to reduce them. If we get rid of this one and add > the ->alloc_cmd() stuff, we can kill one more. The bio path already lost > one. So in the IO stack, we went from 6 allocations to 3 for a piece of > IO. And then it starts to add up. Even at just 30-50k iops, that's more > than 1% of time in the testing I did. I see, thanks. Hmm, possibly slab becomes slower. ;) Then I think that we need something like the ->alloc_cmd() method. Let's ask James. I don't think that it's just about simply adding the hook; there are some issues that we need to think about. Though Boaz worries too much a bit, I think. I'm not sure about this patch if we add ->alloc_cmd(). I doubt that there are any llds don't use ->alloc_cmd() worry about the overhead of the separated sense buffer allocation. If a lld doesn't define the own alloc_cmd, then I think it's fine to use the generic command allocator that does the separate sense buffer allocation.