Re: [patch] Cache align futex hash buckets

[Andrew Morton <akpm@osdl.org>]

Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
>
> On Mon, Feb 20, 2006 at 04:23:17PM -0800, Andrew Morton wrote:
> > Ravikiran G Thirumalai <kiran@scalex86.org> wrote:
> > >
> > > On Mon, Feb 20, 2006 at 03:34:19PM -0800, Andrew Morton wrote:
> > > > Andrew Morton <akpm@osdl.org> wrote:
> > > > >
> > > > > > @@ -100,9 +100,10 @@ struct futex_q {
> > > > > >  struct futex_hash_bucket {
> > > > > >         spinlock_t              lock;
> > > > > >         struct list_head       chain;
> > > > > > -};
> > > > > > +} ____cacheline_internodealigned_in_smp;
> > > > > >  
> > > > > > -static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS];
> > > > > > +static struct futex_hash_bucket futex_queues[1<<FUTEX_HASHBITS] 
> > > > > > +				__cacheline_aligned_in_smp;
> > > > > >  
> > > > > 
> > > > > How much memory does that thing end up consuming?
> > > > 
> > > > I think a megabyte?
> > > 
> > > On most machines it would be 256 * 128 = 32k. or 16k on arches with 64B 
> > > cachelines.  This looked like a simpler solution for spinlocks falling on
> > > the same cacheline.  So is 16/32k unreasonable?
> > > 
> > 
> > CONFIG_X86_VSMP enables 4096-byte padding for
> > ____cacheline_internodealigned_in_smp.    It's a megabyte.
> 
> Yes, only on vSMPowered systems.  Well, we have a large 
> internode cacheline, but these machines have lots of memory too.  I 
> thought a  simpler padding solution might be acceptable as futex_queues 
> would be large only on our boxes.

Well it's your architecture...  As long as you're finding this to be a
sufficiently large problem in testing to justify consuming a meg of memory
then fine, let's do it.

But your initial changelog was rather benchmark-free?  It's always nice to
see numbers accompanying a purported optimisation patch.