From: Ravikiran G Thirumalai <kiran@scalex86.org>
To: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Andrew Morton <akpm@osdl.org>,
linux-kernel@vger.kernel.org,
"Shai Fultheim (Shai@scalex86.org)" <shai@scalex86.org>
Subject: Re: [patch] Cache align futex hash buckets
Date: Tue, 21 Feb 2006 12:20:24 -0800 [thread overview]
Message-ID: <20060221202024.GA3635@localhost.localdomain> (raw)
In-Reply-To: <43FA8938.70006@yahoo.com.au>
On Tue, Feb 21, 2006 at 02:30:00PM +1100, Nick Piggin wrote:
> Ravikiran G Thirumalai wrote:
> >Following change places each element of the futex_queues hashtable on a
> >different cacheline. Spinlocks of adjacent hash buckets lie on the same
> >cacheline otherwise.
> >
>
> It does not make sense to add swaths of unused memory into a hashtable for
> this purpose, does it?
I don't know if having two (or more) spinlocks on the same cacheline is a good
idea. Right now, on a 128 B cacheline we have 10 spinlocks on the
same cacheline here!! Things get worse if two futexes from different nodes
hash on to adjacent, or even nearly adjacent hash buckets.
>
> For a minimal, naive solution you just increase the size of the hash table.
> This will (given a decent hash function) provide the same reduction in
> cacheline contention, while also reducing collisions.
Given a decent hash function. I am not sure the hashing function is smart
enough as of now. Hashing is not a function of nodeid, and we have some
instrumentation results which show hashing on NUMA is not good as yet, and
there are collisions from other nodes onto the same hashbucket; Nearby
buckets have high hit rates too.
I think some sort of NUMA friendly hashing, where futexes from same nodes
hash onto a node local hash table, would be a decent solution here.
As I mentioned earlier, we are working on that, and we can probably allocate
the spinlock from nodelocal memory then and avoid this bloat.
We are hoping to have this as a stop gap fix until we get there.
Thanks,
Kiran
next prev parent reply other threads:[~2006-02-21 20:19 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2006-02-20 23:32 [patch] Cache align futex hash buckets Ravikiran G Thirumalai
2006-02-20 23:33 ` Andrew Morton
2006-02-20 23:34 ` Andrew Morton
2006-02-21 0:09 ` Ravikiran G Thirumalai
2006-02-21 0:23 ` Andrew Morton
2006-02-21 1:04 ` Ravikiran G Thirumalai
2006-02-21 1:09 ` Andrew Morton
2006-02-21 1:39 ` Ravikiran G Thirumalai
2006-02-21 14:44 ` Andi Kleen
2006-02-21 3:30 ` Nick Piggin
2006-02-21 18:33 ` Christoph Lameter
2006-02-21 23:14 ` Christoph Lameter
2006-02-22 0:40 ` Nick Piggin
2006-02-22 2:08 ` Andrew Morton
2006-02-22 2:35 ` Ravikiran G Thirumalai
2006-02-22 2:37 ` Nick Piggin
2006-02-22 20:17 ` Ravikiran G Thirumalai
2006-02-22 20:50 ` Andrew Morton
[not found] ` <20060223015144.GC3663@localhost.localdomain>
2006-02-23 2:08 ` Ravikiran G Thirumalai
2006-02-21 20:20 ` Ravikiran G Thirumalai [this message]
2006-02-22 0:45 ` Nick Piggin
2006-02-22 2:09 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20060221202024.GA3635@localhost.localdomain \
--to=kiran@scalex86.org \
--cc=akpm@osdl.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nickpiggin@yahoo.com.au \
--cc=shai@scalex86.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).