Re: [Lse-tech] Re: [ckrm-tech] Re: [PATCH 00/01] Move Exit Connectors

[Matt Helsley <matthltc@us.ibm.com>]

On Fri, 2006-01-13 at 04:35 -0500, Jes Sorensen wrote:
> >>>>> "Matt" == Matt Helsley <matthltc@us.ibm.com> writes:
> 
> Matt> On Thu, 2006-01-12 at 05:01 -0500, Jes Sorensen wrote:
> >> It all depends on the specific location of the locks and how often
> >> they are taken. As long as something is taken by the same CPU all
> >> the time is not going to be a major issue, but if we end up with
> >> anything resembling a global lock, even if it is only held for a
> >> very short time, it is going to bite badly. On a 4-way you probably
> >> won't notice, but go to a 64-way and it bites, on a 512-way it eats
> >> you alive (we had a problem in the timer code quite a while back
> >> that prevented the machine from booting - it wasn't a lock that was
> >> held for a long time, just the fact that every CPU would take it
> >> every HZ was enough).
> 
> Matt> 	OK, so you've established that global locks in timer paths are
> Matt> Bad.  However you haven't established that timer paths are
> Matt> almost the same as the task paths we're talking about. I suspect
> Matt> timer paths are used much more frequently than fork, exec, or
> Matt> exit.
> 
> Hi Matt,
> 
> I wasn't trying to make it sound like this was an apples to apples
> comparison, what I am saying is simply that locks aren't free.

OK. I'm not sure what gave you the impression I thought they were.

> You are totally right that fork/exec should be called a lot less
> frequently, but the delay account data collection points will be in far
> more places than that and they will all go for the lock.

	All of these places are highly likely to also be in the context of the
task that has the lock in it's task->delays struct. This could be a
strongly recommended practice for taking the lock -- we can add a
comment next to the lock suggesting as much.

> Matt> 	I've appended a small patch that adds a global lock to the
> Matt> task_notify paths for testing purposes. I'm curious to know what
> Matt> kind of a performance difference you would see on your 64 or
> Matt> 512-way if you were to run with it.
> 
> I don't have a 512-way to play with at the moment, but again I don't
> think it makes sense to benchmark things which aren't matching what we
> are looking at. If we can avoid the locks in the first place then
> there's really no reason for not doing that.

	It could be good for establishing a lower bound on contention resulting
from locking implementations of task_notify. I think in that respect
it's comparing two species of apples.

	We need to clarify what locking we're talking about -- locking in
task_notify to protect the notification list vs. locking in delayacct to
protect the accounting data consistency. I think that a simple lockless
approach may only be possible for task_notify.

Cheers,
	-Matt Helsley