Re: threading question (results after thread pooling)

Re: threading question (results after thread pooling)

[Dieter Nützel]

> Hello,
>
> I have implemented thread pooling (with an environment variable
> where I can give the number of threads to be created). Results:
>
> 1. Linux, no change in the times (not under 2.2.x or 2.4)
[snip]
> I am now pretty much inclined to believe that it is either a) hardware
> issue (someone mentioned that SPARCs and MIPSes handle things differently)
> or b) Linux for some reason just cant give me what IRIX/Solaris can in
> this particular case
[snip]

Hello Ognen,

can you get your hands on an dual AMD Athlon MP 1/1.2 GHz system?
The only mobo currently on the marked is the AMD 760MP based Tyan Thunder K7.
It has (all) the good stuff (Point-to-Point bus, crossbar) which former only 
the (big) Alphas/SUN/SGI etc. had.

http://www.amd.com/products/cpg/server/athlon/index.html
http://www.tyan.com/products/html/thunderk7.html

Regards,
	Dieter
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
Cognitive Systems Group
Vogt-Kölln-Straße 30
D-22527 Hamburg, Germany

email: nuetzel@kogs.informatik.uni-hamburg.de
@home: Dieter.Nuetzel@hamburg.de

Re: threading question (results after thread pooling)

[Mike Castle]

On Thu, Jun 14, 2001 at 04:42:29PM -0600, ognen@gene.pbi.nrc.ca wrote:
> 2. The main thread sets up the data (which are global) and then signals
> that there is work to be done on the same condition variable. The first
> thread to get awaken takes the work. the remaining threads keep waiting.

For curiosities sake, at what point would this technique result in a
thundering herd issue?  Does it happen near the level at which the number of
schedulable entities equal the number of processors or does it have to be
much greater than that?

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: threading question (results after thread pooling)

[ognen]

Hello,

I have implemented thread pooling (with an environment variable
where I can give the number of threads to be created). Results:

1. Linux, no change in the times (not under 2.2.x or 2.4)

2. SGI/Solaris/OSF/1: times decrease when the number of threads matched
the number of processors available. The times were the same as my
previous version or couple of percents better when I exhaggerated the
number of threads to create, say, 128 threads on a 2 CPU.

3. The load on the machines has decreased considerably with the new
solution. I consider this to be the only positive impact I have seen from
this solution.

The solution is basically designed in the following way:

1. Threads are created and they wait on a condition with pthread_cond_wait
2. The main thread sets up the data (which are global) and then signals
that there is work to be done on the same condition variable. The first
thread to get awaken takes the work. the remaining threads keep waiting.
3. Go to 2. until there is work to distribute

I am now pretty much inclined to believe that it is either a) hardware
issue (someone mentioned that SPARCs and MIPSes handle things differently)
or b) Linux for some reason just cant give me what IRIX/Solaris can in
this particular case

Regretfully, the organization I work for prohibits me from releasing the
code I am talking about until the lawyers decide what to do with it. My
hope is to be able to release it for free to anyone interested since this
sequence alignment tool is used a lot :). This kind of defeats the purpose
of my question(s) since without the code it is difficult to talk.

Best regards,
Ognen Duzlevski

On Thu, 14 Jun 2001, Alan Cox wrote:

> > they are done. This should help it (and avoid the pthread_create,
> > pthread_exit). I will implement this and report my results if there is
> > interest.
>
> You should also check up the cache colouring. X86 boxes have relatively poor
> memory performance and most x86 chips have lousy behaviour when data bounces
> between processors or is driven out of cache

-- 
Ognen Duzlevski
Plant Biotechnology Institute
National Research Council of Canada
Bioinformatics team