From: Daniel Phillips <phillips@arcor.de>
To: Steven Cole <elenstev@mesatop.com>, Antonio Vargas <wind@cocodriloo.com>
Cc: Larry McVoy <lm@bitmover.com>, CaT <cat@zip.com.au>,
Anton Blanchard <anton@samba.org>,
linux-kernel@vger.kernel.org
Subject: Re: Scaling noise
Date: Thu, 4 Sep 2003 03:50:31 +0200 [thread overview]
Message-ID: <200309040350.31949.phillips@arcor.de> (raw)
In-Reply-To: <1062603063.1723.91.camel@spc9.esa.lanl.gov>
On Wednesday 03 September 2003 17:31, Steven Cole wrote:
> On Wed, 2003-09-03 at 06:47, Antonio Vargas wrote:
> > As you may probably know, CC-clusters were heavily advocated by the
> > same Larry McVoy who has started this thread.
>
> Yes, thanks. I'm well aware of that. I would like to get a discussion
> going again on CC-clusters, since that seems to be a way out of the
> scaling spiral. Here is an interesting link:
> http://www.opersys.com/adeos/practical-smp-clusters/
As you know, the argument is that locking overhead grows by some factor worse
than linear as the size of an SMP cluster increases, so that the locking
overhead explodes at some point, and thus it would be more efficient to
eliminate the SMP overhead entirely and run a cluster of UP kernels,
communicating through the high bandwidth channel provided by shared memory.
There are other arguments, such as how complex locking is, and how it will
never work correctly, but those are noise: it's pretty much done now, the
complexity is still manageable, and Linux has never been more stable.
There was a time when SMP locking overhead actually cost something in the high
single digits on Linux, on certain loads. Today, you'd have to work at it to
find a real load where the 2.5/6 kernel spends more than 1% of its time in
locking overhead, even on a large SMP machine (sample size of one: I asked
Bill Irwin how his 32 node Numa cluster is running these days). This blows
the ccCluster idea out of the water, sorry. The only way ccCluster gets to
live is if SMP locking is pathetic and it's not.
As for Karim's work, it's a quintessentially flashy trick to make two UP
kernels run on a dual processor. It's worth doing, but not because it blazes
the way forward for ccClusters. It can be the basis for hot kernel swap:
migrate all the processes to one of the two CPUs, load and start a new kernel
on the other one, migrate all processes to it, and let the new kernel restart
the first processor, which is now idle.
Regards,
Daniel
next prev parent reply other threads:[~2003-09-04 1:47 UTC|newest]
Thread overview: 154+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-09-03 4:03 Scaling noise Larry McVoy
2003-09-03 4:12 ` Roland Dreier
2003-09-03 4:20 ` Larry McVoy
2003-09-03 15:12 ` Martin J. Bligh
2003-09-03 4:18 ` Anton Blanchard
2003-09-03 4:29 ` Larry McVoy
2003-09-03 4:33 ` CaT
2003-09-03 5:08 ` Larry McVoy
2003-09-03 5:44 ` Mikael Abrahamsson
2003-09-03 6:12 ` Bernd Eckenfels
2003-09-03 12:09 ` Alan Cox
2003-09-03 15:10 ` Martin J. Bligh
2003-09-03 16:01 ` Jörn Engel
2003-09-03 16:21 ` Martin J. Bligh
2003-09-03 19:41 ` Mike Fedyk
2003-09-03 20:11 ` Martin J. Bligh
2003-09-04 20:36 ` Rik van Riel
2003-09-04 20:47 ` Martin J. Bligh
2003-09-04 21:30 ` William Lee Irwin III
2003-09-03 8:11 ` Giuliano Pochini
2003-09-03 14:25 ` Steven Cole
2003-09-03 12:47 ` Antonio Vargas
2003-09-03 15:31 ` Steven Cole
2003-09-04 1:50 ` Daniel Phillips [this message]
2003-09-04 1:52 ` Larry McVoy
2003-09-04 4:42 ` David S. Miller
2003-09-08 19:40 ` bill davidsen
2003-09-04 2:18 ` William Lee Irwin III
2003-09-04 2:19 ` Steven Cole
2003-09-04 2:35 ` William Lee Irwin III
2003-09-04 2:40 ` Steven Cole
2003-09-04 3:20 ` Nick Piggin
2003-09-04 3:07 ` Daniel Phillips
2003-09-08 19:27 ` bill davidsen
2003-09-08 19:12 ` bill davidsen
2003-09-03 16:37 ` Kurt Wall
2003-09-06 15:08 ` Pavel Machek
2003-09-08 13:38 ` Alan Cox
2003-09-09 6:11 ` Rob Landley
2003-09-09 16:07 ` Ricardo Bugalho
2003-09-10 5:14 ` Rob Landley
2003-09-10 5:45 ` David Mosberger
2003-09-10 10:10 ` Ricardo Bugalho
2003-09-03 6:28 ` Anton Blanchard
2003-09-03 6:55 ` Nick Piggin
2003-09-03 15:23 ` Martin J. Bligh
2003-09-03 15:39 ` Larry McVoy
2003-09-03 15:50 ` Martin J. Bligh
2003-09-04 0:49 ` Larry McVoy
2003-09-04 2:21 ` Daniel Phillips
2003-09-04 2:35 ` Martin J. Bligh
2003-09-04 2:46 ` Larry McVoy
2003-09-04 4:58 ` David S. Miller
2003-09-10 15:47 ` Lock EVERYTHING (for testing) [was: Re: Scaling noise] Timothy Miller
2003-09-04 4:49 ` Scaling noise David S. Miller
2003-09-08 19:50 ` bill davidsen
2003-09-08 23:39 ` Peter Chubb
2003-09-03 17:16 ` William Lee Irwin III
2003-09-03 15:51 ` UP Regression (was) " Cliff White
2003-09-03 17:21 ` William Lee Irwin III
2003-09-03 18:53 ` Cliff White
2003-09-04 0:54 ` Nick Piggin
2003-09-03 5:02 Samium Gromoff
2003-09-03 7:10 John Bradford
2003-09-03 7:38 ` Mike Fedyk
2003-09-03 11:14 ` Larry McVoy
2003-09-08 20:05 ` bill davidsen
2003-09-03 9:41 Brown, Len
2003-09-03 11:02 ` Geert Uytterhoeven
2003-09-03 11:19 ` Larry McVoy
2003-09-03 11:47 ` Matthias Andree
2003-09-03 18:00 ` William Lee Irwin III
2003-09-03 18:05 ` Larry McVoy
2003-09-03 18:15 ` William Lee Irwin III
2003-09-03 18:15 ` Larry McVoy
2003-09-03 18:26 ` William Lee Irwin III
2003-09-03 18:32 ` Alan Cox
2003-09-03 19:46 ` William Lee Irwin III
2003-09-03 20:13 ` Alan Cox
2003-09-03 20:31 ` William Lee Irwin III
2003-09-03 20:48 ` Martin J. Bligh
2003-09-03 21:21 ` William Lee Irwin III
2003-09-03 21:29 ` Martin J. Bligh
2003-09-03 21:51 ` William Lee Irwin III
2003-09-03 21:46 ` Martin J. Bligh
2003-09-04 0:07 ` Mike Fedyk
2003-09-04 1:06 ` Larry McVoy
2003-09-04 1:10 ` Larry McVoy
2003-09-04 1:32 ` William Lee Irwin III
2003-09-04 1:46 ` David Lang
2003-09-04 1:51 ` William Lee Irwin III
2003-09-04 2:31 ` Martin J. Bligh
2003-09-04 2:40 ` Mike Fedyk
2003-09-04 2:50 ` Martin J. Bligh
2003-09-04 3:49 ` Mike Fedyk
2003-09-04 2:48 ` Steven Cole
2003-09-04 17:05 ` Daniel Phillips
2003-09-07 21:18 ` Eric W. Biederman
2003-09-07 23:07 ` Larry McVoy
2003-09-07 23:47 ` Eric W. Biederman
2003-09-08 0:57 ` Larry McVoy
2003-09-08 3:55 ` Eric W. Biederman
2003-09-08 4:47 ` Stephen Satchell
2003-09-08 5:25 ` Larry McVoy
2003-09-08 8:32 ` Eric W. Biederman
2003-09-04 0:58 ` Larry McVoy
2003-09-04 1:12 ` William Lee Irwin III
2003-09-04 2:49 ` Larry McVoy
2003-09-04 3:15 ` William Lee Irwin III
2003-09-04 3:38 ` Nick Piggin
2003-09-05 1:34 ` Robert White
2003-09-03 19:11 ` Steven Cole
2003-09-03 19:36 ` William Lee Irwin III
[not found] <rx83.88x.5@gated-at.bofh.it>
[not found] ` <rxrp.8wt.1@gated-at.bofh.it>
[not found] ` <rxB3.gg.1@gated-at.bofh.it>
[not found] ` <rxB6.gg.5@gated-at.bofh.it>
[not found] ` <rydL.17V.1@gated-at.bofh.it>
[not found] ` <rGXO.5g9.7@gated-at.bofh.it>
2003-09-03 15:33 ` Ihar 'Philips' Filipau
2003-09-03 17:07 Brown, Len
2003-09-03 17:32 ` Larry McVoy
2003-09-03 18:07 ` William Lee Irwin III
2003-09-03 18:07 ` Larry McVoy
2003-09-03 18:25 ` William Lee Irwin III
2003-09-03 23:47 ` Larry McVoy
2003-09-03 23:52 ` William Lee Irwin III
2003-09-03 23:55 ` Martin J. Bligh
2003-09-03 18:28 ` Valdis.Kletnieks
2003-09-03 18:31 ` Alan Cox
2003-09-03 20:11 ` Diego Calleja García
2003-09-03 18:11 ` Alan Cox
2003-09-03 19:56 ` Daniel Gryniewicz
2003-09-03 18:17 ` Martin J. Bligh
2003-09-04 0:36 ` Larry McVoy
2003-09-04 2:21 ` Martin J. Bligh
2003-09-04 2:34 ` Larry McVoy
2003-09-04 2:48 ` Martin J. Bligh
2003-09-04 3:02 ` Larry McVoy
2003-09-04 3:46 ` Gerrit Huizenga
2003-09-04 4:41 ` Martin J. Bligh
2003-09-10 15:02 ` Timothy Miller
2003-09-10 15:12 ` Larry McVoy
2003-09-28 1:51 ` Paul Jakma
2003-09-28 3:13 ` Steven Cole
2003-09-29 0:47 ` Paul Jakma
2003-10-22 1:22 ` Paul Jakma
2003-10-22 3:46 ` Steven Cole
2003-09-04 3:16 ` David Lang
2003-09-04 3:45 ` William Lee Irwin III
2003-09-04 4:51 ` Martin J. Bligh
2003-09-04 3:47 ` Davide Libenzi
2003-09-04 4:16 ` Larry McVoy
2003-09-04 7:43 ` Davide Libenzi
2003-09-08 6:21 Brown, Len
2003-09-08 9:21 ` Eric W. Biederman
2003-09-10 10:01 John Bradford
2003-09-10 11:35 ` Alan Cox
2003-09-10 13:46 ` Bill Davidsen
2003-09-10 15:14 John Bradford
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200309040350.31949.phillips@arcor.de \
--to=phillips@arcor.de \
--cc=anton@samba.org \
--cc=cat@zip.com.au \
--cc=elenstev@mesatop.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lm@bitmover.com \
--cc=wind@cocodriloo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).