Re: Linux scalability?

Re: Linux scalability?

[Dan Kegel]

Sean Hunter wrote:
> On Sat, May 19, 2001 at 10:31:01AM +0200, Sasi Peter wrote: 
> > On Fri, 18 May 2001, Sean Hunter wrote: 
> > 
> > > Why would you want to run a web server with 8 processors rather than four 
> > > webservers with 2 each? 
> > 
> > As you might already know, after the interviews to Mingo I assumed, that a 
> > major portion of the achievements was enabled by the 2.4 scalability 
> > enhacements. That is why I wrote to LKML, to ask about the 2.4 
> > scalability, if anybody out there could tell us about the linux kernel's 
> > scalability possibily compared to W2k scalability... 
>
> Yup. The problem is that you're trying to measure scalability in performance 
> of an i/o-bound task by comparing a machine with greater i/o resource but less 
> processing power with one with greater processing but poorer i/o. Surprisingly 
> enough, the one with the best i/o wins. This isn't really a fair comparison 
> between the two platforms. 

The document tree (21 - 26 GB) is small enough to fit in RAM (32 GB),
so the speed of the disk is not likely to have a noticable impact.
(See http://boudicca.tux.org/hypermail/linux-kernel/2001week20/1276.html )

A lot of people during the Mindcraft discussion made the mistake
of calling the test unfair.
Regardless of whether the initial test was fair, it actually showed 
interesting performance weaknesses in Linux, ones the kernel team
has successfully addressed.

> My point was that in the real world having this configuration for a webserver 
> is unlikely to be sensible at all. 

That's certainly true.  On the other hand, worrying about how many
nanoseconds a system call takes isn't really an issue in the
real world, but kernel hackers love to optimize system call overhead
anyway.  This is the same sort of intellectual challenge.  Plus,
it impresses the beancounters, and they're the ones who buy the
systems and keep us all employed.

- Dan

Re: Linux scalability?

[Sean Hunter]

Yup.  The problem is that you're trying to measure scalability in performance
of an i/o-bound task by comparing a machine with greater i/o resource but less
processing power with one with greater processing but poorer i/o.  Surprisingly
enough, the one with the best i/o wins.  This isn't really a fair comparison
between the two platforms.

If you put the same disk array on both machines and got the same results, then
you'd have a point.

My point was that in the real world having this configuration for a webserver
is unlikely to be sensible at all.

Sean

On Sat, May 19, 2001 at 10:31:01AM +0200, Sasi Peter wrote:
> On Fri, 18 May 2001, Sean Hunter wrote:
> 
> > Why would you want to run a web server with 8 processors rather than four
> > webservers with 2 each?
> 
> As you might already know, after the interviews to Mingo I assumed, that a
> major portion of the achievements was enabled by the 2.4 scalability
> enhacements. That is why I wrote to LKML, to ask about the 2.4
> scalability, if anybody out there could tell us about the linux kernel's
> scalability possibily compared to W2k scalability...
> 
> -- 
> SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Linux scalability?

[Sasi Peter]

On Fri, 18 May 2001, Sean Hunter wrote:

> Why would you want to run a web server with 8 processors rather than four
> webservers with 2 each?

As you might already know, after the interviews to Mingo I assumed, that a
major portion of the achievements was enabled by the 2.4 scalability
enhacements. That is why I wrote to LKML, to ask about the 2.4
scalability, if anybody out there could tell us about the linux kernel's
scalability possibily compared to W2k scalability...

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/

Re: Linux scalability?

[Sasi Peter]

On 18 May 2001, reiser.angus wrote:

> not really the same box
> look at the disk subsystem
> 7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
> 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
> this is sufficient for a such difference

Don't you think that all the really needed stuff could just fit in the
enormous ram of the boxes?

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/

Re: Linux scalability?

[Rodger Donaldson]

On Fri, May 18, 2001 at 09:17:11AM +0100, Sean Hunter wrote:

[Discussion of SPECWeb results]

> Why would you want to run a web server with 8 processors rather than four
> webservers with 2 each?

Because you want to win benchmarketing exercises, not demonstrate that your
architecture has any value in the real world whatsoever.  Because you know
that you can induce people with financial approval to make stupid and
irrational decisions based on irrelevant data.

-- 
Rodger Donaldson		rodgerd@diaspora.gen.nz
Klingons do *not* make good programmers.  They make good PFWs and abuse
staff, though.  "You have dishonoured our Ascend!  I should kill you
where you stand!"	-- Malcolm Ray

Re: Linux scalability?

[Dan Kegel]

Sasi Peter <sape@iq.rulez.org> wrote:
> I am just writing an essay, an have mentioned TUX as a performance and 
> scalability linearity recort holder with TUX, referencing the specweb99 
> website summary page: 
> 
> http://www.spec.org/osg/web99/results/web99.html 
> 
> However, taking a closer look, it turns out, that the above statement 
> holds true only for 1 and 2 processor machines. Scalability already 
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten 
> by IIS 5.0 (8001), and these were measured on the same kind of box! 
>
> How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this 
> bad? 

Let's look at the scores.  (BTW, SPECweb99 gets harder
as the scores get better; the document tree required to achieve a score of
3222 is twice as large as that required to achieve a score of 1438.)

  SPECweb99 result summary:
date    #cpu  #nics L2 cache/cpu  RAM  tree score  sw   model                MHz
1/2001  1     1     256K          2G    5G  1438   tux2 Compq Proliant DL320 800
6/2000  1     1     256K          2G    4G  1270   tux1 Dell Poweredge 6400  667
6/2000  2     2     256K          4G    7G  2200   tux1 Dell Poweredge 4400  800
3/2001  2     4     256K          4G    10G 3222   tux2 Dell Poweredge 2500  1000

2/2001  1     3     2M            8G    9G  2700   tux2 IBM xSeries 370      900
2/2001  2     4     2M            16G   13G 3999   tux2 IBM xSeries 370      900
6/2000  4     4     2M            8G    14G 4200   tux1 Dell Poweredge 6400  700
7/2000  8     8     2M            32G   21G 6387   tux1 Dell Poweredge 8450  700
11/2000 8     8     2M            32G   24G 7500   tux2 Dell Poweredge 8450  700
12/2000 8     8     2M            32G   21G 6407   tux1 IBM Netfinity 8500R  700

3/2001  2     3     256K          4G    8G  2499   IIS5/SWC HP NetserverLP2000r  1000
4/2001  8     8     2M            32G   26G 8000   IIS5/SWC Dell Poweredge 8450  700

IIS5/SWC only has two results on record, at 2 and 8 CPUs.  They're hard
to compare, because they have different cache and RAM sizes and CPU speeds,
but it's safe to say that it performs poorly at 2 CPUs (compared to the 3/2001 
results from Dell) and scales nearly linearly to perform comparatively well at 8 CPUs.

Looking at the IBM 1 and 2 CPU results, twice the CPU only got 1.4 times
the performance.  Not sure TUX is scaling especially well even at 2 CPU's.
(And you can't blame this on disk drives, please don't try.)

So I agree, Tux doesn't seem to scale as well to multiple CPUs as does IIS5/SWC.

About comparing the Tux and IIS/SWC results on the Dell 8 CPU box:
the Tux measurement is 5 months older than the IIS/SWC measurement.
It's interesting to speculate how tux2 would do if tested today; 
It looks like tux2 is about 12% faster than tux1 on 8-CPU machines.
In other words, 5 months of further development on tux and the 2.4 kernel yielded 
a 12% speedup.  Since IIS was only 4% faster than TUX, If Tux were measured today, 
it might have improved enough to beat IIS/SWC, who knows.

- Dan

Re: Linux scalability?

[Ronald Bultje]

On 18 May 2001 10:30:40 +0200, reiser.angus wrote:
> TUX does not exist on 2.2 kernel
> They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX,
> zerocopy, etc..)

I am pretty sure the C'T article mentioned that TUX did use a 2.2.x
kernel - so it does exist. How else could they make a 2.2.x-kernel based
TUX vs. 2.4-test-kernel based TUX comparison?

Too bad the website doesn't mention the kernel number.

--
Ronald

Re: Linux scalability?

[Ronald Bultje]

On 18 May 2001 10:12:34 +0200, reiser.angus wrote:
> > However, taking a closer look, it turns out, that the above statement
> > holds true only for 1 and 2 processor machines. Scalability already
> > suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> > by IIS 5.0 (8001), and these were measured on the same kind of box!
> not really the same box
> look at the disk subsystem
> 7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
> 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
> 
> this is sufficient for a such difference

I read an article about TUX in the dutch C'T a few months ago (nov/dec
2000, I think) - the real difference (according to the article) was the
2.2.x kernel used in TUX. Look at the stats of the website, they used
Redhat 7.0 as base, with kernel 2.2.16. In the C'T, they also used a
2.4-test kernel for TUX, and this one didn't have these "scalibility
problems". The problem seemed to be SMP problems with systems with more
than two cpus in the 2.2.x-based kernel series. 2.4.x kernels didn't
seem to have this problem.

And as far as I know, TUX with 2.4.x kernel was faster than win2k on all
SMP-combinations.

--
Ronald

Re: Linux scalability?

[reiser.angus]

> I read an article about TUX in the dutch C'T a few months ago (nov/dec
> 2000, I think) - the real difference (according to the article) was the
> 2.2.x kernel used in TUX. Look at the stats of the website, they used
> Redhat 7.0 as base, with kernel 2.2.16.

TUX does not exist on 2.2 kernel
They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX,
zerocopy, etc..)

-David

Re: Linux scalability?

[Sean Hunter]

Why would you want to run a web server with 8 processors rather than four
webservers with 2 each?

Sean

On Fri, May 18, 2001 at 09:24:48AM +0200, Sasi Peter wrote:
> Hi!
> 
> I am just writing an essay, an have mentioned TUX as a performance and
> scalability linearity recort holder with TUX, referencing the specweb99
> website summary page:
> 
> http://www.spec.org/osg/web99/results/web99.html
> 
> However, taking a closer look, it turns out, that the above statement
> holds true only for 1 and 2 processor machines. Scalability already
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> by IIS 5.0 (8001), and these were measured on the same kind of box!
> 
> How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this
> bad?
> 
> -- 
> SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Linux scalability?

[reiser.angus]

> However, taking a closer look, it turns out, that the above statement
> holds true only for 1 and 2 processor machines. Scalability already
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> by IIS 5.0 (8001), and these were measured on the same kind of box!
not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0

this is sufficient for a such difference

-David

Linux scalability?

[Sasi Peter]

Hi!

I am just writing an essay, an have mentioned TUX as a performance and
scalability linearity recort holder with TUX, referencing the specweb99
website summary page:

http://www.spec.org/osg/web99/results/web99.html

However, taking a closer look, it turns out, that the above statement
holds true only for 1 and 2 processor machines. Scalability already
suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
by IIS 5.0 (8001), and these were measured on the same kind of box!

How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this
bad?

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/