Reiser vs EXT3

Reiser vs EXT3

[Robert L. Harris]

  Still working on that replacement mail server and a new rumor has hit
the mix.  It follows that reiserfs is much faster than ext3 (made ext3,
not converted from ext2 if it matters) and this is causing some
problems.  On a 200Gig filesystem is this truely an issue?

Thanks,
  Robert


:wq!
---------------------------------------------------------------------------
Robert L. Harris                
                               
DISCLAIMER:
      These are MY OPINIONS ALONE.  I speak for no-one else.
FYI:
 perl -e 'print $i=pack(c5,(41*2),sqrt(7056),(unpack(c,H)-2),oct(115),10);'

Re: Reiser vs EXT3

[David C. Hansen]

On Thu, 2002-10-31 at 06:19, Robert L. Harris wrote:
> 
>   Still working on that replacement mail server and a new rumor has hit
> the mix.  It follows that reiserfs is much faster than ext3 (made ext3,
> not converted from ext2 if it matters) and this is causing some
> problems.  On a 200Gig filesystem is this truely an issue?

ext3 has some SMP scalability problems.  The BKL is used to protect many
journal operations, and we see huge amounts of CPU spent spinning on it
on 4/8/16 proc machines.  So much CPU, that it masks anything else we're
doing on the system.  But, on a single-proc or just a 2-way, you
probably won't see much of this to be significant.  

We haven't tested reiser extensively here, but from what I've seen it
scales much, much better than ext3 (as does jfs and probably xfs too).
-- 
Dave Hansen
haveblue@us.ibm.com

Re: Reiser vs EXT3

[Samuel Flory]

Robert L. Harris wrote:

>  Still working on that replacement mail server and a new rumor has hit
>the mix.  It follows that reiserfs is much faster than ext3 (made ext3,
>not converted from ext2 if it matters) and this is causing some
>problems.  On a 200Gig filesystem is this truely an issue?
>
>  
>

  Have you tried different ext3 journalling modes?  Ordered is pretty 
slow in many cases.  You might want to try writeback instead.  The 
downside is that you might end up losing resently written changes in the 
event of a crash.  Try mounting with "-o data=writeback".

Re: Reiser vs EXT3

[David Lang]

note that breaking up this locking bottleneckhas been done in the 2.5
kernel series so when 2.6 is released this should be much less significant
(Q2 2003 is the current thought, but don't count on it until it's out)

David Lang

On 31 Oct 2002, David C. Hansen wrote:

> Date: 31 Oct 2002 11:02:49 -0800
> From: David C. Hansen <haveblue@us.ibm.com>
> To: Robert L. Harris <Robert.L.Harris@rdlg.net>
> Cc: Linux-Kernel <linux-kernel@vger.kernel.org>
> Subject: Re: Reiser vs EXT3
>
> On Thu, 2002-10-31 at 06:19, Robert L. Harris wrote:
> >
> >   Still working on that replacement mail server and a new rumor has hit
> > the mix.  It follows that reiserfs is much faster than ext3 (made ext3,
> > not converted from ext2 if it matters) and this is causing some
> > problems.  On a 200Gig filesystem is this truely an issue?
>
> ext3 has some SMP scalability problems.  The BKL is used to protect many
> journal operations, and we see huge amounts of CPU spent spinning on it
> on 4/8/16 proc machines.  So much CPU, that it masks anything else we're
> doing on the system.  But, on a single-proc or just a 2-way, you
> probably won't see much of this to be significant.
>
> We haven't tested reiser extensively here, but from what I've seen it
> scales much, much better than ext3 (as does jfs and probably xfs too).
> --
> Dave Hansen
> haveblue@us.ibm.com
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Reiser vs EXT3

[Hans Reiser]

If you want to talk about 2.6 then you should talk about reiser4 not 
reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 for 
30 copies of the linux kernel source code using modern IDE drives and 
modern processors on a dual-CPU box, so I don't think any amount of 
improved scalability will make ext3 competitive with reiser4 for 
performance usages.  

We haven't had anyone test performance using RAID yet for reiser4, that 
could be fun.

Best,

Hans


David Lang wrote:

>note that breaking up this locking bottleneckhas been done in the 2.5
>kernel series so when 2.6 is released this should be much less significant
>(Q2 2003 is the current thought, but don't count on it until it's out)
>
>David Lang
>
>On 31 Oct 2002, David C. Hansen wrote:
>
>  
>
>>Date: 31 Oct 2002 11:02:49 -0800
>>From: David C. Hansen <haveblue@us.ibm.com>
>>To: Robert L. Harris <Robert.L.Harris@rdlg.net>
>>Cc: Linux-Kernel <linux-kernel@vger.kernel.org>
>>Subject: Re: Reiser vs EXT3
>>
>>On Thu, 2002-10-31 at 06:19, Robert L. Harris wrote:
>>    
>>
>>>  Still working on that replacement mail server and a new rumor has hit
>>>the mix.  It follows that reiserfs is much faster than ext3 (made ext3,
>>>not converted from ext2 if it matters) and this is causing some
>>>problems.  On a 200Gig filesystem is this truely an issue?
>>>      
>>>
>>ext3 has some SMP scalability problems.  The BKL is used to protect many
>>journal operations, and we see huge amounts of CPU spent spinning on it
>>on 4/8/16 proc machines.  So much CPU, that it masks anything else we're
>>doing on the system.  But, on a single-proc or just a 2-way, you
>>probably won't see much of this to be significant.
>>
>>We haven't tested reiser extensively here, but from what I've seen it
>>scales much, much better than ext3 (as does jfs and probably xfs too).
>>--
>>Dave Hansen
>>haveblue@us.ibm.com
>>
>>-
>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>Please read the FAQ at  http://www.tux.org/lkml/
>>
>>    
>>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml
>
>
>  
>


-- 
Hans

Re: Reiser vs EXT3

[Jeff Garzik]

Hans Reiser wrote:

> If you want to talk about 2.6 then you should talk about reiser4 not 
> reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 
> for 30 copies of the linux kernel source code using modern IDE drives 
> and modern processors on a dual-CPU box, so I don't think any amount 
> of improved scalability will make ext3 competitive with reiser4 for 
> performance usages. 


What is the read performance like?

write performance isn't the end-all be-all of useful benchmarks, because 
most servers do far more reading in a day than they will ever write. 
 And like Andrew has pointed out on more than one occasion, reads are 
usually synchronous, because applications are typically blocking until 
each read is satisfied.

    Jeff

Re: Reiser vs EXT3

[David C. Hansen]

On Thu, 2002-10-31 at 13:51, Hans Reiser wrote:
> If you want to talk about 2.6 then you should talk about reiser4 not 
> reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 for 
> 30 copies of the linux kernel source code using modern IDE drives and 
> modern processors on a dual-CPU box, so I don't think any amount of 
> improved scalability will make ext3 competitive with reiser4 for 
> performance usages.  
> 
> We haven't had anyone test performance using RAID yet for reiser4, that 
> could be fun.

I have a 14-drive hardware RAID array on an 8-proc box.  Is that the
kind of thing you want testing on?  If you want to send me some testing
scripts, I'll run them.  

-- 
Dave Hansen
haveblue@us.ibm.com

Re: Reiser vs EXT3

[David C. Hansen]

On Thu, 2002-10-31 at 12:49, David Lang wrote:
> note that breaking up this locking bottleneckhas been done in the 2.5
> kernel series so when 2.6 is released this should be much less significant
> (Q2 2003 is the current thought, but don't count on it until it's out)

Actually, ext3 has been immune from most of the lock breakups in 2.5. 
ext2 used to have a lot of problems with BKL contention resulting from 
ext2_get_block() and some other assorted functions.  Al Viro cleaned
these up in early 2.5, but ext3 never got the cleanup.  It still scales
horribly, even 2.5.45.  

-- 
Dave Hansen
haveblue@us.ibm.com

Re: Reiser vs EXT3

[Hans Reiser]

David C. Hansen wrote:

>On Thu, 2002-10-31 at 13:51, Hans Reiser wrote:
>  
>
>>If you want to talk about 2.6 then you should talk about reiser4 not 
>>reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 for 
>>30 copies of the linux kernel source code using modern IDE drives and 
>>modern processors on a dual-CPU box, so I don't think any amount of 
>>improved scalability will make ext3 competitive with reiser4 for 
>>performance usages.  
>>
>>We haven't had anyone test performance using RAID yet for reiser4, that 
>>could be fun.
>>    
>>
>
>I have a 14-drive hardware RAID array on an 8-proc box.  Is that the
>kind of thing you want testing on?  If you want to send me some testing
>scripts, I'll run them.  
>
>  
>
Yes, that would be cool.

Green, please respond to this email with details for him.

-- 
Hans

Re: Reiser vs EXT3

[Matthew Kirkwood]

On Thu, 31 Oct 2002, Jeff Garzik wrote:

> > reiser4 is 7.6 times the write performance of ext3
> > for 30 copies of the linux kernel source code using modern IDE drives

> What is the read performance like?
>
> write performance isn't the end-all be-all of useful benchmarks,
> because most servers do far more reading in a day than they will ever
> write.

I'm not sure how true that is these days.  OLTP DB servers
with a lot of RAM will typically do more write traffic, all
pushed by fsync, than reads.  (Some may claim that that
means your server is overspecced, of course.)

Other servers, too, look rather like that -- mail servers
do a lot of fsync, web servers generally have smallish web
trees but write a lot of logs...

Even on data warehousing apps, there will be a fairly high
level of writes due to use of temporary relations..

Matthew.

Re: Reiser vs EXT3

[David Lang]

One thing that concerns me is the warning to only use a filesystem created
in a certin way for the benchmark, don't use a tar of an ext2/3 filesystem
as that will kill performance.

they say taht there will be a tool to fix this in 4.1, but this makes me
treat the benchmark as a 'best possible' test case and expect the
real-world performance to be considerably worse (how much worse who knows,
I haven't seen anyone try to do worst-case performance tests on it)

David Lang


 On Thu, 31 Oct 2002, Dieter [iso-8859-15] Nützel wrote:

> Date: Thu, 31 Oct 2002 23:52:07 +0100
> From: "Dieter [iso-8859-15] Nützel" <Dieter.Nuetzel@hamburg.de>
> To: Jeff Garzik <jgarzik@pobox.com>
> Cc: Hans Reiser <reiser@namesys.com>,
>      Linux Kernel <linux-kernel@vger.kernel.org>, Reiserfs-List@namesys.com
> Subject: Re: Reiser vs EXT3
>
> Am Donnerstag, 31. Oktober 2002 22:05 schrieb Jeff Garzik:
> > Hans Reiser wrote:
> >
> > > If you want to talk about 2.6 then you should talk about reiser4 not
> > > reiserfs v3, and reiser4 is 7.6 times the write performance of ext3
> > > for 30 copies of the linux kernel source code using modern IDE drives
> > > and modern processors on a dual-CPU box, so I don't think any amount
> > > of improved scalability will make ext3 competitive with reiser4 for
> > > performance usages.
> >
> > What is the read performance like?
>
> From his mentioned paper http://www.namesys.com/v4/fast_reiser4.html, it is
> more then doubled compared to ext3 and ReiserFS v3.
>
> To be fair he should explain if it was compared to the latest ext3 (htree)
> stuff or not, yet.
>
> It looks truly impressive.
>
> Regards,
>         Dieter
>
> --
> Dieter Nützel
> Graduate Student, Computer Science
>
> University of Hamburg
> Department of Computer Science
> @home: Dieter.Nuetzel at hamburg.de (replace at with @)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Reiser vs EXT3

[Kevin Brosius]

> David C. Hansen wrote:
> 
> >On Thu, 2002-10-31 at 13:51, Hans Reiser wrote:
> >  
> >
> >>If you want to talk about 2.6 then you should talk about reiser4 not 
> >>reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 for 
> >>30 copies of the linux kernel source code using modern IDE drives and 
> >>modern processors on a dual-CPU box, so I don't think any amount of 
> >>improved scalability will make ext3 competitive with reiser4 for 
> >>performance usages.  
> >>
> >>We haven't had anyone test performance using RAID yet for reiser4, that 
> >>could be fun.
> >>    
> >>
> >
> >I have a 14-drive hardware RAID array on an 8-proc box.  Is that the
> >kind of thing you want testing on?  If you want to send me some testing
> >scripts, I'll run them.  
> >
> >  
> >
> Yes, that would be cool.
> 
> Green, please respond to this email with details for him.


I have access to a 3 drive hw RAID system with dual processors if you'd
like some more testing.  Do you have info on recommended stripe sizes
vs. performance for Reiser using RAID?

-- 
Kevin

Re: Reiser vs EXT3

[Dieter Nützel]

Am Donnerstag, 31. Oktober 2002 22:05 schrieb Jeff Garzik:
> Hans Reiser wrote:
>
> > If you want to talk about 2.6 then you should talk about reiser4 not 
> > reiserfs v3, and reiser4 is 7.6 times the write performance of ext3 
> > for 30 copies of the linux kernel source code using modern IDE drives 
> > and modern processors on a dual-CPU box, so I don't think any amount 
> > of improved scalability will make ext3 competitive with reiser4 for 
> > performance usages. 
>
> What is the read performance like?

>From his mentioned paper http://www.namesys.com/v4/fast_reiser4.html, it is 
more then doubled compared to ext3 and ReiserFS v3.

To be fair he should explain if it was compared to the latest ext3 (htree) 
stuff or not, yet.

It looks truly impressive.

Regards,
        Dieter

-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)