EXT3 Testing w/ rmap14a with contest results

EXT3 Testing w/ rmap14a with contest results

[Shawn Starr]

2.4.19-rmap14a - on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used 

with hdparm tweaks

noload		4:57.93		99%
cpuload		6:06.85		79%
memload		6:47.27		76%
ioloadhalf		10:36.93		47%
ioloadfull		8:06.46		62%

without hdparm tweaks

noload		4:57.53		99%
cpuload		6:06.68		79%
memload		6:44.22		77%
ioloadhalf		10:02.30		50%
ioloadfull		8:14.96		61%

-------------------------------------------------

on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used 
2.4.20-pre7-rmap14a-xfs-uml-shawn12d - with hdparm tweaks

noload		4:56.86         99%
cpuload		6:06.35         79%
memload		6:27.43         79%
ioloadhalf		9:00.57         55%
ioloadfull		8:11.82         61%

on Athlon MP 2000+ w/ 512MB RAM EXT3 Filesystem used 
2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks

noload		5:02.87         97%
cpuload		6:07.29         79%
memload		6:24.29         80%
ioloadhalf		8:05.11         61%
ioloadfull		7:55.27         63%


For comparsion from my older machine
--------------------------------------------------------------

2.4.20-pre7-rmap14a-xfs-uml-shawn12d - Pentium 200Mhz 64MB Ram  EXT3 
Filesystem

noload		52:46.43		99%
cpuload		1:05:14		79%
memload		1:11:12		79%
ioloadhalf		1:10:14		78%
ioloadfull		1:10:31		78%

--------------------------------------------------------------

on Athlon MP 2000+ w/ 512MB RAM
2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks ___WITH EXT3___

noload Time: 265.82  CPU: 96%  Major Faults: 755922  Minor Faults: 1150416
process_load Time: 314.80  CPU: 80%  Major Faults: 727159  Minor Faults: 
1146210
io_halfmem Time: 444.25  CPU: 58%  Major Faults: 726724  Minor Faults: 1146174
Was writing number 33 of a 257Mb sized io_load file after 449 seconds
Was writing number 9 of a 514Mb sized io_load file after 262 seconds

----------------------------------------------------------------------------------------------------

on Athlon MP 2000+ w/ 512MB RAM
2.4.20-pre7-rmap14a-xfs-uml-shawn12d - without hdparm tweaks  __WITHOUT 
EXT3___

noload Time: 259.41  CPU: 99%  Major Faults: 770937  Minor Faults: 1173712
process_load Time: 318.63  CPU: 80%  Major Faults: 742261  Minor Faults: 
1169518
io_halfmem Time: 305.75  CPU: 87%  Major Faults: 742000  Minor Faults: 1169497
Was writing number 32 of a 257Mb sized io_load file after 306 seconds
io_fullmem Time: 321.77  CPU: 83%  Major Faults: 742000  Minor Faults: 1169497
Was writing number 16 of a 514Mb sized io_load file after 324 seconds
mem_load Time: 345.10  CPU: 78%  Major Faults: 743807  Minor Faults: 1170248

As you can see, there's something DEFINATELY wrong here =)


* NOTES:

The old test at the top with the old format are from contest 0.22, the new 
results seen below are 0.34.

 <conman_work> yes, but results from different versions of contest are 
incompatible

<ShawnCONSOLE> riel uses EXT3
<riel> my cpu is slower
<ShawnCONSOLE> but you have fast disks?
<riel> so it doesn't fall idle as quickly as yours, when waiting on the disk
<riel> not very fast ;)
<riel> old 8 GB IDE disk
<ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait 
longer cause the disk finishes its tasks much faster then the cpu expects?
<ShawnCONSOLE> mem_load final test = 78%
<ShawnCONSOLE> so final numbers:
<ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
<riel> yes, a very fast CPU falls idle more quickly
<riel> but it's very curious that ext3 is _that_ much worse than ext2
<ShawnCONSOLE> thats much better.
<riel> definately worth pointing out to the ext3 maintainers

Re: EXT3 Testing w/ rmap14a with contest results

[Andrew Morton]

Shawn Starr wrote:
> 
> As you can see, there's something DEFINATELY wrong here =)

Well I can't see what it is.

Please, try not to dump tons of raw numbers onto dazed onlookers.

There is great value in you performing the initial processing;
distilling useful information and the "means to reproduce" out
of the raw data.  The old human touch.

Ideally: just two numbers, and the barest, simplest way of
demonstrating the problem.

Re: EXT3 Testing w/ rmap14a with contest results

[Shawn Starr]

I'm reformatting the email with useless stuff dropped sorry =)


On September 18, 2002 11:10 pm, Andrew Morton wrote:
> Shawn Starr wrote:
> > As you can see, there's something DEFINATELY wrong here =)
>
> Well I can't see what it is.
>
> Please, try not to dump tons of raw numbers onto dazed onlookers.
>
> There is great value in you performing the initial processing;
> distilling useful information and the "means to reproduce" out
> of the raw data.  The old human touch.
>
> Ideally: just two numbers, and the barest, simplest way of
> demonstrating the problem.

[BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Shawn Starr]

Sorry about the confusing email before. This should make more sense =)

These results compare EXT3 against EXT2 with rmap using the contest tool
you can get it at: http://contest.kolivas.net

These tests are from a Athlon MP 2000+ w/ 512MB RAM

noload:

Kernel									Time            	CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	259.47		99%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	267.66          	97%

process_load:

Kernel                  							Time            	CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	318.91          	80%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	324.44          	79%

io_halfmem:

Kernel                  							Time			CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	306.82          	87%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	461.74          	57%

io full mem:

Kernel									Time			CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	325.39          	82%
2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	411.47          	64%

full logs of the tests are:

WITH EXT2
------------
noload Time: 259.47  CPU: 99%  Major Faults: 770937  Minor Faults: 1173705
process_load Time: 318.91  CPU: 80%  Major Faults: 742261  Minor Faults: 
1169516
io_halfmem Time: 306.82  CPU: 87%  Major Faults: 742000  Minor Faults: 1169497
Was writing number 33 of a 257Mb sized io_load file after 307 seconds
io_fullmem Time: 325.39  CPU: 82%  Major Faults: 742000  Minor Faults: 1169494
Was writing number 16 of a 514Mb sized io_load file after 337 seconds
mem_load Time: 340.32  CPU: 79%  Major Faults: 743307  Minor Faults: 1170011


WITH EXT3
-----------

noload Time: 267.66  CPU: 97%  Major Faults: 771111  Minor Faults: 1173722
process_load Time: 324.44  CPU: 79%  Major Faults: 742261  Minor Faults: 
1169518
io_halfmem Time: 461.74  CPU: 57%  Major Faults: 742000  Minor Faults: 1169496
Was writing number 34 of a 257Mb sized io_load file after 465 seconds
io_fullmem Time: 411.47  CPU: 64%  Major Faults: 742000  Minor Faults: 1169494
Was writing number 15 of a 514Mb sized io_load file after 425 seconds
mem_load Time: 333.99  CPU: 81%  Major Faults: 743320  Minor Faults: 1170021

NOTES:
====

As you can see, there's something DEFINATELY wrong here.  EXT3 is much slower
then EXT2. I converted the EXT3 disk back to EXT2 to do the second test.

Also, I specified no mount options for EXT3 (which means it uses ordered 
mode). The journal was created with tune2fs -j /dev/hda#


>From #Kernelnewbies (snip)
==============
<ShawnCONSOLE> riel uses EXT3
<riel> my cpu is slower
<ShawnCONSOLE> but you have fast disks?
<riel> so it doesn't fall idle as quickly as yours, when waiting on the disk
<riel> not very fast ;)
<riel> old 8 GB IDE disk
<ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait 
longer cause the disk finishes its tasks much faster then the cpu expects? 
<ShawnCONSOLE> mem load final test = 78%
<ShawnCONSOLE> so final numbers:
<ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
<riel> yes, a very fast CPU falls idle more quickly
<riel> but it's very curious that ext3 is  that  much worse than ext2
<ShawnCONSOLE> thats much better.
<riel> definately worth pointing out to the ext3 maintainers.

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Shawn Starr]

Legend:

Kernel									Time            	CPU
2.4.20-pre7-rmap14a-xfs-uml-shawn12d(EXT2)       	
2.4.20-pre7-rmap14a-xfs-uml-shawn12d(EXT3)            	


On September 19, 2002 12:16 am, Shawn Starr wrote:
> Sorry about the confusing email before. This should make more sense =)
>
> These results compare EXT3 against EXT2 with rmap using the contest tool
> you can get it at: http://contest.kolivas.net
>
> These tests are from a Athlon MP 2000+ w/ 512MB RAM
>
> noload:
>
> Kernel									Time            	CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	259.47		99%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	267.66          	97%
>
> process load:
>
> Kernel                  							Time            	CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	318.91          	80%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	324.44          	79%
>
> io halfmem:
>
> Kernel                  							Time			CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	306.82          	87%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	461.74          	57%
>
> io full mem:
>
> Kernel									Time			CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	325.39          	82%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            	411.47          	64%
>
> full logs of the tests are:
>
> WITH EXT2
> ------------
> noload Time: 259.47  CPU: 99%  Major Faults: 770937  Minor Faults: 1173705
> process load Time: 318.91  CPU: 80%  Major Faults: 742261  Minor Faults:
> 1169516
> io halfmem Time: 306.82  CPU: 87%  Major Faults: 742000  Minor Faults:
> 1169497 Was writing number 33 of a 257Mb sized io load file after 307
> seconds io fullmem Time: 325.39  CPU: 82%  Major Faults: 742000  Minor
> Faults: 1169494 Was writing number 16 of a 514Mb sized io load file after
> 337 seconds mem load Time: 340.32  CPU: 79%  Major Faults: 743307  Minor
> Faults: 1170011
>
>
> WITH EXT3
> -----------
>
> noload Time: 267.66  CPU: 97%  Major Faults: 771111  Minor Faults: 1173722
> process load Time: 324.44  CPU: 79%  Major Faults: 742261  Minor Faults:
> 1169518
> io halfmem Time: 461.74  CPU: 57%  Major Faults: 742000  Minor Faults:
> 1169496 Was writing number 34 of a 257Mb sized io load file after 465
> seconds io fullmem Time: 411.47  CPU: 64%  Major Faults: 742000  Minor
> Faults: 1169494 Was writing number 15 of a 514Mb sized io load file after
> 425 seconds mem load Time: 333.99  CPU: 81%  Major Faults: 743320  Minor
> Faults: 1170021
>
> NOTES:
> ====
>
> As you can see, there's something DEFINATELY wrong here.  EXT3 is much
> slower then EXT2. I converted the EXT3 disk back to EXT2 to do the second
> test.
>
> Also, I specified no mount options for EXT3 (which means it uses ordered
> mode). The journal was created with tune2fs -j /dev/hda#
>
>
> From #Kernelnewbies (snip)
> ==============
> <ShawnCONSOLE> riel uses EXT3
> <riel> my cpu is slower
> <ShawnCONSOLE> but you have fast disks?
> <riel> so it doesn't fall idle as quickly as yours, when waiting on the
> disk <riel> not very fast ;)
> <riel> old 8 GB IDE disk
> <ShawnCONSOLE> so having a fast disk and a fast CPU causes the cpu to wait
> longer cause the disk finishes its tasks much faster then the cpu expects?
> <ShawnCONSOLE> mem load final test = 78%
> <ShawnCONSOLE> so final numbers:
> <ShawnCONSOLE> 99, 80%, 87%, 83%, 75%
> <riel> yes, a very fast CPU falls idle more quickly
> <riel> but it's very curious that ext3 is  that  much worse than ext2
> <ShawnCONSOLE> thats much better.
> <riel> definately worth pointing out to the ext3 maintainers.

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Andreas Dilger]

On Sep 19, 2002  00:16 -0400, Shawn Starr wrote:
> These results compare EXT3 against EXT2 with rmap using the contest tool
> you can get it at: http://contest.kolivas.net
> 
> These tests are from a Athlon MP 2000+ w/ 512MB RAM
> 
> noload:
> 
> Kernel					Time            	CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            259.47		99%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            267.66          	97%
> 
> process_load:
> 
> Kernel                  			Time            	CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            318.91          	80%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            324.44          	79%
> 
> io_halfmem:
> 
> Kernel                  			Time			CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            306.82          	87%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            461.74          	57%
> 
> io full mem:
> 
> Kernel					Time			CPU
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            325.39          	82%
> 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            411.47          	64%

I don't see this as hugely surprising.  ext3 uses more CPU than ext2.
If you are using up the CPU doing other things, then naturally ext3
will take a longer wall-clock time to complete the same tasks as ext2.

I know that Andrew has been doing a bunch of work to reduce ext3 CPU
usage/locking/etc., but I think that is all in 2.5 kernels.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Shawn Starr]

Well, all the tests were done in single user mode =)

On September 19, 2002 02:13 am, Andreas Dilger wrote:
> On Sep 19, 2002  00:16 -0400, Shawn Starr wrote:
> > These results compare EXT3 against EXT2 with rmap using the contest tool
> > you can get it at: http://contest.kolivas.net
> >
> > These tests are from a Athlon MP 2000+ w/ 512MB RAM
> >
> > noload:
> >
> > Kernel					Time            	CPU
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            259.47		99%
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            267.66          	97%
> >
> > process_load:
> >
> > Kernel                  			Time            	CPU
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            318.91          	80%
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            324.44          	79%
> >
> > io_halfmem:
> >
> > Kernel                  			Time			CPU
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            306.82          	87%
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            461.74          	57%
> >
> > io full mem:
> >
> > Kernel					Time			CPU
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            325.39          	82%
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            411.47          	64%
>
> I don't see this as hugely surprising.  ext3 uses more CPU than ext2.
> If you are using up the CPU doing other things, then naturally ext3
> will take a longer wall-clock time to complete the same tasks as ext2.
>
> I know that Andrew has been doing a bunch of work to reduce ext3 CPU
> usage/locking/etc., but I think that is all in 2.5 kernels.
>
> Cheers, Andreas
> --
> Andreas Dilger
> http://www-mddsp.enel.ucalgary.ca/People/adilger/
> http://sourceforge.net/projects/ext2resize/

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Andrew Morton]

Andreas Dilger wrote:
> 
> ...
> > Kernel                                        Time                    CPU
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            325.39                82%
> > 2.4.20-pre7-rmap14a-xfs-uml-shawn12d            411.47                64%
> 
> I don't see this as hugely surprising.  ext3 uses more CPU than ext2.
> If you are using up the CPU doing other things, then naturally ext3
> will take a longer wall-clock time to complete the same tasks as ext2.

Yup.  But here the CPU load is less; obviously some more seeking
was done.  That's fairly normal for ext3 - it has to write the journal
as well as the filesystem....
 
> I know that Andrew has been doing a bunch of work to reduce ext3 CPU
> usage/locking/etc., but I think that is all in 2.5 kernels.
> 

I had a little patch.  Stephen is working on the big fix.

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Stephen C. Tweedie]

Hi,

On Wed, Sep 18, 2002 at 11:32:19PM -0700, Andrew Morton wrote:

> I had a little patch.  Stephen is working on the big fix.

It passed an overnight Cerberus at the end of last week.  :-)
Checking into CVS shortly, then I need to set up a pile of recovery
tests to make sure it's still writing everything it needs to in time.

--Stephen

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Shawn Starr]

Which branch of the kernel is this going into? an -ac branch or 2.5 bk?

On September 23, 2002 03:03 pm, Stephen C. Tweedie wrote:
> Hi,
>
> On Wed, Sep 18, 2002 at 11:32:19PM -0700, Andrew Morton wrote:
> > I had a little patch.  Stephen is working on the big fix.
>
> It passed an overnight Cerberus at the end of last week.  :-)
> Checking into CVS shortly, then I need to set up a pile of recovery
> tests to make sure it's still writing everything it needs to in time.
>
> --Stephen

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Andrew Morton]

Shawn Starr wrote:
> 
> On September 23, 2002 03:03 pm, Stephen C. Tweedie wrote:
> > Hi,
> >
> > On Wed, Sep 18, 2002 at 11:32:19PM -0700, Andrew Morton wrote:
> > > I had a little patch.  Stephen is working on the big fix.
> >
> > It passed an overnight Cerberus at the end of last week.  :-)
> > Checking into CVS shortly, then I need to set up a pile of recovery
> > tests to make sure it's still writing everything it needs to in time.
> >
>
> Which branch of the kernel is this going into? an -ac branch or 2.5 bk?
> 

The way it usually goes is that Stephen checks it into ext3 CVS (based
off current Marcelo kernel) and I port it up to 2.5.

Re: [BENCHMARK] EXT3 vs EXT2 results with rmap14a and testing with contest 0.34

[Stephen C. Tweedie]

Hi,

On Mon, Sep 23, 2002 at 04:00:03PM -0400, Shawn Starr wrote:
 
> Which branch of the kernel is this going into? an -ac branch or 2.5 bk?

Ext3 CVS, but I'll also make a clean, broken-down patchset for
2.4.19 bk. 

In fact, the only reason it's not in ext3 cvs already is because it's
such a pain keeping discrete broken-down patches separate using a form
of SCM like bk or cvs; having decent scripts to track multiple patches
in a working source tree just works a lot better for one developer
when it comes to merging stuff upstream.

Cheers,
 Stephen