RE: RAID 5 performance problems

RE: RAID 5 performance problems

[Jonathan Vardy]

> That's the one.  Your 120GB drives are being seen as UDMA-33. 
>  Whatever is
> causing this is slowing you down.  Fix this, and the 
> performance should
> improve.
> 
> > but after the boot I set hdparm manually for each drive 
> with the following
> > settings:
> >
> > hdparm -a8 -A1 -c1 -d1 -m16 -u1 /dev/hdc.
> 
> According to your single-drive benchmarks, it didn't do the 
> job.  You'll
> have to find the CAUSE of the UDMA-33 identification, and fix it.  An
> example (not necessarily your problem) is that if a 
> 40-conductor cable is
> used, you CAN'T set the drive to UDMA-66/100/133.  There may 
> also be some
> settings in the drive or controller, or some jumpers, that 
> are keeping the
> drive from switching to the fast modes.
> 
> Once you get the drives being identified at a fast UDMA, you 
> then need to
> again verify the array performance.  It should have climbed 
> significantly.
> 
> Good luck.
> 				Peter Ashford
> 

I've rebooted with the original Red Hat kernel (2.4.20) and it
recognizes the drives correctly now (UDMA100) but the performance is
still poor

Here's the latest info:

Bootlog:

ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
PIIX4: IDE controller at PCI slot 00:04.1
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xd800-0xd807, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xd808-0xd80f, BIOS settings: hdc:pio, hdd:pio
PDC20270: IDE controller at PCI slot 02:01.0
PDC20270: chipset revision 2
PDC20270: not 100% native mode: will probe irqs later
    ide2: BM-DMA at 0x9040-0x9047, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0x9048-0x904f, BIOS settings: hdg:pio, hdh:pio
    ide4: BM-DMA at 0x90c0-0x90c7, BIOS settings: hdi:pio, hdj:pio
    ide5: BM-DMA at 0x90c8-0x90cf, BIOS settings: hdk:pio, hdl:pio
hda: Maxtor 2B020H1, ATA DISK drive
blk: queue c0453420, I/O limit 4095Mb (mask 0xffffffff)
hdc: WDC WD1200BB-00CAA1, ATA DISK drive
blk: queue c04538a0, I/O limit 4095Mb (mask 0xffffffff)
hde: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0453d20, I/O limit 4095Mb (mask 0xffffffff)
hdg: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c04541a0, I/O limit 4095Mb (mask 0xffffffff)
hdi: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0454620, I/O limit 4095Mb (mask 0xffffffff)
hdk: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0454aa0, I/O limit 4095Mb (mask 0xffffffff)
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
ide2 at 0x9000-0x9007,0x9012 on irq 19
ide3 at 0x9020-0x9027,0x9032 on irq 19
ide4 at 0x9080-0x9087,0x9092 on irq 19
ide5 at 0x90a0-0x90a7,0x90b2 on irq 19
hda: host protected area => 1
hda: 39062500 sectors (20000 MB) w/2048KiB Cache, CHS=2431/255/63,
UDMA(33)
hdc: host protected area => 1
hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
hde: host protected area => 1
hde: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdg: host protected area => 1
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdi: host protected area => 1
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdk: host protected area => 1
hdk: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)

The bonny results are as followed:

Version  1.03       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
store.datzegik.c 1G  2717  99 10837  30  4912  10  2747  95 17228  14
114.5   1
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16   367  98 +++++ +++ 13163  99   368  98 +++++ +++
1386  93
store.datzegik.com,1G,2717,99,10837,30,4912,10,2747,95,17228,14,114.5,1,
16,367,98,+++++,+++,13163,99,368,98,+++++,+++,1386,93

The results per drive with hdparm:

/dev/hda:
 	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
 	Timing buffered disk reads:  64 MB in  4.32 seconds = 14.81
MB/sec

/dev/hdc:
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  4.55 seconds = 14.07
MB/sec

/dev/hde:
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.50 seconds = 25.60
MB/sec

/dev/hdg:
	Timing buffer-cache reads:   128 MB in  1.12 seconds =114.29
MB/sec
	Timing buffered disk reads:  64 MB in  2.30 seconds = 27.83
MB/sec

/dev/hdi:
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.27 seconds = 28.19
MB/sec

/dev/hdk:
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.54 seconds = 25.20
MB/sec

The raid device:

/dev/md0:
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.19 seconds = 29.22
MB/sec

RE: RAID 5 performance problems

[Ezra Nugroho]

> hdc: host protected area => 1
> hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
> UDMA(33)


Your hdc is still running at udma(33). This is also part of the raid,
right? This will slow the whole thing down since in raid 5 write is done
to all disks simultaneously. Before the system finishes writing to the
slow drive, the write is not done yet.

Re: RAID 5 performance problems

[Jonathan Vardy]

RE: RAID 5 performance problems> I've rebooted with the original Red Hat
kernel (2.4.20) and it
> recognizes the drives correctly now (UDMA100) but the performance is
> still poor

Good news; I've finally got the performance up to a good level. Bonnie++ now
gives 77MB/sec read and 25MB/sec write speeds.

The problem was being caused by a setting in the mainbaord BIOS; "PCI
latency timer" had a default value of 0, after changing this to 32 the drive
speed (hdparm) doubled and the raid finally performed as it should.

I'd like to thank everybody who responded for giving me advice on the
problem, I apreciate it greatly.

Yours sincerely, Jonathan Vardy

Re: RAID 5 performance problems

[H. Peter Anvin]

Followup to:  <Pine.GSO.4.30.0304030858080.20118-100000@multivac.sdsc.edu>
By author:    "Peter L. Ashford" <ashford@sdsc.edu>
In newsgroup: linux.dev.kernel
> 
> The ONLY reason that I can think of to use round cables would be for
> looks.  From a performance or reliability standpoint, they are a waste of
> money.  I routinely build systems with dual 8-channel IDE RAID cards
> (3Ware 7500-8) and 16 disks, and ONLY use flat cables.
> 

In some chassis they make routing cables a lot easier.  In others,
they make it harder.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

RE: RAID 5 performance problems

[Jonathan Vardy]

> I've rebooted with the original Red Hat kernel (2.4.20) and it
> recognizes the drives correctly now (UDMA100) but the performance is
> still poor

Added to this I've replaced the rounded cables with the promise cables
to make sure this is not an issue. I've put in two 500Mhz processors so
it is comparable with my friends orignal setup of 2x500Mhz, 6x80GB (2x
Udma33, 4x Udma66) which had around 80MB/sec for reads.

The dual 500Mhz setup with 2.4.20 does:

/dev/md0:
	Timing buffer-cache reads:   128 MB in  0.96 seconds =133.33
MB/sec
	Timing buffered disk reads:  64 MB in  3.05 seconds = 20.98
MB/sec

Latest Bonnie++ results:

Version  1.03       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
store.datzegik.c 1G  3800  99 11114  22  4910   7  3632  92 17681  10
105.9   0
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16   484  98 +++++ +++ 15714  86   518  98 +++++ +++
1985  92
store.datzegik.com,1G,3800,99,11114,22,4910,7,3632,92,17681,10,105.9,0,1
6,484,98,+++++,+++,15714,86,518,98,+++++,+++,1985,92

Bootmessages:

ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
PIIX4: IDE controller at PCI slot 00:04.1
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xd800-0xd807, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xd808-0xd80f, BIOS settings: hdc:pio, hdd:pio
PDC20270: IDE controller at PCI slot 02:01.0
PDC20270: chipset revision 2
PDC20270: not 100% native mode: will probe irqs later
    ide2: BM-DMA at 0x9040-0x9047, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0x9048-0x904f, BIOS settings: hdg:pio, hdh:pio
    ide4: BM-DMA at 0x90c0-0x90c7, BIOS settings: hdi:pio, hdj:pio
    ide5: BM-DMA at 0x90c8-0x90cf, BIOS settings: hdk:pio, hdl:pio
hda: Maxtor 2B020H1, ATA DISK drive
blk: queue c0453420, I/O limit 4095Mb (mask 0xffffffff)
hdc: WDC WD1200BB-00CAA1, ATA DISK drive
blk: queue c04538a0, I/O limit 4095Mb (mask 0xffffffff)
hde: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0453d20, I/O limit 4095Mb (mask 0xffffffff)
hdg: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c04541a0, I/O limit 4095Mb (mask 0xffffffff)
hdi: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0454620, I/O limit 4095Mb (mask 0xffffffff)
hdk: WDC WD1200BB-60CJA1, ATA DISK drive
blk: queue c0454aa0, I/O limit 4095Mb (mask 0xffffffff)
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
ide2 at 0x9000-0x9007,0x9012 on irq 19
ide3 at 0x9020-0x9027,0x9032 on irq 19
ide4 at 0x9080-0x9087,0x9092 on irq 19
ide5 at 0x90a0-0x90a7,0x90b2 on irq 19
hda: host protected area => 1
hda: 39062500 sectors (20000 MB) w/2048KiB Cache, CHS=2431/255/63,
UDMA(33)
hdc: host protected area => 1
hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
hde: host protected area => 1
hde: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdg: host protected area => 1
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdi: host protected area => 1
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)
hdk: host protected area => 1
hdk: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(100)

RE: RAID 5 performance problems

[Jonathan Vardy]

> From: Ezra Nugroho [mailto:ezran@goshen.edu] 

> Your hdc is still running at udma(33). This is also part of the raid,
> right? This will slow the whole thing down since in raid 5 
> write is done
> to all disks simultaneously. Before the system finishes writing to the
> slow drive, the write is not done yet.

 > Your hdc is still running at udma(33). This is also part of the raid,
> right? This will slow the whole thing down since in raid 5 
> write is done
> to all disks simultaneously. Before the system finishes writing to the
> slow drive, the write is not done yet.

This should not cripple the performance to what I get. My friend had
6x80GB 5400rpm drives with two of those on udma33 and four on udma66 and
he managed 80MB/sec (dual 500Mhz)

Jonathan

Re: RAID 5 performance problems

[Peter L. Ashford]

Jonathan,

> > OK.  We've found a potential issue.  Are the disks being identified as
> > UDMA-33 or UDMA-66/100/133?  The performance numbers agree too closely for
> > this to be a coincidence.  Check the boot logs.
>
> I looked into /var/log/dmesg and found this:
>
> blk: queue c03934a8, I/O limit 4095Mb (mask 0xffffffff)
> hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
> UDMA(33)
>
> This is what you meant?

That's the one.  Your 120GB drives are being seen as UDMA-33.  Whatever is
causing this is slowing you down.  Fix this, and the performance should
improve.

> but after the boot I set hdparm manually for each drive with the following
> settings:
>
> hdparm -a8 -A1 -c1 -d1 -m16 -u1 /dev/hdc.

According to your single-drive benchmarks, it didn't do the job.  You'll
have to find the CAUSE of the UDMA-33 identification, and fix it.  An
example (not necessarily your problem) is that if a 40-conductor cable is
used, you CAN'T set the drive to UDMA-66/100/133.  There may also be some
settings in the drive or controller, or some jumpers, that are keeping the
drive from switching to the fast modes.

Once you get the drives being identified at a fast UDMA, you then need to
again verify the array performance.  It should have climbed significantly.

Good luck.
				Peter Ashford

Re: RAID 5 performance problems

[Jonathan Vardy]

All info regarding the drive part:

ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX4: IDE controller on PCI bus 00 dev 21
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xd800-0xd807, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0xd808-0xd80f, BIOS settings: hdc:DMA, hdd:pio
PDC20270: IDE controller on PCI bus 02 dev 08
PDC20270: chipset revision 2
PDC20270: not 100% native mode: will probe irqs later
PDC20270: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER
Mode.
    ide2: BM-DMA at 0x9040-0x9047, BIOS settings: hde:pio, hdf:pio
    ide3: BM-DMA at 0x9048-0x904f, BIOS settings: hdg:pio, hdh:pio
PDC20270: IDE controller on PCI bus 02 dev 10
PDC20270: chipset revision 2
PDC20270: not 100% native mode: will probe irqs later
PDC20270: ROM enabled at 0x000dc000
PDC20270: (U)DMA Burst Bit ENABLED Primary MASTER Mode Secondary MASTER
Mode.
    ide4: BM-DMA at 0x90c0-0x90c7, BIOS settings: hdi:pio, hdj:pio
    ide5: BM-DMA at 0x90c8-0x90cf, BIOS settings: hdk:pio, hdl:pio
hda: Maxtor 2B020H1, ATA DISK drive
hdc: WDC WD1200BB-00CAA1, ATA DISK drive
hde: WDC WD1200BB-60CJA1, ATA DISK drive
hdg: WDC WD1200BB-60CJA1, ATA DISK drive
hdi: WDC WD1200BB-60CJA1, ATA DISK drive
hdk: WDC WD1200BB-60CJA1, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
ide2 at 0x9000-0x9007,0x9012 on irq 19
ide3 at 0x9020-0x9027,0x9032 on irq 19
ide4 at 0x9080-0x9087,0x9092 on irq 19
ide5 at 0x90a0-0x90a7,0x90b2 on irq 19
blk: queue c0393144, I/O limit 4095Mb (mask 0xffffffff)
hda: 39062500 sectors (20000 MB) w/2048KiB Cache, CHS=2431/255/63, UDMA(33)
blk: queue c03934a8, I/O limit 4095Mb (mask 0xffffffff)
hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c039380c, I/O limit 4095Mb (mask 0xffffffff)
hde: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0393b70, I/O limit 4095Mb (mask 0xffffffff)
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0393ed4, I/O limit 4095Mb (mask 0xffffffff)
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0394238, I/O limit 4095Mb (mask 0xffffffff)
hdk: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)

Re: RAID 5 performance problems

[Peter L. Ashford]

Jonathan,

> > The ONLY reason that I can think of to use round cables would be for
> > looks.  From a performance or reliability standpoint, they are a waste of
> > money.  I routinely build systems with dual 8-channel IDE RAID cards
> > (3Ware 7500-8) and 16 disks, and ONLY use flat cables.
>
> I use rounded cables in my case for a few reasons:
> - The distance between my promise and my drives is small yet the promise
> cables are long, the rounded cables I have are 12" long and fit very neatly
> - The promise cables had two IDE connectors but I only wanted to put one
> drive per channel; the rounded cables are single cables
> - Air flow; because of my small casing the flat promise cables were
> contricting the airflow quite a bit, the rounded less
> - flexibility; I found the flat cables hard to bend in to place whereas the
> round cables you could twist easily
>
> I've added a link which should make it clear that rounded cables in my case
> are a benefit to me. What I was worried about was that they could be
> inferior quality and thus be a factor in my raid performance.
>
> http://www.datzegik.com/DSC00056.JPG

Check out 'http://www.accs.com/p_and_p/TeraByte/cables.html', to see why
round cables are not needed.  Careful cable routing can easilly overcome
the issues you have.  When you have a large number of cables, the flat
cables can stack, but the round cables just make a big bundle.

Also, 3Ware sells 80-conductor/40-pin cables with two connectors in 18",
24" and 36" lengths.

I've built systems in cases that are similar to yours (Antec or Chen-Ming)
with similar numbers of drives, and had no problems with flat cables to
five disks, a CDROM drive and a floppy drive.

Good luck.
				Peter Ashford

Re: RAID 5 performance problems

[Jonathan Vardy]

> OK.  We've found a potential issue.  Are the disks being identified as
> UDMA-33 or UDMA-66/100/133?  The performance numbers agree too closely for
> this to be a coincidence.  Check the boot logs.

I looked into /var/log/dmesg and found this:

blk: queue c0393144, I/O limit 4095Mb (mask 0xffffffff)
hda: 39062500 sectors (20000 MB) w/2048KiB Cache, CHS=2431/255/63, UDMA(33)
blk: queue c03934a8, I/O limit 4095Mb (mask 0xffffffff)
hdc: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c039380c, I/O limit 4095Mb (mask 0xffffffff)
hde: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0393b70, I/O limit 4095Mb (mask 0xffffffff)
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0393ed4, I/O limit 4095Mb (mask 0xffffffff)
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)
blk: queue c0394238, I/O limit 4095Mb (mask 0xffffffff)
hdk: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=232581/16/63,
UDMA(33)

This is what you meant?

but after the boot I set hdparm manually for each drive with the following
settings:

hdparm -a8 -A1 -c1 -d1 -m16 -u1 /dev/hdc.

Re: RAID 5 performance problems

[Jonathan Vardy]

> The ONLY reason that I can think of to use round cables would be for
> looks.  From a performance or reliability standpoint, they are a waste of
> money.  I routinely build systems with dual 8-channel IDE RAID cards
> (3Ware 7500-8) and 16 disks, and ONLY use flat cables.

I use rounded cables in my case for a few reasons:
- The distance between my promise and my drives is small yet the promise
cables are long, the rounded cables I have are 12" long and fit very neatly
- The promise cables had two IDE connectors but I only wanted to put one
drive per channel; the rounded cables are single cables
- Air flow; because of my small casing the flat promise cables were
contricting the airflow quite a bit, the rounded less
- flexibility; I found the flat cables hard to bend in to place whereas the
round cables you could twist easily

I've added a link which should make it clear that rounded cables in my case
are a benefit to me. What I was worried about was that they could be
inferior quality and thus be a factor in my raid performance.

http://www.datzegik.com/DSC00056.JPG

Re: RAID 5 performance problems

[Peter L. Ashford]

Stephan,

> here benchmark from my system with an fasttrak-tx4 (same card jonathan
> has) with WD1800BB :

The WD1800BB and the WD1200BB use the same technology (same data density,
fewer platters).  The streaming speed should be very similar.

> /dev/hde:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  1.37 seconds = 46.72 MB/sec
>
> WD1800BB on onboard ultra100 controller :
>
> /dev/hdo:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  1.58 seconds = 40.51 MB/sec
>
> WD1800BB on onboard ultra33 controller :
>
> /dev/hda:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  2.55 seconds = 25.10 MB/sec

OK.  We've found a potential issue.  Are the disks being identified as
UDMA-33 or UDMA-66/100/133?  The performance numbers agree too closely for
this to be a coincidence.  Check the boot logs.

Good luck.
				Peter Ashford

Re: RAID 5 performance problems

[Ezra Nugroho]

That looks like a slow build!
Mine is about 15 - 20M/sec.

Another observation I saw, I raided the raw devices instead of the
partitions; meaning my array is build from hde, hdg, hdi and hdk
as opposed to hde1, hdg1, hdi1 and hdk1.


On Thu, 2003-04-03 at 16:10, Jonathan Vardy wrote:
> you're right, it should be. When I was writing the original mail I had it
> running in degraded mode so I edited the values that /proc/mdstat gave me to
> match the array in normal mode. I forgot to make [_UUUU]  [UUUUU]. I'm
> currently rebuilding the array but it's taking some time...
> 
> md0 : active raid5 hdc1[5] hdk1[4] hdi1[3] hdg1[2] hde1[1]
>       468872704 blocks level 5, 128k chunk, algorithm 0 [5/4] [_UUUU]
>       [================>....]  recovery = 83.1% (97429504/117218176)
> finish=65.5min speed=5034K/sec
> 
> ----- Original Message -----
> From: "Andy Arvai" <arvai@scripps.edu>
> To: <linux-raid@vger.kernel.org>
> Cc: <linux-kernel@vger.kernel.org>
> Sent: Thursday, April 03, 2003 9:49 PM
> Subject: Re: RAID 5 performance problems
> 
> 
> >
> > Shouldn't /proc/mdstat have [UUUUU] instead of [_UUUU]? Perhaps
> > this is running in degraded mode. Also, you have 'algorithm 0',
> > whereas my raid5 has 'algorithm 2', which is the left-symmetric
> > parity algorithm.
> >
> > Andy
> >
> > > cat /proc/mdstat gives:
> > >
> > > Personalities : [raid0] [raid5]
> > > read_ahead 1024 sectors
> > > md0 : active raid5 hdk1[4] hdi1[3] hdg1[2] hde1[1] hdc1[0]
> > > 468872704 blocks level 5, 128k chunk, algorithm 0 [5/5] [_UUUU]
> > > unused devices: <none>
> >
> >
> >
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: RAID 5 performance problems

[Stephan van Hienen]

On Thu, 3 Apr 2003, Ezra Nugroho wrote:

> What should I expect from a 8 drive raid 5 on 3ware 7500-8?
> I get only
> /dev/sda:
>  Timing buffer-cache reads:   128 MB in  0.74 seconds =172.97 MB/sec
>  Timing buffered disk reads:  64 MB in  1.55 seconds = 41.29 MB/sec
>
> which seems to be slow compared to the numbers above

I think you should look at the '3ware bad write speed' thread on
linux-raid
looks like you are running in 3ware raid mode ?

Re: RAID 5 performance problems

[Jonathan Vardy]

> On Thu, 2003-04-03 at 17:45, Jonathan Vardy wrote:
> > I'm having trouble with getting the right performance out of my software
> > raid 5 system. I've installed Red Hat 9.0 with kernel 2.4.20 compiled
> > myself to match my harware (had the same problem with the default
> > kernel). When I test the raid device's speed using 'hdparm -Tt /dev/hdx'
> > I get this:
> > /dev/md0:
> > Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28 MB/sec
> > Timing buffered disk reads:  64 MB in  2.39 seconds = 26.78 MB/sec
>
> Well, if I'm not wrong, you're testing physical, individual drives, not
> the RAID5, combined, logical volume. So, your values are pretty normal.

/dev/md0 is the raid device and as you can see from the bonny++ results in
my original mail the, performance is not really normal.

Re: RAID 5 performance problems

[Jonathan Vardy]

you're right, it should be. When I was writing the original mail I had it
running in degraded mode so I edited the values that /proc/mdstat gave me to
match the array in normal mode. I forgot to make [_UUUU]  [UUUUU]. I'm
currently rebuilding the array but it's taking some time...

md0 : active raid5 hdc1[5] hdk1[4] hdi1[3] hdg1[2] hde1[1]
      468872704 blocks level 5, 128k chunk, algorithm 0 [5/4] [_UUUU]
      [================>....]  recovery = 83.1% (97429504/117218176)
finish=65.5min speed=5034K/sec

----- Original Message -----
From: "Andy Arvai" <arvai@scripps.edu>
To: <linux-raid@vger.kernel.org>
Cc: <linux-kernel@vger.kernel.org>
Sent: Thursday, April 03, 2003 9:49 PM
Subject: Re: RAID 5 performance problems


>
> Shouldn't /proc/mdstat have [UUUUU] instead of [_UUUU]? Perhaps
> this is running in degraded mode. Also, you have 'algorithm 0',
> whereas my raid5 has 'algorithm 2', which is the left-symmetric
> parity algorithm.
>
> Andy
>
> > cat /proc/mdstat gives:
> >
> > Personalities : [raid0] [raid5]
> > read_ahead 1024 sectors
> > md0 : active raid5 hdk1[4] hdi1[3] hdg1[2] hde1[1] hdc1[0]
> > 468872704 blocks level 5, 128k chunk, algorithm 0 [5/5] [_UUUU]
> > unused devices: <none>
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: RAID 5 performance problems

[Felipe Alfaro Solana]

On Thu, 2003-04-03 at 17:45, Jonathan Vardy wrote:
> I'm having trouble with getting the right performance out of my software
> raid 5 system. I've installed Red Hat 9.0 with kernel 2.4.20 compiled
> myself to match my harware (had the same problem with the default
> kernel). When I test the raid device's speed using 'hdparm -Tt /dev/hdx'
> I get this:
> /dev/md0:
> Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28 MB/sec
> Timing buffered disk reads:  64 MB in  2.39 seconds = 26.78 MB/sec

Well, if I'm not wrong, you're testing physical, individual drives, not
the RAID5, combined, logical volume. So, your values are pretty normal.

________________________________________________________________________
Linux Registered User #287198

Re: RAID 5 performance problems

[Ezra Nugroho]

> here benchmark from my system with an fasttrak-tx4 (same card jonathan
> has) with WD1800BB :
> 
> /dev/hde:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  1.37 seconds = 46.72 MB/sec
> 
> WD1800BB on onboard ultra100 controller :
> 
> /dev/hdo:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  1.58 seconds = 40.51 MB/sec
> 
> WD1800BB on onboard ultra33 controller :
> 
> /dev/hda:
>  Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
>  Timing buffered disk reads:  64 MB in  2.55 seconds = 25.10 MB/sec

What should I expect from a 8 drive raid 5 on 3ware 7500-8?
I get only 
/dev/sda:
 Timing buffer-cache reads:   128 MB in  0.74 seconds =172.97 MB/sec
 Timing buffered disk reads:  64 MB in  1.55 seconds = 41.29 MB/sec

which seems to be slow compared to the numbers above
 

Re: RAID 5 performance problems

[Mike Dresser]

On Thu, 3 Apr 2003, Andy Arvai wrote:

> Shouldn't /proc/mdstat have [UUUUU] instead of [_UUUU]? Perhaps
> this is running in degraded mode. Also, you have 'algorithm 0',
> whereas my raid5 has 'algorithm 2', which is the left-symmetric
> parity algorithm.

he's running it in degraded mode intentionally, the setup is so broken
that the performance loss is masked by the rest of the system.

RAID 5 performance problems

[Chuck Ebbert]

>Tos be certain that this was not the bottleneck, I removed it
>from the raid so that is runs in degraded mode. This did not
>change the performance much


Don't do that and expect a valid performance test.

On average, one-fifth of all array reads will cause reads from all four
remaining disks in order to reconstruct the missing data when you run
in that mode.

--
Chuck

Re: RAID 5 performance problems

[Andy Arvai]

Shouldn't /proc/mdstat have [UUUUU] instead of [_UUUU]? Perhaps
this is running in degraded mode. Also, you have 'algorithm 0',
whereas my raid5 has 'algorithm 2', which is the left-symmetric
parity algorithm.

Andy 

> cat /proc/mdstat gives: 
> 
> Personalities : [raid0] [raid5] 
> read_ahead 1024 sectors
> md0 : active raid5 hdk1[4] hdi1[3] hdg1[2] hde1[1] hdc1[0]
> 	468872704 blocks level 5, 128k chunk, algorithm 0 [5/5] [_UUUU]
> unused devices: <none>

Re: RAID 5 performance problems

[Alan Cox]

On Iau, 2003-04-03 at 20:20, Stephan van Hienen wrote:
> > 2.  I/O bandwidth (multiple PCI buses)
> sure this helps, but he is not even getting near the pci bus
> limitations...

With mirrored software raid you are writing twice the data over
the PCI bus.

Re: RAID 5 performance problems

[Stephan van Hienen]

On Thu, 3 Apr 2003, Ross Vandegrift wrote:

> > I've benchmarked RAW disk speeds with an Ultra-100 and
> > WD1200JB drives, and gotten 50MB/S from each disk, as opposed to your
> > 26MB/S (there should be almost no difference for the BB drives).
>
> Makes perfect sense - he's getting 50M/s to each channel, but it's split
> over two drives, so drive throughput is about halved.
the fasttrak-tx4 is an 4 channel controller (it is an dual ultra-100 on 1
pci board)
Jonathans WD1200BB's should be a bit faster than 30MB/sec maybe, but the
main problem is his /dev/md0...

> Absolutely correct - you should *never* run IDE RAID on a channel that
> has both a master and slave.  When one disk on an IDE channel has an
Jonathan is only using master devices, no slaves...

Re: RAID 5 performance problems

[Stephan van Hienen]

On Thu, 3 Apr 2003, Peter L. Ashford wrote:

> 1.  Disk controllers (get off the motherboard IDE, dump the FastTrak, use
> 	Master only)
fasttrak is nothing more than an ultra version with an special bios
if you don't use this raid bios, there is no difference (for linux)

> 2.  I/O bandwidth (multiple PCI buses)
sure this helps, but he is not even getting near the pci bus
limitations...

> 3.  Memory bandwidth (switch to something like a P4)
my old dual p2-500 had didn't have this problem
(i'm 'the friend' look at the last benchmark (5*80gb raid5)

> 4.  Disks (this is where your friend's dual Xeon system is)
eh, disks are not really his problem
(he is getting 30mb/sec with hdparm for each disk, but getting same result
for his /dev/md0)

>
> Your network probably belongs in this list somewhere, but you didn't
he didn't test any network performance yet, this is just plain disk
benchmarking...


> 26MB/S (there should be almost no difference for the BB drives).  My
> suggestion would be to remove the FastTrak, and get two Ultra-100s and an
> Ultra-133 (or one Ultra-100 and two Ultra-133s).  Use one disk per channel
there is absolutly no performance difference between an fasttrak-100 and
an ultra-100

here benchmark from my system with an fasttrak-tx4 (same card jonathan
has) with WD1800BB :

/dev/hde:
 Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
 Timing buffered disk reads:  64 MB in  1.37 seconds = 46.72 MB/sec

WD1800BB on onboard ultra100 controller :

/dev/hdo:
 Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
 Timing buffered disk reads:  64 MB in  1.58 seconds = 40.51 MB/sec

WD1800BB on onboard ultra33 controller :

/dev/hda:
 Timing buffer-cache reads:   128 MB in  0.28 seconds =457.14 MB/sec
 Timing buffered disk reads:  64 MB in  2.55 seconds = 25.10 MB/sec

> > His raid:
>
> <SNIP>
>
> 99MB/S block write, 178MB/S block read
>
> This MUST have had a better disk controller system AND multiple PCI buses.
> The limit of one PCI bus (32-bit/33MHz, as used by the Promise
> controllers) is 133MB/S, and they are exceeding that.
correct, 3 pci busses

but my old system was only 1 pci bus (same mainbord Jonathan is using)
but was getting way higher performance (his raid is getting lower
performance than an single disk)
with only 150mhz more per cpu

Re: RAID 5 performance problems

[Neil Schemenauer]

Ross Vandegrift <ross@willow.seitz.com> wrote:
> Absolutely correct - you should *never* run IDE RAID on a channel that
> has both a master and slave.  When one disk on an IDE channel has an
> error, the whole channel is reset - this makes both disks
> inaccessible,
> and RAID5 now has two failed disks => you data is gone!  *ALWAYS* use
> separate IDE channels.

I think it's okay to use both channels if you use RAID0+1 (also
known as RAID10), just be sure to mirror across channels.  As a
bonus, RAID0+1 is significantly faster than RAID5.

  Neil

Re: RAID 5 performance problems

[Ross Vandegrift]

[CC list trimmed]

On Thu, Apr 03, 2003 at 10:05:27AM -0800, Peter L. Ashford wrote:
> > 4 of the raid HD's are connected to the Promise controller and the other
> > raid HD is master on the second onboard IDE channel (udma2/ATA33)
> 
> I've NEVER heard that the FastTrak runs fast (I have been told that they
> run VERY slowly).

The FastTrack cards are identical to the UltraDMA cards from promise -
you can turn one into the other with a resistor and a firmware flash.

> I've benchmarked RAW disk speeds with an Ultra-100 and
> WD1200JB drives, and gotten 50MB/S from each disk, as opposed to your
> 26MB/S (there should be almost no difference for the BB drives).

Makes perfect sense - he's getting 50M/s to each channel, but it's split
over two drives, so drive throughput is about halved.

> Use one disk per channel
> (Master only), and move the system disk onto one of the Promise cards.

Absolutely correct - you should *never* run IDE RAID on a channel that
has both a master and slave.  When one disk on an IDE channel has an
error, the whole channel is reset - this makes both disks inaccessible,
and RAID5 now has two failed disks => you data is gone!  *ALWAYS* use
separate IDE channels.


-- 
Ross Vandegrift
ross@willow.seitz.com

A Pope has a Water Cannon.                               It is a Water Cannon.
He fires Holy-Water from it.                        It is a Holy-Water Cannon.
He Blesses it.                                 It is a Holy Holy-Water Cannon.
He Blesses the Hell out of it.          It is a Wholly Holy Holy-Water Cannon.
He has it pierced.                It is a Holey Wholly Holy Holy-Water Cannon.
He makes it official.       It is a Canon Holey Wholly Holy Holy-Water Cannon.
Batman and Robin arrive.                                       He shoots them.

Re: RAID 5 performance problems

[Peter L. Ashford]

Jonathan,

> I'm having trouble with getting the right performance out of my software
> raid 5 system. I've installed Red Hat 9.0 with kernel 2.4.20 compiled
> myself to match my harware (had the same problem with the default
> kernel). When I test the raid device's speed using 'hdparm -Tt /dev/hdx'
> I get this:
>
> /dev/md0:
> Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28 MB/sec
> Timing buffered disk reads:  64 MB in  2.39 seconds = 26.78 MB/sec
>
> This is really low for a raid system and I can't figure out what is
> causing this. I use rounded cables but I also tried the original Promise
> cables and there was no difference in performance.

The ONLY reason that I can think of to use round cables would be for
looks.  From a performance or reliability standpoint, they are a waste of
money.  I routinely build systems with dual 8-channel IDE RAID cards
(3Ware 7500-8) and 16 disks, and ONLY use flat cables.

> But still the raid performs slow. Does anybody have an idea how I could
> improve the performance? I've seen raid systems like my own performing
> much better (speeds of around 80MB/sec). I've added as many specs as I
> could find below so you can see what my configuration is.
>
> The hardware in the machine is as following:
>
> Maiboard   	: Asus P2B-d dual PII slot 1 with latest bios update
> Processors 	: 2x Intel PII 350Mhz
> Memory     	: 512MB SDRam 100Mhz ECC
> Controller 	: Promise FastTrak100 TX4 with latest firmware update
> Boot HD	: Maxtor 20GB 5400rpm
> Raid HD's	: 5x WDC1200BB 120GB 7200rpm ATA100

Your bottlenecks are probably (in order):

1.  Disk controllers (get off the motherboard IDE, dump the FastTrak, use
	Master only)
2.  I/O bandwidth (multiple PCI buses)
3.  Memory bandwidth (switch to something like a P4)
4.  Disks (this is where your friend's dual Xeon system is)

Your network probably belongs in this list somewhere, but you didn't
provide enough information for me to determine where it belongs.  The
necessary information would be the network type, switch/hub, and the type
and quantity of processing being performed prior to putting the data on
the net.

> 4 of the raid HD's are connected to the Promise controller and the other
> raid HD is master on the second onboard IDE channel (udma2/ATA33)

I've NEVER heard that the FastTrak runs fast (I have been told that they
run VERY slowly).  I've benchmarked RAW disk speeds with an Ultra-100 and
WD1200JB drives, and gotten 50MB/S from each disk, as opposed to your
26MB/S (there should be almost no difference for the BB drives).  My
suggestion would be to remove the FastTrak, and get two Ultra-100s and an
Ultra-133 (or one Ultra-100 and two Ultra-133s).  Use one disk per channel
(Master only), and move the system disk onto one of the Promise cards.
This change should get you up to the performance of your friend's old
system.

If you switch to a dual bus system, make sure that the two controllers
with two RAID drives are on separate buses.  In a triple bus system, each
of the three controllers would be on a different bus (put the NIC with the
system drive).

An inexpensive P4 Celeron with FAST memory would probably run faster.
FYI, The primary reason that the large file servers use the dual Xeon
boards is to get the high memory and I/O bandwidth.  The high CPU
bandwidth is also useful, primarily with TCP/IP encapsulation.

<SNIP>

> As you see /dev/hdc is on the onboard IDE channel. Tos be certain that
> this was not the bottleneck, I removed it from the raid so that is runs
> in degraded mode. This did not change the performance much. Rebuild
> speed is laso very slow, it is round 6MB/sec.
>
> The drives originally came from a friend's file server where they were
> also employed in a raid configuration. I've compared my results in
> Bonny++ to his results:
>
> My raid:

<SNIP>

14MB/S block write, 30MB/S block read

> His raid:

<SNIP>

99MB/S block write, 178MB/S block read

This MUST have had a better disk controller system AND multiple PCI buses.
The limit of one PCI bus (32-bit/33MHz, as used by the Promise
controllers) is 133MB/S, and they are exceeding that.

> His machine has dual P Xeon 2000Mhz processors but that shouldn't be the
> reason that the results are so different. My processor isn't at 100%
> while testing. Even his older system which had 80gig drives and dual
> 500Mhz processors is much faster:

<SNIP>

35MB/S block write, 72MB/S block read

As mentioned above, the CPU isn't the primary limit in I/O.  You have to
consider I/O bandwidth (number, width and clock of PCI buses), and memory
bandwidth.

Good luck.
				Peter Ashford

RAID 5 performance problems

[Jonathan Vardy]

Hi,

I'm having trouble with getting the right performance out of my software
raid 5 system. I've installed Red Hat 9.0 with kernel 2.4.20 compiled
myself to match my harware (had the same problem with the default
kernel). When I test the raid device's speed using 'hdparm -Tt /dev/hdx'
I get this:

/dev/md0:
Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28 MB/sec
Timing buffered disk reads:  64 MB in  2.39 seconds = 26.78 MB/sec

Using bonny I get similar speeds (complete info further below):
Write 13668 K/sec, CPU load = 35%
Read  29752 K/sec, CPU load = 45%

This is really low for a raid system and I can't figure out what is
causing this. I use rounded cables but I also tried the original Promise
cables and there was no difference in performance. I've set the
parameters for hdparm so that the harddrives are optmised = 'hdparm -a8
-A1 -c1 -d1 -m16 -u1 /dev/hdc' which results in these settings:

 multcount    = 16 (on)
 IO_support   =  1 (32-bit)
 unmaskirq    =  1 (on)
 using_dma    =  1 (on)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    =  8 (on)
 geometry     = 14593/255/63, sectors = 234441648, start = 0

But still the raid performs slow. Does anybody have an idea how I could
improve the performance? I've seen raid systems like my own performing
much better (speeds of around 80MB/sec). I've added as many specs as I
could find below so you can see what my configuration is.

cat /proc/mdstat gives: 

Personalities : [raid0] [raid5] 
read_ahead 1024 sectors
md0 : active raid5 hdk1[4] hdi1[3] hdg1[2] hde1[1] hdc1[0]
      468872704 blocks level 5, 128k chunk, algorithm 0 [5/5] [_UUUU]
unused devices: <none>

The hardware in the machine is as following:

Maiboard   	: Asus P2B-d dual PII slot 1 with latest bios update
Processors 	: 2x Intel PII 350Mhz
Memory     	: 512MB SDRam 100Mhz ECC
Controller 	: Promise FastTrak100 TX4 with latest firmware update
Boot HD	: Maxtor 20GB 5400rpm
Raid HD's	: 5x WDC1200BB 120GB 7200rpm ATA100

4 of the raid HD's are connected to the Promise controller and the other
raid HD is master on the second onboard IDE channel (udma2/ATA33)

Here are the speeds found using 'hdparm -Tt /dev/hdx'

hda (boot on onboard ATA33)
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  4.33 seconds = 14.78
MB/sec

hdc (raid on onboard ATA33)
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  4.56 seconds = 14.04
MB/sec

hde (raid on promise )
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.42 seconds = 26.45
MB/sec

hdg (raid on promise )
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.43 seconds = 26.34
MB/sec

hdi (raid on promise )
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.41 seconds = 26.56
MB/sec

hdk (raid on promise )
	Timing buffer-cache reads:   128 MB in  1.14 seconds =112.28
MB/sec
	Timing buffered disk reads:  64 MB in  2.42 seconds = 26.45
MB/sec

As you see /dev/hdc is on the onboard IDE channel. Tos be certain that
this was not the bottleneck, I removed it from the raid so that is runs
in degraded mode. This did not change the performance much. Rebuild
speed is laso very slow, it is round 6MB/sec.

The drives originally came from a friend's file server where they were
also employed in a raid configuration. I've compared my results in
Bonny++ to his results:

My raid:

Version  1.03       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
store.datzegik.c 1G  3712  99 13668  37  6432  28  3948  96 29752  45
342.8   6
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16   318  91 +++++ +++ 13157 100   351  98 +++++ +++
1332  92
store.datzegik.com,1G,3712,99,13668,37,6432,28,3948,96,29752,45,342.8,6,
16,318,91,+++++,+++,13157,100,351,98,+++++,+++,1332,92

His raid:

---raid0 5*120gb sw raid:

Version 1.02c       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
storagenew.a2000 1G 19931  99 99258  73 61688  31 23784  98 178478  41
616.9   2
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16  1966  96 +++++ +++ +++++ +++  2043  99 +++++ +++
5518  99
storagenew.a2000.nu,1G,19931,99,99258,73,61688,31,23784,98,178478,41,616
.9,2,16,1966,96,+++++,+++,+++++,+++,2043,99,+++++,+++,5518,99


His machine has dual P Xeon 2000Mhz processors but that shouldn't be the
reason that the results are so different. My processor isn't at 100%
while testing. Even his older system which had 80gig drives and dual
500Mhz processors is much faster:

---raid5 6*80gb sw raid:
Version 1.02c       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
storage.a2000.nu 2G  7935  98 34886  35 15362  25  8073  97 71953  53
180.1   2
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16   711 100 +++++ +++ +++++ +++   710  99 +++++ +++
2856 100
storage.a2000.nu,2G,7935,98,34886,35,15362,25,8073,97,71953,53,180.1,2,1
6,711,100,+++++,+++,+++++,+++,710,99,+++++,+++,2856,100

Yours sincerely, Jonathan Vardy