* Re: Strange delete performance using XFS
2007-04-04 13:47 ` Thomas Kaehn
@ 2007-04-04 13:51 ` Justin Piszcz
2007-04-04 13:57 ` Justin Piszcz
2007-04-04 13:57 ` Justin Piszcz
2007-04-04 18:36 ` Justin Piszcz
2 siblings, 1 reply; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 13:51 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2100 bytes --]
On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> Hi Justin,
>
> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>> Please see below for "time" output.
>>>
>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>> /dev/null 2>&1; done
>>> |
>>> | real 6m6.814s
>>> | user 0m30.290s
>>> | sys 2m42.562s
>>> | # time rm -rf y
>>> |
>>> | real 5m18.034s
>>> | user 0m0.036s
>>> | sys 0m8.169s
>
>> Deletes on XFS is one area that is a little slower than other filesystems.
>> You can increase the log size during the creation of the filesystem and
>> also increase logbufs to 8 and that might help.
>
> Thanks for your suggestions.
>
> I also tried to increase the log size and logbufs mount option. This
> optimizes create and delete times to the above values (with default options
> both are around 9-10 minutes).
>
> The strange thing is that on a similar Dell machines using XFS, too,
> deletes take only ten seconds which would match user and system time.
>
> More than five minutes for deleting 100000 files where ext3 needs
> 3 seconds on the same machine is actually more than a little bit slower
> - to my mind there must be something wrong. JFS needs around 18 seconds.
>
> However I am not sure if the problem is hardware or software related.
> I've also tried to use the newest 3ware firmware - but this did not lead
> to an improvement.
>
> Ciao,
> Thomas
> --
> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
> Technik CISCO Systems Partner - Authorized Reseller
> Im Süsterfeld 6 Tel 0241/701333-18
> tk@westend.com D-52072 Aachen Fax 0241/911879
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>
>
I am running some benchmarks with SW raid and will prevent my findings
shortly.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 13:51 ` Justin Piszcz
@ 2007-04-04 13:57 ` Justin Piszcz
0 siblings, 0 replies; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 13:57 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2288 bytes --]
On Wed, 4 Apr 2007, Justin Piszcz wrote:
>
>
> On Wed, 4 Apr 2007, Thomas Kaehn wrote:
>
>> Hi Justin,
>>
>> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>>> Please see below for "time" output.
>>>>
>>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>>> /dev/null 2>&1; done
>>>> |
>>>> | real 6m6.814s
>>>> | user 0m30.290s
>>>> | sys 2m42.562s
>>>> | # time rm -rf y
>>>> |
>>>> | real 5m18.034s
>>>> | user 0m0.036s
>>>> | sys 0m8.169s
>>
>>> Deletes on XFS is one area that is a little slower than other filesystems.
>>> You can increase the log size during the creation of the filesystem and
>>> also increase logbufs to 8 and that might help.
>>
>> Thanks for your suggestions.
>>
>> I also tried to increase the log size and logbufs mount option. This
>> optimizes create and delete times to the above values (with default options
>> both are around 9-10 minutes).
>>
>> The strange thing is that on a similar Dell machines using XFS, too,
>> deletes take only ten seconds which would match user and system time.
>>
>> More than five minutes for deleting 100000 files where ext3 needs
>> 3 seconds on the same machine is actually more than a little bit slower
>> - to my mind there must be something wrong. JFS needs around 18 seconds.
>>
>> However I am not sure if the problem is hardware or software related.
>> I've also tried to use the newest 3ware firmware - but this did not lead
>> to an improvement.
>>
>> Ciao,
>> Thomas
>> --
>> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
>> Technik CISCO Systems Partner - Authorized Reseller
>> Im Süsterfeld 6 Tel 0241/701333-18
>> tk@westend.com D-52072 Aachen Fax 0241/911879
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
>> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>>
>>
>
> I am running some benchmarks with SW raid and will prevent my findings
> shortly.
Removal tests coming shortly, benchmarking is always interesting.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 13:47 ` Thomas Kaehn
2007-04-04 13:51 ` Justin Piszcz
@ 2007-04-04 13:57 ` Justin Piszcz
2007-04-04 14:12 ` Justin Piszcz
2007-04-04 14:13 ` Justin Piszcz
2007-04-04 18:36 ` Justin Piszcz
2 siblings, 2 replies; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 13:57 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2448 bytes --]
On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> Hi Justin,
>
> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>> Please see below for "time" output.
>>>
>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>> /dev/null 2>&1; done
>>> |
>>> | real 6m6.814s
>>> | user 0m30.290s
>>> | sys 2m42.562s
>>> | # time rm -rf y
>>> |
>>> | real 5m18.034s
>>> | user 0m0.036s
>>> | sys 0m8.169s
>
>> Deletes on XFS is one area that is a little slower than other filesystems.
>> You can increase the log size during the creation of the filesystem and
>> also increase logbufs to 8 and that might help.
>
> Thanks for your suggestions.
>
> I also tried to increase the log size and logbufs mount option. This
> optimizes create and delete times to the above values (with default options
> both are around 9-10 minutes).
>
> The strange thing is that on a similar Dell machines using XFS, too,
> deletes take only ten seconds which would match user and system time.
>
> More than five minutes for deleting 100000 files where ext3 needs
> 3 seconds on the same machine is actually more than a little bit slower
> - to my mind there must be something wrong. JFS needs around 18 seconds.
>
> However I am not sure if the problem is hardware or software related.
> I've also tried to use the newest 3ware firmware - but this did not lead
> to an improvement.
>
> Ciao,
> Thomas
> --
> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
> Technik CISCO Systems Partner - Authorized Reseller
> Im Süsterfeld 6 Tel 0241/701333-18
> tk@westend.com D-52072 Aachen Fax 0241/911879
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>
>
The benchmark:
$ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>/dev/null 2>&1; done
1. Six 400GB SATA drives using SW RAID5:
real 6m24.411s
user 0m43.097s
sys 2m17.350s
2. Four Raptor 150 ADFD drives using SW RAID5:
real 3m16.962s
user 0m42.899s
sys 2m15.420s
3. Two Raptor 74GB *GD drives using SW RAID1:
real 3m19.241s
user 0m41.731s
sys 2m15.873s
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 13:57 ` Justin Piszcz
@ 2007-04-04 14:12 ` Justin Piszcz
2007-04-04 14:21 ` Thomas Kaehn
2007-04-04 14:13 ` Justin Piszcz
1 sibling, 1 reply; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 14:12 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2965 bytes --]
On Wed, 4 Apr 2007, Justin Piszcz wrote:
>
>
> On Wed, 4 Apr 2007, Thomas Kaehn wrote:
>
>> Hi Justin,
>>
>> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>>> Please see below for "time" output.
>>>>
>>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>>> /dev/null 2>&1; done
>>>> |
>>>> | real 6m6.814s
>>>> | user 0m30.290s
>>>> | sys 2m42.562s
>>>> | # time rm -rf y
>>>> |
>>>> | real 5m18.034s
>>>> | user 0m0.036s
>>>> | sys 0m8.169s
>>
>>> Deletes on XFS is one area that is a little slower than other filesystems.
>>> You can increase the log size during the creation of the filesystem and
>>> also increase logbufs to 8 and that might help.
>>
>> Thanks for your suggestions.
>>
>> I also tried to increase the log size and logbufs mount option. This
>> optimizes create and delete times to the above values (with default options
>> both are around 9-10 minutes).
>>
>> The strange thing is that on a similar Dell machines using XFS, too,
>> deletes take only ten seconds which would match user and system time.
>>
>> More than five minutes for deleting 100000 files where ext3 needs
>> 3 seconds on the same machine is actually more than a little bit slower
>> - to my mind there must be something wrong. JFS needs around 18 seconds.
>>
>> However I am not sure if the problem is hardware or software related.
>> I've also tried to use the newest 3ware firmware - but this did not lead
>> to an improvement.
>>
>> Ciao,
>> Thomas
>> --
>> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
>> Technik CISCO Systems Partner - Authorized Reseller
>> Im Süsterfeld 6 Tel 0241/701333-18
>> tk@westend.com D-52072 Aachen Fax 0241/911879
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
>> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>>
>>
>
> The benchmark:
> $ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>> /dev/null 2>&1; done
>
> 1. Six 400GB SATA drives using SW RAID5:
> real 6m24.411s
> user 0m43.097s
> sys 2m17.350s
>
> 2. Four Raptor 150 ADFD drives using SW RAID5:
> real 3m16.962s
> user 0m42.899s
> sys 2m15.420s
>
> 3. Two Raptor 74GB *GD drives using SW RAID1:
> real 3m19.241s
> user 0m41.731s
> sys 2m15.873s
>
>
The removals:
The benchmark:
$ time rm -rf test
1. Six 400GB SATA drives using SW RAID5:
real 0m33.996s
user 0m0.057s
sys 0m8.101s
2. Four Raptor 150 ADFD drives using SW RAID5:
real 0m43.967s
user 0m0.071s
sys 0m8.340s
3. Two Raptor 74GB *GD drives using SW RAID1:
real 0m32.965s
user 0m0.049s
sys 0m6.307s
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 14:12 ` Justin Piszcz
@ 2007-04-04 14:21 ` Thomas Kaehn
2007-04-04 14:24 ` Justin Piszcz
0 siblings, 1 reply; 24+ messages in thread
From: Thomas Kaehn @ 2007-04-04 14:21 UTC (permalink / raw)
To: Justin Piszcz; +Cc: xfs
Hi Justin,
On Wed, Apr 04, 2007 at 10:12:48AM -0400, Justin Piszcz wrote:
> On Wed, 4 Apr 2007, Justin Piszcz wrote:
> >On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> >$ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
> >>/dev/null 2>&1; done
> >
> >1. Six 400GB SATA drives using SW RAID5:
> >real 6m24.411s
> >user 0m43.097s
> >sys 2m17.350s
> >
>
> The removals:
> The benchmark:
> $ time rm -rf test
>
> 1. Six 400GB SATA drives using SW RAID5:
> real 0m33.996s
> user 0m0.057s
> sys 0m8.101s
thanks for your bechmark. To my mind this clearly shows that my
setup is wrong at some point. I'll try again with your mount options.
Ciao,
Thomas
--
Thomas Kähn WESTEND GmbH | Internet-Business-Provider
Technik CISCO Systems Partner - Authorized Reseller
Im Süsterfeld 6 Tel 0241/701333-18
tk@westend.com D-52072 Aachen Fax 0241/911879
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 14:21 ` Thomas Kaehn
@ 2007-04-04 14:24 ` Justin Piszcz
2007-04-04 14:35 ` Thomas Kaehn
0 siblings, 1 reply; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 14:24 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1588 bytes --]
On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> Hi Justin,
>
> On Wed, Apr 04, 2007 at 10:12:48AM -0400, Justin Piszcz wrote:
>> On Wed, 4 Apr 2007, Justin Piszcz wrote:
>>> On Wed, 4 Apr 2007, Thomas Kaehn wrote:
>>> $ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>> /dev/null 2>&1; done
>>>
>>> 1. Six 400GB SATA drives using SW RAID5:
>>> real 6m24.411s
>>> user 0m43.097s
>>> sys 2m17.350s
>>>
>>
>> The removals:
>> The benchmark:
>> $ time rm -rf test
>>
>> 1. Six 400GB SATA drives using SW RAID5:
>> real 0m33.996s
>> user 0m0.057s
>> sys 0m8.101s
>
> thanks for your bechmark. To my mind this clearly shows that my
> setup is wrong at some point. I'll try again with your mount options.
>
> Ciao,
> Thomas
> --
> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
> Technik CISCO Systems Partner - Authorized Reseller
> Im Süsterfeld 6 Tel 0241/701333-18
> tk@westend.com D-52072 Aachen Fax 0241/911879
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>
>
My guess is mkfs.xfs cannot optimzie for your array like it can with a SW
RAID device because it cannot see what is undereath it. Have you tried
making a SW RAID? I also use optimized parameters for my SW RAID1/5 as
well FYI.
Justin.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 14:24 ` Justin Piszcz
@ 2007-04-04 14:35 ` Thomas Kaehn
2007-04-04 20:45 ` Justin Piszcz
0 siblings, 1 reply; 24+ messages in thread
From: Thomas Kaehn @ 2007-04-04 14:35 UTC (permalink / raw)
To: Justin Piszcz; +Cc: xfs
Hi Justin,
On Wed, Apr 04, 2007 at 10:24:42AM -0400, Justin Piszcz wrote:
> >On Wed, Apr 04, 2007 at 10:12:48AM -0400, Justin Piszcz wrote:
> >>On Wed, 4 Apr 2007, Justin Piszcz wrote:
> >>>On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> >>>$ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
> >>>>/dev/null 2>&1; done
> >>>
> My guess is mkfs.xfs cannot optimzie for your array like it can with a SW
> RAID device because it cannot see what is undereath it. Have you tried
> making a SW RAID? I also use optimized parameters for my SW RAID1/5 as
> well FYI.
I guess this might be the problem. I've already tried to alter
the stripe unit to match the RAID stripe size: "-d su=64k,sw=2 -l su=64k".
Maybe the 3ware controller can't deal with the kind of read and write
patterns needed by XFS. But in this case other people should have
realized similar problems.
On a different system with a 3ware 9500S-4LP using 4 disks as RAID5
setup I get a better (but not really good) result for delete
performance (I've taken only 50000 files in this case as the system's
CPU is much slower):
| # time for i in `seq 1 50000`; do dd if=/dev/zero of=$i
| bs=1k count=20 >/dev/null 2>&1; done
|
| real 18m21.643s
| user 0m55.727s
| sys 3m12.140s
| backup:/srv/x# cd ..
| backup:/srv# rm -rf x
|
| # time rm -rf x
|
| real 5m7.845s
| user 0m0.160s
| sys 0m11.369s
Ciao,
Thomas
--
Thomas Kähn WESTEND GmbH | Internet-Business-Provider
Technik CISCO Systems Partner - Authorized Reseller
Im Süsterfeld 6 Tel 0241/701333-18
tk@westend.com D-52072 Aachen Fax 0241/911879
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 14:35 ` Thomas Kaehn
@ 2007-04-04 20:45 ` Justin Piszcz
0 siblings, 0 replies; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 20:45 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2183 bytes --]
On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> Hi Justin,
>
> On Wed, Apr 04, 2007 at 10:24:42AM -0400, Justin Piszcz wrote:
>>> On Wed, Apr 04, 2007 at 10:12:48AM -0400, Justin Piszcz wrote:
>>>> On Wed, 4 Apr 2007, Justin Piszcz wrote:
>>>>> On Wed, 4 Apr 2007, Thomas Kaehn wrote:
>>>>> $ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>>>> /dev/null 2>&1; done
>>>>>
>> My guess is mkfs.xfs cannot optimzie for your array like it can with a SW
>> RAID device because it cannot see what is undereath it. Have you tried
>> making a SW RAID? I also use optimized parameters for my SW RAID1/5 as
>> well FYI.
>
> I guess this might be the problem. I've already tried to alter
> the stripe unit to match the RAID stripe size: "-d su=64k,sw=2 -l su=64k".
>
> Maybe the 3ware controller can't deal with the kind of read and write
> patterns needed by XFS. But in this case other people should have
> realized similar problems.
>
> On a different system with a 3ware 9500S-4LP using 4 disks as RAID5
> setup I get a better (but not really good) result for delete
> performance (I've taken only 50000 files in this case as the system's
> CPU is much slower):
>
> | # time for i in `seq 1 50000`; do dd if=/dev/zero of=$i
> | bs=1k count=20 >/dev/null 2>&1; done
> |
> | real 18m21.643s
> | user 0m55.727s
> | sys 3m12.140s
> | backup:/srv/x# cd ..
> | backup:/srv# rm -rf x
> |
> | # time rm -rf x
> |
> | real 5m7.845s
> | user 0m0.160s
> | sys 0m11.369s
>
>
> Ciao,
> Thomas
> --
> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
> Technik CISCO Systems Partner - Authorized Reseller
> Im Süsterfeld 6 Tel 0241/701333-18
> tk@westend.com D-52072 Aachen Fax 0241/911879
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>
What do you get with ext3 when using time bash -c 'rm -f file; sync'
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 13:57 ` Justin Piszcz
2007-04-04 14:12 ` Justin Piszcz
@ 2007-04-04 14:13 ` Justin Piszcz
2007-04-05 8:17 ` Thomas Kaehn
1 sibling, 1 reply; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 14:13 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2968 bytes --]
On Wed, 4 Apr 2007, Justin Piszcz wrote:
>
>
> On Wed, 4 Apr 2007, Thomas Kaehn wrote:
>
>> Hi Justin,
>>
>> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>>> Please see below for "time" output.
>>>>
>>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>>> /dev/null 2>&1; done
>>>> |
>>>> | real 6m6.814s
>>>> | user 0m30.290s
>>>> | sys 2m42.562s
>>>> | # time rm -rf y
>>>> |
>>>> | real 5m18.034s
>>>> | user 0m0.036s
>>>> | sys 0m8.169s
>>
>>> Deletes on XFS is one area that is a little slower than other filesystems.
>>> You can increase the log size during the creation of the filesystem and
>>> also increase logbufs to 8 and that might help.
>>
>> Thanks for your suggestions.
>>
>> I also tried to increase the log size and logbufs mount option. This
>> optimizes create and delete times to the above values (with default options
>> both are around 9-10 minutes).
>>
>> The strange thing is that on a similar Dell machines using XFS, too,
>> deletes take only ten seconds which would match user and system time.
>>
>> More than five minutes for deleting 100000 files where ext3 needs
>> 3 seconds on the same machine is actually more than a little bit slower
>> - to my mind there must be something wrong. JFS needs around 18 seconds.
>>
>> However I am not sure if the problem is hardware or software related.
>> I've also tried to use the newest 3ware firmware - but this did not lead
>> to an improvement.
>>
>> Ciao,
>> Thomas
>> --
>> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
>> Technik CISCO Systems Partner - Authorized Reseller
>> Im Süsterfeld 6 Tel 0241/701333-18
>> tk@westend.com D-52072 Aachen Fax 0241/911879
>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
>> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>>
>>
>
> The benchmark:
> $ time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>> /dev/null 2>&1; done
>
> 1. Six 400GB SATA drives using SW RAID5:
> real 6m24.411s
> user 0m43.097s
> sys 2m17.350s
>
> 2. Four Raptor 150 ADFD drives using SW RAID5:
> real 3m16.962s
> user 0m42.899s
> sys 2m15.420s
>
> 3. Two Raptor 74GB *GD drives using SW RAID1:
> real 3m19.241s
> user 0m41.731s
> sys 2m15.873s
>
>
I used the DEFAULT create options for XFS as I find it highly optimizes
itself (at least with SW raid) with the exception of the ROOT FS, I had
that optimized awhile ago and I kept it:
/dev/md2 / xfs
logbufs=8,logbsize=262144,biosize=16,noatime,nodiratime,nobarrier 0 1
For my regular RAID5s though I use defaults,noatime.
Justin.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 14:13 ` Justin Piszcz
@ 2007-04-05 8:17 ` Thomas Kaehn
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Kaehn @ 2007-04-05 8:17 UTC (permalink / raw)
To: Justin Piszcz; +Cc: xfs
Hi Justin,
On Wed, Apr 04, 2007 at 10:13:36AM -0400, Justin Piszcz wrote:
> I used the DEFAULT create options for XFS as I find it highly optimizes
> itself (at least with SW raid) with the exception of the ROOT FS, I had
> that optimized awhile ago and I kept it:
>
> /dev/md2 / xfs
> logbufs=8,logbsize=262144,biosize=16,noatime,nodiratime,nobarrier 0 1
>
>
> For my regular RAID5s though I use defaults,noatime.
I've disabled barriers, too, and performance increased dramatically.
However I am not aware of the consequences of disabling write barriers.
The FAQ generally recommends using write barriers except when having
a battery-backed cache (this 3ware has not).
| # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20 >/dev/null 2>&1; done
|
| real 3m52.182s
| user 0m30.482s
| sys 3m16.152s
|
| # time \rm -rf y
|
| real 0m16.327s
| user 0m0.052s
| sys 0m8.305s
So I am unsure if disabling is an option for me. I could imagine
that write barriers are not properly supported by 3ware or have
to be fine tuned at the kernel or SCSI level.
Ciao,
Thomas
--
Thomas Kähn WESTEND GmbH | Internet-Business-Provider
Technik CISCO Systems Partner - Authorized Reseller
Im Süsterfeld 6 Tel 0241/701333-18
tk@westend.com D-52072 Aachen Fax 0241/911879
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 13:47 ` Thomas Kaehn
2007-04-04 13:51 ` Justin Piszcz
2007-04-04 13:57 ` Justin Piszcz
@ 2007-04-04 18:36 ` Justin Piszcz
2007-04-05 7:37 ` Thomas Kaehn
2 siblings, 1 reply; 24+ messages in thread
From: Justin Piszcz @ 2007-04-04 18:36 UTC (permalink / raw)
To: Thomas Kaehn; +Cc: xfs
[-- Attachment #1: Type: TEXT/PLAIN, Size: 2068 bytes --]
On Wed, 4 Apr 2007, Thomas Kaehn wrote:
> Hi Justin,
>
> On Wed, Apr 04, 2007 at 09:29:46AM -0400, Justin Piszcz wrote:
>>> Please see below for "time" output.
>>>
>>> | # time for i in `seq 1 100000`; do dd if=/dev/zero of=$i bs=1k count=20
>>>> /dev/null 2>&1; done
>>> |
>>> | real 6m6.814s
>>> | user 0m30.290s
>>> | sys 2m42.562s
>>> | # time rm -rf y
>>> |
>>> | real 5m18.034s
>>> | user 0m0.036s
>>> | sys 0m8.169s
>
>> Deletes on XFS is one area that is a little slower than other filesystems.
>> You can increase the log size during the creation of the filesystem and
>> also increase logbufs to 8 and that might help.
>
> Thanks for your suggestions.
>
> I also tried to increase the log size and logbufs mount option. This
> optimizes create and delete times to the above values (with default options
> both are around 9-10 minutes).
>
> The strange thing is that on a similar Dell machines using XFS, too,
> deletes take only ten seconds which would match user and system time.
>
> More than five minutes for deleting 100000 files where ext3 needs
> 3 seconds on the same machine is actually more than a little bit slower
> - to my mind there must be something wrong. JFS needs around 18 seconds.
>
> However I am not sure if the problem is hardware or software related.
> I've also tried to use the newest 3ware firmware - but this did not lead
> to an improvement.
>
> Ciao,
> Thomas
> --
> Thomas Kähn WESTEND GmbH | Internet-Business-Provider
> Technik CISCO Systems Partner - Authorized Reseller
> Im Süsterfeld 6 Tel 0241/701333-18
> tk@westend.com D-52072 Aachen Fax 0241/911879
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
> Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
>
For the ext3, try time bash -c 'rm -rf test; sync'
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: Strange delete performance using XFS
2007-04-04 18:36 ` Justin Piszcz
@ 2007-04-05 7:37 ` Thomas Kaehn
0 siblings, 0 replies; 24+ messages in thread
From: Thomas Kaehn @ 2007-04-05 7:37 UTC (permalink / raw)
To: Justin Piszcz; +Cc: xfs
Hi Justin,
On Wed, Apr 04, 2007 at 02:36:37PM -0400, Justin Piszcz wrote:
> For the ext3, try time bash -c 'rm -rf test; sync'
# time bash -c 'rm -rf y; sync'
real 0m1.592s
user 0m0.032s
sys 0m1.408s
Ciao,
Thomas
--
Thomas Kähn WESTEND GmbH | Internet-Business-Provider
Technik CISCO Systems Partner - Authorized Reseller
Im Süsterfeld 6 Tel 0241/701333-18
tk@westend.com D-52072 Aachen Fax 0241/911879
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Die Gesellschaft ist eingetragen im Handelsregister Aachen unter HRB 7608
Geschäftsführer: Thomas Neugebauer, Thomas Heller, Michael Kolb
^ permalink raw reply [flat|nested] 24+ messages in thread