From: Nick Piggin <piggin@cyberone.com.au>
To: Kevin Jacobs <jacobs@penguin.theopalgroup.com>
Cc: akpm@digeo.com, linux-kernel@vger.kernel.org
Subject: Re: Ext3 meta-data performance
Date: Sat, 31 May 2003 01:04:21 +1000 [thread overview]
Message-ID: <3ED772F5.8060100@cyberone.com.au> (raw)
In-Reply-To: <Pine.LNX.4.44.0305290923330.11990-100000@penguin.theopalgroup.com>
Kevin Jacobs wrote:
>On Thu, 29 May 2003, Nick Piggin wrote:
>
>>Kevin Jacobs wrote:
>>
>>>[...]
>>>Since these rsync backups are done in addition to traditional daily tape
>>>backups, we've taken the system out of production use and opened the door
>>>for experimentation. So, the next logical step was to try a 2.5 kernel.
>>>After some work, I've gotten 2.5.70-mm2 booting and it is _much_ better than
>>>the Redhat 2.4 kernels, and the system interactivity is flawless. However,
>>>the speed of creating hard-links is still three and a half times slower than
>>>with the old 2.2 kernel. It now takes ~14 minutes to create the links, and
>>>
>>>from what I can tell, the bottlenecks is not the CPU or the disk-throughput.
>>
>>Its probably seek bound.
>>Provide some more information about your disk/partition setup, and external
>>journals, and data= mode. Remember ext3 will generally always have to do
>>more work than ext2.
>>
>
> SCSI ID 1 3ware 7500-8 ATA RAID Controller
>
> * Array Unit 0 Mirror (RAID 1) 40.01 GB OK
> + Port 0 WDC WD400BB-00DEA0 40.02 GB OK
> + Port 1 WDC WD400BB-00DEA0 40.02 GB OK
> * Array Unit 4 Striped with Parity 64K (RAID 5) 555.84 GB OK
> + Port 4 IC35L180AVV207-1 185.28 GB OK
> + Port 5 IC35L180AVV207-1 185.28 GB OK
> + Port 6 IC35L180AVV207-1 185.28 GB OK
> + Port 7 IC35L180AVV207-1 185.28 GB OK
>
>Disk /dev/sda: 40.0 GB, 40019615744 bytes
>255 heads, 63 sectors/track, 4865 cylinders
>Units = cylinders of 16065 * 512 = 8225280 bytes
>
> Device Boot Start End Blocks Id System
>/dev/sda1 * 1 261 2096451 83 Linux
>/dev/sda2 262 1566 10482412+ 83 Linux
>/dev/sda3 1567 4570 24129630 83 Linux
>/dev/sda4 4571 4865 2369587+ f Win95 Ext'd (LBA)
>/dev/sda5 4571 4589 152586 83 Linux
>/dev/sda6 4590 4734 1164681 83 Linux
>/dev/sda7 4735 4865 1052226 83 Linux
>
>Disk /dev/sdb: 555.8 GB, 555847581696 bytes
>255 heads, 63 sectors/track, 67577 cylinders
>Units = cylinders of 16065 * 512 = 8225280 bytes
>
> Device Boot Start End Blocks Id System
>/dev/sdb1 * 1 67577 542812221 83 Linux
>
>Unit 0 is /dev/sda and the journal is /dev/sda5. Unit 1 is /dev/sdb and the
>backup filesystem is /dev/sdb1. The data= mode is whatever is default,
>/dev/sdb1 is mounted noatime. I've also applied the journal_refile_buffer
>patch posted by AKPM yesterday morning.
>
I think you should have your journal on its own spinde if
you are going to the trouble of having an external one.
>
>>If you want to play with the scheduler, try set
>>/sys/block/blockdev*/queue/nr_requests = 8192
>>
>
>This killed the entire system -- livelocking it with no disk activity to the
>point that I had to hit the reset button. So does setting nr_requests on
>sda and sdb from 128 to 256. The problems hit before the rsync, during a
>'rm -Rf' on a previously copied tree.
>
OK I'll have a look into that.
>
>>then try
>>/sys/block/blockdev*/queue/iosched/antic_expire = 0
>>
>
>This seemed to make no difference.
>
Thats alright then.
>
>>Try the above combinations with and without a big TCQ depth. You should
>>be able to set them on the fly and see what happens to throughput during
>>the operation. Let me know what you see.
>>
>
>I'm not sure how to change TCQ depth on the fly. Last I knew, it was a
>compiled-in parameter.
>
Don't worry too much about this. Its probably not a big issue.
>
>I have some more time to experiment, so please let me know if there is
>anything else you think I should try.
>
Andrew might be able to suggest some worthwhile tests, if nothing
else, try mounting your filesystems as ext2, so you can get a
baseline figure.
next prev parent reply other threads:[~2003-05-30 14:51 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-05-29 12:49 Ext3 meta-data performance Kevin Jacobs
2003-05-29 13:04 ` Nick Piggin
2003-05-30 10:09 ` Kevin Jacobs
2003-05-30 15:04 ` Nick Piggin [this message]
2003-05-30 16:21 ` Andrew Morton
2003-05-30 17:44 ` Andreas Dilger
2003-06-04 16:57 ` Petro
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3ED772F5.8060100@cyberone.com.au \
--to=piggin@cyberone.com.au \
--cc=akpm@digeo.com \
--cc=jacobs@penguin.theopalgroup.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).