From mboxrd@z Thu Jan 1 00:00:00 1970 From: Liam Kurmos Subject: Re: mdadm raid1 read performance Date: Thu, 5 May 2011 09:39:55 +0100 Message-ID: References: <20110504105822.21e23bc3@notabene.brown> <4DC0F2B6.9050708@fnarfbargle.com> <20110505100610.66c93e08@natsu> <4DC25A71.6090705@oldum.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Return-path: In-Reply-To: <4DC25A71.6090705@oldum.net> Sender: linux-raid-owner@vger.kernel.org To: Nikolay Kichukov Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids On Thu, May 5, 2011 at 9:06 AM, Nikolay Kichukov wrote: > -----BE > It seems like he is reading directly from the raid device and not through the filesystem. So there are no filesystem > caches in this way. phew! ... i think (see below) I installed ubuntu 11.04 on the new system last night. this morning i went to reconnect the old system drive (id disconnected it for safety) and 'pop' a small piece of metal must have touched the back of the loose drive and fried the board! ... oh joy. --luckily its an old drive and i have an identical spare so im hoping i can swap the board and save all my work since last git commit. anyway.. this look different in 11.04. mdadm 3.1.4 /dev/md0: Timing buffered disk reads: 594 MB in 3.00 seconds = 197.79 MB/sec zoizoi@shankara:~$ sudo dd if=/dev/md0 of=/dev/null bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 5.84755 s, 179 MB/s zoizoi@shankara:~$ sudo dd if=/dev/md0 of=/dev/null bs=1M count=1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB) copied, 0.152353 s, 6.9 GB/s even though im accessing the block device directly it does look like im getting buffering in natty! I don't think i was in 10.10 and certainly wasnt getting 7GB/s raid10 f2 performance is right down vs what i got last night (i got 470MB/s first try after creating the array so dont think there was buffering. my md1 raid5 was also slow again. readahead on both defaulted down to 256 on 11.04 I applied Neil's x8 fix to both md0 and md1 and now the dd test look much better. sudo dd if=/dev/md0 of=/dev/null bs=1M count=4000 4000+0 records in 4000+0 records out 4194304000 bytes (4.2 GB) copied, 7.62018 s, 550 MB/s a little too good! i could see in the system monitor that i didnt have the large 4G buffer (i do post this test). Something i did must have reset the buffer. I could see a small amount of buffer in system monitor so maybe it was 1GB. I appreciate these are not the best test but not that hdparm is much worse. Once i set the 2048 sector readahead i gets totally unrealistic. zoizoi@shankara:~$ sudo blockdev --setra 2048 /dev/md0 zoizoi@shankara:~$ sudo hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 5294 MB in 3.00 seconds = 1762.34 MB/sec zoizoi@shankara:~$ sudo blockdev --setra 256 /dev/md0 zoizoi@shankara:~$ sudo hdparm -t /dev/md0 /dev/md0: Timing buffered disk reads: 582 MB in 3.00 seconds = 193.78 MB/sec anyway, it looks like i'm getting good read speed now with 2048 lookahead, ill do another dd test on reboot. thanks to you all for the helpful responses, Liam