Re: high throughput storage server?

[Mattias Wadenstein <maswan@acc.umu.se>]

On Fri, 18 Feb 2011, Stan Hoeppner wrote:

> Mattias Wadenstein put forth on 2/18/2011 7:49 AM:
>
>> Here you would either maintain a large list of nfs mounts for the read
>> load, or start looking at a distributed filesystem. Sticking them all
>> into one big fileserver is easier on the administration part, but
>> quickly gets really expensive when you look to put multiple 10GE
>> interfaces on it.
>
> This really depends on one's definition of "really expensive".  Taking
> the total cost of such a system/infrastructure into account, these two
> Intel dual port 10 GbE NICs seem rather cheap at $650-$750 USD:
>
> http://www.newegg.com/Product/Product.aspx?Item=N82E16833106037
> http://www.newegg.com/Product/Product.aspx?Item=N82E16833106075
>
> 20 Gb/s (40 both ways) raw/peak throughput at this price seems like a
> bargain to me (plus the switch module cost obviously, if required,
> usually not for RJ-45 or CX4, thus my motivation for mentioning these).
>
> The storage infrastructure on the back end required to keep these pipes
> full will be the "really expensive" piece.

Exactly my point, a storage server that can sustain 20-200MB/s is rather 
cheap, but one that can sustain 2GB/s is really expensive. Possibly to the 
point where 10-100 smaller file servers are much cheaper. The worst case 
here is very small random reads, and then you're screwed cost-wise 
whatever you choose, if you want to get the 2GB/s number.

[snip]

> RAID 5/6 need not apply due the abysmal RMW partial stripe write
> penalty, unless of course you're doing almost no writes.  But in that
> case, how did the data get there in the first place? :)

Actually, that's probably the common case for data analysis load. Lots of 
random reads, but only occasional sequential writes when you add a new 
file/fileset. So raid 5/6 performance-wise works out pretty much as a 
stripe of n-[12] disks.

/Mattias Wadenstein