Re: SATA-performance: Linux vs. FreeBSD

[Andi Kleen <andi@firstfloor.org>]

"Martin A. Fink" <fink@mpe.mpg.de> writes:

Your mailer seems to be broken. It drops cc.
> 
> If you call fsync in BSD then you get what you expect. anything that is still 
> not on disk will be written. Afterwards fsync returns... So this should be 
> the same like with linux?!

Not necessarily.  The disk may buffer additionally. Handling that
differs widely, but modern Linux forces flushes to platter if the hardware support 
it.

> But the big question still is -- buffered or not -- where do the big 
> variations within linux come frome? I am not writing small blocks. I write 
> huge amounts of data.

1MB is nowhere near huge by modern standards. Many IO subsystems are
only happy with multi MB requests. 

> So the buffer will always be full.

Hardly. Especially not if you do synchronous fsync inbetween.

> If I use a normal SATA-II disk, there are no differences between BSD and Linux 
> when writing to the raw device... So it cant be a buffer-problem alone.

Yes that is something that needs to be investigated. That is why I suggested
oprofile if your assertation of a more CPU overhead on Linux is true.

> I still don't understand the buffer argument. If one writes 25 GB in blocks of 
> 1 MB your buffer should be always full...

Your mental model of a IO subsystem seems to be quite off.
Think what happens when you fsync and submit synchronously.

It's like sending something down a long pipe and waiting until it arrives
at the bottom and you hear the echo of the impact. Then only then you send again. 
There will be always long periods when the pipe will be empty.

If you use large enough blocks these gaps will be quite small and
might effectively become unimportant, but 1MB is nowhere near big enough 
for that.

> Is there a buffered io device that I can use, but that does not use a 
> filesystem?

/dev/sdX*. However it has some other issues that also don't make
it ideal. File systems are usually best.

-Andi