Re: Optimal SCSI Commands per device (TCQ) ?

[Helge Hafting <helgehaf@aitel.hist.no>]

Margit Schubert-While wrote:
> 
> What is the optimal TCQ for eg. AIC7xxxx on Adaptec 39160
> with 2 x U160 disks on each channel ? (MAN3184MP and
> DDYS-T18350N)
> What does it depend on ?

The optimal depends on what you do, so the best
way to get a good answer is to try several
settings and see what works best.

Generally, a deeper tagged queue gives
better throughput.  The difference
is biggest when going from 0 to 1.
Making the queues progressively deeper
gives even more throughput, but the 
improvement diminishes quickly.

You have to look up how deep queues your disks
support, because this is variable among 
different models.  Some might max out at 8,
others can handle 256. There seems to be little
benefit at all beyond 32.

There is currently a downside to deep queues.
Linux v. 2.5 has a fair io scheduler that ensures
that no process waits "too long" for its
disk accesses.  This is particularly
important for reads.  A deep tagged
queue may improve total throughput, but
this may defeat the io scheduler so
processes doing random reads get starved
waiting for processes doing large linear
reads or writes.  

If this don't bother you, go for a deep
queue like 32 perhaps. If you want fairness
(users wait for response, i.e. desktop
machine or file/web server for multiple
people) go for something more shallow.

I have seen recommendations as low as 2-4 tags
to let the io scheduler enforce fairness.
2-4 tags is a lot better than none, going
from 4 to 32 probably makes less difference
than from 0 to 4. 

For an example of what a fair scheduler does,
look at previously posted benchmarks with
kernel compiles during a dbench run on the same disk.
Without fairness the compile takes many times longer.
with fairness the compile time is almost the same
as when compiling without the additional burden of a dbench.

Helge Hafting