Re: [PATCH] cfq-iosched: non-rot devices do not need read queue merging

From: Jeff Moyer <jmoyer@redhat.com>
To: Corrado Zoccolo <czoccolo@gmail.com>
Cc: Vivek Goyal <vgoyal@redhat.com>,
	Jens Axboe <jens.axboe@oracle.com>,
	Linux-Kernel <linux-kernel@vger.kernel.org>,
	Shaohua Li <shaohua.li@intel.com>,
	Gui Jianfeng <guijianfeng@cn.fujitsu.com>
Subject: Re: [PATCH] cfq-iosched: non-rot devices do not need read queue  merging
Date: Tue, 05 Jan 2010 09:58:52 -0500	[thread overview]
Message-ID: <x49zl4s8uoj.fsf@segfault.boston.devel.redhat.com> (raw)
In-Reply-To: <4e5e476b1001041237v71952c8ewaaef3778353f7521@mail.gmail.com> (Corrado Zoccolo's message of "Mon, 4 Jan 2010 21:37:05 +0100")

Corrado Zoccolo <czoccolo@gmail.com> writes:

> On Mon, Jan 4, 2010 at 8:04 PM, Jeff Moyer <jmoyer@redhat.com> wrote:
>> Vivek Goyal <vgoyal@redhat.com> writes:
>>>> >>> Hi Corrado,
>>>> >>>
>>>> >>> What's the reason that reads don't benefit from merging queues and hence
>>>> >>> merging requests and only writes do on SSD?
>>>> >>
>>>> >> On SSDs, reads are just limited by the maximum transfer rate, and
>>>> >> larger (i.e. merged) reads will just take proportionally longer.
>>>> >
>>>> > This is simply not true.  You can get more bandwidth from an SSD (I just
>>>> > checked numbers for 2 vendors' devices) by issuing larger read requests,
>>>> > no matter whether the access pattern is sequential or random.
>>>> I know, but the performance increase given the size is sublinear, and
>>>> the situation here is slightly different.
>>>> In order for the requests to be merged, they have to be submitted concurrently.
>>>> So you have to compare 2 concurrent requests of size x with one
>>>> request of size 2*x (with some CPU overhead).
>>>> Moreover, you always pay the CPU overhead, even if you can't do the
>>>> merging, and you must be very lucky to keep merging, because it means
>>>> the two processes are working in lockstep; it is not sufficient that
>>>> the requests are just nearby, as for rotational disks.
>>>>
>>>
>>> For jeff, at least "dump" utility threads were kind of working in lockstep
>>> for writes and he gained significantly by merging these queues together.
>>
>> Actually, it was for reads.
>>
>>> So the argument is that CPU overhead saving in this case is more substantial
>>> as compared to gains made by lockstep read threads. I think we shall have to
>>> have some numbers to justify that.
>>
>> Agreed.  Corrado, I know you don't have the hardware, so I'll give this
>> a run through the read-test2 program and see if it regresses at all.
> Great.

I ran the test program 50 times, and here are the results:

==> vanilla <==
Mean: 163.22728
Population Std. Dev.: 0.55401

==> patched <==
Mean: 162.91558
Population Std. Dev.: 1.08612

This looks acceptable to me.

Cheers,
Jeff