From: "Zhang, Yanmin" <yanmin_zhang@linux.intel.com>
To: czoccolo@gmail.com, Jens Axboe <jens.axboe@oracle.com>
Cc: Shaohua Li <shaohua.li@intel.com>,
"jmoyer@redhat.com" <jmoyer@redhat.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: fio mmap randread 64k more than 40% regression with 2.6.33-rc1
Date: Thu, 31 Dec 2009 17:16:00 +0800 [thread overview]
Message-ID: <1262250960.1819.68.camel@localhost> (raw)
Comparing with kernel 2.6.32, fio mmap randread 64k has more than 40% regression with
2.6.33-rc1.
The test scenario: 1 JBOD has 12 disks and every disk has 2 partitions. Create
8 1-GB files per partition and start 8 processes to do rand read on the 8 files
per partitions. There are 8*24 processes totally. randread block size is 64K.
We found the regression on 2 machines. One machine has 8GB memory and the other has
6GB.
Bisect is very unstable. The related patches are many instead of just one.
1) commit 8e550632cccae34e265cb066691945515eaa7fb5
Author: Corrado Zoccolo <czoccolo@gmail.com>
Date: Thu Nov 26 10:02:58 2009 +0100
cfq-iosched: fix corner cases in idling logic
This patch introduces about less than 20% regression. I just reverted below section
and this part regression disappear. It shows this regression is stable and not impacted
by other patches.
@@ -1253,9 +1254,9 @@ static void cfq_arm_slice_timer(struct cfq_data *cfqd)
return;
/*
- * still requests with the driver, don't idle
+ * still active requests from this queue, don't idle
*/
- if (rq_in_driver(cfqd))
+ if (cfqq->dispatched)
return;
2) How about other 20%~30% regressions? It's complicated. My bisect plus
Li Shaohua's investigation located 3 patches,
df5fe3e8e13883f58dc97489076bbcc150789a21,
b3b6d0408c953524f979468562e7e210d8634150,
5db5d64277bf390056b1a87d0bb288c8b8553f96.
tiobench also has regression and Li Shaohua located the same patches. See link
http://lkml.indiana.edu/hypermail/linux/kernel/0912.2/03355.html.
Shaohua worked about patches to fix the tiobench regression. However, his patch
doesn't work for fio randread 64k regression.
I retried bisect manually and eventually located below patch,
commit 718eee0579b802aabe3bafacf09d0a9b0830f1dd
Author: Corrado Zoccolo <czoccolo@gmail.com>
Date: Mon Oct 26 22:45:29 2009 +0100
cfq-iosched: fairness for sync no-idle queues
The patch is a little big. After many try, I found below section is the key.
@@ -2218,13 +2352,10 @@ cfq_update_idle_window(struct cfq_data *cfqd, struct cfq_queue *cfqq,
enable_idle = old_idle = cfq_cfqq_idle_window(cfqq);
if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
- (!cfqd->cfq_latency && cfqd->hw_tag && CFQQ_SEEKY(cfqq)))
+ (sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
enable_idle = 0;
That section deletes the condition checking of !cfqd->cfq_latency, so
enable_idle=0 with more possibility.
I wrote a testing patch which just overlooks the original 3 patches related
to tiobench regression, and a patch which adds back the checking of !cfqd->cfq_latency.
Then, all regression of fio randread 64k disappears.
Then, instead of working around the original 3 patches, I applied Shaohua's 2 patches
and added the checking of !cfqd->cfq_latency while also reverting the patch mentioned in 1).
But the result still has more than 20% regression. So Shaohua's patches couldn't improve
fio rand read 64k regression.
fio_mmap_randread_4k has about 10% improvement instead of regression. I checked
that my patch plus the debugging patch have no impact on this improvement.
randwrite 64k has about 25% regression. My method also restores its performance.
I worked out a patch to add the checking of !cfqd->cfq_latency back in
function cfq_update_idle_window.
In addition, as for item 1), could we just revert the section in cfq_arm_slice_timer?
As Shaohua's patches don't work for this regression, we might continue to find
better methods. I will check it next week.
---
With kernel 2.6.33-rc1, fio rand read 64k has more than 40% regression. Located
below patch.
commit 718eee0579b802aabe3bafacf09d0a9b0830f1dd
Author: Corrado Zoccolo <czoccolo@gmail.com>
Date: Mon Oct 26 22:45:29 2009 +0100
cfq-iosched: fairness for sync no-idle queues
It introduces for more than 20% regression. The reason is function cfq_update_idle_window
forgets to check cfqd->cfq_latency, so enable_idle=0 with more possibility.
Below patch against 2.6.33-rc1 adds the checking back.
Signed-off-by: Zhang Yanmin <yanmin_zhang@linux.intel.com>
---
diff -Nraup linux-2.6.33_rc1/block/cfq-iosched.c linux-2.6.33_rc1_rand64k/block/cfq-iosched.c
--- linux-2.6.33_rc1/block/cfq-iosched.c 2009-12-23 14:12:03.000000000 +0800
+++ linux-2.6.33_rc1_rand64k/block/cfq-iosched.c 2009-12-31 16:26:32.000000000 +0800
@@ -3064,8 +3064,8 @@ cfq_update_idle_window(struct cfq_data *
cfq_mark_cfqq_deep(cfqq);
if (!atomic_read(&cic->ioc->nr_tasks) || !cfqd->cfq_slice_idle ||
- (!cfq_cfqq_deep(cfqq) && sample_valid(cfqq->seek_samples)
- && CFQQ_SEEKY(cfqq)))
+ (!cfqd->cfq_latency && !cfq_cfqq_deep(cfqq) &&
+ sample_valid(cfqq->seek_samples) && CFQQ_SEEKY(cfqq)))
enable_idle = 0;
else if (sample_valid(cic->ttime_samples)) {
if (cic->ttime_mean > cfqd->cfq_slice_idle)
next reply other threads:[~2009-12-31 9:15 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-12-31 9:16 Zhang, Yanmin [this message]
2009-12-31 10:34 ` fio mmap randread 64k more than 40% regression with 2.6.33-rc1 Corrado Zoccolo
2010-01-01 10:12 ` Zhang, Yanmin
2010-01-01 16:32 ` Corrado Zoccolo
2010-01-02 12:33 ` Zhang, Yanmin
2010-01-02 18:52 ` Corrado Zoccolo
2010-01-04 8:18 ` Zhang, Yanmin
2010-01-04 18:28 ` Corrado Zoccolo
2010-01-16 16:27 ` Corrado Zoccolo
2010-01-18 3:06 ` Zhang, Yanmin
2010-01-19 20:10 ` Corrado Zoccolo
2010-01-19 20:42 ` Jeff Moyer
2010-01-19 21:40 ` Vivek Goyal
2010-01-19 21:58 ` Corrado Zoccolo
2010-01-20 19:18 ` Vivek Goyal
2010-01-20 1:29 ` Shaohua Li
2010-01-20 14:00 ` Jeff Moyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1262250960.1819.68.camel@localhost \
--to=yanmin_zhang@linux.intel.com \
--cc=czoccolo@gmail.com \
--cc=jens.axboe@oracle.com \
--cc=jmoyer@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=shaohua.li@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).