From: Joseph Qi <joseph.qi@linux.alibaba.com>
To: Theodore Ts'o <tytso@mit.edu>, Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org,
Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Subject: [RFC] performance regression with "ext4: Allow parallel DIO reads"
Date: Fri, 19 Jul 2019 17:22:24 +0800 [thread overview]
Message-ID: <ab7cf51b-6b52-d151-e22c-6f4400a14589@linux.alibaba.com> (raw)
Hi Ted & Jan,
I've observed an significant performance regression with the following
commit in my Intel P3600 NVMe SSD.
16c54688592c ext4: Allow parallel DIO reads
From my initial investigation, it may be because of the
inode_lock_shared (down_read) consumes more than inode_lock (down_write)
in mixed random read write workload.
Here is my test result.
ioengine=psync
direct=1
rw=randrw
iodepth=1
numjobs=8
size=20G
runtime=600
w/ parallel dio reads : kernel 5.2.0
w/o parallel dio reads: kernel 5.2.0, then revert the following commits:
1d39834fba99 ext4: remove EXT4_STATE_DIOREAD_LOCK flag (related)
e5465795cac4 ext4: fix off-by-one error when writing back pages before dio read (related)
16c54688592c ext4: Allow parallel DIO reads
bs=4k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 30898KB/s, 7724, 555.00us | WRITE 30875KB/s, 7718, 479.70us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 117915KB/s, 29478, 248.18us | WRITE 117854KB/s，29463, 21.91us
-------------------------------------------------------------------------------------------
bs=16k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 58961KB/s, 3685, 835.28us | WRITE 58877KB/s, 3679, 1335.98us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 218409KB/s, 13650, 554.46us | WRITE 218257KB/s，13641, 29.22us
-------------------------------------------------------------------------------------------
bs=64k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 119396KB/s, 1865, 1759.38us | WRITE 119159KB/s, 1861, 2532.26us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 422815KB/s, 6606, 1146.05us | WRITE 421619KB/s, 6587, 60.72us
-------------------------------------------------------------------------------------------
bs=512k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 392973KB/s, 767, 5046.35us | WRITE 393165KB/s, 767, 5359.86us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 590266KB/s, 1152, 4312.01us | WRITE 590554KB/s, 1153, 2606.82us
-------------------------------------------------------------------------------------------
bs=1M:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 487779KB/s, 476, 8058.55us | WRITE 485592KB/s, 474, 8630.51us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 593927KB/s, 580, 7623.63us | WRITE 591265KB/s, 577, 6163.42us
-------------------------------------------------------------------------------------------
Thanks,
Joseph
next reply other threads:[~2019-07-19 9:22 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-19 9:22 Joseph Qi [this message]
2019-07-23 11:17 ` [RFC] performance regression with "ext4: Allow parallel DIO reads" Joseph Qi
2019-07-25 21:20 ` Andreas Dilger
2019-07-26 1:12 ` Joseph Qi
2019-07-27 1:57 ` Andreas Dilger
2019-07-27 2:16 ` Joseph Qi
2019-07-28 22:51 ` Dave Chinner
2019-07-30 1:34 ` Joseph Qi
2019-08-15 15:13 ` Jan Kara
2019-08-16 13:23 ` Joseph Qi
2019-08-16 14:57 ` Jan Kara
2019-08-20 3:00 ` Joseph Qi
2019-08-20 16:08 ` Theodore Y. Ts'o
2019-08-21 1:04 ` Joseph Qi
2019-08-21 3:34 ` Theodore Y. Ts'o
2019-08-22 6:45 ` Joseph Qi
2019-08-22 5:40 ` Dave Chinner
2019-08-23 7:57 ` Joseph Qi
2019-08-23 8:07 ` Joseph Qi
2019-08-23 10:16 ` Dave Chinner
2019-08-23 13:08 ` Joseph Qi
2019-08-24 2:18 ` Dave Chinner
2019-08-26 8:39 ` Jan Kara
2019-08-26 19:10 ` Andreas Dilger
2019-08-27 1:00 ` Joseph Qi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ab7cf51b-6b52-d151-e22c-6f4400a14589@linux.alibaba.com \
--to=joseph.qi@linux.alibaba.com \
--cc=jack@suse.cz \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=xiaoguang.wang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).