From: Joseph Qi <jiangqi903@gmail.com>
To: Joseph Qi <joseph.qi@linux.alibaba.com>,
Theodore Ts'o <tytso@mit.edu>, Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org,
Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>,
Liu Bo <bo.liu@linux.alibaba.com>
Subject: Re: [RFC] performance regression with "ext4: Allow parallel DIO reads"
Date: Tue, 23 Jul 2019 19:17:19 +0800 [thread overview]
Message-ID: <29d50d24-f8e7-5ef4-d4d8-3ea6fb1c6ed3@gmail.com> (raw)
In-Reply-To: <ab7cf51b-6b52-d151-e22c-6f4400a14589@linux.alibaba.com>
Hi Ted & Jan,
Could you please give your valuable comments?
Thanks,
Joseph
On 19/7/19 17:22, Joseph Qi wrote:
> Hi Ted & Jan,
> I've observed an significant performance regression with the following
> commit in my Intel P3600 NVMe SSD.
> 16c54688592c ext4: Allow parallel DIO reads
>
> From my initial investigation, it may be because of the
> inode_lock_shared (down_read) consumes more than inode_lock (down_write)
> in mixed random read write workload.
>
> Here is my test result.
>
> ioengine=psync
> direct=1
> rw=randrw
> iodepth=1
> numjobs=8
> size=20G
> runtime=600
>
> w/ parallel dio reads : kernel 5.2.0
> w/o parallel dio reads: kernel 5.2.0, then revert the following commits:
> 1d39834fba99 ext4: remove EXT4_STATE_DIOREAD_LOCK flag (related)
> e5465795cac4 ext4: fix off-by-one error when writing back pages before dio read (related)
> 16c54688592c ext4: Allow parallel DIO reads
>
> bs=4k:
> -------------------------------------------------------------------------------------------
> w/ parallel dio reads | READ 30898KB/s, 7724, 555.00us | WRITE 30875KB/s, 7718, 479.70us
> -------------------------------------------------------------------------------------------
> w/o parallel dio reads| READ 117915KB/s, 29478, 248.18us | WRITE 117854KB/s，29463, 21.91us
> -------------------------------------------------------------------------------------------
>
> bs=16k:
> -------------------------------------------------------------------------------------------
> w/ parallel dio reads | READ 58961KB/s, 3685, 835.28us | WRITE 58877KB/s, 3679, 1335.98us
> -------------------------------------------------------------------------------------------
> w/o parallel dio reads| READ 218409KB/s, 13650, 554.46us | WRITE 218257KB/s，13641, 29.22us
> -------------------------------------------------------------------------------------------
>
> bs=64k:
> -------------------------------------------------------------------------------------------
> w/ parallel dio reads | READ 119396KB/s, 1865, 1759.38us | WRITE 119159KB/s, 1861, 2532.26us
> -------------------------------------------------------------------------------------------
> w/o parallel dio reads| READ 422815KB/s, 6606, 1146.05us | WRITE 421619KB/s, 6587, 60.72us
> -------------------------------------------------------------------------------------------
>
> bs=512k:
> -------------------------------------------------------------------------------------------
> w/ parallel dio reads | READ 392973KB/s, 767, 5046.35us | WRITE 393165KB/s, 767, 5359.86us
> -------------------------------------------------------------------------------------------
> w/o parallel dio reads| READ 590266KB/s, 1152, 4312.01us | WRITE 590554KB/s, 1153, 2606.82us
> -------------------------------------------------------------------------------------------
>
> bs=1M:
> -------------------------------------------------------------------------------------------
> w/ parallel dio reads | READ 487779KB/s, 476, 8058.55us | WRITE 485592KB/s, 474, 8630.51us
> -------------------------------------------------------------------------------------------
> w/o parallel dio reads| READ 593927KB/s, 580, 7623.63us | WRITE 591265KB/s, 577, 6163.42us
> -------------------------------------------------------------------------------------------
>
> Thanks,
> Joseph
>
next prev parent reply other threads:[~2019-07-23 11:17 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-19 9:22 [RFC] performance regression with "ext4: Allow parallel DIO reads" Joseph Qi
2019-07-23 11:17 ` Joseph Qi [this message]
2019-07-25 21:20 ` Andreas Dilger
2019-07-26 1:12 ` Joseph Qi
2019-07-27 1:57 ` Andreas Dilger
2019-07-27 2:16 ` Joseph Qi
2019-07-28 22:51 ` Dave Chinner
2019-07-30 1:34 ` Joseph Qi
2019-08-15 15:13 ` Jan Kara
2019-08-16 13:23 ` Joseph Qi
2019-08-16 14:57 ` Jan Kara
2019-08-20 3:00 ` Joseph Qi
2019-08-20 16:08 ` Theodore Y. Ts'o
2019-08-21 1:04 ` Joseph Qi
2019-08-21 3:34 ` Theodore Y. Ts'o
2019-08-22 6:45 ` Joseph Qi
2019-08-22 5:40 ` Dave Chinner
2019-08-23 7:57 ` Joseph Qi
2019-08-23 8:07 ` Joseph Qi
2019-08-23 10:16 ` Dave Chinner
2019-08-23 13:08 ` Joseph Qi
2019-08-24 2:18 ` Dave Chinner
2019-08-26 8:39 ` Jan Kara
2019-08-26 19:10 ` Andreas Dilger
2019-08-27 1:00 ` Joseph Qi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=29d50d24-f8e7-5ef4-d4d8-3ea6fb1c6ed3@gmail.com \
--to=jiangqi903@gmail.com \
--cc=bo.liu@linux.alibaba.com \
--cc=jack@suse.cz \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=xiaoguang.wang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).