Linux-ext4 Archive on lore.kernel.org
 help / color / Atom feed
From: Joseph Qi <joseph.qi@linux.alibaba.com>
To: Theodore Ts'o <tytso@mit.edu>, Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org,
	Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Subject: [RFC] performance regression with "ext4: Allow parallel DIO reads"
Date: Fri, 19 Jul 2019 17:22:24 +0800
Message-ID: <ab7cf51b-6b52-d151-e22c-6f4400a14589@linux.alibaba.com> (raw)

Hi Ted & Jan,
I've observed an significant performance regression with the following
commit in my Intel P3600 NVMe SSD.
16c54688592c ext4: Allow parallel DIO reads

From my initial investigation, it may be because of the
inode_lock_shared (down_read) consumes more than inode_lock (down_write)
in mixed random read write workload.

Here is my test result.

ioengine=psync
direct=1
rw=randrw
iodepth=1
numjobs=8
size=20G
runtime=600

w/ parallel dio reads : kernel 5.2.0
w/o parallel dio reads: kernel 5.2.0, then revert the following commits:
  1d39834fba99 ext4: remove EXT4_STATE_DIOREAD_LOCK flag (related)
  e5465795cac4 ext4: fix off-by-one error when writing back pages before dio read (related)
  16c54688592c ext4: Allow parallel DIO reads

bs=4k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 30898KB/s, 7724, 555.00us   | WRITE 30875KB/s, 7718, 479.70us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 117915KB/s, 29478, 248.18us | WRITE 117854KB/s,29463, 21.91us
-------------------------------------------------------------------------------------------

bs=16k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 58961KB/s, 3685, 835.28us   | WRITE 58877KB/s, 3679, 1335.98us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 218409KB/s, 13650, 554.46us | WRITE 218257KB/s,13641, 29.22us
-------------------------------------------------------------------------------------------

bs=64k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 119396KB/s, 1865, 1759.38us | WRITE 119159KB/s, 1861, 2532.26us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 422815KB/s, 6606, 1146.05us | WRITE 421619KB/s, 6587, 60.72us
-------------------------------------------------------------------------------------------

bs=512k:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 392973KB/s, 767, 5046.35us  | WRITE 393165KB/s, 767, 5359.86us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 590266KB/s, 1152, 4312.01us | WRITE 590554KB/s, 1153, 2606.82us
-------------------------------------------------------------------------------------------

bs=1M:
-------------------------------------------------------------------------------------------
w/ parallel dio reads | READ 487779KB/s, 476, 8058.55us  | WRITE 485592KB/s, 474, 8630.51us
-------------------------------------------------------------------------------------------
w/o parallel dio reads| READ 593927KB/s, 580, 7623.63us  | WRITE 591265KB/s, 577, 6163.42us
-------------------------------------------------------------------------------------------

Thanks,
Joseph

             reply index

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-19  9:22 Joseph Qi [this message]
2019-07-23 11:17 ` Joseph Qi
2019-07-25 21:20   ` Andreas Dilger
2019-07-26  1:12     ` Joseph Qi
2019-07-27  1:57       ` Andreas Dilger
2019-07-27  2:16         ` Joseph Qi
2019-07-28 22:51       ` Dave Chinner
2019-07-30  1:34         ` Joseph Qi
2019-08-15 15:13           ` Jan Kara
2019-08-16 13:23             ` Joseph Qi
2019-08-16 14:57               ` Jan Kara
2019-08-20  3:00                 ` Joseph Qi
2019-08-20 16:08                   ` Theodore Y. Ts'o
2019-08-21  1:04                     ` Joseph Qi
2019-08-21  3:34                       ` Theodore Y. Ts'o
2019-08-22  6:45                         ` Joseph Qi
2019-08-22  5:40                       ` Dave Chinner
2019-08-23  7:57                         ` Joseph Qi
2019-08-23  8:07                           ` Joseph Qi
2019-08-23 10:16                           ` Dave Chinner
2019-08-23 13:08                             ` Joseph Qi
2019-08-24  2:18                               ` Dave Chinner
2019-08-26  8:39                                 ` Jan Kara
2019-08-26 19:10                                   ` Andreas Dilger
2019-08-27  1:00                                     ` Joseph Qi

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ab7cf51b-6b52-d151-e22c-6f4400a14589@linux.alibaba.com \
    --to=joseph.qi@linux.alibaba.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=tytso@mit.edu \
    --cc=xiaoguang.wang@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ext4 Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-ext4/0 linux-ext4/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-ext4 linux-ext4/ https://lore.kernel.org/linux-ext4 \
		linux-ext4@vger.kernel.org
	public-inbox-index linux-ext4

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-ext4


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git