From: Ritesh Harjani <riteshh@linux.ibm.com>
To: "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>,
Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>,
Ext4 Developers List <linux-ext4@vger.kernel.org>,
joseph.qi@linux.alibaba.com, Liu Bo <bo.liu@linux.alibaba.com>
Subject: Re: Discussion: is it time to remove dioread_nolock?
Date: Thu, 16 Jan 2020 15:16:52 +0530 [thread overview]
Message-ID: <20200116094653.BF97FA405B@b06wcsmtp001.portsmouth.uk.ibm.com> (raw)
In-Reply-To: <20200115164829.GB165687@mit.edu>
[-- Attachment #1: Type: text/plain, Size: 5058 bytes --]
Hello Ted,
On 1/15/20 10:18 PM, Theodore Y. Ts'o wrote:
> On Wed, Jan 15, 2020 at 05:00:53AM +0530, Ritesh Harjani wrote:
>>
>> I too collected some performance numbers on my x86 box with
>> --direct=1, bs=4K/1M & ioengine=libaio, with default opt v/s dioread_nolock
>> opt on latest ext4 git tree.
>>
>> I found the delta to be within +/- 6% in all of the runs which includes, seq
>> read, mixed rw & mixed randrw.
>
> Thanks for taking the performance measurements!
>
> Are you able to release more detail of what the deltas were for those
> tests? And how stable were those results across repeated runs?
>
I have collected these numbers using fio. All of the column values
shown in the attachment & pasted below are average of 3 runs.
I checked all those individual runs too and saw that even in the run-to-
run variations the delta was within +/- 5% only.
The naming of those individual runs *.json files are a bit weird and
will take sometime if I have to publish the individual runs. But do let
me know if that as well is required. I can make some changes in the
scripts to print individual run's numbers too.
I have pasted the individual fio files too which I used for my testing.
I have used libaio as ioengine (with iodepth=16) in the SeqRead case and
psync in others.
Performance numbers:
===================
(seq) read (4K) - libaio
============
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- --------------------- ----------------------------
1 138.928059895833 138.824869791667
8 129.345052083333 124.472005208333
24 71.6555989583333 72.275390625
(Not sure why 24 thread case has lower perf numbers, but on increasing
the iodepth=32, I could see an increase in bw with 24 threads case
too.)
(seq) read (1M) - libaio
===========
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- --------------------- ---------------
1 138.905598958333 138.832682291667
8 111.263997395833 109.301106770833
24 121.895182291667 127.75390625
randrw(4K) - read (psync)
==========================
threads default_opt_read [KB/s] dioread_opt_read [KB/s]
---------- ---------------- ----------------
1 414.666666666667 410.0
8 780.0 792.333333333333
24 967.333333333333 991.666666666667
randrw(4K) - write (psync)
===========================
threads default_opt_write [KB/s] dioread_opt_write [KB/s]
---------- ----------------- -----------------
1 418.0 413.666666666667
8 796.666666666667 809.666666666667
24 981.333333333333 1007.66666666667
randrw(1M) - read (psync)
=================
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- ----------------- -----------------------
1 39.5693359375 39.7288411458333
8 44.5179036458333 47.9098307291667
24 50.2861328125 51.720703125
randrw(1M) - write (psync)
==================
threads default_opt_write(MB/s) dioread_nolock_opt_write(MB/s)
---------- ---------------------- -----------------------
1 41.4583333333333 41.5068359375
8 46.0768229166667 49.568359375
24 49.5947265625 50.7083333333333
rw(1M) - read (psync)
=============
threads default_opt_read(MB/s) dioread_nolock_opt_read(MB/s)
---------- ---------------------- -------------------
1 43.6458333333333 43.6770833333333
8 48.1178385416667 49.2718098958333
24 50.5703125 53.7890625
rw(1M) - write (psync)
==============
threads default_opt_write(MB/s) dioread_nolock_opt_write(MB/s)
---------- ----------------------- ------------------
1 45.5065104166667 45.5654296875
8 49.7431640625 51.0690104166667
24 50.2493489583333 53.3463541666667
FIO FILES
=========
dio_read.fio
============
[global]
ioengine=libaio
rw=read
runtime=60
iodepth=16
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
dio_randrw.fio
==============
[global]
ioengine=psync
rw=randrw
runtime=60
iodepth=1
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
dio_rw.fio
==========
[global]
ioengine=psync
rw=rw
runtime=60
iodepth=1
direct=1
size=10G
filename=./testfile
group_reporting=1
thread=1
;bs=$bs
;numjobs=24
[fio-run]
Thanks
-ritesh
[-- Attachment #2: datafile.txt --]
[-- Type: text/plain, Size: 3311 bytes --]
SeqRead (4K) - libaio
============
threads default_opt[read_4K(MB/s)] dioread_nolock_opt [read_4K(MB/s)]
---------- --------------------------- ------------------------------
1 138.928059895833 138.824869791667
8 129.345052083333 124.472005208333
24 71.6555989583333 72.275390625
SeqRead(1M) - libaio
===========
threads default_opt [read_1M(MB/s)] dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- --------------------------------
1 138.905598958333 138.832682291667
8 111.263997395833 109.301106770833
24 121.895182291667 127.75390625
randrw(4K) - read (psync)
==========================
threads default_opt_read [KB/s] dioread_opt_read [KB/s]
---------- ---------------- ----------------
1 414.666666666667 410.0
8 780.0 792.333333333333
24 967.333333333333 991.666666666667
randrw(4K) - write (psync)
===========================
threads default_opt_write [KB/s] dioread_opt_write [KB/s]
---------- ----------------- -----------------
1 418.0 413.666666666667
8 796.666666666667 809.666666666667
24 981.333333333333 1007.66666666667
randrw(1M) - read (psync)
=================
threads default_opt [read_1M(MB/s) dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- ---------------------------------
1 39.5693359375 39.7288411458333
8 44.5179036458333 47.9098307291667
24 50.2861328125 51.720703125
randrw(1M) - write (psync)
==================
threads default_opt [write_1M(MB/s)] dioread_nolock_opt [write_1M(MB/s)]
---------- ---------------------------- ----------------------------------
1 41.4583333333333 41.5068359375
8 46.0768229166667 49.568359375
24 49.5947265625 50.7083333333333
rw(1M) - read (psync)
=============
threads default_opt [read_1M(MB/s)] dioread_nolock_opt [read_1M(MB/s)]
---------- --------------------------- ---------------------------------
1 43.6458333333333 43.6770833333333
8 48.1178385416667 49.2718098958333
24 50.5703125 53.7890625
rw(1M) - write (psync)
==============
threads default_opt [write_1M(MB/s)] dioread_nolock_opt [write_1M(MB/s)
---------- ----------------------------- ---------------------------------
1 45.5065104166667 45.5654296875
8 49.7431640625 51.0690104166667
24 50.2493489583333 53.3463541666667
next prev parent reply other threads:[~2020-01-16 9:47 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-12-26 15:31 Discussion: is it time to remove dioread_nolock? Theodore Y. Ts'o
2019-12-27 13:10 ` Joseph Qi
2019-12-29 15:03 ` Xiaoguang Wang
2020-01-06 12:24 ` Ritesh Harjani
2020-01-07 0:43 ` Theodore Y. Ts'o
2020-01-07 8:22 ` Jan Kara
2020-01-07 17:11 ` Theodore Y. Ts'o
2020-01-07 17:22 ` Jan Kara
2020-01-08 10:45 ` Ritesh Harjani
2020-01-08 17:42 ` Theodore Y. Ts'o
2020-01-09 9:21 ` Ritesh Harjani
2020-01-09 16:38 ` Theodore Y. Ts'o
2020-01-14 23:30 ` Ritesh Harjani
2020-01-15 16:48 ` Theodore Y. Ts'o
2020-01-16 9:46 ` Ritesh Harjani [this message]
2020-01-09 12:34 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200116094653.BF97FA405B@b06wcsmtp001.portsmouth.uk.ibm.com \
--to=riteshh@linux.ibm.com \
--cc=bo.liu@linux.alibaba.com \
--cc=jack@suse.cz \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=xiaoguang.wang@linux.alibaba.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).