Linux-ext4 Archive on lore.kernel.org
 help / color / Atom feed
From: Ritesh Harjani <riteshh@linux.ibm.com>
To: "Theodore Y. Ts'o" <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>,
	Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>,
	Ext4 Developers List <linux-ext4@vger.kernel.org>,
	joseph.qi@linux.alibaba.com, Liu Bo <bo.liu@linux.alibaba.com>
Subject: Re: Discussion: is it time to remove dioread_nolock?
Date: Thu, 16 Jan 2020 15:16:52 +0530
Message-ID: <20200116094653.BF97FA405B@b06wcsmtp001.portsmouth.uk.ibm.com> (raw)
In-Reply-To: <20200115164829.GB165687@mit.edu>

[-- Attachment #1: Type: text/plain, Size: 5058 bytes --]

Hello Ted,

On 1/15/20 10:18 PM, Theodore Y. Ts'o wrote:
> On Wed, Jan 15, 2020 at 05:00:53AM +0530, Ritesh Harjani wrote:
>>
>> I too collected some performance numbers on my x86 box with
>> --direct=1, bs=4K/1M & ioengine=libaio, with default opt v/s dioread_nolock
>> opt on latest ext4 git tree.
>>
>> I found the delta to be within +/- 6% in all of the runs which includes, seq
>> read, mixed rw & mixed randrw.
> 
> Thanks for taking the performance measurements!
> 
> Are you able to release more detail of what the deltas were for those
> tests?  And how stable were those results across repeated runs?
> 

I have collected these numbers using fio. All of the column values
shown in the attachment & pasted below are average of 3 runs.
I checked all those individual runs too and saw that even in the run-to-
run variations the delta was within +/- 5% only.

The naming of those individual runs *.json files are a bit weird and 
will take sometime if I have to publish the individual runs. But do let
me know if that as well is required. I can make some changes in the 
scripts to print individual run's numbers too.

I have pasted the individual fio files too which I used for my testing.
I have used libaio as ioengine (with iodepth=16) in the SeqRead case and
psync in others.


Performance numbers:
===================

(seq) read (4K) - libaio
============
threads     default_opt_read(MB/s)   dioread_nolock_opt_read(MB/s)
----------  ---------------------    ----------------------------
1           138.928059895833             138.824869791667 

8           129.345052083333             124.472005208333 

24          71.6555989583333             72.275390625


(Not sure why 24 thread case has lower perf numbers, but on increasing
the iodepth=32, I could see an increase in bw with 24 threads case
too.)


(seq) read (1M) - libaio
===========
threads     default_opt_read(MB/s)  dioread_nolock_opt_read(MB/s)
----------  ---------------------   ---------------
1           138.905598958333             138.832682291667 

8           111.263997395833             109.301106770833 

24          121.895182291667             127.75390625


randrw(4K)  - read (psync) 
 

========================== 

threads     default_opt_read [KB/s]  dioread_opt_read [KB/s] 

----------  ----------------    ---------------- 

1           414.666666666667    410.0 

8           780.0               792.333333333333 

24          967.333333333333    991.666666666667 


 
 

randrw(4K)  - write (psync) 

=========================== 

threads     default_opt_write [KB/s]  dioread_opt_write [KB/s] 

----------  -----------------    ----------------- 

1           418.0                413.666666666667 

8           796.666666666667     809.666666666667 

24          981.333333333333     1007.66666666667


randrw(1M) - read (psync)
=================
threads     default_opt_read(MB/s)   dioread_nolock_opt_read(MB/s)
----------  -----------------        -----------------------
1           39.5693359375                39.7288411458333 

8           44.5179036458333             47.9098307291667 

24          50.2861328125                51.720703125


randrw(1M) - write (psync)
==================
threads     default_opt_write(MB/s)  dioread_nolock_opt_write(MB/s)
----------  ----------------------   -----------------------
1           41.4583333333333              41.5068359375 

8           46.0768229166667              49.568359375 

24          49.5947265625                 50.7083333333333


rw(1M) - read (psync)
=============
threads     default_opt_read(MB/s)  dioread_nolock_opt_read(MB/s)
----------  ----------------------  -------------------
1           43.6458333333333             43.6770833333333 

8           48.1178385416667             49.2718098958333 

24          50.5703125                   53.7890625


rw(1M) - write (psync)
==============
threads     default_opt_write(MB/s)   dioread_nolock_opt_write(MB/s)
----------  -----------------------   ------------------
1           45.5065104166667               45.5654296875 

8           49.7431640625                  51.0690104166667 

24          50.2493489583333               53.3463541666667



FIO FILES
=========

dio_read.fio
============
[global]
         ioengine=libaio
         rw=read
         runtime=60
         iodepth=16
         direct=1
         size=10G
         filename=./testfile
         group_reporting=1
         thread=1
         ;bs=$bs
         ;numjobs=24

[fio-run]


dio_randrw.fio
==============
[global]
         ioengine=psync
         rw=randrw
         runtime=60
         iodepth=1
         direct=1
         size=10G
         filename=./testfile
         group_reporting=1
         thread=1
         ;bs=$bs
         ;numjobs=24

[fio-run]

dio_rw.fio
==========
[global]
         ioengine=psync
         rw=rw
         runtime=60
         iodepth=1
         direct=1
         size=10G
         filename=./testfile
         group_reporting=1
         thread=1
         ;bs=$bs
         ;numjobs=24

[fio-run]


Thanks
-ritesh



[-- Attachment #2: datafile.txt --]
[-- Type: text/plain, Size: 3311 bytes --]

SeqRead (4K) - libaio
============
threads     default_opt[read_4K(MB/s)]   dioread_nolock_opt [read_4K(MB/s)]
----------  ---------------------------  ------------------------------
1           138.928059895833             138.824869791667                                    
8           129.345052083333             124.472005208333                                    
24          71.6555989583333             72.275390625   


SeqRead(1M) - libaio
===========
threads     default_opt [read_1M(MB/s)]  dioread_nolock_opt [read_1M(MB/s)]
----------  ---------------------------  --------------------------------
1           138.905598958333             138.832682291667                                    
8           111.263997395833             109.301106770833                                    
24          121.895182291667             127.75390625      


randrw(4K)  - read (psync)
==========================
threads     default_opt_read [KB/s]  dioread_opt_read [KB/s]
----------  ----------------  	----------------
1           414.666666666667  	410.0
8           780.0             	792.333333333333
24          967.333333333333  	991.666666666667


randrw(4K)  - write (psync)
===========================
threads     default_opt_write [KB/s]  dioread_opt_write [KB/s]
----------  ----------------- 	 -----------------                                      
1           418.0             	 413.666666666667
8           796.666666666667  	 809.666666666667
24          981.333333333333  	 1007.66666666667

randrw(1M) - read (psync)
=================
threads     default_opt [read_1M(MB/s)   dioread_nolock_opt [read_1M(MB/s)]
----------  ---------------------------  ---------------------------------
1           39.5693359375                39.7288411458333                                      
8           44.5179036458333             47.9098307291667                                      
24          50.2861328125                51.720703125       


randrw(1M) - write (psync)
==================
threads     default_opt [write_1M(MB/s)]  dioread_nolock_opt [write_1M(MB/s)]
----------  ----------------------------  ----------------------------------
1           41.4583333333333              41.5068359375                                          
8           46.0768229166667              49.568359375                                           
24          49.5947265625                 50.7083333333333         


rw(1M) - read (psync)
=============
threads     default_opt [read_1M(MB/s)]  dioread_nolock_opt [read_1M(MB/s)]
----------  ---------------------------  ---------------------------------
1           43.6458333333333             43.6770833333333                                  
8           48.1178385416667             49.2718098958333                                  
24          50.5703125                   53.7890625


rw(1M) - write (psync)
==============
threads     default_opt [write_1M(MB/s)]   dioread_nolock_opt [write_1M(MB/s)
----------  -----------------------------  ---------------------------------
1           45.5065104166667               45.5654296875                                      
8           49.7431640625                  51.0690104166667                                   
24          50.2493489583333               53.3463541666667         


  reply index

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-26 15:31 Theodore Y. Ts'o
2019-12-27 13:10 ` Joseph Qi
2019-12-29 15:03 ` Xiaoguang Wang
2020-01-06 12:24   ` Ritesh Harjani
2020-01-07  0:43     ` Theodore Y. Ts'o
2020-01-07  8:22       ` Jan Kara
2020-01-07 17:11         ` Theodore Y. Ts'o
2020-01-07 17:22           ` Jan Kara
2020-01-08 10:45             ` Ritesh Harjani
2020-01-08 17:42               ` Theodore Y. Ts'o
2020-01-09  9:21                 ` Ritesh Harjani
2020-01-09 16:38                   ` Theodore Y. Ts'o
2020-01-14 23:30                     ` Ritesh Harjani
2020-01-15 16:48                       ` Theodore Y. Ts'o
2020-01-16  9:46                         ` Ritesh Harjani [this message]
2020-01-09 12:34                 ` Jan Kara

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200116094653.BF97FA405B@b06wcsmtp001.portsmouth.uk.ibm.com \
    --to=riteshh@linux.ibm.com \
    --cc=bo.liu@linux.alibaba.com \
    --cc=jack@suse.cz \
    --cc=joseph.qi@linux.alibaba.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=tytso@mit.edu \
    --cc=xiaoguang.wang@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ext4 Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-ext4/0 linux-ext4/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-ext4 linux-ext4/ https://lore.kernel.org/linux-ext4 \
		linux-ext4@vger.kernel.org
	public-inbox-index linux-ext4

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-ext4


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git