Re: Discussion: is it time to remove dioread_nolock?

[Ritesh Harjani <riteshh@linux.ibm.com>]

On 1/8/20 11:12 PM, Theodore Y. Ts'o wrote:
> On Wed, Jan 08, 2020 at 04:15:13PM +0530, Ritesh Harjani wrote:
>> Hello Ted/Jan,
>>
>> On 1/7/20 10:52 PM, Jan Kara wrote:
>>> On Tue 07-01-20 12:11:09, Theodore Y. Ts'o wrote:
>>>> Hmm..... There's actually an even more radical option we could use,
>>>> given that Ritesh has made dioread_nolock work on block sizes < page
>>>> size.  We could make dioread_nolock the default, until we can revamp
>>>> ext4_writepages() to write the data blocks first....
>>
>> Agreed. I guess it should be a straight forward change to make.
>> It should be just removing test_opt(inode->i_sb, DIOREAD_NOLOCK) condition
>> from ext4_should_dioread_nolock().
> 
> Actually, it's simpler than that.  In fs/ext4/super.c, around line
> 3730, after the comment:
> 
> 	/* Set defaults before we parse the mount options */
> 
> Just add:
> 
>       set_opt(sb, DIOREAD_NOLOCK);

Yes, silly me.


> 
> This will allow system administrators to revert back to the original
> method using the someone confusingly named mount option,
> "dioread_lock".  (Maybe we can add a alias for that mount option so
> it's less confusing).
> 
>>> Yes, that's a good point. And I'm not opposed to that if it makes the life
>>> simpler. But I'd like to see some performance numbers showing how much is
>>> writeback using unwritten extents slower so that we don't introduce too big
>>> regression with this...
>>>
>>
>> Yes, let me try to get some performance numbers with dioread_nolock as
>> the default option for buffered write on my setup.
> 
> I started running some performance runs last night, and the
> interesting thing that I found was that fs_mark actually *improved*
> with dioread_nolock (with fsync enabled).  That may be an example of
> where fixing the commit latency caused by writeback can actually show
> up in a measurable way with benchmarks.
> 
> Dbench was slightly impacted; I didn't see any real differences with
> compilebench or postmark.  dioread_nolock did improve fio with
> sequential reads; which is interesting, since I would have expected

IIUC, this Seq. read numbers are with --direct=1 & bs=2MB & 
ioengine=libaio, correct?
So essentially it will do a DIO AIO sequential read.

Yes, it *does shows* a big delta in the numbers. I also noticed a higher
deviation between the two runs with dioread_nolock.


> with the inode_lock improvements, there shouldn't have been any
> difference.  So that may be a bit of wierdness that we should try to
> understand.

So inode_lock patches gives improvement in mixed read/write workload
where inode exclusive locking was causing the bottleneck earlier.


In this run, was encryption or fsverity enabled?
If yes then in that case I see that ext4_dio_supported() will return
false and it will fallback to bufferedRead.
Though with that also can't explain the delta with only enabling
dioread_nolock.


> 
> See the attached tar file; open ext4-modes/index.html in a browser to
> see the pretty graphs.  The raw numbers are in ext4/composite.xml.

The graphs and overview looks really good. I will also check about PTS
sometime. Will be good to capture such reports.


ritesh