From: Mingkai Dong <mingkaidong@gmail.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: Mikulas Patocka <mpatocka@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Jan Kara <jack@suse.cz>, Steven Whitehouse <swhiteho@redhat.com>,
Eric Sandeen <esandeen@redhat.com>,
Dave Chinner <dchinner@redhat.com>, Theodore Ts'o <tytso@mit.edu>,
Wang Jianchao <jianchao.wan9@gmail.com>,
"Tadakamadla, Rajesh" <rajesh.tadakamadla@hpe.com>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-nvdimm@lists.01.org, sunrise_l@sjtu.edu.cn
Subject: Re: Expense of read_iter
Date: Fri, 8 Jan 2021 00:43:20 +0800 [thread overview]
Message-ID: <17045315-CC1F-4165-B8E3-BA55DD16D46B@gmail.com> (raw)
In-Reply-To: <20210107151125.GB5270@casper.infradead.org>
Hi Matthew,
We have also discovered the expense of `->read_iter` in our study on Ext4-DAX.
In single-thread 4K-reads, the `->read` version could outperform `->read_iter`
by 41.6% in terms of throughput.
According to our observation and evaluation, at least for Ext4-DAX, the cost
also comes from the invocation of `->iomap_begin` (`ext4_iomap_begin`),
which might not be simply avoided by adding a new iter_type.
The slowdown is more significant when multiple threads reading different files
concurrently, due to the scalability issue (grabbing a read lock to check the
status of the journal) in `ext4_iomap_begin`.
In our solution, we implemented the `->read` and `->write` interfaces for
Ext4-DAX. Thus, we also think it would be good if both `->read` and `->read_iter`
could exist.
By the way, besides the implementation of `->read` and `->write`, we have
some other optimizations for Ext4-DAX and would like to share them once our
patches are prepared.
Thanks,
Mingkai
> On Jan 7, 2021, at 23:11, Matthew Wilcox <willy@infradead.org> wrote:
>
> On Thu, Jan 07, 2021 at 08:15:41AM -0500, Mikulas Patocka wrote:
>> I'd like to ask about this piece of code in __kernel_read:
>> if (unlikely(!file->f_op->read_iter || file->f_op->read))
>> return warn_unsupported...
>> and __kernel_write:
>> if (unlikely(!file->f_op->write_iter || file->f_op->write))
>> return warn_unsupported...
>>
>> - It exits with an error if both read_iter and read or write_iter and
>> write are present.
>>
>> I found out that on NVFS, reading a file with the read method has 10%
>> better performance than the read_iter method. The benchmark just reads the
>> same 4k page over and over again - and the cost of creating and parsing
>> the kiocb and iov_iter structures is just that high.
>
> Which part of it is so expensive? Is it worth, eg adding an iov_iter
> type that points to a single buffer instead of a single-member iov?
>
> +++ b/include/linux/uio.h
> @@ -19,6 +19,7 @@ struct kvec {
>
> enum iter_type {
> /* iter types */
> + ITER_UBUF = 2,
> ITER_IOVEC = 4,
> ITER_KVEC = 8,
> ITER_BVEC = 16,
> @@ -36,6 +36,7 @@ struct iov_iter {
> size_t iov_offset;
> size_t count;
> union {
> + void __user *buf;
> const struct iovec *iov;
> const struct kvec *kvec;
> const struct bio_vec *bvec;
>
> and then doing all the appropriate changes to make that work.
> _______________________________________________
> Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
> To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org
next prev parent reply other threads:[~2021-01-07 16:43 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-07 13:15 [RFC v2] nvfs: a filesystem for persistent memory Mikulas Patocka
2021-01-07 15:11 ` Expense of read_iter Matthew Wilcox
2021-01-07 16:43 ` Mingkai Dong [this message]
[not found] ` <2041983017.5681521.1610459100858.JavaMail.zimbra@sjtu.edu.cn>
2021-01-12 14:06 ` David Laight
2021-01-13 16:44 ` Mikulas Patocka
[not found] ` <1224425872.715547.1610703643424.JavaMail.zimbra@sjtu.edu.cn>
2021-01-20 4:47 ` Dave Chinner
2021-01-20 14:18 ` Jan Kara
2021-01-20 15:12 ` Mikulas Patocka
2021-01-20 15:44 ` David Laight
2021-01-21 15:47 ` Matthew Wilcox
2021-01-21 16:06 ` Mikulas Patocka
2021-01-07 18:59 ` Mikulas Patocka
2021-01-10 6:13 ` Matthew Wilcox
2021-01-10 21:19 ` Mikulas Patocka
2021-01-11 0:18 ` Matthew Wilcox
2021-01-11 21:10 ` Mikulas Patocka
2021-01-11 10:11 ` David Laight
2021-01-10 16:20 ` [RFC v2] nvfs: a filesystem for persistent memory Al Viro
2021-01-10 16:51 ` Al Viro
2021-01-10 21:14 ` Mikulas Patocka
2021-01-10 23:40 ` Al Viro
2021-01-11 11:41 ` Mikulas Patocka
2021-01-11 10:29 ` David Laight
2021-01-11 11:44 ` Mikulas Patocka
2021-01-11 11:57 ` David Laight
2021-01-11 14:43 ` Al Viro
2021-01-11 14:54 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=17045315-CC1F-4165-B8E3-BA55DD16D46B@gmail.com \
--to=mingkaidong@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=dchinner@redhat.com \
--cc=esandeen@redhat.com \
--cc=jack@suse.cz \
--cc=jianchao.wan9@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=mpatocka@redhat.com \
--cc=rajesh.tadakamadla@hpe.com \
--cc=sunrise_l@sjtu.edu.cn \
--cc=swhiteho@redhat.com \
--cc=tytso@mit.edu \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).