From: Changheun Lee <nanich.lee@samsung.com>
To: damien.lemoal@wdc.com
Cc: Johannes.Thumshirn@wdc.com, axboe@kernel.dk,
jisoo2146.oh@samsung.com, junho89.kim@samsung.com,
linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
mj0123.lee@samsung.com, nanich.lee@samsung.com,
seunghwan.hyun@samsung.com, sookwan7.kim@samsung.com,
tj@kernel.org, yt0928.kim@samsung.com
Subject: Re: Re: [PATCH] bio: limit bio max size.
Date: Tue, 12 Jan 2021 20:58:46 +0900 [thread overview]
Message-ID: <20210112115846.31415-1-nanich.lee@samsung.com> (raw)
In-Reply-To: <BL0PR04MB6514F7F944D70DE024494544E7AA0@BL0PR04MB6514.namprd04.prod.outlook.com>
>On 2021/01/12 17:52, Changheun Lee wrote:
>> From: "Changheun Lee" <nanich.lee@samsung.com>
>>
>> bio size can grow up to 4GB when muli-page bvec is enabled.
>> but sometimes it would lead to inefficient behaviors.
>> in case of large chunk direct I/O, - 64MB chunk read in user space -
>> all pages for 64MB would be merged to a bio structure if memory address is
>> continued phsycally. it makes some delay to submit until merge complete.
>> bio max size should be limited as a proper size.
>
>But merging physically contiguous pages into the same bvec + later automatic bio
>split on submit should give you better throughput for large IOs compared to
>having to issue a bio chain of smaller BIOs that are arbitrarily sized and will
>likely need splitting anyway (because of DMA boundaries etc).
>
>Do you have a specific case where you see higher performance with this patch
>applied ? On Intel, BIO_MAX_SIZE would be 1MB... That is arbitrary and too small
>considering that many hardware can execute larger IOs than that.
>
When I tested 32MB chunk read with O_DIRECT in android, all pages of 32MB
is merged into a bio structure.
And elapsed time to merge complete was about 2ms.
It means first bio-submit is after 2ms.
If bio size is limited with 1MB with this patch, first bio-submit is about
100us by bio_full operation.
It's not large delay and can't be observed with low speed device.
But it's needed to reduce merge delay for high speed device.
I improved 512MB sequential read performance from 1900MB/s to 2000MB/s
with this patch on android platform.
As you said, 1MB might be small for some device.
But method is needed to re-size, or select the bio max size.
>
>>
>> Signed-off-by: Changheun Lee <nanich.lee@samsung.com>
>> ---
>> block/bio.c | 2 +-
>> include/linux/bio.h | 3 ++-
>> 2 files changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/block/bio.c b/block/bio.c
>> index 1f2cc1fbe283..dbe14d675f28 100644
>> --- a/block/bio.c
>> +++ b/block/bio.c
>> @@ -877,7 +877,7 @@ bool __bio_try_merge_page(struct bio *bio, struct page *page,
>> struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1];
>>
>> if (page_is_mergeable(bv, page, len, off, same_page)) {
>> - if (bio->bi_iter.bi_size > UINT_MAX - len) {
>> + if (bio->bi_iter.bi_size > BIO_MAX_SIZE - len) {
>> *same_page = false;
>> return false;
>> }
>> diff --git a/include/linux/bio.h b/include/linux/bio.h
>> index 1edda614f7ce..0f49b354b1f6 100644
>> --- a/include/linux/bio.h
>> +++ b/include/linux/bio.h
>> @@ -20,6 +20,7 @@
>> #endif
>>
>> #define BIO_MAX_PAGES 256
>> +#define BIO_MAX_SIZE (BIO_MAX_PAGES * PAGE_SIZE)
>>
>> #define bio_prio(bio) (bio)->bi_ioprio
>> #define bio_set_prio(bio, prio) ((bio)->bi_ioprio = prio)
>> @@ -113,7 +114,7 @@ static inline bool bio_full(struct bio *bio, unsigned len)
>> if (bio->bi_vcnt >= bio->bi_max_vecs)
>> return true;
>>
>> - if (bio->bi_iter.bi_size > UINT_MAX - len)
>> + if (bio->bi_iter.bi_size > BIO_MAX_SIZE - len)
>> return true;
>>
>> return false;
>>
>
>
>--
>Damien Le Moal
>Western Digital Research
next prev parent reply other threads:[~2021-01-12 12:15 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20210112084819epcas1p2389fe5fd665e4ff7b41ad92344547294@epcas1p2.samsung.com>
2021-01-12 8:33 ` [PATCH] bio: limit bio max size Changheun Lee
2021-01-12 9:16 ` Damien Le Moal
[not found] ` <CGME20210112121356epcas1p124baa93f10eb3400539ba4db27c18955@epcas1p1.samsung.com>
2021-01-12 11:58 ` Changheun Lee [this message]
2021-01-13 1:16 ` Damien Le Moal
[not found] ` <CGME20210113040146epcas1p230596c7c3760471dca442d1f7ce4dc55@epcas1p2.samsung.com>
2021-01-13 3:46 ` Changheun Lee
2021-01-13 5:53 ` Damien Le Moal
[not found] ` <CGME20210113065444epcas1p4b8ee3edb314a06b1a9f92fd0e38ca856@epcas1p4.samsung.com>
2021-01-13 6:39 ` Changheun Lee
2021-01-13 7:12 ` Damien Le Moal
2021-01-13 9:19 ` Ming Lei
2021-01-13 9:28 ` Damien Le Moal
2021-01-13 10:24 ` Ming Lei
2021-01-13 11:16 ` Damien Le Moal
2021-01-13 11:47 ` Ming Lei
2021-01-13 12:02 ` Damien Le Moal
2021-01-14 3:52 ` Ming Lei
2021-01-14 4:00 ` Damien Le Moal
[not found] ` <CGME20210114045019epcas1p16d4f5f258c2a3b290540ac640745764d@epcas1p1.samsung.com>
2021-01-14 4:35 ` Changheun Lee
2021-01-17 12:47 ` [bio] 70c9aa94e8: WARNING:at_block/bio.c:#bio_iov_iter_get_pages kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210112115846.31415-1-nanich.lee@samsung.com \
--to=nanich.lee@samsung.com \
--cc=Johannes.Thumshirn@wdc.com \
--cc=axboe@kernel.dk \
--cc=damien.lemoal@wdc.com \
--cc=jisoo2146.oh@samsung.com \
--cc=junho89.kim@samsung.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mj0123.lee@samsung.com \
--cc=seunghwan.hyun@samsung.com \
--cc=sookwan7.kim@samsung.com \
--cc=tj@kernel.org \
--cc=yt0928.kim@samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).