From: Jaewon Kim <jaewon31.kim@samsung.com>
To: John Stultz <jstultz@google.com>
Cc: "sumit.semwal@linaro.org" <sumit.semwal@linaro.org>,
"daniel.vetter@ffwll.ch" <daniel.vetter@ffwll.ch>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
"mhocko@kernel.org" <mhocko@kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"jaewon31.kim@gmail.com" <jaewon31.kim@gmail.com>,
"T.J. Mercier" <tjmercier@google.com>
Subject: Re: [PATCH] dma-buf: system_heap: avoid reclaim for order 4
Date: Wed, 18 Jan 2023 16:30:43 +0900 [thread overview]
Message-ID: <20230118073043epcms1p8b5cfde0d773e296770775581b6ce7105@epcms1p8> (raw)
In-Reply-To: <CANDhNCpKY5Af059ok8ZcgJ=wt7NaorZxqQXaTS848CwY0LNFiw@mail.gmail.com>
>On Tue, Jan 17, 2023 at 12:31 AM Jaewon Kim <jaewon31.kim@samsung.com> wrote:
>> > Using order 4 pages would be helpful for many IOMMUs, but it could spend
>> > quite much time in page allocation perspective.
>> >
>> > The order 4 allocation with __GFP_RECLAIM may spend much time in
>> > reclaim and compation logic. __GFP_NORETRY also may affect. These cause
>> > unpredictable delay.
>> >
>> > To get reasonable allocation speed from dma-buf system heap, use
>> > HIGH_ORDER_GFP for order 4 to avoid reclaim.
>
>Thanks for sharing this!
>The case where the allocation gets stuck behind reclaim under pressure
>does sound undesirable, but I'd be a bit hesitant to tweak numbers
>that have been used for a long while (going back to ion) without a bit
>more data.
>
>It might be good to also better understand the tradeoff of potential
>on-going impact to performance from using low order pages when the
>buffer is used. Do you have any details like or tests that you could
>share to help ensure this won't impact other users?
>
>TJ: Do you have any additional thoughts on this?
>
>thanks
>-john
Let me share what I tested with trace events of compaction, direct_reclaim.
This log was captured within one dma-buf alloc for 1,957,888 byte size.
It shows direct reclaim spent 4.973 msec total so that it could reclaim 209 pages total.
But it spent 12.606 msec in compaction.
I might miss preempted time but I think compaction is quite time consuming job.
Sorry I don't know how much the current gfp is helpful to IOMMU.
I think the default should be set like my change to improve allocation,
and the current gfp could be set if need through other options like cmdline.
-5972 [006] .... 7971.248176: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.248394: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xac800 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.248399: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.248922: mm_vmscan_direct_reclaim_end: nr_reclaimed=17
-5972 [006] .... 7971.248974: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xacc00 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.249063: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xb5c00 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.249067: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.249768: mm_vmscan_direct_reclaim_end: nr_reclaimed=18
-5972 [006] .... 7971.249843: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xb6000 free_pfn=0xafc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.249845: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xb6000 free_pfn=0xafc00 zone_end=0x100000, mode=async status=partial_skipped
-5972 [006] .... 7971.249849: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.250116: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.250152: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xaf800 zone_end=0x100000, mode=async
-5972 [006] .... 7971.250305: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xac400 free_pfn=0xaf800 zone_end=0x100000, mode=async status=success
-5972 [006] .... 7971.250375: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xac800 free_pfn=0xaf800 zone_end=0x100000, mode=async
-5972 [006] .... 7971.250380: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xaf800 free_pfn=0xaf800 zone_end=0x100000, mode=async status=partial_skipped
-5972 [006] .... 7971.250384: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.250597: mm_vmscan_direct_reclaim_end: nr_reclaimed=20
-5972 [006] .... 7971.251289: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xaf000 zone_end=0x100000, mode=async
-5972 [006] .... 7971.251497: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xac400 free_pfn=0xaf000 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.251502: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.251797: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.251890: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x5b400 free_pfn=0xb0400 zone_end=0x100000, mode=sync
-5972 [006] .... 7971.252740: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x63800 free_pfn=0xb0400 zone_end=0x100000, mode=sync status=success
-5972 [006] .... 7971.252774: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x63c00 free_pfn=0xb0000 zone_end=0x100000, mode=async
-5972 [006] .... 7971.253325: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x74e20 free_pfn=0xade92 zone_end=0x100000, mode=async status=success
-5972 [006] .... 7971.253333: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x83400 free_pfn=0xafc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.253683: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xac400 free_pfn=0xafc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.253688: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.254134: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.254206: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xad000 free_pfn=0xafc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.254289: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xae1c0 free_pfn=0xae000 zone_end=0x100000, mode=async status=partial_skipped
-5972 [006] .... 7971.254304: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.254510: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x74db0 free_pfn=0xadcc1 zone_end=0x100000, mode=async status=success
-5972 [006] .... 7971.254519: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x75000 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.254612: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xac400 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.254617: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.255303: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.255307: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x75000 free_pfn=0xffc00 zone_end=0x100000, mode=sync
-5972 [006] .... 7971.260198: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x751e5 free_pfn=0xaa83f zone_end=0x100000, mode=sync status=success
-5972 [006] .... 7971.260236: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.260883: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xb5c00 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.260891: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.261174: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.261178: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x75000 free_pfn=0xffc00 zone_end=0x100000, mode=sync
-5972 [006] .... 7971.264861: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x75681 free_pfn=0xffc00 zone_end=0x100000, mode=sync status=success
-5972 [006] .... 7971.264873: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xbbc00 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.264889: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xc1400 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [006] .... 7971.264891: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [006] .... 7971.265141: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [002] .... 7971.265585: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0xc1800 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [002] .... 7971.265669: mm_compaction_end: zone_start=0x40000 migrate_pfn=0xc2c00 free_pfn=0xffc00 zone_end=0x100000, mode=async status=contended
-5972 [002] .... 7971.265678: mm_vmscan_direct_reclaim_begin: order=4 gfp_flags=GFP_HIGHUSER|__GFP_NOWARN|__GFP_COMP|__GFP_ZERO
-5972 [002] .... 7971.266987: mm_vmscan_direct_reclaim_end: nr_reclaimed=22
-5972 [006] .... 7971.267466: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x40000 free_pfn=0xffc00 zone_end=0x100000, mode=sync
-5972 [006] .... 7971.267778: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x5d000 free_pfn=0xffc00 zone_end=0x100000, mode=sync status=success
-5972 [006] .... 7971.267927: mm_compaction_begin: zone_start=0x40000 migrate_pfn=0x5c000 free_pfn=0xffc00 zone_end=0x100000, mode=async
-5972 [006] .... 7971.268092: mm_compaction_end: zone_start=0x40000 migrate_pfn=0x74d70 free_pfn=0xffc00 zone_end=0x100000, mode=async status=success
Thank you
Jaewon Kim
next prev parent reply other threads:[~2023-01-18 8:03 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcas1p2.samsung.com>
2023-01-17 8:25 ` [PATCH] dma-buf: system_heap: avoid reclaim for order 4 Jaewon Kim
[not found] ` <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcms1p6>
2023-01-17 8:31 ` Jaewon Kim
2023-01-18 6:54 ` John Stultz
2023-01-18 19:55 ` T.J. Mercier
[not found] ` <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcms1p2>
2023-01-25 9:56 ` Jaewon Kim
2023-01-25 10:19 ` Jaewon Kim
2023-01-25 20:32 ` John Stultz
[not found] ` <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcms1p3>
2023-01-26 4:42 ` 김재원
2023-01-26 5:04 ` (2) " John Stultz
[not found] ` <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcms1p8>
2023-01-18 7:30 ` Jaewon Kim [this message]
2023-02-04 15:02 ` Jaewon Kim
2023-02-07 4:37 ` (2) " John Stultz
[not found] ` <CGME20230117082521epcas1p22a709521a9e6d2346d06ac220786560d@epcms1p1>
2023-02-07 7:33 ` Jaewon Kim
2023-02-07 16:56 ` John Stultz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230118073043epcms1p8b5cfde0d773e296770775581b6ce7105@epcms1p8 \
--to=jaewon31.kim@samsung.com \
--cc=akpm@linux-foundation.org \
--cc=daniel.vetter@ffwll.ch \
--cc=hannes@cmpxchg.org \
--cc=jaewon31.kim@gmail.com \
--cc=jstultz@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=sumit.semwal@linaro.org \
--cc=tjmercier@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).