From: Jaewon Kim <jaewon31.kim@samsung.com>
To: Michal Hocko <mhocko@suse.com>
Cc: "jstultz@google.com" <jstultz@google.com>,
"tjmercier@google.com" <tjmercier@google.com>,
"sumit.semwal@linaro.org" <sumit.semwal@linaro.org>,
"daniel.vetter@ffwll.ch" <daniel.vetter@ffwll.ch>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"hannes@cmpxchg.org" <hannes@cmpxchg.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"jaewon31.kim@gmail.com" <jaewon31.kim@gmail.com>
Subject: RE: [PATCH v3] dma-buf/heaps: system_heap: avoid too much allocation
Date: Wed, 12 Apr 2023 20:37:59 +0900 [thread overview]
Message-ID: <20230412113759epcms1p8cb15b54e3a96c7616419cb030d16f804@epcms1p8> (raw)
In-Reply-To: <ZDaP4/PYyb9tKGQi@dhcp22.suse.cz>
>On Wed 12-04-23 18:44:40, Jaewon Kim wrote:
>> >On Wed 12-04-23 17:57:26, Jaewon Kim wrote:
>> >> >Sorry for being late. I know there was some pre-existing discussion
>> >> >around that but I didn't have time to participate.
>> >> >
>> >> >On Mon 10-04-23 16:32:28, Jaewon Kim wrote:
>> >> >> @@ -350,6 +350,9 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
>> >> >> struct page *page, *tmp_page;
>> >> >> int i, ret = -ENOMEM;
>> >> >>
>> >> >> + if (len / PAGE_SIZE > totalram_pages())
>> >> >> + return ERR_PTR(-ENOMEM);
>> >> >> +
>> >> >
>> >> >This is an antipattern imho. Check 7661809d493b ("mm: don't allow
>> >> >oversized kvmalloc() calls") how kvmalloc has dealt with a similar
>> >>
>> >> Hello Thank you for the information.
>> >>
>> >> I tried to search the macro of INT_MAX.
>> >>
>> >> include/vdso/limits.h
>> >> #define INT_MAX ((int)(~0U >> 1))
>> >>
>> >> AFAIK the dma-buf system heap user can request that huge size more than 2GB.
>> >
>> >Do you have any pointers? This all is unreclaimable memory, right? How
>> >are those users constrained to not go overboard?
>>
>> Correct dma-buf system heap memory is unreclaimable. To avoid that huge request,
>> this patch includes __GFP_RETRY_MAYFAIL.
>
>__GFP_RETRY_MAYFAIL doesn't avoud huge requests. It will drain the free
>available memory to the edge of OOM (especially for low order requests)
>so effectively anybody else requesting any memory (GFP_KERNEL like req.)
>will hit the oom killer very likely).
>
>> #define LOW_ORDER_GFP (GFP_HIGHUSER | __GFP_ZERO | __GFP_RETRY_MAYFAIL)
>>
>> >
>> >> So
>> >> I think totalram_pages() is better than INT_MAX in this case.
>> >>
>> >> >issue. totalram_pages doesn't really tell you anything about incorrect
>> >> >users. You might be on a low memory system where the request size is
>> >> >sane normally, it just doesn't fit into memory on that particular
>> >> >machine.
>> >>
>> >> Sorry maybe I'm not fully understand what you meant. User may requested
>> >> a huge size like 3GB on 2GB ram device. But I think that should be rejected
>> >> because it is bigger than the device ram size.
>> >
>> >Even totalram_pages/10 can be just unfeasible amount of data to be
>> >allocated without a major disruption. totalram_pages is no measure of
>> >the memory availability.
>> >If you want to have a ballpark estimation then si_mem_available might be
>> >something you are looking for. But I thought the sole purpose of this
>> >patch is to catch obviously buggy callers (like sign overflow lenght
>> >etc) rather than any memory consumption sanity check.
>>
>> Yes if we want to avoid some big size, si_mem_available could be one option.
>> Actually I tried to do totalram_pages() / 2 like the old ion system heap in
>> the previous patch version. Anyway totalram_pages in this patch is used to
>> avoid the buggy size.
>
>So let me repeat that totalram_pages is a wrong thing to do(tm).
>
>This is not a subsystem I would feel like nacking a patch, but consider
>this feedback as strong of a rejection as somebody external can give
>you. A mm internal allocator would get an outright nack.
>
>What you are doing is just wrong and an antipattern to what other
>allocators do. Either use something like INT_MAX to catch overflows or
>do not try to catch buggy code but pretend a better memory consumer
>citizen by using something like si_mem_available (ideally think of
>other potential memory users so do not allow any request to use all
>of it). The later might require much more involved interface and I do
>rememeber some attempts to account and limit dmabuf memory better.
>
>> And as we discussed in v2 patch, __GFP_RETRY_MAYFAIL was added. And I think
>> the gfp makes us feel better in memory perspective.
>
>wishful thinking that is.
>--
>Michal Hocko
>SUSE Labs
Yes I think you're right. As a allocator, dma-buf system heap looks to be loose
in memory allocation. Limiting dmabuf memory may be required. But I think there
is no nice and reasonable way so far. And the dma-buf system heap is being
widely used in Android mobile system. AFAIK the camera consumes huge memory
through this dma-buf system heap. I actually even looked a huge size request
over 2GB in one dma-buf request.
Jaewon Kim
next prev parent reply other threads:[~2023-04-12 11:38 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcas1p4.samsung.com>
2023-04-10 7:32 ` [PATCH v3] dma-buf/heaps: system_heap: avoid too much allocation Jaewon Kim
2023-04-12 8:22 ` Michal Hocko
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p7>
2023-04-12 8:57 ` Jaewon Kim
2023-04-12 9:23 ` Michal Hocko
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p4>
2023-04-12 9:44 ` Jaewon Kim
2023-04-12 11:02 ` Michal Hocko
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p8>
2023-04-12 11:37 ` Jaewon Kim [this message]
2023-04-12 11:51 ` Michal Hocko
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p2>
2023-04-12 12:35 ` Jaewon Kim
2023-04-12 13:01 ` Michal Hocko
2023-04-12 16:49 ` T.J. Mercier
2023-04-12 22:10 ` Jaewon Kim
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p6>
2023-04-13 0:16 ` Jaewon Kim
2023-04-13 6:55 ` Michal Hocko
[not found] ` <CGME20230410073304epcas1p4cf3079b096994d69472b7801bd530bc7@epcms1p1>
2023-04-13 7:01 ` Jaewon Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230412113759epcms1p8cb15b54e3a96c7616419cb030d16f804@epcms1p8 \
--to=jaewon31.kim@samsung.com \
--cc=akpm@linux-foundation.org \
--cc=daniel.vetter@ffwll.ch \
--cc=hannes@cmpxchg.org \
--cc=jaewon31.kim@gmail.com \
--cc=jstultz@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=sumit.semwal@linaro.org \
--cc=tjmercier@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).