From: Roman Gushchin <guro@fb.com>
To: Mike Kravetz <mike.kravetz@oracle.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Zi Yan <ziy@nvidia.com>, Joonsoo Kim <iamjoonsoo.kim@lge.com>,
<linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>,
<kernel-team@fb.com>
Subject: Re: [PATCH rfc 0/2] mm: cma: make cma_release() non-blocking
Date: Thu, 22 Oct 2020 10:16:28 -0700 [thread overview]
Message-ID: <20201022171628.GE300658@carbon.dhcp.thefacebook.com> (raw)
In-Reply-To: <91779b4c-378d-66ee-2df0-edb270dd4d04@oracle.com>
On Thu, Oct 22, 2020 at 09:42:11AM -0700, Mike Kravetz wrote:
> On 10/21/20 7:33 PM, Roman Gushchin wrote:
> > On Wed, Oct 21, 2020 at 05:15:53PM -0700, Mike Kravetz wrote:
> >> On 10/16/20 3:52 PM, Roman Gushchin wrote:
> >>> This small patchset makes cma_release() non-blocking and simplifies
> >>> the code in hugetlbfs, where previously we had to temporarily drop
> >>> hugetlb_lock around the cma_release() call.
> >>>
> >>> It should help Zi Yan on his work on 1 GB THPs: splitting a gigantic
> >>> THP under a memory pressure requires a cma_release() call. If it's
> >>> a blocking function, it complicates the already complicated code.
> >>> Because there are at least two use cases like this (hugetlbfs is
> >>> another example), I believe it's just better to make cma_release()
> >>> non-blocking.
> >>>
> >>> It also makes it more consistent with other memory releasing functions
> >>> in the kernel: most of them are non-blocking.
> >>
> >> Thanks for looking into this Roman.
> >
> > Hi Mike,
> >
> >>
> >> I may be missing something, but why does cma_release have to be blocking
> >> today? Certainly, it takes the bitmap in cma_clear_bitmap and could
> >> block. However, I do not see why cma->lock has to be a mutex. I may be
> >> missing something, but I do not see any code protected by the mutex doing
> >> anything that could sleep?
> >>
> >> Could we simply change that mutex to a spinlock?
> >
> > I actually have suggested it few months ago, but the idea was
> > opposed by Joonsoo: https://lkml.org/lkml/2020/4/3/12 .
> >
> > The time of a bitmap operation is definitely not an issue in my context,
> > but I can't speak for something like an embedded/rt case.
> >
>
> I wonder if it may be time to look into replacing the cma area bitmap
> with some other data structure? Joonsoo was concerned about the time
> required to traverse the bitmap for an 8GB area. With new support for
> allocating 1GB hugetlb pages from cma, I can imagine someone setting
> up a cma area that is hundreds of GB if not TB in size. It is going
> take some time to traverse a bitmap describing such a huge area.
If the cma area is used exclusively for 1 GB allocations, the bitmap can
have only 1 bit per GB, so it shouldn't be a big problem.
Long-term I have some hopes to be able to allocate 1 GB pages without
a need to reserve a cma area: we can try to group pages based on their mobility
on a 1 GB scale, so that all non-movable pages will reside in few 1 GB blocks.
I'm looking into that direction, but don't have any results yet.
If this idea fails and we'll end up allocating a large cma area unconditionally
and shrink it on demand (I think Rik suggested something like this),
replacing the bitmap with something else sounds like a good idea to me.
As now, I want to unblock Zi Yan on his work on 1 GB THPs, so maybe
we can go with introducing cma_release_nowait(), as I suggested in
the other e-mail in this thread? Do you have any objections?
Thanks!
next prev parent reply other threads:[~2020-10-22 17:16 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-16 22:52 [PATCH rfc 0/2] mm: cma: make cma_release() non-blocking Roman Gushchin
2020-10-16 22:52 ` [PATCH rfc 1/2] " Roman Gushchin
2020-10-16 22:52 ` [PATCH rfc 2/2] mm: hugetlb: don't drop hugetlb_lock around cma_release() call Roman Gushchin
2020-10-22 0:15 ` [PATCH rfc 0/2] mm: cma: make cma_release() non-blocking Mike Kravetz
2020-10-22 2:33 ` Roman Gushchin
2020-10-22 16:42 ` Mike Kravetz
2020-10-22 17:16 ` Roman Gushchin [this message]
2020-10-22 17:25 ` Mike Kravetz
2020-10-22 1:54 ` Xiaqing (A)
2020-10-22 2:45 ` Roman Gushchin
2020-10-22 3:47 ` Xiaqing (A)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201022171628.GE300658@carbon.dhcp.thefacebook.com \
--to=guro@fb.com \
--cc=akpm@linux-foundation.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).