From: Balbir Singh <bsingharora@gmail.com>
To: js1304@gmail.com, Andrew Morton <akpm@linux-foundation.org>
Cc: Rik van Riel <riel@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
mgorman@techsingularity.net, Laura Abbott <lauraa@codeaurora.org>,
Minchan Kim <minchan@kernel.org>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Michal Nazarewicz <mina86@mina86.com>,
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com>,
Vlastimil Babka <vbabka@suse.cz>,
Rui Teng <rui.teng@linux.vnet.ibm.com>,
linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Joonsoo Kim <iamjoonsoo.kim@lge.com>
Subject: Re: [PATCH v3 0/6] Introduce ZONE_CMA
Date: Mon, 27 Jun 2016 21:25:45 +1000 [thread overview]
Message-ID: <57710D39.4060109@gmail.com> (raw)
In-Reply-To: <1464243748-16367-1-git-send-email-iamjoonsoo.kim@lge.com>
On 26/05/16 16:22, js1304@gmail.com wrote:
> From: Joonsoo Kim <iamjoonsoo.kim@lge.com>
>
> Hello,
>
> Changes from v2
> o Rebase on next-20160525
> o No other changes except following description
>
> There was a discussion with Mel [1] after LSF/MM 2016. I could summarise
> it to help merge decision but it's better to read by yourself since
> if I summarise it, it would be biased for me. But, if anyone hope
> the summary, I will do it. :)
>
> Anyway, Mel's position on this patchset seems to be neutral. He said:
> "I'm not going to outright NAK your series but I won't ACK it either"
>
> We can fix the problems with any approach but I hope to go a new zone
> approach because it is less error-prone. It reduces some corner case
> handling for now and remove need for potential corner case handling to fix
> problems.
>
> Note that our company is already using ZONE_CMA for a years and
> there is no problem.
>
> If anyone has a different opinion, please let me know and let's discuss
> together.
>
> Andrew, if there is something to do for merge, please let me know.
>
> [1] https://lkml.kernel.org/r/20160425053653.GA25662@js1304-P5Q-DELUXE
>
> Changes from v1
> o Separate some patches which deserve to submit independently
> o Modify description to reflect current kernel state
> (e.g. high-order watermark problem disappeared by Mel's work)
> o Don't increase SECTION_SIZE_BITS to make a room in page flags
> (detailed reason is on the patch that adds ZONE_CMA)
> o Adjust ZONE_CMA population code
>
> This series try to solve problems of current CMA implementation.
>
> CMA is introduced to provide physically contiguous pages at runtime
> without exclusive reserved memory area. But, current implementation
> works like as previous reserved memory approach, because freepages
> on CMA region are used only if there is no movable freepage. In other
> words, freepages on CMA region are only used as fallback. In that
> situation where freepages on CMA region are used as fallback, kswapd
> would be woken up easily since there is no unmovable and reclaimable
> freepage, too. If kswapd starts to reclaim memory, fallback allocation
> to MIGRATE_CMA doesn't occur any more since movable freepages are
> already refilled by kswapd and then most of freepage on CMA are left
> to be in free. This situation looks like exclusive reserved memory case.
I am afraid I don't understand the problem statement completely understand.
Is this the ALLOC_CMA case or the !ALLOC_CMA one? I also think one other
problem is that in my experience and observation all CMA allocations seem
to come from one node-- the highest node on the system
>
> In my experiment, I found that if system memory has 1024 MB memory and
> 512 MB is reserved for CMA, kswapd is mostly woken up when roughly 512 MB
> free memory is left. Detailed reason is that for keeping enough free
> memory for unmovable and reclaimable allocation, kswapd uses below
> equation when calculating free memory and it easily go under the watermark.
>
> Free memory for unmovable and reclaimable = Free total - Free CMA pages
>
> This is derivated from the property of CMA freepage that CMA freepage
> can't be used for unmovable and reclaimable allocation.
>
> Anyway, in this case, kswapd are woken up when (FreeTotal - FreeCMA)
> is lower than low watermark and tries to make free memory until
> (FreeTotal - FreeCMA) is higher than high watermark. That results
> in that FreeTotal is moving around 512MB boundary consistently. It
> then means that we can't utilize full memory capacity.
>
OK.. so you are suggesting that we are under-utilizing the memory in the
CMA region?
> To fix this problem, I submitted some patches [1] about 10 months ago,
> but, found some more problems to be fixed before solving this problem.
> It requires many hooks in allocator hotpath so some developers doesn't
> like it. Instead, some of them suggest different approach [2] to fix
> all the problems related to CMA, that is, introducing a new zone to deal
> with free CMA pages. I agree that it is the best way to go so implement
> here. Although properties of ZONE_MOVABLE and ZONE_CMA is similar, I
> decide to add a new zone rather than piggyback on ZONE_MOVABLE since
> they have some differences. First, reserved CMA pages should not be
> offlined.
Why? Why are they special? Even if they are offlined by user action,
one would expect the following to occur
1. User would mark/release the cma region associated with them
2. User would then hotplug the memory
> If freepage for CMA is managed by ZONE_MOVABLE, we need to keep
> MIGRATE_CMA migratetype and insert many hooks on memory hotplug code
> to distiguish hotpluggable memory and reserved memory for CMA in the same
> zone. It would make memory hotplug code which is already complicated
> more complicated.
Again why treat it special, one could potentially deny the hotplug based
on the knowledge of where the CMA region is allocated from
> Second, cma_alloc() can be called more frequently
> than memory hotplug operation and possibly we need to control
> allocation rate of ZONE_CMA to optimize latency in the future.
> In this case, separate zone approach is easy to modify. Third, I'd
> like to see statistics for CMA, separately. Sometimes, we need to debug
> why cma_alloc() is failed and separate statistics would be more helpful
> in this situtaion.
>
> Anyway, this patchset solves four problems related to CMA implementation.
>
Balbir
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-06-27 11:26 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-05-26 6:22 [PATCH v3 0/6] Introduce ZONE_CMA js1304
2016-05-26 6:22 ` [PATCH v3 1/6] mm/page_alloc: recalculate some of zone threshold when on/offline memory js1304
2016-06-24 13:20 ` Vlastimil Babka
2016-06-28 8:12 ` Joonsoo Kim
2016-05-26 6:22 ` [PATCH v3 2/6] mm/cma: introduce new zone, ZONE_CMA js1304
2016-05-26 6:22 ` [PATCH v3 3/6] mm/cma: populate ZONE_CMA js1304
2016-06-22 9:23 ` Chen Feng
2016-06-23 2:52 ` Joonsoo Kim
2016-06-28 11:23 ` Chen Feng
2016-06-29 8:00 ` Joonsoo Kim
2016-06-27 8:24 ` Vlastimil Babka
2016-06-28 8:31 ` Joonsoo Kim
2016-05-26 6:22 ` [PATCH v3 4/6] mm/cma: remove ALLOC_CMA js1304
2016-06-27 9:30 ` Vlastimil Babka
2016-06-28 8:16 ` Joonsoo Kim
2016-05-26 6:22 ` [PATCH v3 5/6] mm/cma: remove MIGRATE_CMA js1304
2016-05-27 1:42 ` Chen Feng
2016-05-27 5:32 ` Joonsoo Kim
2016-06-27 9:46 ` Vlastimil Babka
2016-06-28 8:17 ` Joonsoo Kim
2016-05-26 6:22 ` [PATCH v3 6/6] mm/cma: remove per zone CMA stat js1304
2016-06-27 9:54 ` Vlastimil Babka
2016-05-26 8:04 ` [PATCH v3 0/6] Introduce ZONE_CMA Feng Tang
2016-05-27 5:28 ` Joonsoo Kim
2016-05-27 6:25 ` Feng Tang
2016-05-27 6:42 ` Joonsoo Kim
2016-05-27 7:27 ` Feng Tang
2016-05-30 5:45 ` Joonsoo Kim
2016-06-17 7:38 ` Chen Feng
2016-06-20 6:48 ` Joonsoo Kim
2016-06-21 2:08 ` Chen Feng
2016-06-21 6:56 ` Joonsoo Kim
2016-06-27 11:25 ` Balbir Singh [this message]
2016-06-29 7:57 ` Joonsoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=57710D39.4060109@gmail.com \
--to=bsingharora@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=hannes@cmpxchg.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=js1304@gmail.com \
--cc=lauraa@codeaurora.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=m.szyprowski@samsung.com \
--cc=mgorman@techsingularity.net \
--cc=mina86@mina86.com \
--cc=minchan@kernel.org \
--cc=riel@redhat.com \
--cc=rui.teng@linux.vnet.ibm.com \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).