Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

["孙世龙 sunshilong" <sunshilong369@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6039 bytes --]

Hi,David Rientjes, David Hildenbrand
Thanks a lot to both of you.

>Grouping pages by mobility simply means that we attempt (best
>effort) to allocate unmovable pages from the same pageblocks and movable
>pages from the same pageblocks.

How can I understand it in the right way, especially for "from the same
pageblocks"? Could you please explain that in more detail?

>Yes, you can use kernelcore= (or movablecore=) on the kernel command line
>to set this up: allocations from this zone must have __GFP_MOVABLE so
>they "should" be migratable.  This is typically only useful when
>CONFIG_COMPACTION is enabled, however, to do that defragmentation work so
>that higher order memory becomes available.
I was very hopeful for this method(i.e. ZONE_MOVABLE).
It's really bad news. As the subject, I have no choice other than disabling
these options(i.e. CONFIG-MIGRATION and CONFIG-COMPACTION)
since I am using a real-time OS(i.e. it needs a patch to the Linux kernel
and some kconfig options should be disabled).

Are there still some methods that could be used by the Linux kernel
to reduce memory fragmentation?


>> Is there some potential problems that I should be aware of if I enable
>> "ZONE_MOVABLE" on real-time system?
>>

>Yes, if ZONE_MOVABLE is made too big then you can risk out of memory
>kills: movable allocations can fallback to ZONE_NORMAL but unmovable
>allocations cannot graduate to ZONE_MOVABLE.  So if ZONE_NORMAL is full
>(either because you have too much unmovable memory in-use or too much
>movable fell back to ZONE_NORMAL), and you have more unmovable
>allocations, you'll get reclaim in ZONE_NORMAL and, at worst case, oom
>kills.
I can draw the conclusion that ZONE_NORMAL is only used to allocate
unmovable memory(i.e. no movable memory could be allocated from
ZONE_NORMAL) while "kernelcore= (or movablecore=)" option is set.
Am I right?


As the kernel without such option(i.e. kernelcore= or movablecore=), there
are
both movable memory and unmovable memory in ZONE_NORMAL.
For details, see the footnote.

It's different with the option and without the option:
Only unmovable memory could be allocated from ZONE_NORMAL while the option
is set whereas both unmovable and movable could be allocated from it
without
setting the option.
Am I right?


Here is part of the output when executing "sudo cat /proc/pagetypeinfo" on
the platform
without the option(i.e. kernelcore= or movablecore=)：
Free pages count per migrate type at order
                           0   1   2   3   4   5   6   7   8   9   10
   DMA, type    Unmovable  0   1   1   0   2   1   1   0   1   0    0
   DMA, type      Movable  0   0   0   0   0   0   0   0   0   1    3
   DMA, type  Reclaimable  0   0   0   0   0   0   0   0   0   0    0
   DMA, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
   DMA, type      Isolate  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type    Unmovable  1   0   0   0   0   0   1   1   1   1    0
 DMA32, type      Movable  8   6   7   5   5   4   6   5   2   2  723
 DMA32, type  Reclaimable  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type      Isolate  0   0   0   0   0   0   0   0   0   0    0
Normal, type    Unmovable  0  23   9   2   1   1   0   1  10  11    0
Normal, type      Movable  1 791 767 630 111  11   5   5   2   0  929
Normal, type  Reclaimable  0   2   4   2   2   1   1   1   1   1    0
Normal, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
Normal, type      Isolate  0   0   0   0   0   0   0   0   0   0    0

Number of blocks type Unmovable Movable Reclaimable HighAtomic Isolate
Node 0, zone      DMA        1       7           0          0       0
Node 0, zone    DMA32        2    1526           0          0       0
Node 0, zone   Normal      160    2318          74          0       0

Best regards.

David Rientjes <rientjes@google.com> 于2020年6月25日周四 上午1:22写道：

> On Wed, 24 Jun 2020, 孙世龙 sunshilong wrote:
>
> > >> Are there still some methods that could be used by the Linux kernel
> > >> to reduce memory fragmentation while both CONFIG-MIGRATION
> > >> and CONFIG-COMPACTION are disabled?
> > >
> > >
> > >We do have mobility grouping on pageblock order.
> > >Also, I think you can use ZONE_MOVABLE without migration and compaction,
> > >to at least locally limit unmovable fragmentation.
> > It's a good news.
> >
> > Could you please explain that in more detail for me or suggest some
> documents
> > for me to go through.
> >
>
> /proc/buddyinfo and /proc/pagetypeinfo will show the various pageblocks on
> the system.  Grouping pages by mobility simply means that we attempt (best
> effort) to allocate unmovable pages from the same pageblocks and movable
> pages from the same pageblocks.
>
> > As per the post(
> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06782.html),
> > I think it's something like ZONE_NORMAL. And I find that
> > "ZONE_MOVEABLE" is available on Linux-v4.9.
> >
>
> Yes, you can use kernelcore= (or movablecore=) on the kernel command line
> to set this up: allocations from this zone must have __GFP_MOVABLE so they
> "should" be migratable.  This is typically only useful when
> CONFIG_COMPACTION is enabled, however, to do that defragmentation work so
> that higher order memory becomes available.
>
> > Is there some potential problems that I should be aware of if I enable
> > "ZONE_MOVABLE" on real-time system?
> >
>
> Yes, if ZONE_MOVABLE is made too big then you can risk out of memory
> kills: movable allocations can fallback to ZONE_NORMAL but unmovable
> allocations cannot graduate to ZONE_MOVABLE.  So if ZONE_NORMAL is full
> (either because you have too much unmovable memory in-use or too much
> movable fell back to ZONE_NORMAL), and you have more unmovable
> allocations, you'll get reclaim in ZONE_NORMAL and, at worst case, oom
> kills.

[-- Attachment #2: Type: text/html, Size: 7311 bytes --]