Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[孙世龙 sunshilong]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

Are there still some methods that could be used by the Linux kernel
to reduce memory fragmentation while both CONFIG-MIGRATION
and CONFIG-COMPACTION are disabled?

Are there some system settings that could make for this goal?

[-- Attachment #2: Type: text/html, Size: 296 bytes --]

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[David Hildenbrand]

On 23.06.20 17:52, 孙世龙 sunshilong wrote:
> Are there still some methods that could be used by the Linux kernel 
> to reduce memory fragmentation while both CONFIG-MIGRATION 
> and CONFIG-COMPACTION are disabled?

We do have mobility grouping on pageblock order.

Also, I think you can use ZONE_MOVABLE without migration and compaction,
to at least locally limit unmovable fragmentation.

> 
> Are there some system settings that could make for this goal?

Why don't you want page compaction/migration?

-- 
Thanks,

David / dhildenb

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[孙世龙 sunshilong]

Thank you for your attention to this matter.

>> Are there still some methods that could be used by the Linux kernel
>> to reduce memory fragmentation while both CONFIG-MIGRATION
>> and CONFIG-COMPACTION are disabled?
>
>
>We do have mobility grouping on pageblock order.
>Also, I think you can use ZONE_MOVABLE without migration and compaction,
>to at least locally limit unmovable fragmentation.
It's a good news.

Could you please explain that in more detail for me or suggest some documents
for me to go through.

As per the post(http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06782.html),
I think it's something like ZONE_NORMAL. And I find that
"ZONE_MOVEABLE" is available on Linux-v4.9.

>>
>> Are there some system settings that could make for this goal?
>
>Why don't you want page compaction/migration?

Because I am using a real-time system(and I have to apply a patch
to Linux kernel), I have no choice other than disabling these
aforementioned options.

Is there some potential problems that I should be aware of if I enable
"ZONE_MOVABLE" on real-time system?

Best Regards.

David Hildenbrand <david@redhat.com> 于2020年6月24日周三 下午5:53写道：
>
> On 23.06.20 17:52, 孙世龙 sunshilong wrote:
> > Are there still some methods that could be used by the Linux kernel
> > to reduce memory fragmentation while both CONFIG-MIGRATION
> > and CONFIG-COMPACTION are disabled?
>
> We do have mobility grouping on pageblock order.
>
> Also, I think you can use ZONE_MOVABLE without migration and compaction,
> to at least locally limit unmovable fragmentation.
>
> >
> > Are there some system settings that could make for this goal?
>
> Why don't you want page compaction/migration?
>
> --
> Thanks,
>
> David / dhildenb
>

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[David Rientjes]

[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]

On Wed, 24 Jun 2020, 孙世龙 sunshilong wrote:

> >> Are there still some methods that could be used by the Linux kernel
> >> to reduce memory fragmentation while both CONFIG-MIGRATION
> >> and CONFIG-COMPACTION are disabled?
> >
> >
> >We do have mobility grouping on pageblock order.
> >Also, I think you can use ZONE_MOVABLE without migration and compaction,
> >to at least locally limit unmovable fragmentation.
> It's a good news.
> 
> Could you please explain that in more detail for me or suggest some documents
> for me to go through.
> 

/proc/buddyinfo and /proc/pagetypeinfo will show the various pageblocks on 
the system.  Grouping pages by mobility simply means that we attempt (best 
effort) to allocate unmovable pages from the same pageblocks and movable 
pages from the same pageblocks.

> As per the post(http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06782.html),
> I think it's something like ZONE_NORMAL. And I find that
> "ZONE_MOVEABLE" is available on Linux-v4.9.
> 

Yes, you can use kernelcore= (or movablecore=) on the kernel command line 
to set this up: allocations from this zone must have __GFP_MOVABLE so they 
"should" be migratable.  This is typically only useful when 
CONFIG_COMPACTION is enabled, however, to do that defragmentation work so 
that higher order memory becomes available.

> Is there some potential problems that I should be aware of if I enable
> "ZONE_MOVABLE" on real-time system?
> 

Yes, if ZONE_MOVABLE is made too big then you can risk out of memory 
kills: movable allocations can fallback to ZONE_NORMAL but unmovable 
allocations cannot graduate to ZONE_MOVABLE.  So if ZONE_NORMAL is full 
(either because you have too much unmovable memory in-use or too much 
movable fell back to ZONE_NORMAL), and you have more unmovable 
allocations, you'll get reclaim in ZONE_NORMAL and, at worst case, oom 
kills.

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[孙世龙 sunshilong]

[-- Attachment #1: Type: text/plain, Size: 6039 bytes --]

Hi,David Rientjes, David Hildenbrand
Thanks a lot to both of you.

>Grouping pages by mobility simply means that we attempt (best
>effort) to allocate unmovable pages from the same pageblocks and movable
>pages from the same pageblocks.

How can I understand it in the right way, especially for "from the same
pageblocks"? Could you please explain that in more detail?

>Yes, you can use kernelcore= (or movablecore=) on the kernel command line
>to set this up: allocations from this zone must have __GFP_MOVABLE so
>they "should" be migratable.  This is typically only useful when
>CONFIG_COMPACTION is enabled, however, to do that defragmentation work so
>that higher order memory becomes available.
I was very hopeful for this method(i.e. ZONE_MOVABLE).
It's really bad news. As the subject, I have no choice other than disabling
these options(i.e. CONFIG-MIGRATION and CONFIG-COMPACTION)
since I am using a real-time OS(i.e. it needs a patch to the Linux kernel
and some kconfig options should be disabled).

Are there still some methods that could be used by the Linux kernel
to reduce memory fragmentation?


>> Is there some potential problems that I should be aware of if I enable
>> "ZONE_MOVABLE" on real-time system?
>>

>Yes, if ZONE_MOVABLE is made too big then you can risk out of memory
>kills: movable allocations can fallback to ZONE_NORMAL but unmovable
>allocations cannot graduate to ZONE_MOVABLE.  So if ZONE_NORMAL is full
>(either because you have too much unmovable memory in-use or too much
>movable fell back to ZONE_NORMAL), and you have more unmovable
>allocations, you'll get reclaim in ZONE_NORMAL and, at worst case, oom
>kills.
I can draw the conclusion that ZONE_NORMAL is only used to allocate
unmovable memory(i.e. no movable memory could be allocated from
ZONE_NORMAL) while "kernelcore= (or movablecore=)" option is set.
Am I right?


As the kernel without such option(i.e. kernelcore= or movablecore=), there
are
both movable memory and unmovable memory in ZONE_NORMAL.
For details, see the footnote.

It's different with the option and without the option:
Only unmovable memory could be allocated from ZONE_NORMAL while the option
is set whereas both unmovable and movable could be allocated from it
without
setting the option.
Am I right?


Here is part of the output when executing "sudo cat /proc/pagetypeinfo" on
the platform
without the option(i.e. kernelcore= or movablecore=)：
Free pages count per migrate type at order
                           0   1   2   3   4   5   6   7   8   9   10
   DMA, type    Unmovable  0   1   1   0   2   1   1   0   1   0    0
   DMA, type      Movable  0   0   0   0   0   0   0   0   0   1    3
   DMA, type  Reclaimable  0   0   0   0   0   0   0   0   0   0    0
   DMA, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
   DMA, type      Isolate  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type    Unmovable  1   0   0   0   0   0   1   1   1   1    0
 DMA32, type      Movable  8   6   7   5   5   4   6   5   2   2  723
 DMA32, type  Reclaimable  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
 DMA32, type      Isolate  0   0   0   0   0   0   0   0   0   0    0
Normal, type    Unmovable  0  23   9   2   1   1   0   1  10  11    0
Normal, type      Movable  1 791 767 630 111  11   5   5   2   0  929
Normal, type  Reclaimable  0   2   4   2   2   1   1   1   1   1    0
Normal, type   HighAtomic  0   0   0   0   0   0   0   0   0   0    0
Normal, type      Isolate  0   0   0   0   0   0   0   0   0   0    0

Number of blocks type Unmovable Movable Reclaimable HighAtomic Isolate
Node 0, zone      DMA        1       7           0          0       0
Node 0, zone    DMA32        2    1526           0          0       0
Node 0, zone   Normal      160    2318          74          0       0

Best regards.

David Rientjes <rientjes@google.com> 于2020年6月25日周四 上午1:22写道：

> On Wed, 24 Jun 2020, 孙世龙 sunshilong wrote:
>
> > >> Are there still some methods that could be used by the Linux kernel
> > >> to reduce memory fragmentation while both CONFIG-MIGRATION
> > >> and CONFIG-COMPACTION are disabled?
> > >
> > >
> > >We do have mobility grouping on pageblock order.
> > >Also, I think you can use ZONE_MOVABLE without migration and compaction,
> > >to at least locally limit unmovable fragmentation.
> > It's a good news.
> >
> > Could you please explain that in more detail for me or suggest some
> documents
> > for me to go through.
> >
>
> /proc/buddyinfo and /proc/pagetypeinfo will show the various pageblocks on
> the system.  Grouping pages by mobility simply means that we attempt (best
> effort) to allocate unmovable pages from the same pageblocks and movable
> pages from the same pageblocks.
>
> > As per the post(
> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06782.html),
> > I think it's something like ZONE_NORMAL. And I find that
> > "ZONE_MOVEABLE" is available on Linux-v4.9.
> >
>
> Yes, you can use kernelcore= (or movablecore=) on the kernel command line
> to set this up: allocations from this zone must have __GFP_MOVABLE so they
> "should" be migratable.  This is typically only useful when
> CONFIG_COMPACTION is enabled, however, to do that defragmentation work so
> that higher order memory becomes available.
>
> > Is there some potential problems that I should be aware of if I enable
> > "ZONE_MOVABLE" on real-time system?
> >
>
> Yes, if ZONE_MOVABLE is made too big then you can risk out of memory
> kills: movable allocations can fallback to ZONE_NORMAL but unmovable
> allocations cannot graduate to ZONE_MOVABLE.  So if ZONE_NORMAL is full
> (either because you have too much unmovable memory in-use or too much
> movable fell back to ZONE_NORMAL), and you have more unmovable
> allocations, you'll get reclaim in ZONE_NORMAL and, at worst case, oom
> kills.

[-- Attachment #2: Type: text/html, Size: 7311 bytes --]

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[David Rientjes]

[-- Attachment #1: Type: text/plain, Size: 1630 bytes --]

On Thu, 25 Jun 2020, 孙世龙 sunshilong wrote:

> >Grouping pages by mobility simply means that we attempt (best
> >effort) to allocate unmovable pages from the same pageblocks and movable
> >pages from the same pageblocks.
> 
> How can I understand it in the right way, especially for "from the same
> pageblocks"? Could you please explain that in more detail?
> 

Movable page allocations (__GFP_MOVABLE) are first attempted from 
MIGRATE_MOVABLE pageblocks; unmovable page allocations (~__GFP_MOVABLE) 
are first attempted from MIGRATE_UNMOVABLE.  That doesn't necessarily help 
if migration and compaction are disabled.

__GFP_RECLAIMABLE (like dentries and inodes, for instance) have their own 
pageblocks, however: MIGRATE_RECLAIMABLE.  If we can reclaim all memory 
from these pageblocks, and it is all reclaimable (we didn't have to 
allocate unmovable pages from it, for example), it should free up the 
whole pageblock. 

Other than that, the page allocator tries to grab the smallest free page 
when necessary to leave higher order pages available.

> Are there still some methods that could be used by the Linux kernel
> to reduce memory fragmentation?
> 

Without compaction or migration, that's just about it.

> I can draw the conclusion that ZONE_NORMAL is only used to allocate
> unmovable memory(i.e. no movable memory could be allocated from
> ZONE_NORMAL) while "kernelcore= (or movablecore=)" option is set.
> Am I right?
> 

Movable memory can be allocated from ZONE_NORMAL on fallback.  The reverse 
is not true: unmovable memory cannot be allocated from ZONE_MOVABLE.

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[孙世龙 sunshilong]

Hi, David Rientjes

      Thank you for the clarification.
      My current understanding of this matter is at a different level
with your help.

     >>Are there still some methods that could be used by the Linux kernel
     >> to reduce memory fragmentation?
     >
     >Without compaction or migration, that's just about it.
     Do you mean there are no other methods that could be used to
     achieve this goal?

     Thank you for your attention to this matter.
     Best Regards.
     sunshilong

David Rientjes <rientjes@google.com> 于2020年6月28日周日 上午11:27写道：
>
> On Thu, 25 Jun 2020, 孙世龙 sunshilong wrote:
>
> > >Grouping pages by mobility simply means that we attempt (best
> > >effort) to allocate unmovable pages from the same pageblocks and movable
> > >pages from the same pageblocks.
> >
> > How can I understand it in the right way, especially for "from the same
> > pageblocks"? Could you please explain that in more detail?
> >
>
> Movable page allocations (__GFP_MOVABLE) are first attempted from
> MIGRATE_MOVABLE pageblocks; unmovable page allocations (~__GFP_MOVABLE)
> are first attempted from MIGRATE_UNMOVABLE.  That doesn't necessarily help
> if migration and compaction are disabled.
>
> __GFP_RECLAIMABLE (like dentries and inodes, for instance) have their own
> pageblocks, however: MIGRATE_RECLAIMABLE.  If we can reclaim all memory
> from these pageblocks, and it is all reclaimable (we didn't have to
> allocate unmovable pages from it, for example), it should free up the
> whole pageblock.
>
> Other than that, the page allocator tries to grab the smallest free page
> when necessary to leave higher order pages available.
>
> > Are there still some methods that could be used by the Linux kernel
> > to reduce memory fragmentation?
> >
>
> Without compaction or migration, that's just about it.
>
> > I can draw the conclusion that ZONE_NORMAL is only used to allocate
> > unmovable memory(i.e. no movable memory could be allocated from
> > ZONE_NORMAL) while "kernelcore= (or movablecore=)" option is set.
> > Am I right?
> >
>
> Movable memory can be allocated from ZONE_NORMAL on fallback.  The reverse
> is not true: unmovable memory cannot be allocated from ZONE_MOVABLE.

Re: Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[David Rientjes]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

On Sun, 28 Jun 2020, 孙世龙 sunshilong wrote:

>      >>Are there still some methods that could be used by the Linux kernel
>      >> to reduce memory fragmentation?
>      >
>      >Without compaction or migration, that's just about it.
>      Do you mean there are no other methods that could be used to
>      achieve this goal?
> 

Without compaction and migration, we don't have the ability to move pages 
around for defragmentation so any strategy can only be used at allocation 
time.

__rmqueue_smallest() will attempt to take the smallest possible free page 
to leave higher order pages available in an attempt to consolidate 
allocations on the smallest set of pageblocks as possible given the 
allocation migratetype.

When we do have to fallback to a pageblock of a different migratetype, we 
take the largest possible free page and convert it over if it's at least 
half free for subsequent allocations of the desired migratetype.

That's the extent that we can do this at allocation time.

Are there still some methods that could be used by the Linux kernel to reduce memory fragmentation while both CONFIG-MIGRATION and CONFIG-COMPACTION are disabled?

[孙世龙 sunshilong]

[-- Attachment #1: Type: text/plain, Size: 226 bytes --]

Are there still some methods that could be used by the Linux kernel
to reduce memory fragmentation while both CONFIG-MIGRATION
and CONFIG-COMPACTION are disabled?

Are there some system settings that could make for this goal?

[-- Attachment #2: Type: text/html, Size: 296 bytes --]