Re: PageBlocks and Migrate Types

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> Hello,
> 
> 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?

Pageblocks are a simple grouping of physically contiguous pages with
common set of flags. I haven't checked closely recently so I might
misremember but my recollection is that only the migrate type is stored
there. Normally we would store that information into page flags but
there is not enough room there.

MIGRATE_CMA represent pages allocated for the CMA allocator. There are
other migrate types denoting unmovable/movable allocations or pages that
are isolated from the page allocator.

Very broadly speaking, the migrate type groups pages with similar
movability properties to reduce fragmentation that compaction cannot
do anything about because there are objects of different properties
around. Please note that pageblock might contain objects of a different
migrate type in some cases (e.g. low on memory).

Have a look at gfpflags_to_migratetype and how the gfp mask is converted
to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
to see how it is used in the code.

> How many movable/unmovable pages are defined by default?

There is nothing like that. It depends on how many objects of a specific
type are allocated.

HTH
-- 
Michal Hocko
SUSE Labs

Re: PageBlocks and Migrate Types

[Pankaj Suryawanshi]

On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
>
> On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > Hello,
> >
> > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?
>
> Pageblocks are a simple grouping of physically contiguous pages with
> common set of flags. I haven't checked closely recently so I might
> misremember but my recollection is that only the migrate type is stored
> there. Normally we would store that information into page flags but
> there is not enough room there.
>
> MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> other migrate types denoting unmovable/movable allocations or pages that
> are isolated from the page allocator.
>
> Very broadly speaking, the migrate type groups pages with similar
> movability properties to reduce fragmentation that compaction cannot
> do anything about because there are objects of different properti
> around. Please note that pageblock might contain objects of a different
> migrate type in some cases (e.g. low on memory).
>
> Have a look at gfpflags_to_migratetype and how the gfp mask is converted
> to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
> to see how it is used in the code.
>
> > How many movable/unmovable pages are defined by default?
>
> There is nothing like that. It depends on how many objects of a specific
> type are allocated.


It means that it started creating pageblocks after allocation of
different objects, but from which block it allocate initially when
there is nothing like pageblocks ? (when memory subsystem up)

Pageblocks and its type dynamically changes ?
>
>
> HTH
> --
> Michal Hocko
> SUSE Labs

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Thu 22-08-19 23:54:19, Pankaj Suryawanshi wrote:
> On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> >
> > On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > > Hello,
> > >
> > > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux memory ?
> >
> > Pageblocks are a simple grouping of physically contiguous pages with
> > common set of flags. I haven't checked closely recently so I might
> > misremember but my recollection is that only the migrate type is stored
> > there. Normally we would store that information into page flags but
> > there is not enough room there.
> >
> > MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> > other migrate types denoting unmovable/movable allocations or pages that
> > are isolated from the page allocator.
> >
> > Very broadly speaking, the migrate type groups pages with similar
> > movability properties to reduce fragmentation that compaction cannot
> > do anything about because there are objects of different properti
> > around. Please note that pageblock might contain objects of a different
> > migrate type in some cases (e.g. low on memory).
> >
> > Have a look at gfpflags_to_migratetype and how the gfp mask is converted
> > to a migratetype for the allocation. Also follow different MIGRATE_$TYPE
> > to see how it is used in the code.
> >
> > > How many movable/unmovable pages are defined by default?
> >
> > There is nothing like that. It depends on how many objects of a specific
> > type are allocated.
> 
> 
> It means that it started creating pageblocks after allocation of
> different objects, but from which block it allocate initially when
> there is nothing like pageblocks ? (when memory subsystem up)

Pageblocks are just a way to group physically contiguous pages. They
just exist along with the physically contiguous memory. The migrate type
for most of the memory is set to MIGRATE_MOVABLE. Portion of the memory
might be reserved by CMA then that memory has MIGRATE_CMA. Following
set_pageblock_migratetype call paths will give you a good picture. 

> Pageblocks and its type dynamically changes ?

Yes as the purpose of the underlying memory for the block changes.
-- 
Michal Hocko
SUSE Labs

Re: PageBlocks and Migrate Types

[Michal Hocko]

On Mon 26-08-19 22:35:08, Pankaj Suryawanshi wrote:
> On Mon, Aug 26, 2019 at 12:34 PM Michal Hocko <mhocko@kernel.org> wrote:
> >
> > On Thu 22-08-19 23:54:19, Pankaj Suryawanshi wrote:
> > > On Thu, Aug 22, 2019 at 6:22 PM Michal Hocko <mhocko@kernel.org> wrote:
> > > >
> > > > On Wed 21-08-19 22:23:44, Pankaj Suryawanshi wrote:
> > > > > Hello,
> > > > >
> > > > > 1. What are Pageblocks and migrate types(MIGRATE_CMA) in Linux
> memory ?
> > > >
> > > > Pageblocks are a simple grouping of physically contiguous pages with
> > > > common set of flags. I haven't checked closely recently so I might
> > > > misremember but my recollection is that only the migrate type is
> stored
> > > > there. Normally we would store that information into page flags but
> > > > there is not enough room there.
> > > >
> > > > MIGRATE_CMA represent pages allocated for the CMA allocator. There are
> > > > other migrate types denoting unmovable/movable allocations or pages
> that
> > > > are isolated from the page allocator.
> > > >
> > > > Very broadly speaking, the migrate type groups pages with similar
> > > > movability properties to reduce fragmentation that compaction cannot
> > > > do anything about because there are objects of different properti
> > > > around. Please note that pageblock might contain objects of a
> different
> > > > migrate type in some cases (e.g. low on memory).
> > > >
> > > > Have a look at gfpflags_to_migratetype and how the gfp mask is
> converted
> > > > to a migratetype for the allocation. Also follow different
> MIGRATE_$TYPE
> > > > to see how it is used in the code.
> > > >
> > > > > How many movable/unmovable pages are defined by default?
> > > >
> > > > There is nothing like that. It depends on how many objects of a
> specific
> > > > type are allocated.
> > >
> > >
> > > It means that it started creating pageblocks after allocation of
> > > different objects, but from which block it allocate initially when
> > > there is nothing like pageblocks ? (when memory subsystem up)
> >
> > Pageblocks are just a way to group physically contiguous pages. They
> > just exist along with the physically contiguous memory. The migrate type
> > for most of the memory is set to MIGRATE_MOVABLE. Portion of the memory
> > might be reserved by CMA then that memory has MIGRATE_CMA. Following
> > set_pageblock_migratetype call paths will give you a good picture.
> 
> it means if i have 4096 continuous pages = 1 pageblock
> then all the 4096 pages of same type. but if any one page is different than
> block type then ? it changed the block type or something else ?

That really depends on the specific migrate type. CMA, ISOLATE migrate
types are all or nothing IIRC. I would have to check the code to tell
exactly when MOVABLE/UNMOVABLE pageblocks transitions are done.
steal_suitable_fallback sounds like a good start to look at.
-- 
Michal Hocko
SUSE Labs