How to handle page allocation when memory exceeds a local node

How to handle page allocation when memory exceeds a local node

[Won-Kyo Choe]

Suppose that there are two nodes and each node has 16GiB memory size.

When a process needs pages from the kernel, I understand that
__alloc_pages_nodemask() will do allocation for the process.

In the function, I noticed that if there is no available page on
a zone (or a node) while it loops for_next_zone_zonelist_nodemask()
(a macro in get_page_from_freelist()), the kernel would try to reclaim
reclaimable page from the zone. This loops will continue until the
kernel gets a free page. This means that the kernel would try to find a
page from the remote node if there is no free page in the local node.

My question is that when the kernel starts to allocate a page from the
remote node due to no enough memory in the local node, does the kernel
always check there is a reclaimable page from the local node?
In other words, does the kernel always start to find a page from the
very first entry (which would be the local node and the first zone)
in the loop?

In my perspective, if the kernel starts to allocate in the remote node,
I think the scheduler should move the process to the remote node and it
will allocate a page in the remote node at first in the loop (in the
process view, the node would be the local now since it is moved). Would
the scheduler do that?


Regards,
WK Choe


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to handle page allocation when memory exceeds a local node

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 684 bytes --]

On Thu, 22 Aug 2019 17:41:22 +0900, Won-Kyo Choe said:

> In my perspective, if the kernel starts to allocate in the remote node,
> I think the scheduler should move the process to the remote node and it
> will allocate a page in the remote node at first in the loop (in the
> process view, the node would be the local now since it is moved). Would
> the scheduler do that?

That's not the scheduler's job to do that.  Plus... what do you do about the
case where a process already has 12G of memory on one node, that node runs out
and 1 4K page gets allocated on another node.  Which is better, move the 12G,
or every once in a while try to relocate that 1 4K page to a better node?


[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: How to handle page allocation when memory exceeds a local node

[Won-Kyo Choe]

On Thu, Aug 22, 2019 at 06:00:01AM -0400, Valdis Klētnieks wrote:
> On Thu, 22 Aug 2019 17:41:22 +0900, Won-Kyo Choe said:
> 
> > In my perspective, if the kernel starts to allocate in the remote node,
> > I think the scheduler should move the process to the remote node and it
> > will allocate a page in the remote node at first in the loop (in the
> > process view, the node would be the local now since it is moved). Would
> > the scheduler do that?
> 
> That's not the scheduler's job to do that.  Plus... what do you do about the
> case where a process already has 12G of memory on one node, that node runs out
> and 1 4K page gets allocated on another node.  Which is better, move the 12G,
> or every once in a while try to relocate that 1 4K page to a better node?
> 
I appreciate it for your answer.

Firstly, I would say that the latter is better because of the small size
but what do you exactly refer to a better node? Could you explain more
specifically? I am not sure but I think that a better node means a node
where the process can show better performance and this would be done by
the scheduler in a certain way.

Secondly, I think that you assume there should always be migration when
the process has allocated pages in the remote node to try to exploit
performance benefits on the local node. Is it a default policy of the
kernel? My concern is that what if the process uses the whole memory
capacity with two nodes and the process has one thread? In this case
could I still get expectation that there is a myriad of migration or
exchange between two nodes?

Regards,
WK Choe

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies