RE: Re: RE(2): FW: [LSF/MM/BPF TOPIC] SMDK inspired MM changes for CXL

[Kyungsan Kim <ks0204.kim@samsung.com>]

Hi Matthew Wilcox. 
We appreciate you join this topic and revise our sentences sophisticatedly.

>On Thu, Mar 23, 2023 at 07:51:05PM +0900, Kyungsan Kim wrote:
>> One problem we experienced was occured in the combination of hot-remove and kerelspace allocation usecases.
>> ZONE_NORMAL allows kernel context allocation, but it does not allow hot-remove because kernel resides all the time.
>> ZONE_MOVABLE allows hot-remove due to the page migration, but it only allows userspace allocation.
>
>No, that's not true.  You can allocate kernel memory from ZONE_MOVABLE.
>You have to be careful when you do that, but eg filesystems put symlinks
>and directories in ZONE_MOVABLE, and zswap allocates memory from
>ZONE_MOVABLE.  Of course, then you have to be careful that the kernel
>doesn't try to move it while you're accessing it.  That's the tradeoff.

You are correct.
In fact, the intention of the sentence was to generally explain the movability preference of a kernel and user context.
We have been aware that a kernel context is able to allocate from ZONE_MOVABLE
using GFP_MOVABLE and implementing the movable callbacks, migrate_page(), putback_page(), isolate_page().
We had studied that the z3fold/zsmalloc allocator of zswap also allocate from ZONE_MOVABLE.
But we did not aware that symlinks and dentries are allocated from ZONE_MOVABLE.
Thank you for letting us know the additional cases.

Let me revise the part. In regards to page movability, 
a kernel context prefers unmovable in general, but some kernel contexts are movable such as symlink, dentry, and zswap.
an user context prefers movable in general, but some user contexts are unmovable such as DMA buffer.

>
>> Alternatively, we allocated a kernel context out of ZONE_MOVABLE by adding GFP_MOVABLE flag.
>> In case, oops and system hang has occasionally occured because ZONE_MOVABLE can be swapped.
>
>I think you mean "migrated".  It can't be swapped unless you put the
>page on the LRU list, inviting the kernel to swap it.

"migrated" is correct.

>
>> We resolved the issue using ZONE_EXMEM by allowing seletively choice of the two usecases.
>
>This sounds dangerously confused.  Do you want the EXMEM to be removable
>or not?  If you do, then allocations from it have to be movable.  If
>you don't, why go to all this trouble?

I'm sorry to make you confused. We will try more to clearly explain our thought.
We think the CXL DRAM device should be removable along with HW pluggable nature.
For MM point of view, we think a page of CXL DRAM can be both movable and unmovable. 
An user or kernel context should be able to determine it. Thus, we think dedication on the ZONE_NORMAL or the ZONE_MOVABLE is not enough.