Felix Kuehling <felix.kuehling@amd.com> writes:

> Am 2022-02-16 um 07:26 schrieb Jason Gunthorpe:
>> The other place that needs careful audit is all the callers using
>> vm_normal_page() - they must all be able to accept a ZONE_DEVICE page
>> if we don't set pte_devmap.
>
> How much code are we talking about here? A quick search finds 26 call-sites in
> 12 files in current master:
>
>    fs/proc/task_mmu.c
>    mm/hmm.c
>    mm/gup.c
>    mm/huge_memory.c (vm_normal_page_pmd)
>    mm/khugepaged.c
>    mm/madvise.c
>    mm/mempolicy.c
>    mm/memory.c
>    mm/mlock.c
>    mm/migrate.c
>    mm/mprotect.c
>    mm/memcontrol.c
>
> I'm thinking of a more theoretical approach: Instead of auditing all users, I'd
> ask, what are the invariants that a vm_normal_page should have. Then check,
> whether our DEVICE_COHERENT pages satisfy them. But maybe the concept of a
> vm_normal_page isn't defined clearly enough for that.
>
> That said, I think we (Alex and myself) made an implicit assumption from the
> start, that a DEVICE_COHERENT page should behave a lot like a normal page in
> terms of VMA mappings, even if we didn't know what that means in detail.

Yes I'm afraid I made a similar mistake when reviewing this, forgetting that
DEVICE_COHERENT pages are not LRU pages and therefore need special treatment in
some places. So for now I will have to withdraw my reviewed-by until this has
been looked at more closely, because as you note below accidentally treating
them as LRU pages leads to a bad time.

> I can now at least name some differences between DEVICE_COHERENT and normal
> pages: how the memory is allocated, how data is migrated into DEVICE_COHERENT
> pages and that it can't be on any LRU list (because the lru list_head in struct
> page is aliased by pgmap and zone_device_data). Maybe I'll find more differences
> if I keep digging.
>
> Regards,
>   Felix
>
>
>>
>> Jason