Re: [PATCH v6 01/10] mm: add zone device coherent type memory support

[Felix Kuehling <felix.kuehling@amd.com>]

Am 2022-02-18 um 14:26 schrieb Jason Gunthorpe:
> On Fri, Feb 18, 2022 at 02:20:45PM -0500, Felix Kuehling wrote:
>> Am 2022-02-17 um 19:19 schrieb Jason Gunthorpe:
>>> On Thu, Feb 17, 2022 at 04:12:20PM -0500, Felix Kuehling wrote:
>>>
>>>> I'm thinking of a more theoretical approach: Instead of auditing all users,
>>>> I'd ask, what are the invariants that a vm_normal_page should have. Then
>>>> check, whether our DEVICE_COHERENT pages satisfy them. But maybe the concept
>>>> of a vm_normal_page isn't defined clearly enough for that.
>>> I would say the expectation is that only 'page cache and anon' pages
>>> are returned - ie the first union in struct page
>>>
>>> This is because the first file in your list I looked at:
>>>
>>> static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
>>> 				unsigned long addr, unsigned long end,
>>> 				struct mm_walk *walk)
>>>
>>> {
>>> 		page = vm_normal_page(vma, addr, ptent);
>>> [..]
>>> 		if (pageout) {
>>> 			if (!isolate_lru_page(page)) {
>>>
>>> Uses the LRU field, so this is incompatible with all the other page
>>> types.
>>>
>>> One mitigation of this might be to formally make vm_normal_page() ==
>>> 'pte to page cache and anon page' and add a new function that is 'pte
>>> to any struct page'
>>>
>>> Then go through and sort callers as appropriate.
>>>
>>> The 'pte to page cache and anon page' can detect ZONE_DEVICE by
>>> calling is_zone_device_page() insted of pte_devmap() and then continue
>>> to return NULL. This same trick will fix GUP_fast.
>> Sounds good to me. What about vm_normal_page_pmd? Should we remove the
>> pmd_devmap check from that function as well. I'm not even sure what a huge
>> zone_device page would look like, but maybe that's a worthwhile future
>> optimization for our driver.
> IIRC there are other problems here as PMDs are currently wired to THPs
> and not general at all..
>
> We have huge zone_device pages, it is just any compound page of
> contiguous pfns - you should be aggregating any contiguous string of
> logical PFNs together into a folio for performance. If the folio is
> stuffed into a PMD or not is a different question..
>
>> I'd propose the function names vm_normal_page and vm_normal_or_device_page
>> for the two functions you described.
> I wouldn't say device_page, it should be any type of page - though
> device_page is the only other option ATM, AFIAK.

Ok, then how about vm_normal_lru_page and vm_normal_any_page?

Regards,
   Felix


>
>> current vm_normal_page with the pte_devmap check removed. vm_normal_page
>> could be implemented as a wrapper around vm_normal_or_device_page, which
>> just adds the !is_zone_device_page() check.
> Yes
>
> Jason