Re: [PATCH v7 04/14] mm: add device coherent vma selection for memory migration

From: Alistair Popple <apopple@nvidia.com>
To: David Hildenbrand <david@redhat.com>
Cc: Alex Sierra <alex.sierra@amd.com>,
	jgg@nvidia.com, Felix.Kuehling@amd.com, linux-mm@kvack.org,
	rcampbell@nvidia.com, linux-ext4@vger.kernel.org,
	linux-xfs@vger.kernel.org, amd-gfx@lists.freedesktop.org,
	dri-devel@lists.freedesktop.org, hch@lst.de, jglisse@redhat.com,
	willy@infradead.org, akpm@linux-foundation.org
Subject: Re: [PATCH v7 04/14] mm: add device coherent vma selection for memory migration
Date: Thu, 30 Jun 2022 21:44:55 +1000	[thread overview]
Message-ID: <878rpe73t3.fsf@nvdebian.thelocal> (raw)
In-Reply-To: <956b1c51-b8f1-0480-81ca-5d03b45110f7@redhat.com>

David Hildenbrand <david@redhat.com> writes:

> On 29.06.22 05:54, Alex Sierra wrote:
>> This case is used to migrate pages from device memory, back to system
>> memory. Device coherent type memory is cache coherent from device and CPU
>> point of view.
>>
>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
>> Reviewed-by: Alistair Poppple <apopple@nvidia.com>
>> Signed-off-by: Christoph Hellwig <hch@lst.de>
>
>
> I'm not too familiar with this code, please excuse my naive questions:
>
>> @@ -148,15 +148,21 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,
>>  			if (is_writable_device_private_entry(entry))
>>  				mpfn |= MIGRATE_PFN_WRITE;
>>  		} else {
>> -			if (!(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM))
>> -				goto next;
>
> Why not exclude MIGRATE_VMA_SELECT_DEVICE_PRIVATE here? IIRC that would
> have happened before this change.

I might be missing something as I don't quite follow - this path is for
normal system pages so we only want to skip selecting them if
MIGRATE_VMA_SELECT_SYSTEM or MIGRATE_VMA_SELECT_DEVICE_COHERENT aren't
set.

Note that MIGRATE_VMA_SELECT_DEVICE_PRIVATE doesn't apply here because
we already know it's not a device private page by virtue of
pte_present(pte) == True.

>>  			pfn = pte_pfn(pte);
>> -			if (is_zero_pfn(pfn)) {
>> +			if (is_zero_pfn(pfn) &&
>> +			    (migrate->flags & MIGRATE_VMA_SELECT_SYSTEM)) {
>>  				mpfn = MIGRATE_PFN_MIGRATE;
>>  				migrate->cpages++;
>>  				goto next;
>>  			}
>>  			page = vm_normal_page(migrate->vma, addr, pte);
>> +			if (page && !is_zone_device_page(page) &&
>
> I'm wondering if that check logically belongs into patch #2.

I don't think so as it would break functionality until the below
conditionals are added - we explicitly don't want to skip
is_zone_device_page(page) == False here because that is the pages we are
trying to select.

You could add in this:

>> +			    !(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM))

But then in patch 2 we know this can never be true because we've already
checked for !MIGRATE_VMA_SELECT_SYSTEM there.

>> +				goto next;
>> +			else if (page && is_device_coherent_page(page) &&
>> +			    (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_COHERENT) ||
>> +			     page->pgmap->owner != migrate->pgmap_owner))
>
>
> In general LGTM